Problem: What is the vector x such that A \cdot x = b?
Excerpt from The Algorithm Design Manual: Solving linear systems is a problem of such scientific and commercial importance that excellent codes are readily available. There is likely no good reason to implement your own solver, even though the basic algorithm (Gaussian elimination) is one we learned in high school. This is especially true if you are working with large systems.
Gaussian elimination is based on the fact that the solution to a system of linear equations is invariant under scaling (multiplying both sides by a constant; i.e. if x=y, then 2x=2y) and adding equations (i.e. the solution to the equations x=y and w=z is the same as the solution to x=y and x+w=y+z). Gaussian elimination scales and adds equations so as to eliminate each variable from all but one equation, leaving the system in such a state that the solution can just be read off from the equations.
|Parallel Numerical Algorithms by D. Keyes and A. Sameh and V. Venkatarishnan
|Matrix Computations by Gene H. Golub and Charles F. Van Loan
|Elementary Numerical computing with Mathematica by R. Skeel and J. Keiper
|Numerical methods and analysis by J. Buchanan and P. Turner
|Parallel Algorithms for Matrix Computations by K. Gallivan
|Introduction to Algorithms by T. Cormen and C. Leiserson and R. Rivest and C. Stein
|Introduction to Parallel and Vector Solution of Linear Systems by J. Ortega
|Numerical Mathematics and Computing by W. Cheney and D. Kincaid
|The Design and Analysis of Computer Algorithms by A. Aho and J. Hopcroft and J. Ullman