### Using Two Dimensional Arrays: The Gaussian Elimination Algorithm for Simultaneneous Equations

• Introduction: Simultaneous Equations

• In Mathematics, simultaneous equations are a set of equations containing multiple variables.

This set of equations is often referred to as a system of equations.

• Example 1: 2 simultaneous equations (with 2 unknowns x1 and x2 )

 ``` 3 x1 + 7 x2 = 78 8 x1 + 5 x2 = 45 ```

Example 2: 3 simultaneous equations (with 3 unknowns x1, x2 and x3 )

 ``` 5 x1 + 6 x2 - 3 x3 = 23 3 x1 - 4 x2 + 8 x3 = 25 -6 x1 + x2 - 2 x3 = 19 ```

• Matrix/vector representation of simultaneous equations

• From High School, you should have learned about matrices (if not, here is the definition):

 Matrix = a 2-dimensional grid of number

Example:

 ``` +- -+ | 5 6 -3 | A = | 3 -4 8 | | -6 1 -2 | +- -+ ```

• The elements in a matrix are numbered as follows:

 ``` +- -+ | a11 a12 a13 | A = | a21 a22 a23 | | a31 a32 a33 | +- -+ ```

• From High School, you should also have learned about vectors (if not, here is the definition):

 Column vector = a column of number Row vector = a row of number

Example: column vector

 ``` +- -+ | 2 | x = | 5 | | -1 | +- -+ ```

• The elements in a vector are numbered as follows:

 ``` +- -+ | x1 | x = | x2 | | x3 | +- -+ ```

• From High School, you should also have learned about Matrix-vector multiplication (if not, here is the definition):

 ``` +- -+ +- -+ | a11 a12 a13 | | x1 | A×x = | a21 a22 a23 | × | x2 | | a31 a32 a33 | | x3 | +- -+ +- -+ +- -+ def | a11x1 + a12x2 + a13x3 | = | a21x1 + a22x2 + a23x3 | | a31x1 + a32x2 + a33x3 | +- -+ ```

The product A×x is a vector !

• Writing simultaneous equations in matrix/vector form

Example:

 ``` 5 x1 + 6 x2 - 3 x3 = 23 3 x1 - 4 x2 + 8 x3 = 25 -6 x1 + x2 - 2 x3 = 19 View it as a column vector (of 3 elements): +- -+ +- -+ | 5 x1 + 6 x2 - 3 x3 | | 23 | ==> | 3 x1 - 4 x2 + 8 x3 | = | 25 | | -6 x1 + x2 - 2 x3 | | 19 | +- -+ +- -+ The left column vector can be written as a Matrix-vector product: +- -+ +- -+ +- -+ | 5 6 -3 | | x1 | | 23 | <==> | 3 -4 8 | × | x2 | = | 25 | | -6 1 -2 | | x3 | | 19 | +- -+ +- -+ +- -+ Matrix column column vector vector ```

• Solving simultaneous equations

• A very well-known technique used to solve simultaneous equations is the Gaussian Elimination Method

• In this webpage, we will study:

 The elementary row operations used in the Gaussian elemination method A basic Gaussian elmination method to solve systems of simultaneous equations (There are more advanced variants to select pivots to maximize computational accuracy) A computer program that using a two-dimensional array to implement the Gaussian elmination method. You can use this program to solve homework problems in you Linear Algebra class :-)

• Elementary row operations

• Consider the following system of equations:

 ``` 2 x1 + 4 x2 = 16 3 x1 + 2 x2 = 12 ```

The solution is:

 ``` x1 = 2 x1 = 3 Because: 2 × 2 + 4 × 3 = 4 + 12 = 16 3 × 2 + 2 × 3 = 6 + 6 = 12 ```

• Equality of the Left hand Side (LHS) and the Right hand Size (RHS):

• The equality (=) of the LHS and the RHS means that:

 the values in the LHS and RHS of the equal "=" sign are in balance.

• The equations can be represented intuitively in the following manner:

 ``` 2 x1 + 4 x2 = 16 3 x1 + 2 x2 = 12 ```

• Operations that can maintain the equality of LHS and RHS:

• Equality maintaining operation #1:

 If we multiply (or divide) the LHS and the RHS with the same factor, the equality will still hold.

Example: multiplying first equation by 0.5

 ``` 0.5 × (2 x1 + 4 x2) = 0.5 × 16 ==> x1 + 2 x2 = 8 ```

(This makes perfect sense: because take away half from both sides of the scale will keep the scale in balance)

• Equality maintaining operation #2:

 We can add (or subtract) any factor of one equation to another equation, and the resulting equality will still hold

Example:

 ``` x1 + 2 x2 = 8 (eq 1) 3 x1 + 2 x2 = 12 (eq 2) (eq 2) ===> 3 x1 + 2 x2 = 12 3 x (eq 1) ===> 3 x1 + 6 x2 = 24 − --------------------------- 0 x1 − 4 x2 = −12 ```

(This makes perfect sense: because if both scales are in balance, then adding the stuff from the left side of one scale to the left side of the other scale, and simultaneously adding the stuff from the left side of one scale to the left side of the other scale will keep the scale in balance.

We can do this any number of times, even with fractional amounts)

• Elementary row operations: operations that maintain equality

• Row multiplication:

 Each element in a row can be multiplied by a non-zero constant.

Example:

 ``` If: 2 x1 + 4 x2 = 16 Then: x1 + 2 x2 = 8 (multiply each element in the row by 0.5) ```

 A row can be replaced by the sum of that row and a multiple of another row.

Example:

 ``` If: x1 + 2 x2 = 8 (eq 1) 3 x1 + 2 x2 = 12 (eq 2) Then: we can replace (eq2) with (eq2 - 3×eq1): x1 + 2 x2 = 8 (eq 1) -4 x2 = -12 (eq 2 - 3×eq1) ```

• The Gaussian Elimination Method

• Overview:

 The Gaussian elimination method use the two elementary row operations to transform the original system of simultaneous equations into a trivial system of of simultaneous equations

• Trivial system of simultaneous equations:

• The trivial system of simultaneous equations is the following set of equations:

 ``` x1 + 0 × x2 = value1 0 × x1 + x2 = value2 ```

It's called "trivial" because solving this system requires no effort at all...

The solution is:

 ``` x1 = value1 x2 = value2 ```

• A small example of the Gaussian Elimination method

• Solve:

 ``` 2 x1 + 4 x2 = 16 3 x1 + 2 x2 = 12 ```

• Step 1A: make the first coefficient of the first row equal to 1

• Original equations:

 ``` 2 x1 + 4 x2 = 16 3 x1 + 2 x2 = 12 ```

• Multiply first equation by 0.5 (or divide by 2):

 ``` 1 x1 + 2 x2 = 8 3 x1 + 2 x2 = 12 ```

Step 1B: make all other coefficients in the first column equal to ZERO (by adding a multiple of the first row to the other rows)

• Original equations:

 ``` 1 x1 + 2 x2 = 8 (eq1) 3 x1 + 2 x2 = 12 (eq2) ```

• Subtract equation eq1 from equation eq2

 ``` 1 x1 + 2 x2 = 8 (eq1) 0 x1 - 4 x2 = -12 (eq2 - 3×eq1) ```

Repeat

• Step 2A: make the second coefficient of the second row equal to 1

• Original equations:

 ``` x1 + 2 x2 = 8 -4 x2 = -12 ```

(Note: the second coefficient of the equation is −4... I did not show the first coefficient (0x1) because it is ZERO)

• Multiply first equation by −0.25 (or divide by −4):

 ``` x1 + 2 x2 = 8 1 x2 = 3 ```

Step 2B: make all other coefficients in the second column equal to ZERO (by adding a multiple of the second row to the other rows)

• Original equations:

 ``` x1 + 2 x2 = 8 (eq3) 1 x2 = 3 (eq4) ```

• Subtract equation eq4 from equation eq3

 ``` x1 + 0 x2 = 2 (eq3 - 2×eq4) 1 x2 = 3 (eq4) ```

Resulting trivial system of simultaneous equations:

 ``` x1 = 2 x2 = 3 ```

Which also give the solution...

• The pivot

• Dictionary definition of pivot:

 Pivot = the center point of any rotational system To pivot = to cause to rotate

• Computer Science definition:

 Pivot (a.k.a. pivot element) = the coefficient in the set of equations used to make all other coefficients in a column to be equal to ZERO (0)

• Example 1: the pivot element in Step 1 (A/B) above is the coefficient of a11

 ``` Simultaneous equations: 2 x1 + 4 x2 = 16 3 x1 + 2 x2 = 12 ``` ``` In matrix/vector form: +- -+ +- -+ +- -+ | 2 4 | | x1 | = | 16 | | 3 2 | | x2 | | 12 | +- -+ +- -+ +- -+ ```

The element 2 in the matrix is coefficient a11

• Example 2: the pivot element in Step 2 (A/B) above is the coefficient of a22

 ``` Simultaneous equations: x1 + 2 x2 = 8 -4 x2 = -12 ``` ``` In matrix/vector form: +- -+ +- -+ +- -+ | 1 2 | | x1 | = | 8 | | 0 -4 | | x2 | | -12 | +- -+ +- -+ +- -+ ```

The element −4 in the matrix is coefficient a22

• Selecting pivots in the Gaussian Elimination Method

• Selecting the pivot in the Gaussian Elimination Algorithm:

 There are sophisticated ways to select pivots in the Gaussian Elimination Algorithm to reduce computational errors made by the algorithm We will only discussed the basic Gaussian Elimination Algorithm using a very simple pivot selection algorithm

• The basic pivot selection algorithm:

 Pivot used in iteration 1: coefficient a11 Pivot used in iteration 2: coefficient a22 And so on... Pivot used in iteration n: coefficient ann

• A larger example of the Gaussian Elimination method

• Solve:

 ``` 2 x1 + 4 x2 + 6 x3 = 14 x1 + 4 x2 + 3 x3 = 3 3 x1 + 5 x2 + 2 x3 = 2 ```

We do not need to write out the variable x1, x2 and x3

We can represent these simulatneous equation by the following shorter form:

 ``` +- -+ | 2 4 6 | 14 | | 1 4 3 | 3 | | 3 5 2 | 2 | +- -+ ```

Note:

 This is not a matrix It is a short hand used to execute elementary row operations We must remember that column i in this short hand form corresponds to the variable xi.

• The Gaussian Elimination Algorithm on the short hand form:

• Iteration 1:

• Step 1A: select pivot a11

 ``` +- -+ | 2 4 6 | 14 | | 1 4 3 | 3 | | 3 5 2 | 2 | +- -+ ```

• Step 1B: divide the entire pivot row by the pivot a11

 ``` +- -+ | 1 2 3 | 7 | | 1 4 3 | 3 | | 3 5 2 | 2 | +- -+ ```

This step is a.k.a.: normalizing the pivot row

• Step 1C: make all other coefficient under the pivot element ZERO.

 ``` +- -+ | 1 2 3 | 7 | | 1 4 3 | 3 | - 1×(Pivot row) | 3 5 2 | 2 | - 3×(Pivot row) +- -+ Result: +- -+ | 1 2 3 | 7 | | 0 2 0 | -4 | | 0 -1 -7 | -19 | +- -+ ```

This step is a.k.a.: sweeping the pivot column

• Iteration 2:

• Step 2A: select pivot a22

 ``` +- -+ | 1 2 3 | 7 | | 0 2 0 | -4 | | 0 -1 -7 | -19 | +- -+ ```

• Step 2B: normalizing the pivot row by divide the entire pivot row by the pivot a22

 ``` +- -+ | 1 2 3 | 7 | | 0 1 0 | -2 | | 0 -1 -7 | -19 | +- -+ ```

• Step 2C: sweep the pivot column

 ``` +- -+ | 1 2 3 | 7 | - 2×(Pivot row) | 0 1 0 | -2 | | 0 -1 -7 | -19 | -(-1)×(Pivot row) +- -+ Note: - (-1)×(Pivot row) <==> + 1×(Pivot row) Result: +- -+ | 1 0 3 | 11 | | 0 1 0 | -2 | | 0 0 -7 | -21 | +- -+ ```

• Iteration 3:

• Step 3A: select pivot a33

 ``` +- -+ | 1 0 3 | 11 | | 0 1 0 | -2 | | 0 0 -7 | -21 | +- -+ ```

• Step 3B: normalizing the pivot row by divide the entire pivot row by the pivot a33

 ``` +- -+ | 1 0 3 | 11 | | 0 1 0 | -2 | | 0 0 1 | 3 | +- -+ ```

• Step 3C: sweep the pivot column

 ``` +- -+ | 1 0 3 | 11 | - 3×(Pivot row) | 0 1 0 | -2 | - 0×(Pivot row) | 0 0 1 | 3 | +- -+ Result: +- -+ | 1 0 0 | 2 | | 0 1 0 | -2 | | 0 0 1 | 3 | +- -+ ```

• The trivial system's solution:

 ``` x1 = 2 x2 = -2 x3 = 3 ```

• Storing information used in the Gaussian Elmination Algorithm

• Variables used to store information used by the Gaussian Elimination Algorithm:

 ``` double[][] A; // The Coefficient "matrix" double[] b; // The right hand side "vector" int n; // number of unknowns (and size of A and b) ```

• Example:

 ``` Simultaneous equations: 2 x1 + 4 x2 + 6 x3 = 14 x1 + 4 x2 + 3 x3 = 3 3 x1 + 5 x2 + 2 x3 = 2 ``` ``` Representation: n = 3; double[][] A = new double[n][n]; double[] b = new double[n]; A[0][0] = 2; A[0][1] = 4; A[0][2] = 6; b[0] = 14; A[1][0] = 1; A[1][1] = 4; A[1][2] = 3; b[0] = 3; A[2][0] = 3; A[2][1] = 5; A[2][2] = 2; b[0] = 2; ```

• Important note:

• In Mathematics, the indices begin with 1 (ONE):

 a11, a12 and so on...

• However, in Computer Scoence, the array indices begin with 0 (ZERO)

 A[0][0] = a11 A[0][1] = a12 And so on !!!

• The Gaussian Elmination Algorithm in Java

• From the above discussion, we have the following high level description of the Gaussian Elimination algorithm:

 ``` Read in n, A and b; for ( each row i from 0 to n-1 ) do { (Step i A): Select pivot pivot = A[i][i]; (Step i B): Normalize row i for ( each element in row i ) divide element by the pivot; (Step i C): Sweep column i; for row k from 0 to n-1, except row i do { factor = A[k][i]; Subtract "factor" × (row i) from (row k) } } ```

• Refining (Step B) Normalize row i

• Description:

 ``` for ( each element in row i ) divide element by the pivot; ```

• Refinement:

 ``` for ( j = 0; j < n; j++ ) a[i][j] = a[i][j] / pivot; // elements in A b[i] = b[i] / pivot; // element in the RHS ```

• Refining (Step C): Sweep column i

• Description:

 ``` (Row i is the pivot row) for row k from 0 to n-1, except row i do { factor = A[k][i]; Subtract "factor" × (row i) from (row k) } ```

• Refinement:

 ``` (Row i is the pivot row) for ( k = 0, 1, 2, ...., n-1, except when k=i ) { factor = A[k][i]; /* -------------------------------- Go through ALL elements in row k --------------------------------- */ for ( j = 0; j < n; j++ ) { Subtract (factor×a[i][j]) from a[k][j] } Subtract (factor×b[i]) from b[k] } ```

• Java program:

 ``` import java.util.Scanner; public class Gauss1 { public static void main(String[] args) { Scanner in = new Scanner(System.in); double[][] A; // The coefficients double[] b; // The RHS double pivot, factor; int n; // Number of variables int i, j, k; System.out.print("Number of unknows: "); n = in.nextInt(); // Read in problem size A = new double[n][n]; // Reserve space for coefficient matrix b = new double[n]; // Reserve space for RHS vector /* -------------------------------------- Read in the rows, one row at a time -------------------------------------- */ for (i = 0; i < n; i++) { // Read in row i's coefficients for (j = 0; j < n; j++) { System.out.print("A["+i+"]["+j+"] = "); A[i][j] = in.nextDouble(); } // Read in row i's RHS System.out.print("b["+i+"] = "); b[i] = in.nextDouble(); } /* ------------------------------------ The Gaussian elimination method ------------------------------------ */ for (i = 0; i < n; i++) { // Process row i /* --------------------- (A) Select pivot --------------------- */ pivot = A[i][i]; /* --------------------- (B) Normalize row i --------------------- */ for (j = 0; j < n; j++) A[i][j] = A[i][j]/pivot; b[i] = b[i]/pivot; /* --------------------- (C) Sweep using row i --------------------- */ for (k = 0; k < n; k++) { if ( k != i ) { factor = A[k][i]; for (j = 0; j < n; j++) A[k][j] = A[k][j] - factor*A[i][j]; b[k] = b[k] - factor*b[i]; } } } System.out.println("======================"); System.out.println("Solution:"); for (i = 0; i < n; i++) System.out.println("x" + (i+1) + " = " + b[i]); } } ```

• Example Program: (Demo above code)

How to run the program:

 Right click on link and save in a scratch directory To compile:   javac Gauss1.java To run:          java Gauss1

• Making the Gaussian Elimination Algorithm into a Black Box

• Inputs of the Gaussian Elimination Algorithm:

 The coefficient matrix (double[][] A) The RHS (double[] b)

• Output of the Gaussian Elimination Algorithm:

• The solution (vector)

Note:

 We can return the solution stored inside the input parameter double[] b

• The Gaussian Elimination Algorithm written as a method:

 ``` public class MathTools { public static double[] GaussianElim(double[][] A, double[] b) { double pivot, factor; int i, j, k; int n = b.length; // # unknowns /* ----------------------------- Gaussian elimination method ----------------------------- */ for (i = 0; i < n; i++) { // Process row i /* --------------------- (A) Select pivot --------------------- */ pivot = A[i][i]; /* --------------------- (B) Normalize row i --------------------- */ for (j = 0; j < n; j++) A[i][j] = A[i][j]/pivot; b[i] = b[i]/pivot; /* --------------------- (C) Sweep using row i --------------------- */ for (k = 0; k < n; k++) { if ( k != i ) { factor = A[k][i]; for (j = 0; j < n; j++) A[k][j] = A[k][j] - factor*A[i][j]; b[k] = b[k] - factor*b[i]; } } } return(b); // The answer is in double[] b } } ```

• How to use the GaussianElim() method:

 ``` public class Gauss2 { public static void main(String[] args) { Scanner in = new Scanner(System.in); double[][] A; double[] b, x; double pivot, factor; int n; // Number of variables int i, j, k; System.out.print("Number of unknows: "); n = in.nextInt(); // Read in problem size A = new double[n][n]; // Reserve space for coefficient matrix b = new double[n]; // Reserve space for RHS vector // Read in the rows, one row at a time for (i = 0; i < n; i++) { // Read in row i's coefficients for (j = 0; j < n; j++) { System.out.print("A["+i+"]["+j+"] = "); A[i][j] = in.nextDouble(); } // Read in row i's RHS System.out.print("b["+i+"] = "); b[i] = in.nextDouble(); } x = MathTools.GaussianElim(A, b); System.out.println("Solution:"); for (i = 0; i < n; i++) System.out.println("x[" + (i+1) + "] = " + x[i]); } } ```

• Example Program: (Demo above code)

How to run the program:

 Right click on link and save in a scratch directory To compile:   javac Gauss2.java To run:          java Gauss2

How to use the input:

 Right click on link and save the Gauss.inp file in the same scratch directory Run program using:          java Gauss2 < Gauss.inp

• Final note...

• The MathTools.GaussianElim() will modify the actual input parameters because as you know:

 Array elements in the input parameters can be modified by method through the array reference input parameter See: click here

• If you do not want a method to modify an array parameter, then you must do the following:

 Inside the method, create a new array of the same size and dimension as the array that you want to preserve Copy the elements of the input array into the created array Modify the copy