Asnetcomp Tutorials: What is Gradient ?

What is Gradient ?

Gradient is not just a slope of of a certain function. It is an operator to change a scalar field to a vector field. A scalar field has no direction like temperature, density, energy, and Electric Potential of Voltage. It seems simple as you do not have to analyze it (a scalar field) in terms of direction, just it exists there or anywhere around us. But it is sometime inconvenient when you want to analyze it or simply want to know more about it .

Wiki (26-Jan-2012) introduces gradient as

"
In vector calculus, the gradient of a scalar field is a vector field that points in the direction of the greatest rate of increase of the scalar field, and whose magnitude is that rate of increase.

"

It is more like a definition, not an explanation to those who do not know gradient well.

Wiki further introduces some well quoted examples as below:

"
Consider a room in which the temperature is given by a scalar field, $T$ , so at each point $(x, y, z)$ the temperature is $T (x, y, z)$ . (We will assume that the temperature does not change over time.) At each point in the room, the gradient of $T$ at that point will show the direction the temperature rises most quickly. The magnitude of the gradient will determine how fast the temperature rises in that direction.

"

This is the same as the above brief explanation. No enough explanation to those who do not know gradient well. Please note that time is not involved and only in terms of location (x,y,z) in this explanation.

Wiki continues

"
Consider a surface whose height above sea level at a point $(x, y)$ is $H (x, y)$ . The gradient of $H$ at a point is a vector pointing in the direction of the steepest slope or grade at that point. The steepness of the slope at that point is given by the magnitude of the gradient vector.
"

Again, the same thing, but slightly different from the case of temperature of a room.

Temperature of a room - Temperature (scalar field) in 3D (room) --- (A)
Height (scalar) to or from the see level. The see level is regarded as constant --- (B)

(B) is confusing when we compare with the case of (A), in which a scalar quantity of the temperature can be measured at any point and may differ in 3D space (x, y, z). $T (x, y, z).$ While in (B) the height is the quantity in z-direction.

$(A) T (x, y, z) may have some or any value (say 20 deg C or 68 deg F, or simply 1, 2, 3 ...)$
$(B)$ $H (x, y) has some value of height (say 100m or 328 feet, or simply$ $1, 2, 3 ...)$

$But the problem is, can we write in the following ways ?$

$(A) T (x, y, z) = 20 (68)$
$(B) H (x, y) = 100 (328)$

$Yes but this is one type. We can use any value as$

$(A) T (x, y, z) = c (constant)$
$(B) H (x, y) = c (constant)$
$(A) is a bit difficult to visualize to express 'c'.$

$We can use a certain color in 3D space where 'c' occupies. When 'c' changes we use different colors or different tones of gray.$
$While (B) is easier to visualize as we can express it$ $H (x, y) = z or$ $H (x, y, z) = z$

$Level line and Level surface$
$This is an important concept to capture the concept of gradient.$

$(B) H (x, y) = z or$ $H (x, y, z) = z (z = constant)$
$We can plot the points which show a certain constant z value as height above the x-y plane in the 3D space. It will be a line or curve. We also can plot these on$ $the$ $2D x-y plane though there is no height.$ $It will be a line or curve called <Level line> showing the places in x-y plane having the selected same level (value) in the direction of z (height).$ $z (height) can take any value but should remain constant in the function. This is the point.$

$(A) T (x, y, z) = c (constant)$

$We can apply a similar thing to this equation. Now z in a variable of the function, not a constant. The result we will get will be a surface in x-y-z 3D space showing the selected same level (value) of < T > and mostly a curved surface in 3D. Now I think that you can visualize this.$

How about the following cases ?

A (x, 0) = c or A (x, c) = c (constant)   This means x = c
A (0, y) = c or A (c, y) = c (constant)   This means y = c

We can plots <level points>, which produces a straight line on the x-y plane with x being constant for any y value, a vertical straight line, parallel with the y-axis; and the one with y being constant for any x value, a horizontal straight line, parallel with the x-axis. If c = 0, then the line is either y-axis or x-axis itself.

So we can find some relations between dimensions (Ds).

0D (point) --> 1D (line) A (x, 0) = c or A (x, c) = c (constant)
$We can plot$ $Level points in 1D space (on a straight line).$
$Notes:$
$1) x can be either separate spots. But cannot be continuous (which is a line).$
$2) We can say that x depends on c, actually x = c.$

1D --> 2D   $H (x, y) = c or$ $H (x, y, c) = c$ $(z = constant)$
$We can plot$ $Level lines (usually curves) in 2D space (on a plane).$
$Notes:$
$1) H generally indicates a function of x and y here with a constant c.$
$2) We can say that x and y values depend on c, actually H(x,y) = c. But c is constant (can be changed but not a variable element of the function). So c restricts the freedom of variables x and y.$

$Example$

$xy = 1 or H(x,y) - 1 = 0$

x =1, then y =1
x= 2, then y= 1/2
x= 3, then y= 1/3

$These are on the line of y = 1/x.$

$x+y = 0 or F(x,y) = 0$

x =1, then y =-1
x= 2, then y= -2
x= 3, then y= -3

These are on the line of y = -x

Seems stupidly simple but important relations as you can see later.

2D --> 3D $T (x, y, z) = c (constant)$
$We can plot$ $Level surfaces$ $(usually a curved surface) in 3D space.$

Notes

1) $Like above c is constant (can be changed but not a variable element of the function). So c restricts the freedom of variables x, y and z.$

$Derivative$

Wiki shows the calculation formulas of gradient.

Expression in 3-dimensional rectangular coordinates

The form of the gradient depends on the coordinate system used. In Cartesian coordinates, the above expression expands to

$\nabla f(x, y, z) = \left(\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}, \frac{\partial f}{\partial z}\right)$ --- (1)

which is often written using the standard unit vectors $\hat{\mathbf{i}}, \hat{\mathbf{j}}, \hat{\mathbf{k}}$ as

$\frac{\partial f}{\partial x} {\mathbf{\hat i}} + \frac{\partial f}{\partial y} \hat{\mathbf{j}} + \frac{\partial f}{\partial z} \hat{\mathbf{k}}$ ---- (2)

(1) is a partial differentiation of a function f in the three mutually perpendicular directions, x, y and z - ∂f/∂x, ∂f/∂y and ∂f/∂y. Is this a vector function showing a related vector field? It does not seem so as no vector shown. It can be a vector (function) as it shows a derivative of f (scalar function) in each direction of x, y, z. Meanwhile (2) seems more like a vector as three unit vectors are shown and please note <+> sign, which means Grad f is a addition of each partial differentiation of a function f with respect to x, y, and z and multiplied by a i, j, and k unit vector respectively. Grad f (x, y, z) is a addition of of ∂f/∂x, ∂f/∂y and ∂f/∂y.

However, even if you already know the partial differentiation it is difficult to get the idea of the following underlined italic part intuitively.

"
the gradient of a scalar field is a vector field that points in the direction of the greatest rate of increase of the scalar field, and whose magnitude is that rate of increase.

"

Simply why the greatest ?

Some more explanations are needed.

1. What is differentiation? or What is derivative?

Spatially, the derivative is a rate change of a function with respect to a certain direction.

y = 1, the derivative is 0. Though 'x' is not shown in this equation but it is usually regarded as the rate change of y of this function with respect to 'unit x' is 0.
where 'unit x' is the direction in the x-axis and the value is one (1).

Besides 1, any other constant like 1/2, 2,10, the derivative of

y = 1/2, 2, 10 etc

all 0 (zeer). This should be noticed in that derivative of a function having a independent constant the constant portion disappeared. So a constant of the scalar field function does not affect the derivative.
y = x, the derivative is 1. The rate change of y of this function with respect to 'unit x' is 1, constant, no change at any x and the y values on the line of y = x (straight line).

y= x², the derivative is 2x. This time the rate change of 'y' value to 'unit x' of this function is not a number (constant) but a function "y = 2x" with respect to 'unit x'. So

when x = 0, y = 0   --> the derivative at (0, 0) is 0. (y' = 2 x 0 = 0)
when x = 1, y = 1 --> the derivative at (1, 1) is 2. (y' = 2 x 1 = 2)
when x = 2, y = 4 --> the derivative at (2, 4) is 4. (y = 2 x 2 = 4)
when x = 10, y = 100 --> the derivative at (10, 100) is 20. (y' = 2 x 10 = 20)

The above numbers 0, 2, 4, 20 are the tangent value of the tangent line at each point.

How about the first part - direction - the direction of the greatest rate of increase of the scalar field ? How to get this direction? By differentiation (or getting derivative) as well. This is the point of understanding gradient.

The 2nd part of the definition of gradient " (the) magnitude is that rate of increase (or decrease as well or more generally, just change)" is referring to this. So differentiation (partial differentiation in case of gradient) of a scalar function makes sense to get this value.

We have seen the two types of equations above.

y = x
y= x²

y is dependent on x. But we can change them to

x = y
x = +/- √ y

Now x depends on y. But these are actually the same as far as the x-y relation is concerned.

$H (x, y) = c (constant)$
$T (x, y, z) = c (constant)$

$These two are different. x, y, z are independent and controlled by the function of H or T to make the equation correct.$

1) y = x
2) y= x²

We already had derivative rate change of y with respect to x.

1)-a) y' = 1
2)-a) y' = 2x

Another way, we change 1) and 2)

1)-b) y - x = 0
2)-b) y - x² = 0

Then differentiate with respect to x

1)-c) dy/dx - dx/dx = d0/dx
dy/dx - 1= 0
dy/dx = 1 (same result as y' = 1)

2)-c) dy/dx - dx²/dx = d0/dx
    dy/dx - 2x = 0
          dy/dx = 2x (same result as y' = 2x)

The above two are just another presentation of differentiating with respect to x. But we can apply this method to

$H (x, y) = c (constant)$
$T (x, y, z) = c (constant)$

H' $(dx/dx, dy/dx) = dc/dx = 0$ $with respect to x$
H' $(dx/dy, dy/dy) = dc/d = 0$ $with respect to y$

$T' (dx /dx, d y/dx, dz/ dx) = dc/dx = 0$ $with respect to x$
$T' (dx /dy, d y/dy, dz/ dy) = dc/dy= 0$ $with respect to y$
$T' (dx /dz, d y/dz, dz/ dz) = dc/dz= 0$ $with respect to z$

This procedure is wrong. We do not have functions of H and T and x, y and z are values, not equations.

$H (x, y) = c (constant)$
$We applied this to the following very simple equation.$

$y = x$

Then change this to

H (x, y) = y - x = 0
$Which means$
$The function H of x and y is$

$y - x = 0$
$We can write$

$H (x, y) = 0$
$Then let's apply the gradient operation (scalar to vector)$
$grad$ $H (x, y) = ($ $- dx/dx$ $) i + (dy/dy) j = -1 i + 1$ $j (=0)$

$What does this mean? We seem to be getting closer to <$ $the direction of the greatest rate of increase of the scalar field, and whose magnitude is that rate of increase>, at least the direction and some magnitude, the greatest is another issue.$

$<$ $-1 i + 1$ $j > is a vector with the direction and we can calculate its magnitude.$

$Direction: j / - i or - (j / i)$

       \      l
       \    l
           \ l ^
<-- i     \l $l j$
$----------------------$
              l \
              l \
              l    \
              l      \

Magnitude: $\sqrt 2$

The direction is normal to the line of y = x or <x - y = 0>. Let' try one more.

$H (x, y) = y -$ $x 2 (=0)$
$which means$
$The function H of x and y is$

$y - x 2 = 0$

$We can write$

$H (x, y) = 0$

Applying the $gradient operation (scalar to vector)$
$grad$ $H (x, y) = - ($ $dx 2 /dx$ $) i + (dy/dy) j = -2x i + 1$ $j$ $(=0)$

$This is a function of x. The value of x can change.$
$At (0 ,0) : -2 x 0$ $i + 1$ $j =$ $0$ $i + 1$ $j = +$ $1 j Magnitude 1 -- (a)$

$At (1, 1) : -2$ $i + 1 j$ $Magnitude$ $\sqrt 5 -- (b)$

$At (-1, 1) : -2 x (-1)$ $i + 1 j = 2$ $i + 1$ $j$ $Magnitude \sqrt 5 -- (c)$
$At (2, 4): -2 x 2$ $i + 1 j$ $= -4$ $i + 1$ $j$ $Magnitude \sqrt 17 -- (d)$
$At (-2, 4) : -2 x (-2)$ $j$ $= 4$ $i + 1$ $j$ $Magnitude \sqrt 17 -- (e)$

$At each point (0, 0), (1, 1), (2, 4), (-2, 4) of <$ $y - x 2 = 0> we can get the direction and calculate the magnitude of of the vector <grad H (x, y)>. Let's check them if they are correct or not.$
Now we refer to the concept of tangent and the tangent line. Tangent line at any point of y= x² shows the slope of y= x² at this point (the value of y with respect to x = 1 (unit)) of y= x²). For instance,

when x = 0 (then y = 0) the tangent or the slope of the tangent line at (0, 0) = 0 and
the tangent line is y = 0.   --- (a')

When x = 1 (then y = 1), the tangent at (1, 1) is 2 and the tangent line is y = 2x - 1.    --- (b')

When x = -1 (then y = 1), the tangent at (-1, 1) is -2 and the tangent line is y = -2x - 1.    --- (c')

When x = 2 (then y = 4), the tangent at (2, 4) is 4 and the tangent line is y = 4x - 4.    --- (d')

When x = -2 (then y = 4), the tangent at (-2, 4) is -4 and the tangent line is y = -4x - 4.    --- (e')

(a) and (a'), (b) and (b'), (c) and (c'), (d) and (d'), and (e) and (e') are apparently closely related.

(a) and (a')

At (0, 0)

The tangent line is y = 0 while the direction of the gradient vector is $+ 1$ $j .$

             l
             l ^
             l l   $j$
------------------- y= 0

(b) and (b')

At (1, 1)
The tangent line is y = 2x - 1 while the direction of the gradient vector $-2$ $i + 1 j is perpendicular to$
y = 2x - 1.

At (-1, 1)
The tangent line is y = -2x - 1 while the direction of the gradient vector $2$ $i + 1 j is perpendicular to$
y = -2x - 1.

At (2, 4)
The tangent line is y = 4x - 4 while the direction of the gradient vector $-4$ $i + 1$ $j$ $is perpendicular to$
y = 4x - 4.

At (-2, 4)
The tangent line is y = -4x - 4 while the direction of the gradient vector $4$ $i + 1$ $j$ $is perpendicular to$
y = -4x - 4.

What does this (perpendicular) relation mean ? Also <Simply why the greatest ?>

Continue to the next post.

sptt

Asnetcomp Tutorials

Wednesday, January 25, 2012

What is Gradient ? - Basic understanding

Expression in 3-dimensional rectangular coordinates

No comments:

Post a Comment