Lesson 4. Continuous functions: what we can optimize

Continuous functions: those that have chance to be optimized

As I have advertised in the last lesson, we will think about functions that can be maximized / minimized. In order to give you an idea of what "continuous" means, it is better to give you some discontinuous functions. On FooPlot, type "1/x" under "Function $y(x)$". Note that the horizontal line has $x$-values while the vertical line has $y$-values. The plot you just made consists of $(x, y)$ such that $y = 1/x$. For exmpale, we have $1 = 1/1$, so the plot, which is a curve, passes $(1, 1)$. Let's try a harder one. You also have $-2 = 1/(-1/2)$, so your plot passes $(-2, -1/2)$ as well.

Now, in the equation $y = 1/x$, put $x = 0.001$. Then what do you get for $y$? You get $y = 1/0.001 = (1/0.001)(1000/1000) = 1000/1 = 1000$. Trying more numbers, you see as $x$ gets closer from the right side to $x = 0$, the values of $y$ gets bigger.

On the other hand, try $x = -0.001$ in your equation $y = 1/x$. You get $y = -1000$, and you see that when $x$ gets closer from the left side to $x = 0$, the magnitude of $y$ is getting bigger, but with a negative sign! You should really see this in your plot on FooPlot. You shall soon notice that you cannot take $x = 0$ in the equation $y = 1/x$. In this case, we say that the function $x \mapsto 1/x$ is discontinuous at $x = 0$, because the value at $x = 0$ is undefined. Here is an important observation in your picture: the function $x \mapsto 1/x$ has NO maximum. This issue of not having maximum happens at $x = 0$, where the function is discontinuous.

If you have read the discussion about $\sqrt{2}$ in Lesson 2, you might ask whether we have some "legitimate cheating" that resolve this issue. Let's try. For example, we can define the function $f(x)$ by saying $f(x) = 1/x$ for all $x \neq 0$, but then force $f(0) = 0$. This is not plugging $x = 0$ in the expression $1/x$. We are merely coloring $(0, 0)$ in the plot you draw on FooPlot. Notice that there is still discontinuity at $x = 0$. You still cannot find maximum of this function because of this discontinuity at $x = 0$.

Now, consider the function $x \mapsto -x^{2} + 1$. Type "-x^2 + 1" under "Function $y(x)$" on FooPlot to draw the plot of the function, namely all the points $(x, y)$ such that $y = -x^{2} + 1$. Notice that this function is continuous everywhere. That is, on a sheet of paper, you can draw this function at once without having to put your pen off the paper. Unlike the function $x \mapsto 1/x$, notice that the function $x \mapsto -x^{2} + 1$ has a maximum! This maximum happens at $x = 0$, and the value of the maximum is $y = -0^{2} + 1 = 1$.

Optimization needs local constraint

Now, we have a very nice continuous function $x \mapsto -x^{2} + 1$, along with the plot $y = -x^{2} + 1$. We have witnessed that at $x = 0$, the function has maximum $y = 1$. However, can you find its minimum? No, as $x$ becomes $100, 1000, 10000, \dots$, your plot shows that the $y$-value will decrease forever. What do we do in this situation? "Legitimate cheating"! We only think about the function from $x = 0$ to $x = 1$ and forget about any other $x$-values in the picture. Then, notice that the function from $x = 0$ to $x = 1$ take minimum at $x = 1$, because we simply cannot take other values such as $x = 2, 3, 4, \dots$. That is, we realized: continuous functions can be locally maximized / minimized.

Subtle remark: end points and Extreme Value Theorem

We noticed that the function $h(x) = -x^{2} + 1$, when restricted to the interval from $x = 0$ to $x = 1$, obtains minimum at $x = 1$, namely $h(1) = -1 + 1 = 0$. (Note that I named this function using the letter $h$ without telling you.) If you are very subtle, you could ask, what if we talk about the interval from $x = 0$ to $x = 1$, but not including endpoints $x = 0$ and $x = 1$? You can see that this is quite annoying, but we have to stick to the truth: without the end point $x = 0$, the function $h(x)$ does not have maximum. The reason is as follows: we have 

$h(0.1) = -0.01 + 1 = 0.99$ and likewise have 

$h(0.01) = 0.9999$, 

$h(0.001) = 0.999999$, 

$h(0.0001) = 0.99999999, \dots$, 

so as $x$ gets closer to $x = 0$ from the right side, the value of $h(x)$ increases toward $1$, but since you cannot take $x = 0$ (because we are not including this point), you can never hit $1$. Since $h(x)$ can increase forever, this means that it does not take maximum when restricted to $0 < x < 1$ (that is, from $x = 0$ to $x = 1$ without including the end points).

It turns out that once we include the end points, we can always maximize and minimize a continuous function. This is what people call 

Extreme Value Theorem: Any continuous function on an interval including endpoints can obtain maximum and minimum.

Exercise: what you really need to check whether you are following!

One of my good teachers in college told me that "the best way to learn mathematics by doing exercises." You are serious about learning, and you know you want to understand what is going on. Thus, you want to try this exercise. You don't have to get it right. If you feel like your answer is OK, then you are good to go. Otherwise, feel free to leave any question in the comment section below!

Exercise. Let $f(x) = -x^{2} + 1$ defined for $0 \leq x < 1$. That is, the function is defined from $x = 0$ to $x = 1$, including $x = 0$ but not including $x = 1$. 

1. Does $f(x)$ takes maximum in the given interval? If so, at which $x$-value does it take maximum. What is the value of the maximum?
2. Does $f(x)$ takes minimum in the given interval? If not, explain why it does not take the minimum, possibly using the argument I used in the post.