1.3. Basic optimization principles

We begin our study of the basic principals
of optimization by looking at a very

general form of optimization problem which
is given by this. Maximize over a variable

x. An objective function t of x. To
provide some motivation think of the

problem as a one face by a firm that once
maximize total profits given by t of x. So

these are total profits. That's a function
of an action that in can take x, which x

is any number. A little bit of terminology
in a general optimization problem the

variable that we can control is called the
control variable. And this is a variable

that which is over and the function that
we are trying to maximize in this case T

of X is called the objective function.
Using this terminology the goal of the

optimum maximization problem is to select
the level of the control variable that

generates the maximum valuable of the
objective function. To provide some

intuition consider a graphical
presentation of the problem. We have some

sort of function, let's say something that
looks like this, is the function t of x.

And what we are trying to do is find the
level of x in this axis that provides the

maximum level of a function. In this
graphical representation this would be

this level. Fortunately there is a recipe
that you learn in calculus that allow us

to take a function t apply some
derivatives and find the general body of x

star that maximizes the function. And the
recipe is given by what are known as the

first order. Or necessary conditions and
second order or sufficient conditions. The

first order conditions state that at any
point that is a local maximum must satisfy

the condition that each derivative is
equal to zero. However as you probably

remember for calculus necessary conditions
are not sufficient some points may satisfy

them and it's still not the maximum and we
will see an example of this in a second.

However, if a point also satisfies what I
call second order of sufficient conditions

which in this case is given by second
derivative is less than zero so then such

a point is a local max imum.
To warm up a practice one which we have

just learned lets do a simple example.
Consider the function ten minus x minus

five squared. Think of this as a profit
function that we are trying to maximize.

In order to find the maximum we need to
first apply the first order conditions

which would say that the derivative of the
objective function which in this case is

minus two times x minus five. Has to be
equal to zero and doing a little bit of

algebra this implies that x and the
optimum. Has to be equal to five. Now we

know that these conditions are necessary
but not sufficient so lets check the

second order conditions which are the
sufficient ones that requires taking the

second derivative of the function or the
derivative of this which in this case

means. The second derivative of this
function is just minus two which is less

than zero which implies that the second
order conditions are satisfied and that

the unique local optimum or local maximum
therefore the global maximum is at x star

equals five. Let me provide you with three
different types of intuition for why the

first and second order conditions look
like they do. Let's just start with a

graphical intuition of the first order
conditions. Suppose that you have a

function that looks more or less like
this. That is the objective function that

we're trying to maximize. Remember, that
the first order condition states that at

any local maximum, the derivative at the
optimum, at the at local maximum has to be

equal to zero. Now, to see why this has to
be the case, consider a point for example

like at x, at which the derivative.
Clearly, it's not equal to zero. In this

particular case it's positive. It is easy
to see that the value of the function of

that point can not be a maximum. In other
words x can not be maximizing the function

because given that this lobby is positive
if you move a little bit to the right,

let's say to a point like x plus delta x
you can increase the value of the function

approximately by the value this low times
delta x. Now, it's also easy to see, that

if you're at a point, like X-hat, at
which, in this case, the value of the

slope is negative, you can also be, novy
maximizing, because if you move to a point

like X-hat, minus D-X, minus delta X, you
can also increase the value of the

function for similar resource. Now, it's
very easy to see, that the only point at

which you can not play this trick, of
improving the value of the objective

function, by changing the value X, is a
point, like, X-star, at which, the slope,

is zero, because if you move locally,
either to the left, or to the right, the

value of the objective function would not
change. This provides an intuition, for

why, the first order conditions, have to
satisfy, this property at a global, at a

local maximum. Now consider a graphical
intuition for why the second order

conditions look like they do. To remind
you, the second order condition is given

by, the second derivative and the local
optimum has to be less than zero. That

provides together with the first other
conditions, sufficient conditions for

optimization. Now, to see why that has to
be the case, compare the following two

functions. The one on the left looks like
this, and it does satisfy the, Second

order conditions, are the point X star. In
fact, it is easy to see that in this case,

the second derivative, at X star is
greater than zero. This low is becoming

progressive more positive as you move to
the right. Just like the one before and at

which an x star our candidate for the
local optimum the second derivative

satisfies the second order condition let's
see why it has to be the case that x star

is not a maximum here but is a maximum
here and the key. Nature of the difference

is captured by the second order condition.
Now, if you start x star in the field on

the left and move a tiny bit to the right
by delta X, since the slope is zero you

don't really change the value of the
function, so that doesn't help you,

however and this is the key intuition that
you have to see, because the second

derivative is positive the value of t he
log changes when you move from Xstar to

Xstar + delta X. And becomes positive here
at the new point. That means that if you

carry out a further move, let's say by
another delta x since this slope was

positive at x star plus delta x, the
function now has increased, and that means

that by moving two delta x to the right,
you can improve the value of the function

which means that it cannot be an optimum.
In contrast see what happens if you tried

to play the same game at x star in the
field on the right. As before the first

order conditions are satisfied so eh this
lobby zero at x star but if you tried to

move to the right because lets say by
delta x because the second derivative is

less then zero this lobe actually goes
down. Unit becomes negative therefore if

you try to move a farther the x the value
of the function goes down. It should be

easy to see and you should convince
yourself that it doesn't matter for this

argument whether the movement is to the
right or to the left. In both cases they

go through in an analogous way. This
demonstrates the intuition for why the

second order condition t double prime of x
star less than zero provide sufficient

conditions for the point to be a local
maximum. Let's look at intuition for why

the optimization conditions look like they
do from a different perspective in this

case a mathematical one. By the way I'm
trying different intuitions because some

of you may find some of them more natural
then the others. Now you will recall from

basic calculus that the value of the
objective function in fact of any function

at the point x plus dx can be very well
approximated by the value of the function

at x. Class the derivative of the
functional x times delta x class the

second derivative of the function at x
times the x square plus a bunch of high

order terms that are approximate zero for
a small changes in the value of x. Now,

given this, the change in the fun in the
objective function produced by the trans

delta x can be written as DT. The change
in the function is approximately equal to

the derivative at x times delta x, plus
the second derivative at x times the x

squared. Now for a function to be
maximized at X, it has to be the case that

we cannot find a small change delta X,
either positive or negative, that

increased the value of the function, in
other words that makes the change in the

function positive. But notice that if the
point X satisfies the first order

conditions, the necessary conditions, this
term is zero, sees the first derivative is

equal to zero. Furthermore, if it
satisfies the second order conditions,

this term is negative, and therefore, the
doesn't change the function has to be less

than zero. In other words, when the first
order conditions and second order

condition are satisfied, it is not
possible to find a change, a small lock

and chance, delta x, that increases the
value of the function. Finally, consider

the economic intuition behind the
maximization problem. Recall the

reputation of the representative function,
T of X, as some sort of benefit of taking

action x measuring dollars. For example if
you would affirm t of x made the note the

profit of taking action x. Now under this
interpretation of the problem the

derivative of the function x is called the
marginal benefit, which is a concept that

we will use repeatedly throughout the
course and that I will abbreviate with the

acronym MB. Now, the marginal benefit of x
is equal to either the increase in benefit

produced by increasing X by one unit, or
alternative, they decrease in benefit

produced by decreasing X by one unit. Now
an important property of an optimum has to

be that at the optimum or maximum, the
marginal benefit has to be equal to zero.

Now, why is this the case? Well, suppose
that is not the case? For example, suppose

that the marginal benefit is greater than
zero then clearly profits or benefits were

not maximized at x star since you could
increase them by increasing x by a little

bit. So if you increase x, you increase
profits, or the benefit function t.

Similarly, if the marginal benefit is l
ess then zero, you can not be maximizing

the benefit either, because in this case
by decreasing x you can also increase

benefits. It follows that at an optimum
marginal benefit has to be equal to zero.

Now the last principle that we have seen
in the economic intuition is so important

that is water everything. An economic
actor that is performing the optimal

action would choose its level of activity
to the point where marginal benefit is

equal to zero. Marginal benefit equal to
zero ingrain that in your heads because

otherwise it cannot be optimizing if
marginal benefit was not equal to zero it

could increase its benefit or its profits
by either increasing its level of activity

or decreasing its level of activity.