Constrained Optimization

In economics, most optimization problems involve scarcity. We want to maximize something (utility, profit, output), but we face constraints (budget, technology, time, policy).

This chapter develops the tools to solve such problems.

We proceed in three stages:

1. Equality constraints (substitution → Lagrangian)

2. Multiple constraints and bordered Hessian

3. Inequality constraints (Kuhn–Tucker conditions)

I — Equality Constraints¶

8.1 The Economic Logic¶

Think geometrically:

• The objective function gives indifference curves

• The constraint gives a feasible set

• The optimum occurs where the highest attainable indifference curve touches the constraint

Mathematically, this corresponds to:

• Equal slopes (tangency)

• Constraint satisfied exactly

8.2 Solving by Substitution¶

Suppose we maximize

subject to

Step 1: Rewrite the Constraint¶

Step 2: Substitute into Objective¶

Now this is a single-variable problem.

Step 3: First-Order Condition¶

Solve:

Then

Step 4: Second-Order Condition¶

Evaluated at $x_1 = 4$:

Conclusion: The stationary point is a maximum.

Objective value:

Limitation of Substitution¶

We now introduce a more powerful method.

II — Lagrange Multiplier Method¶

An alternative to the substitution method is the Lagrange multiplier method.

Given an optimization problem say, to maximize an objective function

subject to a constraint:

Begin by defining the Lagrangian function:

where $\lambda$ is known as the Lagraingain multiplier.

Then the First-Order Conditions (F.O.C): simply get the first partials of the Lagrangian function

And solve for the critical values of $x_1^*$ and $x_2^*$.

Note that the last condition simply reproduces the constraint.

Let’s continue with the example above: We wish to maximize equation (1) subject to (2). To find the first-order conditions (FOCs) for this specific Lagrangian, we take the partial derivative of $\mathcal{L}$ with respect to each of the three arguments ($x_1$, $x_2$, and $\lambda$) and set them equal to zero.

The Lagrangian Function¶

First-Order Conditions¶

(i) Partial derivative with respect to $x_1$:

Simplifying gives:

(ii) Partial derivative with respect to $x_2$:

Simplifying gives:

(iii) Partial derivative with respect to $\lambda$:

This simply returns the original constraint:

Solving the System¶

A common next step is to solve for $\lambda$ in the first two equations to establish a relationship between $x_1$ and $x_2$:

Setting (4) and (5) equal gives:

Substituting $x_1 = x_2$ into the constraint:

Note that this is the same solution as we saw with the substitution method above.

Additionally,

8.3 Intuition¶

At an interior optimum with one equality constraint:

• The objective function’s gradient must be parallel to the constraint’s gradient.

In two dimensions:

8.4 Second-Order Condition: Bordered Hessian¶

For the constrained problem, we cannot simply examine the Hessian of $f$.

We must use the bordered Hessian.^[1]

The bordered Hessian is:

Hence the second principal minor, we have

which is positive, hence $|\mathbf{\bar{H}}|$ is negative definite and we have met the sufficient condition for a maximum.

That is, evaluate the determinant at the stationary point we found earlier ($x_1 = 4$, $x_2 = 4$), the resulting matrix is:

which is positive, hence $|\mathbf{\bar{H}}|$ is negative definite and we have met the sufficient condition for a maximum. In other words, for a bivariate case with one constraint, a Lagrangian function is negative (positive) definite at a stationary point if the determinant of its bordered Hessian is positive (negative) when evaluated at that point. In this case we have sufficient condition for the stationary point identified by the Lagrangian method to be a maximum (minimum).

8.5 General Case (n variables, m = 1)¶

More generally, for the case of $n$ arguments and $m$ constraints, we have a sufficient condition for a maximum if the determinant of the bordered Hessian ($|\bar{\mathbf{H}}_{n+m}|$) has the same sign as $(-1)^n$, and the largest $n-m$ leading principal minors alternate in sign. In this case, the quadratic form is negative definite subject to the constraint.

Alternatively, if the largest $n-m$ leading principal minors of the bordered Hessian (including the determinant of the bordered Hessian itself) all have the same sign as $(-1)^m$, then the quadratic form is positive definite subject to the constraint, and the stationary point represents a minimum.

As a rule, we essentially need to evaluate the determinants of the largest $n-m$ leading principal minors of the bordered Hessian.

Also it is important to note is that the Lagrangian method generates a variable not obtained with the substitution method i.e., the Lagrangian multiplier λ, which represents the effect of a small change in the constraint on the optimal value of the objective function.

Although we have seen just a simple case, optimization of a bivariate function subject to one constraint, the Lagrangian method is powerful enough to deal with multivariate functions and multiple constraints

III — Multiple Constraints¶

In general, we can write the Lagrangian function for a constrained optimization problem with an objective function with $n$ variables subject to $m$ equality constraints as follows:

where $g_i(x_1, \dots, x_n) = c_i$ represents the $i$th constraint.

After which we find the stationary points and check for whether we have a maximum, minimum or neither.

8.6 Breakfast Example¶

Let;s try maximizing the Utility function:

Subject to a Budget constraint:

The Lagrangian function is:

First-Order Conditions¶

Solving ratios:

And using the budget:

As an exercise, verify that we have a maximum by evaluating the bordered Hessian. Note that in this case $m-n = 3-1$, so we need to evaluate the determinants of the largest two principal minors, $|\mathbf{\bar{H}}| = |\mathbf{\bar{H}_3}|$ and $|\mathbf{\bar{H}_2}|$.

Adding a Second Constraint¶

Suppose we have a second constraint:

Then the Lagrangian function becomes

The F.O.C are

Including the two constraints

From the first 3 equations, we find that

Perhaps we should explain this before moving on? We need to eliminate the Lagrange multipliers ($\lambda_1$ and $\lambda_2$) using the first three equations.

Step 1: Isolate $\lambda_1$: From the second FOC ($\frac{\partial \mathcal{L}}{\partial V}$):

Step 2: Isolate $\lambda_2$: Subtract the third FOC from the first FOC to cancel out $\lambda_2$:

Step 3: Combine and Solve for $V$: Substitute our $\lambda_1$ value ($\frac{2}{3V}$) into that result:

Now, rearrange to solve for $V$:

We proceed by simplify the first constraint by substituting $V = \frac{SJ}{3(J - S)}$ into the first constraint:

To clear the denominators, multiply the entire equation by $12(J - S)$:

Next, from the second constraint, we know $J = 24 - S$. We substitute this into our simplified equation:

And grouping the $S$ terms:

Divide by -6 to make it a standard quadratic:

Using the quadratic formula or factoring $(S - 24)(S - 8) = 0$ gives $S = 24$. This would imply $J = 0$, which is impossible given the $\ln J$ term in our objective function.

$S = 8$: This is our valid solution.

The final Values $S = 8$, $J = 24 - 8 = 16$ and $V = \frac{SJ}{3(J - S)} = \frac{8 \times 16}{3(16 - 8)} = \frac{128}{24} = \frac{16}{3} \text{ or } 5 \frac{1}{3}$

Hence, the optimal point for this system is:

Which gives an utility of 2.17, which is lower than 2.21 (without the second constraint).

Now the second order conditions, or bordered Hessian.

And

which matches the sign of $(-1)^n$ where in this case $n=3$, i.e., both are negative.

Hence the stationary point $S^* = 8$, $V^* = 5 \frac{1}{3}$, and $J^* = 16$ represent a maximum.

But don’t we have to evaluete the determinant other principal minors of the bordered Hessian as well? No, because in this case, we have to evaluate only $n-m = 3-2 = 1$ largest Hessians.

In general, for the case of $n$ arguments and $m$ constraints, we have a sufficient condition for a maximum if the largest $n - m$ leading principal minors of the bordered Hessian alternate in sign, with the sign of the full determinant ($| \mathbf{\bar{H}}|$) being the same as $(-1)^n$.

Alternatively, we have a sufficient condition for a minimum if the largest $n - m$ leading principal minors of the bordered Hessian all have the same sign, which must be the same as the sign of $(-1)^m$. If the minors do not follow either of these patterns, the second-order condition for a local extremum is not satisfied.

IV — Inequality Constraints¶

8.7 Why Inequalities Change Everything¶

So far we have considered constraints which are strictly equalities. We now consider the case when constraints take the form of inequalities rather than equalities. This involves a slight modification to the Lagrangian method, which we present below.

8.8 The Kuhn–Tucker Conditions¶

Maximize:

subject to:

Set up the Lagrangian function as usual:

The solutions to this problem satisfies the following conditions:

1. Stationarity

2. Primal feasibility

3. Dual feasibility

4. Complementary slackness

The above are known as the Kuhn-Tucker conditions for an optimization problem with inequality constraints.

The last two Kuhn-Ticker conditions are known as complementary slackness conditions (meaning that both optimal points and first-partials evaluated at the optimal points cannot simultaneously both be zero).

8.11 Worked Inequality Example¶

Consider the problem of maximizing the objective function:

subject to the constraints:

Note that we have written the constraint that $f(X_1, x_2($ is nonnegative in a way to make it conformable with the setup of the problem above.

The Lagrangian is:

The optimal choice of $x_1$ and $x_2$ satisfies the Kuhn–Tucker conditions

We proceed by considering solutions that arise in four cases corresponding to the four possible ways in which the complementary slackness conditions may be met. These cases are

Case 1: $\lambda_1 = 0, \lambda_2 = 0$¶

In this case the condition $\partial L / \partial x_1 = 0$ shows that $x_1 = 1$, and the condition $\partial L / \partial x_2 = 0$ shows that $x_2 = 0$. The pair that $x_1 = 1$ and that $x_2 = 0$ satisfies the constraint

as well as the constraint $x_2$ which s nonnegative. Hence:

Value:

Case 2: $\lambda_1 \neq 0, \lambda_2 = 0$¶

Here the F.O.C $\partial L / \partial x_2 = 0$ requires that $\lambda_1 = 1$, which satisfies the condition that $\lambda_1 \ge 0$. This in turn requires that $x_1 = 1/2$, and satisfy the condition $\partial L / \partial x_1 = 0$. The complementary slack condition requires that

which is satisfied for $x_2 = 1$ or $x_2 = -1$. But the constraint that $x_2$ is nonnegative allows us to consider only $x_2 = 1$. Hence:

Value:

Case 3: $\lambda_1 = 0, \lambda_2 ≠ 0$¶

Here the condition $\partial L/\partial x_1 = 0$ gives $x_1 = 1$. The condition $\lambda_2 x_2 = 0$ requires that $x_2 = 0$. Therefore this solution is identical to the one found when $\lambda_1 = 0$ and $\lambda_2 = 0$. Same as Case 1:

Case 4: $\lambda_1 ≠ 0, \lambda_2 ≠ 0$¶

Lastly, we consider the case where $\lambda_1 \neq 0$ and $\lambda_2 \neq 0$. The condition $\lambda_2 x_2 = 0$ requires that $x_2 = 0$. But if $x_2 = 0$, the condition $\partial L / \partial x_2 = 0$ requires that $\lambda_2 = 0$. Therefore there is no feasible solution in which $\lambda_1 \neq 0$ and $\lambda_2 \neq 0$.

Conclusion¶

Comparing the valid cases:

• Case 1 & 3: $f(1, 0) = 1/2$

• Case 2: $f(1/2, 1) = 11/8$

Ranking the solutions by the value of the objective function, $11/8 > 1/2$, we find that the optimal solution is $x_1^* = 1/2, x_2^* = 1$ since

corresponding to Case 2 where $\lambda_1 \neq 0$ and $\lambda_2 = 0$.

Furthermore, note that at the optimal solution,

and this constraint is binding, while

and this constraint is not binding.

That is, at

Constraint 1 binds. But constraint 2 does not bind.