Convex Functions

Convex functions are real valued functions which visually can be understood as functions which satisfy the fact that the line segment joining any two points on the graph of the function lie above that of the function. Some familiar examples include $x \mapsto x^2$, $x \mapsto e^x$, etc.

Source: Wikipedia: Eli Osherovich

These functions satisfy a number of interesting properties such as continuity,existence of left and right derivatives which are of usual interests to Analysis. These also lead to the notion of convexity being of interest in Optimization Theory, Probability Theory and the Calculus of Variations.

We first start with a preliminary definition of a convex set.

A set $E \subseteq \mathbb{R}$ is said to be convex if given $x,y \in E$ such that $x < y$ then $[x,y] \subseteq E$

If $a < b$ then $[a,b],[a,b),(a,b],(a,b)$ are all convex sets. Another interesting example is $\left \{ a \right \}$.

But, $\mathbb{Q}$ is not a convex set as $1<\sqrt{2}<2$ but $\sqrt{2} \notin \mathbb{Q}$

We're now ready to define convex functions.

Let $E$ be a convex set and $f : E \to \mathbb{R}$ be a function.

Then $f$ is said to be convex if the following holds :

$$f(\lambda x + (1-\lambda) y) \leq \lambda f(x) + (1-\lambda)f(y) \quad \forall \lambda \in (0,1), x,y \in E$$

Note that $f$ is defined at $\lambda x + (1-\lambda) y$ as $E$ is convex. Also, after some routine computations, it is easy to see that this definition is equivalent to the intuitive definition given in the introduction.

Concave functions are defined as above but the inequality is flipped.

The most popular property of convex functions is the Jensen's Inequality.

Here's another way to check if a function is convex.

Let $f : I \to \mathbb{R}$ be a twice differentiable function.

Then,

• $f$ is convex on $I$ if and only if $f''(x) \geq 0 \quad \forall x \in I$
• $f$ is concave on $I$ if and only if $f''(x) \leq 0 \quad \forall x \in I$

This proof assumes the knowledge of the mean value theorem.

The proof for concave functions follows by the following result ;

If $g$ is concave, $-g$ is convex.

Note that a function is convex if and only if the following holds :

$f(\mu x + (1-\mu) y) \leq \mu f(x) + (1-\mu) f(y) \quad \forall \, \, \mu \in (0,1) ; \, x,y \in I ;\, x < y$

The proof of this result is not difficult and hence is left to the interested reader.

By virtue of this result, we need to show that the above inequality holds.

To continue with the proof, put $c = \mu x + (1-\mu)y$.

Now, consider the following expression :

$$\begin{aligned} \mu f(x) + (1-\mu)f(y) - f(c) &= (1-\mu)(f(y)-f(c)) - \mu(f(c)-f(x)) \quad x < c < y \\ &= (1-\mu)(y-c)f'(\eta_1) - \mu(c-x)f'(\eta_2) \quad x < \eta_2 < c < \eta_1 < y \\ &= \mu(1-\mu)(y-x)(f'(\eta_1)-f'(\eta_2)) \\ &= \mu(1-\mu)(y-x)f''(\xi) \quad \eta_2 < \xi < \eta_1 \\ \end{aligned}$$

This leads to the following:

$$\implies f(c) \leq \mu f(x) + (1-\mu) f(y) \quad \mu \in (0,1) ; x,y \in I; x<y \iff f''(z) \geq 0 \quad \forall z \in I$$

And we're done. $_\square$

---

Note: The condition that $f''$ is non-negative is equivalent to the fact that $f'$ is monotonically increasing. Hence, the theorem can only require the fact that $f$ is differentiable and $f'$ is monotonically increasing.