Minimizing a function - sum of squares
I'm hoping you can help with this problem. I haven't taken calculus in years and I don't know where to start...
The sum of squares of a sample of data is minimized when the sample mean is used as the basis of the calculation.
$$g(c) = \sum_{i=1}^n(X_i-c)^2$$
Show that the function is minimized where $c = \overline {X} $.
$\endgroup$ 16 Answers
$\begingroup$Viewed as a function of $c$, $g(c)$ is differentiable and $$ g'(c)=\sum_{i=1}^{n}-2(X_i-c) $$ If this has a minimum, it must occur where $g'(c)=0$.
$\endgroup$ $\begingroup$We have $g(c) \ge 0$. For relative extrema $c \in \mathbb{R}$ we need: $$ 0 = g'(c) = \sum_{i=1}^n 2 (X_i - c) (-1) = 2 \left( n c - \sum_{i=1}^n X_i\right) \Rightarrow \\ c = \frac{1}{n} \sum_{i=1}^n X_i = \bar{X} $$ Further $$ g''(c) = 2n > 0 $$ so it should be a minimum.
$\endgroup$ $\begingroup$An arithmetic argument,
$$ \begin{align} \sum_{i=1}^{n} (x_i - c)^2 &= \sum_{i=1}^{n} (x_i - \bar{x} + \bar{x} - c)^2 \\ &= \sum_{i=1}^{n} (x_i - \bar{x})^2 + n (\bar{x} - c)^2 - 2 (\bar{x} - c) \sum_{i=1}^{n} (x_i - \bar{x}) \\ &\geq \sum_{i=1}^{n} (x_i - \bar{x})^2 , \end{align} $$
since $\sum_{i=1}^{n}(x_i - \bar{x}) = 0$ and $n (\bar{x} - c)^2 \geq 0$.
$\endgroup$ $\begingroup$Setting the derivative of $g(c)$ equal to $0$ yields: $$ g'(c) = 2\sum_{i} (x_i - c) = 2(\sum_i x_i - nc)= 0\implies \sum_i x_i = nc \implies c^* = \frac{\sum_i x_i}{n} $$
$\endgroup$ $\begingroup$You can know where a minimum exists by using the derivative of the function because its well known that in most of the cases the minimum value is when the a derivative of the function is zero. So if you can find the first derivative of the function and equalized it with zero you can obtain the value that you are looking for.
$\endgroup$ 2 $\begingroup$$\begin{array}\\ g(c) &= \sum_{i=1}^n(X_i-c)^2\\ &= \sum_{i=1}^n(X_i^2-2X_ic+c^2)\\ &= \sum_{i=1}^nX_i^2-2\sum_{i=1}^nX_ic+\sum_{i=1}^nc^2\\ &= \sum_{i=1}^nX_i^2-2cn\bar{X}+nc^2\\ &= \sum_{i=1}^nX_i^2+n(c^2-2c\bar{X})\\ &= \sum_{i=1}^nX_i^2+n(c^2-2c\bar{X}+\bar{X}^2)-n\bar{X}^2\\ &= \sum_{i=1}^nX_i^2+n(c-\bar{X})^2-n\bar{X}^2\\ &\ge \sum_{i=1}^nX_i^2-n\bar{X}^2\\ \end{array} $
with equality if and only if $c=\bar{X} $.
$\endgroup$
