$…$ / $$…$$), leggibile e copiabile con lo screen reader. Scorciatoia: Alt+Shift+L.
The maximum likelihood estimation is one of the most important estimation methods in Econometrics. It can be shown to be consistent and asymptotically efficient under general conditions. Namely, the Maximum Likelihood Estimator (MLE, from now on) of a value is the number that is ’most likely’ or has the maximum likelihood of generating that specific value.
An MLE must be found by first deriving a likelihood function, for example in a form such as \(L=L(\theta , x_1, \ldots , x_N)\), where \(\theta \) is the variable which characterizes the population under consideration.
Let’s take into account the following worked example.
Example 5.
Suppose that our population involves values of a discrete random variable \(X\) having geometric probability distributions: \[p(x_i)=(1-\theta ) \theta ^{x_i},\] where \(x_i\) is a random observation on \(X\). Since observations are independent for a random sample, we can write the probability of obtaining our \(N\) observations as \[L=p(x_1) \cdot p(x_2) \cdot \cdots \cdot p(x_N)=(1-\theta )^N \theta ^{x_1 + \cdots + x_N}.\] Typically, we prefer to take the logarithmic function of \(L\) rather than \(L\) itself, to ease the subsequent calculations. Call \(l(\cdot )=\ln (L(\cdot )\) the log-likelihood function. We have: \[l(\theta )=\ln (L(\theta ))=\ln (1-\theta )+x_1 \ln (\theta )+ \cdots + \ln (1-\theta )+x_N \ln (\theta )=\] \[=N \ln (1-\theta ) + \ln (\theta ) \sum _{i=1}^N x_i.\] Since \(l(\theta )\) is maximized at the same value \(\theta ^*\) as \(L(\theta )\), we can take the FOC: \[\dfrac {\partial l}{\partial \theta }=-\dfrac {N}{1-\theta }+\dfrac {\sum _{i=1}^N x_i}{\theta }=0 \qquad \Longrightarrow \] \begin {equation*} \Longrightarrow \ \cdots \ \Longrightarrow \theta \left (\dfrac {1}{\sum _{i=1}^N x_i}+\dfrac {1}{N}\right )=\dfrac {1}{N} \qquad \Longrightarrow \end {equation*} \begin {equation} \Longrightarrow \qquad \theta ^*=\dfrac {\dfrac {1}{N}}{\dfrac {1}{\sum _{i=1}^N x_i}+\dfrac {1}{N}}=\dfrac {\sum _{i=1}^N x_i}{\sum _{i=1}^N x_i +N}. \label {optimaltheta} \end {equation}On the other hand, the following Example describes an analogous derivation, when \(X\) is a continuous random variable distributed according to a Poisson density having \(\theta \) as its parameter.
Example 6.
Call \(X\) a continuous random variable whose probability density function is a Poisson distribution of the kind: \[p(x)=\theta e^{-\theta x},\] meaning that, as usual: \[Pr\{X \leq x\}=F(x)=\int _{-\infty }^x \theta e^{-\theta t} dt.\] Also in this case, the likelihood function is \begin {equation*} L(\theta )=\prod _{i=1}^N p(x_i)=\theta ^N e^{-\theta (x_1+\cdots + x_N)}, \end {equation*} whereas the corresponding log-likelihood function is \begin {equation} l(\theta )=\ln (L(\theta ))=\sum _{i=1}^N \ln (\theta e^{-\theta x_i})=N \ln \theta – \theta \sum _{i=1}^N x_i. \label {loglikelihoodPoisson} \end {equation} Differentiating (2) with respect to \(\theta \), we obtain: \begin {equation*} l^\prime (\theta )=\dfrac {N}{\theta }- \sum _{i=1}^N x_i=0 \ \ \Longrightarrow \ \ \theta ^*=\dfrac {N}{\sum _{i=1}^N x_i}=\dfrac {1}{\overline {x}}, \end {equation*} where \(\overline {x}\) is the usual arithmetic mean.3.1 Maximum likelihood estimation of regression parameters
Consider a regression line \[y_i=\alpha + \beta x_i + \epsilon _i,\] where the disturbances \(\epsilon _i\) have null mean values and their variance is the same as \(y_i\), i.e. \(Var(y_i)=Var(\epsilon _i)=\sigma ^2\), for all \(i=1, \ldots , N\). The explained variables \(y_i\) are normally distributed and their mean values are given by \[\mathbb {E}[y_i]=\alpha + \beta x_i.\] Hence, we can write the probability density function (p.d.f.) of each variable \(y_i\) as follows: \[p(y_i)=\dfrac {1}{\sqrt {2 \pi \sigma ^2}}e^{-\frac {(y_i-\alpha – \beta x_i)^2}{2 \sigma ^2}}.\] Due to the classical assumptions, the disturbances \(\epsilon _i\) are uncorrelated, normally distributed, and independent of each other. This implies independence for \(y_i\) as well. Hence, the likelihood function will be the usual product of p.d. functions: \[L(\alpha , \beta , \sigma ^2)=p(y_1) \cdot \cdots \cdot p(y_N).\] Taking the logarithm yields: \begin {equation*} l(\alpha , \beta , \sigma ^2)=\ln (L(\alpha , \beta , \sigma ^2))=-\dfrac {N \ln (2\pi )}{2}-\dfrac {N \ln (\sigma ^2)}{2}-\dfrac {1}{2 \sigma ^2}\sum _{i=1}^N [y_i-\alpha -\beta x_i]^2. \end {equation*} Now the standard procedure to find \(\tilde {\alpha }\) and \(\tilde {\beta }\) so as to minimize the sum of the squares must be implemented, given the negative sign of the above expression. Hence, the maximum likelihood estimators \(\alpha \) and \(\beta \) are exactly the same as in the OLS procedure. However, we have to calculate the FOC with respect to the variance to determine the third parameter: \begin {equation*} \dfrac {\partial l}{\partial \sigma ^2}=-\dfrac {N}{2 \sigma ^2}+\dfrac {1}{2 \sigma ^4}\sum _{i=1}^N [y_i-\alpha -\beta x_i]^2=0, \end {equation*} leading to: \begin {equation} \tilde {\sigma }^2=\dfrac {\sum _{i=1}^N [y_i-\alpha -\beta x_i]^2}{N}=\dfrac {\sum _{i=1}^N e_i^2}{N}. \label {optimalvariancebivariate} \end {equation} It is interesting to note that the same happens in the multivariate case.
3.2 Confidence intervals for coefficients
Interval estimation is fundamental to identify the best estimate of a parameter involving an explicit expression of its uncertainty. If we are to select an interval for a parameter \(\theta \), we typically assume that it must be symmetric. This means that, if we found the value \(\hat {\theta }\), a suitable interval might be \([\hat {\theta }-\delta , \ \hat {\theta }+\delta ]\), where \(\delta \) may be chosen equal to \(0.01\) or to \(0.05\), conventionally.
Remembering that by (19) and (20) we have that: \begin {equation*} \beta _k | x \ \sim \ N \left (\beta _k^*, \sigma ^2 S_{kk} \right ), \end {equation*} where \(S_{kk}\) is the \(k\)-th diagonal element of the matrix \((X^T X)^{-1}\). Therefore, taking \(95\) percent (i.e., \(\alpha _c=0.05\)) as the selected confidence level, we have that \begin {equation*} -1.96 \leq \dfrac {\beta _k – \beta _k^*}{\sqrt {\sigma ^2 S_{kk}}} \leq 1.96 \end {equation*} \[\Downarrow \] \begin {equation*} Pr \left \{\beta _k – 1.96 \sqrt {\sigma ^2 S_{kk}} \leq \beta _k^* \leq \beta _k + 1.96 \sqrt {\sigma ^2 S_{kk}} \right \}=0.95, \end {equation*} which is a statement about the probability that the above interval contains \(\beta _k^*\).
If we choose to use \(s^2\) instead of \(\sigma ^2\), we typically use the \(t\) distribution. In that case, given the level \(\alpha _c\): \begin {equation*} Pr \left \{\beta _k – t_{(1-\alpha _c /2), [M-N]}^* \sqrt {s^2 S_{kk}} \leq \beta _k^* \leq \beta _k + t_{(1-\alpha _c /2), [M-N]}^* \sqrt {s^2 S_{kk}} \right \}=1-\alpha _c, \end {equation*} where \(t_{(1-\alpha _c /2), [M-N]}^*\) is the appropriate quantile taken from \(t\) distribution. If \(1-\alpha _c=0.95\), we obtain the confidence interval for each \(\beta ^*_k\), i.e. \[\beta ^*_k \in \ \left (\beta _k – 1.96 \sqrt {s^2 S_{kk}}, \ \beta _k + 1.96 \sqrt {s^2 S_{kk}} \right ).\]
