$…$ / $$…$$), leggibile e copiabile con lo screen reader. Scorciatoia: Alt+Shift+L.
There are several possible tests that are usually carried out in regression models, in particular testing hypotheses is quite an important task to assess the validity of an economic model. This Section is essentially based on Chapter \(5\) of Greene’s book [1], which is suggested for a much more detailed discussion of this topic.
The first example proposed by Greene where a null hypothesis is tested concerns a simple economic model, describing price of paintings in an auction. Its regression equation is \begin {equation} \ln P = \beta _1 + \beta _2 \ln S + \beta _3 AR + \epsilon , \label {simplemodeltotest} \end {equation} where \(P\) is the price of a painting, \(S\) is its size, \(AR\) is its ’aspect ratio’. Namely, we are not sure that this model is correct, because it is questionable whether the size of a painting affects its price (Greene proposes some examples of extraordinary artworks such as Mona Lisa by Leonardo da Vinci which is very small-sized). This means that this is an appropriate case where we can test a null hypothesis, i.e. a hypothesis such that one coefficient (\(\beta _2\)) is equal to \(0\). If we call \(H_0\) the null hypothesis on \(\beta _2\), we also formulate the related alternative hypothesis, \(H_1\), which assumes that \(\beta _2 \neq 0\).
The null hypothesis will be subsequently tested, or measured, against the data, and finally:
- if the data are inconsistent with \(H_0\) with a reasonable degree of certainty, \(H_0\) will be rejected.
- Otherwise, provided the data are consistent with \(H_0\), \(H_0\) will not be rejected.
Note that rejecting the null hypothesis means ruling it out conclusively, whereas not rejecting it does not mean its acceptance, but it may involve further investigation and tests.
The first testing procedure was introduced by Neyman and Pearson (\(1933\)), where the observed data were divided in an acceptance region and in a rejection region.
The so-called general linear hypothesis is a set of restrictions on the basic linear regression model, which are linear equations involving parameters \(\beta _i\). We are going to examine some simple cases, as are listed in [1] (Section \(5.3\)):
- one coefficient is \(0\), i.e. there exists \(j=1, \ldots , N\) such that \(\beta _j=0\);
- two coefficients are equal, i.e. there exist \(j\) and \(k\), \(j \neq k\), such that \(\beta _j = \beta _k\);
- some coefficients sum to \(1\), i.e. (for example) \(\beta _2 + \beta _5 + \beta _6 + \beta _8 = 1\);
- more than one coefficient is \(0\), i.e. (for example) \(\beta _3 = \beta _5 = \beta _9 = 0\).
Then there may be a combination of the above restrictions, for example we can have that \(2\) coefficients are equal to \(0\) and other \(2\) coefficients are equal, and so on. There can also be some non-linear restrictions, in more complex cases.
We will discuss the \(3\) main tests in the following Sections: the Wald Test, the Likelihood Ratio (LR) Test, the Lagrange Multipliers (LM) test.
4.1 Hints on the main distributions in Statistics
We are going to recall two major distributions which are particularly helpful when implementing the tests, especially regression analysis, analysis of variance, and so on: the chi-squared or \(\chi ^2\) distribution with \(k\) degrees of freedom and the Student’s \(t-\)distribution (or \(t\)-distribution).
Given \(k > 1\) independent, normally distributed random variables \(Z_1, \ldots , Z_p\), the sum of their squares is distributed according to the \(\chi ^2\) distribution with \(k\) degrees of freedom, i.e \[\sum _{j=1}^k Z^2_j \ \sim \ \chi ^2 (k).\] The p.d.f. of \(\chi ^2(k)\) is the following one: \begin {equation} f(x;k) = \ \begin {cases} \dfrac {x^{\frac {k}{2}-1} e^{- \frac {x}{2}}}{\sqrt {2^k} \cdot \Gamma \left (\dfrac {k}{2} \right )} \qquad \text {if} \ x > 0 \\ \\ 0 \qquad \qquad \qquad \ \text {otherwise} \end {cases} , \label {pdfchi2} \end {equation} where \(\Gamma (\cdot )\) is Euler’s (and Legendre’s) Gamma function, i.e. \[\Gamma (z) = \int _0^\infty x^{z-1} e^{-x} dx,\] for all \(z \in \mathbb {C} \setminus \mathbb {Z}_-\), and in particular, it is defined on positive integers as follows: \[\Gamma (n)=(n-1)!\] for all \(n \in \mathbb {N}\).
The properties of \(\chi ^2(k)\) are many, and they can be found on any Statistics textbook. A particularly meaningful one is:
if \(X_1, \ldots , X_k\) are independent normally distributed random variables, such that \(X_i \sim N(\mu , \sigma ^2)\), then: \[\sum _{i=1}^k (X_i – \overline {X})^2 \sim \sigma ^2 \chi _{k-1}^2,\] where \(\overline {X}=\dfrac {X_1 + \cdots X_k}{k}\).
A key use of \(\chi ^2\) distributions concerns the \(F\) statistic, especially the construction of the Fisher – Snedecor distribution \(F\), which will be treated in the last Section of the present Chapter.
On the other hand, Student’s \(t\)-distribution is quite important when the sample size at hand is small and when the standard deviation of the population is unknown. The \(t\)-distribution is widely employed in a lot of statistical frameworks, for example the Student’s \(t\)-test to assess the statistical significance of the difference between two sample means, or in the linear regression analysis.
Basically, we assume to take a sample of \(p\) observations from a normal distribution. We already know that a true mean value exists, but we can only calculate the sample mean. Defining \(\nu = p – 1\) as the number of degrees of freedom of the \(t\)-distribution, we can assess the confidence with which a given range would contain the true mean by constructing the distribution with the following p.d.f.: \begin {equation} f(x;\nu ) = \ \begin {cases} \dfrac {\Gamma \left (\frac {\nu + 1}{2}\right ) \left (1+ \dfrac {x^2}{\nu }\right )^{- \frac {\nu + 1}{2}}}{\sqrt {\nu \pi } \cdot \Gamma \left (\dfrac {\nu }{2} \right )} \qquad \text {if} \ x > 0 \\ \\ 0 \qquad \qquad \qquad \ \text {otherwise} \end {cases} , \label {pdftstudent} \end {equation}
4.2 Wald Test
The Wald test is named after the statistician Abraham Wald. It can be used in a number of different contexts to estimate the distance between the estimate \(\hat {\theta }\) of a parameter (that is, its MLE) and the proposed value of the same parameter \(\theta _0\). Substantially, it is a ’significance test’, meaning that its principle is to fit the regression without any restriction, and then assess whether the results seem to agree with the hypothesis.
We begin from a very simple case, referring to the above example on art market. Suppose that we want to test a null hypothesis \(H_0\): \(\beta _2 = \beta ^0_2\), where \(\beta ^0_2\) is the assumed value (in this case, zero) of the regression coefficient. We aim to evaluate the Wald distance \(W_j\) of a coefficient estimate from its hypothesized value: \begin {equation} W_j = \dfrac {b_j – \beta ^0_j}{\sqrt {\sigma ^2 S_{jj}}}, \label {jthWalddistance} \end {equation} where \(s^2\) is given by (18) and \(S_{jj}\) is the \(j\)-th diagonal element of the matrix \((X^T X)^{-1}\). If we assume that \(\mathbb {E}[\beta _j]=\beta ^0_j\), \(W_j\) is normally distributed. We can call \(W_j = t_j\) because it has a distribution with \(M-N\) degrees of freedom.
We first identify a confidence interval with which we would like to verify our model, for example the standard value of \(95\%\). So, we can state that it is unlikely that a single value of \(t_j\) falls outside the interval: \[(-t_{(1-\alpha /2), [M-N]}^*, \ t^*_{(1-\alpha /2), [M-N]}).\] The null hypothesis \(H_0\) should be rejected if \(W_j\) is sufficiently large.
In Greene’s own words, since ’it is so unlikely that we would conclude that it could not happen if the hypothesis were correct, so the hypothesis must be incorrect’.
Back to the results in the previous Chapter, if we compute \(W_k\) using the sample estimate of \(\sigma ^2\), i.e. \(s^2\), we have: \begin {equation} t_j = \dfrac {b_j – \beta ^0_j}{\sqrt {s^2 S_{jj}}}. \label {tjfortests} \end {equation} The variable \(t_j\) in the form (5) has a \(t\) distribution with \(M-N\) degrees of freedom. The \(t\) ratio is the ratio between the estimator \(b_j\) and its standard error, so: \[t_j=\dfrac {b_j}{\sqrt {s^2 S_{jj}}}\] can be used for tests. If it is larger than \(1.96\), this means that the coefficient is significantly different from \(0\) at \(95\%\) confidence level, the null hypothesis should be rejected, so the related coefficient can be considered statistically significant.
In the next Example we are going to deal with an econometric model derived from a study by Mroz [2] (published in \(1987\) on Econometrica), corresponding to Example \(5.2\) ([1], pages \(156-157\)). It will be very useful to outline how to read and understand the regression results in a Table.
Example 7.
Consider the following regression equation which aims to investigate the relation between married women’s earnings and other relevant data such as their age, education and children: \begin {equation} \ln (\textit {Earnings}) = \beta _1 + \beta _2 \cdot \textit {Age} + \beta _3 \cdot \textit {Age}^2 + \beta _4 \cdot \textit {Education} + \beta _5 \cdot \textit {Kids} + \epsilon . \label {Mrozmodel} \end {equation} Note the presence of the same covariate in \(2\) different positions: Age is considered both in the linear and in the quadratic forms. This structure violates the assumption of independence among covariates, but it justified by the well-known effect of age on income, which has a parabolic concave behaviour over time. This scenario is easily explained by the relation between wages and pensions, for example. For this reason, we expect that the coefficient \(\beta _2\) is positive and that \(\beta _3\) is negative.The number of observations is \(428\), corresponding to \(428\) white married women whose age was between \(30\) and \(60\) in \(1975\), and consequently the number of degrees of freedom of the model is \(428-5 = 423\). The following Table presents all the results, including the \(t\) ratio:
| Variable | Coefficient | Standard error | \(t\) ratio |
| Constant | \(3.24009\) | \(1.7674\) | \(1.833\) |
| Age | \(0.20056\) | \(0.08386\) | \(2.392\) |
| Age\(^2\) | \(-0.0023147\) | \(0.00098688\) | \(-2.345\) |
| Education | \(0.067472\) | \(0.025248\) | \(2.672\) |
| Kids | \(-0.35119\) | \(0.14753\) | \(-2.38\) |
To augment the above Table, we also know that the sum of squared residuals \(SSE\) is \(599.4582\), that the standard error of the regression \(s\) is \(1.19044\), and that \(R^2 = 0.040995\). In short, we can summarize the following:
- The \(t\) ratio shows that at \(95\%\) confidence level, all coefficients are statistically significant except the intercept, which is smaller then \(1.96\).
- The signs of all coefficients are consistent with our initial expectations: education affects earnings positively, the presence of children affects earnings negatively. We can estimate that an additional year of schooling yields \(6.7\%\) increase in earnings.
- The age acts as an inverted U on earnings, i.e. \(\beta _2\) is positive and \(\beta _3\) is negative. Specifically, the form of the age profile suggests that the peak of earnings can be approximately found at \(43\) years of age.
4.3 The \(F\) statistic
The F test, or F statistic, is a way to test a hypothesis against another one, for example the null hypothesis against the alternative hypothesis. We are going to treat this fundamental test as easily as possible (for further reading and technical details, see [1], \(157-161\)).
First, we should rigorously define the Fisher – Snedecor distribution: consider \(2\) random variables \(X\) and \(\widetilde {X}\), which are respectively distributed according to the chi-squared distributions \(\chi ^2 (k)\) and \(\chi ^2 (l)\), having \(k\) and \(l\) degrees of freedom. The Fisher – Snedecor distribution \(\mathcal {F}(k,l)\) is the distribution of the random variable \[F = \dfrac {X/k}{\widetilde {X}/l}.\] Its p.d.f. is given by \begin {equation*} f(x;k,l)=\dfrac {1}{x B(k/2, l/2)}\sqrt {\dfrac {k^k l^l x^k}{(kx+l)^{k+l}}}, \end {equation*} where \(B(\cdot )\) is Euler’s Beta function, i.e. \[B \left (\dfrac {k}{2}, \dfrac {l}{2} \right )=\int _0^1 t^{\frac {k}{2}-1} (1-t)^{\frac {l}{2}-1} dt,\] which is connected to Gamma by the following identity: \[B \left (\dfrac {k}{2}, \dfrac {l}{2} \right ) =\dfrac {\Gamma \left (\dfrac {k}{2}\right ) \Gamma \left (\dfrac {l}{2}\right )}{\Gamma \left (\dfrac {k+l}{2}\right )}.\] The mean value of such a random variable is \(\dfrac {l}{l-2}\) for \(l > 2\), and its variance is \(\dfrac {2 l^2 (k+l-2)}{k(l-2)^2 (l-4)}\) for \(l > 4\).
To carry out the \(F\) test, we are going to assume that the \(2\) random variables under consideration are normally distributed with variances \(\sigma ^2_X\) and \(\sigma ^2_{\widetilde {X}}\) and observed standard errors \(s^2_X\) and \(s^2_{\widetilde {X}}\). Now, since the random variables \[\dfrac {(k-1)s^2_X}{\sigma ^2_X} \qquad \text { and } \qquad \dfrac {(l-1)s^2_{\widetilde {X}}}{\sigma ^2_{\widetilde {X}}}\] are respectively distributed according to \(\chi ^2 (k-1)\) and \(\chi ^2 (l-1)\), then the random variable \[F=\dfrac {\sigma ^2_{\widetilde {X}}}{\sigma ^2_X}\dfrac {s^2_X}{s^2_{\widetilde {X}}}\] follows \(\mathcal {F}(k-1,l-1)\).
The easiest way to use the \(F\) test in Econometrics can be described as follows. First, we should also note that the \(F\) ratio quantifies the relationship between the relative increase in the \(SSR\) and the relative increase in degrees of freedom between \(2\) models. Call \(SSR_1\) and \(SSR_2\) the sums of squares of residuals of the \(2\) models, which respectively have \(p_1\) and \(p_2\) degrees of freedom. Model \(1\) is the ’simple’ model, whereas model \(2\) is ’complicated’. Clearly, we can take one of the \(2\) models based on the null hypothesis and the remaining one based on the alternative hypothesis, to test them against one another. We can also write: \begin {equation} F=\dfrac {SSR_1 – SSR_2}{SSR_2}\dfrac {p_2}{p_1 – p_2}, \label {Fratio} \end {equation} under the assumption that model \(1\) is simpler then model \(2\), then it has a larger \(SSR\). What we expect is that if the more complicated model (\(2\)) is correct, the following inequality holds: \[\dfrac {SSR_1 – SSR_2}{SSR_2} > \dfrac {p_1 – p_2}{p_2},\] which is equivalent to saying that if the \(F\) ratio (7) is smaller than \(1\), the simpler model is the correct one. On the other hand, if (7) is larger than \(1\), we can have \(2\) occurrences:
- either the more complicated model is the correct one;
- or the simpler model is the correct one, but the impression of a better fit achieved by the more complicated model is caused by the random scatter.
In order to try to answer this question, we can employ the P value, which provides an assessment of the probability of this last case. Basically, this verification works as follows:
- if the \(P\)-value is low, we can conclude that model \(2\) is significantly better than model \(1\);
- if the \(P\)-value is high, no evidence exists which supports model \(2\), so we accept model \(1\).
To conclude with, a few explanatory words about the P-value, also known as the asymptotic significance, which is the probability that, when the null hypothesis is assumed to be true, a result is obtained which is equal or more extreme than the one which is actually observed. Hence:
the smaller the \(P\)-value \(\ \ \Longrightarrow \ \ \) the higher the significance \(\ \ \Longrightarrow \)
\(\ \ \Longrightarrow \ \ \) the higher the probability that the null hypothesis does not appropriately explain the scenario.
In other words, given a significance level \(\alpha _l\) selected by the investigator, if the \(P\)-value is smaller than \(\alpha _l\), the data are inconsistent with the null hypothesis, so it must be rejected.
To conclude with, here is a very simple example to show how to use the \(P\)-value.
Example 8.
Suppose to flip a coin \(7\) times in a row. If the coin is fair, at every flip we have the following trivial probabilities: \[\text {Prob} \left \{\text {Outcome is Head} \right \}=\text {Prob} \left \{\text {Outcome is Tail} \right \}=\dfrac {1}{2}.\] Assume the fairness of the coin as the null hypothesis, i.e.Null hypothesis: the coin is fair.
Alternative hypothesis: the coin is unfair, or fixed.
Suppose that the \(P\)-value is calculated based on the total number of Heads obtained, and that the confidence cutoff is \(0.05\).
If the researcher gets ’Head’ \(7\) times, the probability of such an event, provided each flip of the coin is independent of the remaining flips, is \[(1/2)^7=0.0078 < 0.05,\] that is the result is significant at this confidence level.
Therefore, the null hypothesis should be rejected. We conclude that a very high probability exists that the coin is fixed.
On the other hand, if the researcher gets ’Head’ \(4\) times and ’Tail’ \(3\) times, the probability of such an event is \[\dfrac {7!}{4!3!}\dfrac {1}{2^7}=0.2734375 > 0.05,\] so this result is not significant. In this case, the null hypothesis can be accepted.
