What is the Wald test?
The Wald Test, a venerable statistical tool, holds applicability across an array of regression models, encompassing linear regression, logistic regression, and generalized linear models (GLM). This test scrutinizes the significance of individual parameters within the model, gauging whether specific coefficients significantly deviate from their hypothesized values, often set at zero.
Its test statistic, obtained by dividing the estimated parameter by its standard error, adheres to an asymptotic standard normal distribution under the assumption of a large sample size. The all-important p-value linked to the test statistic dictates parameter significance. Notably, a low p-value heralds statistical significance, leading to the rejection of the original hypothesis.
To elucidate, suppose we consider a logistic regression model evaluating the effect of medication, a binary predictor variable, on the likelihood of a positive outcome, a binary response. Employing the Wald Test would enable the determination of whether the drug exhibits statistically significant effects on the response variable, thereby providing invaluable insights into drug efficacy.
What is the Likelihood Ratio Test (LRT)?
Conversely, the Likelihood Ratio Test (LRT) proves instrumental when assessing hypotheses in the realm of statistical modeling, particularly in the comparison of nested models, wherein one model represents a simplified version of the other. This test rigorously evaluates whether adding or removing predictor variables from a model results in significant improvements in model fit relative to a simpler model.
The crux of the LRT lies in its test statistic, founded upon the difference in log-likelihood between the two models under scrutiny. More specifically, the log-likelihood of the more complex model (Model 2) is juxtaposed against the log-likelihood of the simpler model (Model 1). The resultant difference is multiplied by two, yielding a test statistic that follows a chi-square distribution, with degrees of freedom contingent upon the disparity in the number of parameters between the two models.
Significantly, the LRT surpasses the Wald Test's potency in scenarios involving two nested models, particularly when confronted with diminutive sample sizes or nonlinear models. As a consequence, it serves as a powerful mechanism to assess the overall significance of a set of parameters.
Fig. 1. Quantile–quantile plot of saddle point approximation (x-axis) against other approximations to the p-value. (Lumley T, et al., 2013)
Differences Between the Wald test and the Likelihood Ratio Test (LRT)
Although both the Wald test and the LRT are used for hypothesis testing, they serve different purposes and compare different aspects of statistical models. The main differences between the two tests are as follows:
Evidently, the Wald Test and the LRT diverge substantially in terms of their primary objectives and the facets of statistical models they compare. Firstly, the Wald Test concentrates on scrutinizing the significance of each parameter within a model, enabling independent assessment of the effect of a particular predictor variable on the response variable, irrespective of other predictor variables. In sharp contrast, the LRT focuses on comparing the overall fit of two nested models, testing the hypothesis that additional parameters in a more complex model significantly enhance model fit relative to a simpler model.
Secondly, the Wald Test is primarily wielded to compare models boasting one or more specific coefficient differences. Thus, it is frequently employed in regression models to gauge the significance of individual predictor variables. Conversely, the LRT is tailored specifically for comparing nested models, providing the means to ascertain whether a more intricate model captures the data more fittingly than a simpler model, boasting fewer parameters.
Additionally, the efficacy of these tests varies significantly. The LRT, as empirically demonstrated, exhibits superior performance compared to the Wald Test, particularly in the presence of limited data or when the underlying data distribution deviates from linearity. Consequently, the LRT emerges as the favored option when grappling with complex models or data constraints.
Finally, assumptions pertaining to these tests warrant meticulous consideration. The Wald Test hinges on the asymptotic normality assumption, which remains valid for sufficiently large sample sizes. However, in scenarios featuring smaller samples or nonlinear models, these assumptions may falter, thereby compromising the reliability of results. On the other hand, the LRT distinguishes itself through its robustness, remaining valid even in the presence of modest sample sizes and demonstrating reduced sensitivity to distributional assumptions.
References
- Tu W, Zhou XH. A Wald test comparing medical costs based on log-normal distributions with zero valued costs. Stat Med. 1999 Oct 30;18(20):2749-61.
- Lumley T, Scott A. Partial likelihood ratio tests for the Cox model under complex sampling[J]. Statistics in Medicine, 2013, 32(1): 110-123.
