The selection of variables for the mixed-effects model has been an active topic in the current literature research. Hypothesis testing procedures are used to infer if an individual random component is significant or not. Based on these testing procedures, a step-by-step process can be constructed to select essential random effects. This paper reviews thirteen literature studies on the selection of variables in mixed-effects models.
The study by Gokalp and Arslan (2019) focuses on the selection of variables in elliptical "linear mixed models" (LMM) with "shrinkage penalty function (SPF)." The choice of variable and estimating the parameter simultaneously applies SPFs. Several studies implement the full LMM definition with elliptic distributions to help overcome outliers and heavy tailedness within the data. In addition to robust estimation, one of the LMM's severe topics is variable selection. Shrinkage methods have recently emerged as an efficient procedure for the selection of the model. For instance, ridge selection as one of the most common shrinkage methods is better than subset regression considering its variance reduction and accuracy. The ridge regression method has its disadvantages, and the subset procedures are not stable when a minor data change may result in a different selection model. Shrinkage methods performing variable selection like the selection operator and least absolute shrinkage overcome such obstacles.
Delegate your assignment to our experts and they will do the rest.
For the simulation study, Gokalp and Arslan (2019) carried out the LMM in the form of
y 1 = X 1 β + Z iui + e i, i=1,….., n,
Where y 1i = [y i1 ,….y ini ] and i is representing the subset observations. Multivariate normal distribution generates independent variables. There is a need to add Intercepts along with 1s to X. Z is generated by the first four X columns.
Two different methods for making comparisons are used to conduct simulation studies. These are the ECM algorithm for the ECM algorithm and the t-distributed LMM for the classical LMM. Using the SCAD to expand the ECM algorithm in elliptical LMM aid in selecting fixed effects effectively. The simulation results show that the method being proposed is better than classical ECM-SCAD, particularly when having longer tailed data. Moreover, LMM’s error terms and multivariate t-distributed random effects help obtain sparse solution sets and more biased coefficients.
In their study, Sciandra and Plaia (2018) indicated that model selection in mixed models concerns the proper identification of the fixed effects parameters, random effect components, and correlation structure. The study explores the problems of hybrid model selection from the graphical viewpoint by only focusing on the parameters of fixed effects and assuming “random effect factors” that do not need to be selected. The simulation study is carried out to show empirically how and when the proposed plot may help identify the right model if the considered candidate model seems to be right. The study further indicated that the algorithm is based on the l1-penalization of the highest chance estimation to select the relevant fixed effect coefficient. The article employs a single scenario simulation since it graphically compares a set of candidate models regarding fixed effects and conditionally on a specification of the randomly selected components. The authors introduced an extension to the deviance plot's mixed model class, proposed in many regression frameworks. A comparison of models is carried out with the aid of "Penalized Weighted Residual Sum of Square." As the comparison is concerned with only the fixed part specification, there is a penalty term for the random part's complexity. The proposed graphical tool allows evaluating the advantages of regression with no consideration of regression introduction order in the linear predictor.
Juming and Shang (2018) have proposed a procedure for variable selection to select the random and fixed effects in “linear mixed models simultaneously.” They employed an adaptive LASSO penalty and a profile log-likelihood for selecting and estimating variables. “Newton Rahson (NR)” optimization algorithm can be used to complete the estimation of parameter s. Furthermore, the error variance does not affect fixed and random effect selection since their noise parameter is not included in the “profile log-likelihood.” Therefore, that improves the selection procedure.
The proposed method in the article can quickly discover important impacts. The model dimension would dramatically reduce after a few NR iterations at the beginning. Hence, the burden of computation is highly increased. In the simulation study, the article asses the empirical procedure performance proposed in the simulation study involving four extensively used simulation settings. The report then compares the performance in accuracy estimation, as well as Photoshop frequency separation selection, with the available approach of joint selection. In each case of the approach, a hundred datasets are granted from the first model. For each simulation measure, both Photoshop frequency separation estimation and the unpenalized MLE are plugged, and ratio medians are computed over a hundred data sets.
Eyck and Cavanaugh (2018), in their article, investigated the natural approach to model section with the aid of pseudo-likelihood based information criteria under the "generalized linear mixed model (GLMM)" approach. They also proposed a new, improved method of comparing the criteria of information between the candidate models. The GLMM approach helps construct the requirements using the pseudo-data, depending on the candidate model. The article further noted that additional “Akaike Information Criterion (AIC)” variants were proposed based on the complexity penalization, which uses computationally intensive algorithms. That consists of cross-validation, bootstrapping, and Monte Carlo simulation.
The article's simulation study focuses on the criteria AIC and “Bayesian Information Criterion (BIC)” for the results following the Bernoulli, gamma distributions, Poisson, and binomial. Each of the simulation study sets is based on generating a thousand samples of data sized N = 100. The simulation is designed as a “factorial experiment,” whereby the factors include the selection criteria, distribution, and modeling construction method for the pseudo data. The construction method dictates the way the terms of “goodness-of-fit” are formulated for BIC and AIC. For each candidate model, the AIC and BIC means are computed after summarizing the model selection to assess each criterion's ability to pick the right model.
Reviewing a study article by Lee and Chen (2019), the study looked at selecting variables with a large p. It is assumed that the sparse mixed-effect vectors for various subpopulation vectors are different from each other to help explore the heterogeneity. That means in various sub-populations, one may select different subgroups of effects to assist in explaining the equivalent response variable. The main objective is to deduce the sparse mixed-effect vectors and identify the crucial subpopulation effects while classifying the observation into the subpopulation. Therefore, different variable selection approaches were imposed for large p and small n to decrease the vast covariance.
The article uses the Bayesian variable selection approach instead of the penalized approaches. The Bayesian method is for variable selection for "finite mixture of linear mixed-effects models (FMLMEMs)" fitting. Additionally, the proposed corresponding algorithm can help determine the importance of variables in the model and classify the observation. The article presented a unified approach for FMLMEMs to predict the members within the component and identify the essential “fixed effects and random effects.” In the simulation study, the article demonstrated that the proposed algorithm better performs in selecting variables and classification, including p>n and big collinearity problems.
Chauvet et al. (2019), in their article, focused on modeling grouped responses through a "Generalized Linear Mixed Models (GLMM)" with a vast explanatory variable number. The paper investigates the descriptive structure and then connects it to the interpretable dimensions. It further proposes combining reduction of dimension and predictor regulation by using supervised components aligned on the most interpretable and predictive directions within the explanatory space. The "Project Iterated Normal Gradient (PING)" algorithm describes the “(t + 1)-th iteration” of the single component mixed "Supervised Component-based Generalized Linear Regression (SCGLR)." The algorithm is repeated until one reaches the parameter stability. Considering structural information, mixed SCGLR performs classical estimates of GLMM and does not converge within the data.
Mixed SCGLR depends on reducing function for the less than p and covering all necessary information about Y contained in X. The algorithm aims to base the “explanatory subspace on structural dimensions” from which interpretability and stabilize predictions can be gained. Furthermore, mixed SCGLR can identify less or more local prediction structures that are associated with all the y k 's and do well in the grounded data with Bernoulli, Gaussian, and Poisson results. Besides, the mixed SCGLR built orthogonal components revealing the “dimensional explanatory predictive dimensions” and greatly help in model interpretation.
Säfken et al. (2018) described an add-on package to the lme4 that facilitates the selection of models based on conditional AIC, using examples in their illustration. The article focused on the stable and fast dependent computation of the "conditional Akaike Information Criterion (cAIC)" in the mixed model. The calculation is estimated with lme4 since they are implemented in the R-package cAIC4. Furthermore, they presented a conditional AICs implementation proposed for the settings in non-Gaussian. With that result, an introduction of a new stepwise conditional variable selection scheme is done. It allows for a full automatic choice of random and fixed effects based on favorable conditional AIC.
The cAIC4-package offers conditional AIC for dependent responses for Poisson distribution and an approximate conditional AIC for the binary data. Poisson cAIC makes use of the bias correlation. Besides, using the fast refit (Ime4-package function), both cAICs can be moderately calculated fast because n – d and n model refits are needed with n representing the number of observations. On the other hand, d represents the number of responses that are null in Poisson responses. The article offers empirical science users the possibility to use conditional AIC without worrying about complex and lengthy calculations. The article goes far beyond just model selection in the mixed models, but it extends to penalized spline smoothing in addition to other structured additive regression models.
Another article by Jakubk (2018) also focused on variable selections, in which the author introduces convex methods for selecting variables in high dimensional linear mixed models. The author further gives proof of the consistency of the variable selection method. The article argues that if q<n , it may be sufficient to treat LMM as a classical linear regression model for variable selection. As indicated by the simulation study in the article, LMM's positive effect in the proper selection of variables is only minor if it is compared to a more simple and effective method. It is useful if the needed component of variance-covariance, which is used in deriving the weighting matrix, is unknown and needs to be estimated from the provided data. The simulation study in the article compares two methods step by step with other related methods since “least absolute shrinkage and selection operator (LASSOP)” and LMMLASSO are solving different LMM type than LMMLASSO. In each of the comparisons done, only a generated problem for which the technique gives precisely the set S 0 is considered out of the hundred issues generated. The success of the simulation methods in this article is that the plans do not aim to estimate the matrix D and vector u . The ways are avowing the double estimation errors, which may arise when one estimates the matrix D first and after that estimates vector u based on the D estimate.
Another article by Ariyo et al. (2019) is a study that aims to ascertain if the vague prior choice invariance and covariance parameters are essential for model selection. The article measures how vague priors defiantly affect the marginal selection technique. In the article, simulation studies were carried out to anchor two sets of data. Within the simulation study, two different selection criteria were used based on absolute and minimum value differences. The model with the lowest selection criteria was selected for the strategy with minimum value. They selected a model that was the simplest for the absolute difference whenever the fundamental difference between the considered models is lower than 5. These have been suggested in the BIC and AIC literature in addition to DIC literature. The same threshold is used for WAIC and “ pseudo-Bayes factor (PSBF).” However, the previous work in the study did not justify for absolute difference beyond DIC. The study results in the article showed that the vague prior choice affected both criteria versions. However, the effect is minimal for the marginal performance compared to the conditional version of the criteria. Besides, the performance of conditional criteria is in a way that does not seem to be consistent, always selecting over-specified models.
Another article by Wu et al. (2016) considers both random and fixed effects within the linear mixed models. The article proposes a novel approach to efficiently selecting variables for models with high correlations between the covariates linked to random and fixed effects. The proposed method orthogonalizes both “fixed and random effects” to separately choose the two sets of impact with less effect on one another. The orthogonalization-based approach is easy to implement since all the selection steps rely on the least-squares and do not require any specific distribution assumption apart from the fourth-order moments. Although the method requires a concrete condition when separating fixed and random effects. Failure to which may result in efficient, fixed effects and the loss of activity. That may happen when one includes a variable as both random and fixed effects at the same time. If the two are different from one another, the numerical studies suggested that O-SCAD is quite efficient in selecting the fixed effect.
In their article, Ghosh and Thoresen (2017) consider a regular selection of essential fixed-effect variables in the “linear mixed effect models” together with the “maximum penalized estimation” of probability for parameters that are both “fixed and random-effect” based on the general “non-concave penalties.” The proposed approach tends to reduce computation efforts by producing the random effect estimations simultaneously to avoid the process requiring two steps. Furthermore, the article considers the penalized probability for both random effects and fixed parameters with general “non-convex penalties” and simultaneously maximize it to achieve their "maximum penalized probability estimators (MPLEs)." The approach further considers the “regularized selection” of variables with fixed-effect through an algorithm that is suitable and computationally efficient. The oracle's properties and consistency are proved for the low classical dimension cases and high dimensions of non-polynomial sample size order.
The paper suggests using a unified approach that employs an objective function’s “local quadratic approximation.” After that, the method uses an “iterative New-Raphson algorithm.” However, the article recommends combining the algorithm with a version of the “coordinate descent algorithm” to help achieve greater computational efficiency when there is a large p . The article proves asymptotic properties such as oracle properties of selecting variables and consistency for the general “non-convex loss” under high and low set-up dimensions. The improved “asymptotic property” and numerical performance over the classical L1 penalty illustrate the importance of SCAD function.
In an article by Cheng et al. (2019), they presented a new "nonlinear mixed-effects scalar-on function regression model" with the Gaussian process before focusing on selecting the variables from a larger number of candidate variables, including both operational and scalar data. The article developed a new and exciting algorithm for selecting variables called "functional least angle regression (FLARS)." With the algorithm, presentation of functional variables using a variety of techniques and the correlation between a mixed and scalar group and functional variables are focused. The selected algorithm is accurate and efficient for selecting variables and estimating the parameters, even if the number of functional variables is enormous. There is a correlation between the variables.
The proposed FLARS algorithm uses a measure of correlation from a functionally modified canonical analysis of correlation. It, therefore, simultaneously gives correlation and a projection. However, the conventional rules for stopping always fail in this algorithm because it depends on the tuning parameters. The article, therefore, proposes a “new stopping rule.” The article's simulation study and real data analysis show good performance in the “new stopping rule” alongside the FLARS simulation. The integration that is used in the calculation for the functional object is done using three different techniques. These are conventional representative data points (RDP), Basic functions (BF), and the new technique based on Gaussian Quadrature.
Finally, in the article by Hossain et al. (2018), they proposed the "non-penalty James-Stein shrinkage and pretest estimation" technique based on the longitudinal data's linear mixed model when some fixed effect parameters are restricted. The article further establishes asymptotic distributional biases and the proposed estimator risks. It then investigates their relative performance concerning the maximum probability estimators that are not limited. A simulation study in the article for the variety of inactive covariate combinations proves that shrinkage estimators are better in performance than the penalty estimators in some parameter space parts. That is possible when there are several inactive covariates within the same model. Besides, it shows that the penalty, shrinkage, and pretest estimators are better than unrestricted maximum likelihood estimators (UE).
Simulation study supports the PSE under a variety of “inactive covariates” over the UE. The simulation study shows that the restricted estimator performs best at or even near the restriction. However, it is dominated by the “positive-part shrinkage estimator (PSE)” as an individual moves far from the point of restriction point. A small movement away from the limitation, however, making the RE inefficient. That, therefore, questions its ability to be applied for practical purposes.
References
Ariyo, O., Lesaffre, E., Verbeke, G., & Quintero, A. (2019). Model selection for Bayesian linear mixed models with longitudinal data: Sensitivity to the choice of priors. Communications in Statistics - Simulation and Computation , 1-25. https://doi.org/10.1080/03610918.2019.1676439
Chauvet, J., Bry, X., & Trottiery, C. (2019). Component-based regularization of multivariate generalized linear mixed models. Journal of Computational and Graphical Statistics . https://doi.org/10.1080/10618600.2019.1598870
Cheng, Y., Shi, J., & Eyre, J. (2019). Nonlinear mixed-effects scalar-on-function models and variable selection. Statistics And Computing , 30 (1), 129-140. https://doi.org/10.1007/s11222-019-09871-3
Eyck, P., & Cavanaugh, J. (2018). An Alternate Approach to Pseudo-Likelihood Model Selection in the Generalized Linear Mixed Modeling Framework. The Indian Journal of Statistics , 80 (1), 98-122. Retrieved 27 October 2020, from.
Ghosh, A., & Thoresen, M. (2017). Non-concave penalization in linear mixed-effect models and regularized selection of fixed effects. Asta Advances In Statistical Analysis , 102 (2), 179-210. https://doi.org/10.1007/s10182-017-0298-z
Gokalp Yavuz, F., & Arslan, O. (2019). Variable selection in elliptical linear mixed model. Journal of Applied Statistics , 47 (11), 2025-2043. https://doi.org/10.1080/02664763.2019.1702928
Hossain, S., Thomson, T., & Ahmed, E. (2018). Shrinkage estimation in linear mixed models for longitudinal data. Metrika , 81 (5), 569-586. https://doi.org/10.1007/s00184-018-0656-1
Jakubk, J. (2018). Convex method for selection of fixed effects in high-dimensional linear mixed models. Retrieved 27 October 2020, from.
Juming, P., & Shang, J. (2018). A simultaneous variable selection methodology for linear mixed models. Journal of Statistical Computation and Simulation , 88 (17), 3323-3337. https://doi.org/10.1080/00949655.2018.1515948
Lee, K., & Chen, R. (2019). Bayesian variable selection in a finite mixture of linear mixed-effects models. Journal of Statistical Computation and Simulation , 89 (13), 2434-2453. https://doi.org/10.1080/00949655.2019.1620746
Säfken, B., Rügamer, D., Greven, S., & Kneib, T. (2018). Conditional Model Selection in Mixed-Effects Models with cAIC4. Retrieved 27 October 2020, from.
Sciandra, M., & Plaia, A. (2018). A graphical model selection tool for mixed models. Communications in Statistics - Simulation And Computation, 47 (9), 2624-2638. https://doi.org/10.1080/03610918.2017.1353617
Wu, P., Luo, X., Xu, P., & Zhu, L. (2016). New variable selection for linear mixed-effects models. Annals Of The Institute Of Statistical Mathematics , 69 (3), 627-646. https://doi.org/10.1007/s10463-016-0555-z
