• Terms and Definitions 1
  • Variable
  • Parameter
  • Model
  • Linear and Nonlinear model
  • Fixed effect parameter
  • Random effect parameter
  • Mixed effects modeling
  • Structural model
  • Random effect model
  • Population PK and PD
• Terms and Definitions 2
  • Maximum likelihood
  • Objective function value
  • Log-likelihood ratio
  • Optimization procedure
  • Point estimate
  • Confidence intervals
  • Maximum a posteriori/post hoc estimates
  • Simulation
  • Clinical trial design
  • Pharmacodynamic terms
• Population analysis methods
  • Naive pooled data approach (NPD)
  • Naive averaged data approach (NAD)
  • Standard two-stage approach (S2S)
  • Three stage analysis (3S)
  • Bayesian estimation
  • One stage analysis
• Estimation methods
  • Ordinary least squares (OLS)
  • Weighted least squares (WLS)
  • Extended least squares (ELS)
• Good Modeling Practices (GMP)
  • Data management
  • Rational modeling process
  • Report

Good Modeling Practices (GMP)

Rational model building

Previous / Next

Evaluation of final parameter estimates and model selection

In order to characterize that the present model is representative of the data at hand, more than one selection criteria needs to be looked at.

Following are few suggestions:

o Diagnostic plots

Predicted vs. observed concentration will give you an overall sense of how well the model is doing. Weighted residuals vs. predicted concentration, residuals vs. time, and weighted residuals vs. time will also help diagnose potential errors related to the model selected and the variance structure.

Normally, the following plots are looked at:

o Individual plots of observed and predicted concentration versus time.

A goodness of fit plot where each individual profiles could be seen. A plot of representative individuals is shown.

Click for individual plots

o PRED & IPRED versus DV (population predicted and individual predicted versus observed concentration)

This graph provides the overall trend in the data. Ideally the concentrations should be uniformly randomly distributed along the line of identity. Any bias seen in the population predicted plot will pinpoint exploring any covariate relationship.Individual predictions are ususally better than population predictions.

Click for a PRED & IPRED vs DV plot

o RES vs PRED (Residuals versus population predicted)

Residuals are difference between an observations and its population predictions. This plot provides a useful picture of the residual error distribution. For a proportional error model, the residuals are cone shaped. For an additive error model the residuals are evenly distributed.

Click for a RES vs PRED plot

o WRES vs PRED (Weighted residuals versus population predicted)

This plot signifies whether the variability is modeled correctly or not. There should be uniform spread of the data without any trend. If any trend is observed like a S-shaped wave, then it is necessary to transform the data or use a different residual error model.

Click for a WRES vs PRED plot

Note:Though residual plots are some of the goodness of fits plot explored, the utility of theses residual plots is not well documneted.

o Difference in Objective function value (DOFV)

In the case of nested models (one model is a subset of other, i.e, null model (without covariates) is a subset of full model (with covariates), the difference in the objective function value or in other words, the log-likelihood ratios (LLR) are assumed to follow a chi-square distribution. The decision on choosing the null model or full model is based on the degrees of freedom (difference in number of parameters), pre-specified significance level and the critical chi-square value at that significance level and degrees of freedom.

For instance, a difference of 3.84 units in OFV between the competing models is considered significant at p<0.05 and with one degrees of freedom.

Plot of a chi-squared distribution for one degree of freedom with critical chisquare value is shown below:

o Akaike information criteria (AIC)

AIC is used in the case of non-nested models.

AIC = Objective function value + 2*p

where p = total number of parameters in the model (stuctural+error). Lower the value of AIC better the model.

o Precision of the parameter estimates

The precision or the standard error of the estimate could be obtained by the modeling process and can be compared between competing models. If the standard error is large, then probably the model could be overparametrized.