• Terms and Definitions 1
  • Variable
  • Parameter
  • Model
  • Linear and Nonlinear model
  • Fixed effect parameter
  • Random effect parameter
  • Mixed effects modeling
  • Structural model
  • Random effect model
  • Population PK and PD
• Terms and Definitions 2
  • Maximum likelihood
  • Objective function value
  • Log-likelihood ratio
  • Optimization procedure
  • Point estimate
  • Confidence intervals
  • Maximum a posteriori/post hoc estimates
  • Simulation
  • Clinical trial design
  • Pharmacodynamic terms
• Population analysis methods
  • Naive pooled data approach (NPD)
  • Naive averaged data approach (NAD)
  • Standard two-stage approach (S2S)
  • Three stage analysis (3S)
  • Bayesian estimation
  • One stage analysis
• Estimation methods
  • Ordinary least squares (OLS)
  • Weighted least squares (WLS)
  • Extended least squares (ELS)
• Good Modeling Practices (GMP)
  • Data management
  • Rational modeling process
  • Report

 

Estimation methods

When fitting a model to a data, the main objective is to obtain estimates of parameters of interest, such that the differences between predictions and observations (residuals) are minimal. This is usually accomplished by minimizing a "sum of squares" objective function. Three of the most commonly used minimization methods are Ordinary least squares (OLS), weighted least squares (WLS) and Extended least squares (ELS) approach.

Ordinary least squares (OLS)

The ordinary least squares estimation minimizes the sum of squared residuals, often called the objective function value. The OLS objective function value is given as

OFV_OLS = Σ_i(y_pred-y_obs)²

where y_pred denotes the predicted value of y_obs based on the model, and y_obs are observed concentrations. The OLS method makes the following assumption that all observations have equal variance. i.e., the residuals are homoscedastic.

Weighted least squares (WLS)

The weighted least squares estimation minimizes the below objection value, given as

OFV_WLS = Σ_i.W_i(y_pred-y_obs)²

where W_i denotes the weight, which is inversely proportional to the variance of the observation. The variance of the observation varies directly with the magnitude of the observation, as in radioactive counting. The WLS approach assumes unequal error variance (heteroscedastic). Thus WLS gives less weight to observations with larger error variance and viceversa.

Extended least squares (ELS)^1,2

In the extended least squares estimation method, the variance model can take any form including non linear functions. The ELS estimation minimizes the below objective function value.

OFV_ELS = Σ_i.[{ (y_pred-y_obs)² /W_i}+ln (Var ( y_pred)]

where Wi is given by Var ( y_pred). The ELS equation can be derived by maximum likelilood method, and under this assumption the objective function value is proportional to twice the negative log likelihood.

References

1. Peck CC, Beal SL, Sheiner LB, Nichols AI.Extended least squares nonlinear regression: a possible solution to the "choice of weights" problem in analysis of individual pharmacokinetic data. J Pharmacokinet Biopharm. 1984 Oct;12(5):545-58.
2. Peck CC, Sheiner LB, Nichols AI.The problem of choosing weights in nonlinear regression analysis of pharmacokinetic data. Drug Metab Rev. 1984;15(1-2):133-48.