The LCGC Blog: A Simplified Guide for Weighted Fitting and its Significance in Separation Science

This blog is focused on explaining the fundamentals of weighted fitting, along with its significance and application in analytical chemistry.

Calibration curves are commonly used for quantification of an analyte in a sample by comparing the analyte’s response to the response of standards at known concentrations. Calibration curves are commonly used for quantitation of target analytes in food chemistry, environmental analysis, and the pharmaceutical industry. All separation scientists are familiar with building a calibration curve through linear regression. The most common calibration function is the affine linear equation:

where f (x) is the modeled instrument response, x is analyte concentration or mass (independent variable), m is the slope, and b is the intercept. Typically, the and parameters in Equation 1 are solved for by minimizing the residual sum of squares between f (x) and the measured instrument response (y, dependent variable). This approach is known as the “ordinary least squares” (OLS) solution.

OLS methodology does not always work for quantitating samples in the lower range of the calibration curve. This is an especially challenging scenario for the pharmaceutical industry, where accurate quantitation of impurities at ≤1% of target analytes is essential. Here, weighted least squares (WLS) regression can significantly improve the quantitation of analytes at the low end of the calibration range. WLS can easily be performed at the click of a button in several specialized software, such as MATLAB, R, etc (1). However, it is vital for separation scientists to have a fundamental understanding of WLS and its rationale for proper use in any software.

In this blog, we provide a simplified step-by-step guide to perform WLS using calibration data of carbamazepine (a small molecule drug commonly utilized for treating epilepsy) analyzed via high-performance liquid chromatography–ultraviolet (HPLC–UV). A fundamental discussion of noise characteristics is provided to better explain why WLS can improve quantitation. The implementation of WLS in Excel (see the DOI link below) is provided, alleviating the need for analysts to buy specialized software. The errors in quantitation via ordinary least square fit and weighted least square fit are compared. Weighted fitting can greatly improve quantitation, making the evaluation of its utility, as described in this blog, a critical step of developing accurate and robust analytical methods.

Click here for the replication data.

Step 1

Define the linear range for your method and prepare a suitable number of standards within that linear range. In this case, HPLC–UV provided a linear response for the concentration range of 0.005 mM to 1 mM carbamazepine. Standards were prepared in MeOH and 1 µL of each sample was injected into the HPLC in triplicate. Here, peak area was used as a measurement of instrument response.

The data in Table 1 can be used to plot a calibration curve (in Excel or similar software), where the average peak area for three injections is plotted on the y-axis versus analyte concentration on the x-axis. Note that some analysts may prefer to plot all three replicates rather than the average value. The linear trend line with fitted equation and R² value can be automatically calculated in Excel (using the OLS solution) as is shown in Figure 1.

Step 2

Next, the calibration curve should be evaluated using quality control solutions or in this case, for simplicity, through back calculation (2). In back calculation, the peak areas of the calibration standards and the fitted calibration function are used to back calculate the calibration standards’ concentration. These back calculated concentrations can then be compared to their true value, and the quality of the calibration curve can be evaluated.

Generally, a R² value of 0.9991 is considered good, and visually inspecting the calibration curve and data points also seems to indicate good linearity. The fitted calibration curve equation is:

Is this fit truly good enough to quantitate analytes within the whole linear range? Let’s see!

As can be seen from Table 2, the errors for Cal 1 and Cal 2 are exceedingly high, and this can result in inaccurate quantitation of samples at low end of the concentration range. This area of the calibration curve is often used when performing impurity analysis, where concentrations of the target analytes are low.

What can be done to resolve this issue? Let’s take a closer look at the analytical data and the assumptions of our calibration.

Step 3

For chromatographic quantitation with OLS, it is assumed that variance due to instrument noise is homogenous. However, this is not always true due to the concept of “heteroscedastic noise” that is often superimposed onto our analytical data. Noise in measurement science, regardless of its probability density or power spectral density, can either be (i) independent of sample concentration or measurement intensity, making it “homoscedastic,” or (ii) be related to sample concentration, making it “heteroscedastic.” Figure 2 shows the difference between a simulated calibration curve with homoscedastic noise (ε) versus heteroscedastic noise (ε (x)). Homoscedastic noise (Figure 2-A) is advantageous, as the probability density of the measurement does not change throughout the concentration range. This allows for the robust use of OLS, as well as associated calculations, such as the popular three times standard deviation method for limit of detection (LOD = 3s). Heteroscedastic noise (Figure 2b) complicates data analysis as the probability density changes throughout the concentration range, making OLS not robust and common calculations, such as LOD = 3s, not statistically valid. In our experience, most chromatography-based quantitation methods, such as LC–UV, LC–MS, and GC–MS, tend to have heteroscedastic noise if calibration is achieved over a wide enough concentration range (~ >2 orders of magnitude).

Now, let’s see how the variation in our data looks. Is the noise homoscedastic or heteroscedastic?

We can see that standard deviation is increasing with the increase in concentration and response, which shows that data is heteroscedastic (standard deviation is square root of variance). One way to improve the calibration of data which has heteroscedastic noise (such as this case) is to use weighted least squares regression. WLS has a similar objective as OLS, and it minimizes a weighted residual sum of squares. There are many options for weighting schemes such as 1/x_i, 1/x_i², or 1/(standard deviation)². Weighting by inverse variance (1/variance) is the most common method, as this allows for more weight to be placed on observations measured with less absolute uncertainty which often leads to more robust calculation of the m and b parameters in Equation 1.

Step 4

When using weighted least squares, the slope and intercept of equation one can be solved using the following equations (3):

We used the weighting (W_i) of 1/variance for calibration level i. Utilizing equation 3 and 4, an m value of 17024.1 and a b value of 8.79 were obtained. The updated parameters were used to repeat the quantitation that was done in Table 2 using:

In Table 3, it can be seen that with the updated parameters in the calibration curve obtained by the weighted fit, there is no bias in error towards the lower range of the calibration curve. The average error for back calculation using WLS was 4.02%, a reduction of more than 6-fold compared to the average back calculated error using OLS of 27.7%.

Concluding Thoughts

This blog shows the advantage of weighted least square fitting for real-world methods. It is crucial for emerging separations scientists to understand the reasoning behind weighted fitting, how to perform it, and its utilization in various industries. Using the four simple steps provided in this blog, analysts can determine if WLS is useful for their method and easily implement WLS by just using Excel, as shown in the Excel sheet.

References

(1) Least-Squares Solution in Presence of Known Covariance (lscov). MATLAB Help Center 2025. https://www.mathworks.com/help/matlab/ref/lscov.html (accessed 2025-5-14)

(2) Jurado, J. M.; Alcázar, A.; Muñiz-Valencia, R.; Ceballos-Magaña, S. G.; Raposo, F. Some Practical Considerations for Linearity Assessment of Calibration Curves as Function of Concentration Levels According to the Fitness-for-Purpose Approach. Talanta 2017, 172, 221–229. DOI: 10.1016/j.talanta.2017.05.049

(3) de Levie, R. When, Why, and How to Use Weighted Least Squares. J. Chem. Educ. 1986, 63 (1), 10. DOI: 10.1021/ed063p10

About the Authors