Fits a multivariate linear regression model using least squares.

The type double function is imsls_d_regression.

If the optional argument IMSLS_NO_INTERCEPT is not used, regression returns a pointer to an array of length n_dependent × (n_independent + 1) containing a least-squares solution for the regression coefficients. The estimated intercept is the initial component of each row, where the i‑th row contains the regression coefficients for the i‑th dependent variable.

Function imsls_f_regression fits a multivariate multiple linear regression model with or without an intercept. The multiple linear regression model is

where the observed values of the yi’s are the responses or values of the dependent variable; the xi1’s, xi2’s, …, xik’s are the settings of the k (input in n_independent) independent variables; β0, β1, …, βk are the regression coefficients whose estimated values are to be output by imsls_f_regression; and the ɛi’s are independently distributed normal errors each with mean 0 and variance s2. Here, n is the sum of the frequencies for all nonmissing observations, i.e.,

where fi is equal to frequencies[i] if optional argument IMSLS_FREQUENCIES is specified and equal to 1.0 otherwise. Note that by default, β0 is included in the model.

More generally, imsls_f_regression fits a multivariate regression model. See the chapter introduction for a description of the multivariate model.

Function imsls_f_regression computes estimates of the regression coefficients by minimizing the sum of squares of the deviations of the observed response yi from the fitted response

for the n observations. This minimum sum of squares (the error sum of squares) is output as one of the analysis of variance statistics if IMSLS_ANOVA_TABLE (or IMSLS_ANOVA_TABLE_USER) is specified and is computed as follows:

Another analysis of variance statistic is the total sum of squares. By default, the total sum of squares is the sum of squares of the deviations of yi from its mean

the so-called corrected total sum of squares. This statistic is computed as follows:

When IMSLS_NO_INTERCEPT is specified, the total sum of squares is the sum of squares of yi, the so-called uncorrected total sum of squares. This is computed as follows:

See Draper and Smith (1981) for a good general treatment of the multiple linear regression model, its analysis, and many examples.

In order to compute a least-squares solution, imsls_f_regression performs an orthogonal reduction of the matrix of regressors to upper-triangular form. The reduction is based on one pass through the rows of the augmented matrix (x, y) using fast Givens transformations. (See Golub  and Van Loan 1983, pp. 156–162; Gentleman 1974.) This method has the advantage that the loss of accuracy resulting from forming the crossproduct matrix used in the normal equations is avoided.

By default, the current means of the dependent and independent variables are used to internally center the data for improved accuracy. Let xi be a column vector containing the j-th row of data for the independent variables. Let xi represent the mean vector for the independent variables given the data for rows 1, 2, …, i.

The current mean vector is defined as follows:

where the wj’s and the fj’s are the weights and frequencies. The i‑th row of data has

subtracted from it and is multiplied by

where

Although a crossproduct matrix is not computed, the validity of this centering operation can be seen from the following formula for the sum of squares and crossproducts matrix:

An orthogonal reduction on the centered matrix is computed. When the final computations are performed, the intercept estimate and the first row and column of the estimated covariance matrix of the estimated coefficients are updated (if IMSLS_COEF_COVARIANCES or IMSLS_COEF_COVARIANCES_USER is specified) to reflect the statistics for the original (uncentered) data. This means that the estimate of the intercept is for the uncentered data.

As part of the final computations, imsls_f_regression checks for linearly dependent regressors. In particular, linear dependence of the regressors is declared if any of the following three conditions are satisfied:

On completion of the final computations, if the i‑th regressor is declared to be linearly dependent upon the previous i − 1 regressors, the i‑th coefficient estimate and all elements in the i‑th row and i‑th column of the estimated variance-covariance matrix of the estimated coefficients (if IMSLS_COEF_COVARIANCES or IMSLS_COEF_COVARIANCES_USER is specified) are set to 0. Finally, if a linear dependence is declared, an informational (error) message, code IMSLS_RANK_DEFICIENT, is issued indicating the model is not full rank.

A regression model

is fitted to data taken from Maindonald (1984, pp. 203–204).

A weighted least-squares fit is computed using the model

and weights 1∕i2 discussed by Maindonald (1984, pp. 67−68).

In the example, IMSLS_WEIGHTS is specified. The minimum sum of squares for error in terms of the original untransformed regressors and responses for this weighted regression is

where wi = 1/i2, represented in the C code as array w.

A multivariate regression is performed for a data set with two dependent variables. Also, usage of the keyword IMSLS_X_INDICES is demonstrated. Note that the required input variable y is not referenced and is declared as a pointer to a float.

Continuing with Example 1data, the example below invokes the regression function using values of IDO greater than 0. Also, usage of the keywords IMSLS_COEF_COVARIANCES and IMSLS_X_MEAN is demonstrated.