Calculates maximum likelihood estimates for the parameters of one of several univariate probability distributions.

Routine MLE calculates maximum likelihood estimates for the parameters of a univariate probability distribution, where the distribution is one specified by IPDF and where the input data X is (assumed to be) a random sample from that distribution.

Let {xi, i = 1, …, N} represent a random sample from a probability distribution with density function f(x∣θ), which depends on a vector θ ∈ ℜp containing the values of the parameters of the distribution. The values in θ are fixed but unknown and the problem is to find an estimate for θ given the sample data.

The likelihood function is defined to be the product

The estimator

That is, the estimator that maximizes L also maximizes log L and is the maximum likelihood estimate, or MLE for θ.

The likelihood problem is in general a constrained non-linear optimization problem, where the constraints are determined by the permissible range of θ. In a few situations, the problem has a closed form solution. Otherwise, MLE uses the numerical method as documented in routine BCONF (see Chapter 8, "Optimization" in the Math Library for details) to solve the likelihood problem. If USEMM is .TRUE. (the default), method of moments estimates serve as starting values of the parameters. In some cases, method of moments estimators may not exist, such as when certain moments of the true distribution do not exist; thus it is possible that the starting values are not truly method of moments estimates. If USEMM is set to .FALSE., input values of PARAM are used as starting values.

Upper and lower bounds, when needed for the optimization, have default values for each selection of IPDF (defaults will vary depending on the allowable range of the parameters). It is possible that the optimization will fail. In such cases, the user may try adjusting upper and lower bounds using the optional parameters PARAMLB, PARAMUB, or adjusting up or down the scaling factors in XSCALE, which can sometimes help the optimization converge.

Standard errors and covariances are supplied, in most cases, using the asymptotic properties of ML estimators. Under some general regularity conditions, ML estimates are consistent and asymptotically normally distributed with variance-covariance equal to the inverse Fisher’s Information matrix evaluated at the true value of the parameter, θ0:

MLE approximates the asymptotic variance using the negative inverse Hessian evaluated at the ML estimate:

The Hessian is approximated numerically for all but a few cases where it can be determined in closed form.

In cases when the asymptotic result does not hold, standard errors may be available from the known sampling distribution. For example, the ML estimate of the Pareto distribution location parameter is the minimum of the sample. The variance is estimated using the known sampling distribution of the minium, or first order‑statistic for the Pareto distribution.

For further details regarding the properties of the estimators and the theory of the maximum likelihood method, see Kendall and Stuart (1979). The different probability distributions have wide coverage in the statistical literature. See Johnson & Kotz (1970a, 1970b, or later editions).

Parameter estimation (including maximum likelihoood) for the generalized Pareto distribution is studied in Hosking and Wallis (1987) and Giles and Feng (2009), and estimation for the generalized extreme value distribution is treated in Hosking, Wallis, and Wood (1985).

The data are N = 100 observations generated from the logistic distribution with location parameter μ = 0.85 and scale parameter σ = 0.5.

The data are N = 100 observations generated from the generalized extreme value distribution with location parameter μ = 0, scale parameter σ = 1.0, and shape parameter ξ = ‑0.25.