Final Report Summary - EMBAF (Estimating monotone boundaries and frontiers)
In this project, the Marie-Currie fellow investigated the problem of estimation of support boundaries and frontiers, as well as its intersections with extreme-value theory, robustness, regression quantiles, polynomial splines, and optimization. This problem has increasing usage in classification and cluster analysis, economics, education, finance, management, physics, public policy, and other arenas. It is also closely related to edge estimation in image reconstruction.
The added value of this study is three-fold. First, the researcher contributes to the less-discussed problem of estimating boundary curves that are believed or required to be monotone and/or concave. The motivating data examples that he has analysed are concerned with various unrelated contexts such as, for instance, the assessment of the economic efficiency of European Air Controllers and American electric utility companies, or master curve prediction in the reliability programs of nuclear reactors. The following results have been obtained.
(i) Frontier modeling is clearly a problem belonging to extreme value theory. Reliable estimation of a joint support extremity from this perspective involves, however, many delicate issues when the sample size is not sufficiently large. This project examines a new extreme-value based model, Gamma-moment called, which provides a valid alternative for completely envelopment frontier models that often suffer from lack of efficiency, and for purely stochastic ones that are known to be sensitive to model misspecification.
The limit distribution of the resulting frontier estimator is derived and the method seems to offer a viable approach under the single monotonicity constraint.
(ii) Nonparametric boundary regression methods often result in unconstrained estimators. A feasible monotonizing technique is to take the largest (smallest) monotone function that lies below (above) the unrestricted estimator of interest or any convex combination of these two envelope functions. When the original unconstrained estimator is asymptotically equicontinuous as an empirical process, the Marie-Currie fellow demonstrates that all its first-order properties are valid for the isotonic projected estimator. Other unrelated motivating applications include the estimation of the biometric mean remaining lifetime function and involve a monotone estimator of the conditional distribution function that has the distributional properties of the local linear regression estimator.
(iii) The researcher also contributes to the expanding literature on robust frontier analysis by relying on a recent concept of a partial order-m frontier well inside the joint support but converging to the true full support boundary as the trimming order m goes to infinity. It is defined as a weighted moment of a nonstandard conditional distribution. To further investigate the study of the theory for robust frontier estimators, he first regularizes the partial frontier estimator by deriving a sequence m(n) which tends to infinity slowly enough, as the sample size n goes to infinity, to recover an asymptotic normal distribution (like the ordinary case of a fixed m). Then, highly unconventional ideas were developed in order to handle the inherent bias when estimating the true frontier itself.
(iv) Perhaps most importantly, he developes a novel constrained fit of the boundary curve which benefits from the smoothness of quadratic spline approximation and the computational efficiency of linear programs. Using cubic splines is also feasible and more attractive under simultaneous multiple shape constraints; computing the optimal spline smoother is then formulated into a second-order cone programming problem. Both constrained quadratic and cubic spline frontiers have a similar level of computational complexity to the unconstrained fits and inherit their asymptotic properties. The utility of this method is illustrated through applications to some real datasets and simulation evidence is also presented to show its superiority over the best known methods.
(v) Many of the findings and prescriptions stemming from this research project can now be directly applied by making use of two available R packages, called ``Frontiles'' and ``npbr'', dedicated to nonparametric frontier modeling in both statistical and econometric literatures. They provide a variety of functions for the most popular and innovative approaches to nonparametric boundary and efficiency estimation. The selected methods are concerned with empirical, smoothed, unrestricted as well as constrained fits under both separate and multiple shape constraints. The routines included in ``npbr'' and ``frontiles'' are user friendly and afford a large degree of flexibility in the estimation specifications. They integrate smoothing parameter selection and seamlessly allow for Monte Carlo comparisons among the implemented estimation procedures. These packages will be extremely useful for statisticians and applied researchers.
Second, in many practical situations, using the single estimate of the support boundary would not hold exhaustive information about the spread of the conditional distribution tails. Instead, the researcher suggests to use a new concept referred to as extremiles. Extremiles are attractive because of their conceptual simplicity, the easy implementation of their estimators and their good properties. They can be defined for a wide range of distributions and summarize a distribution in much the same way as the popular M-quantiles do. As is the case in the duality between the mean and the median, the choice between extremiles and quantiles will usually depend on the application at hand. Quantiles are appealing because of their conventional probability interpretation, whilst extremiles suggest better capability of fitting data when it comes to taking into account the magnitude of extremes. They afford more valuable information about the spread of long-tailed distributions and can serve as a more efficient instrument of risk protection than M-quantiles in actuarial and portfolio allocation problems.
Finally, other closely related research ideas were studied in the present project.
On one hand, the Marie-Currie fellow has elucidated the limit distributions of extremal kernel regression quantiles when they are located in the range of the data or near and even beyond the sample boundary. On the other hand, he has suggested measuring U.S. credit unions performance by using quantile-type distances to the efficient production frontier.