Nonlinear partial least squares

Abstract

Partial Least Squares (PLS) has been shown to be a versatile regression technique with an increasing number of applications in the areas of process control, process monitoring and process analysis. This Thesis considers the area of nonlinear PLS; a nonlinear projection based regression technique. The nonlinearity is introduced as a univariate nonlinear function between projections, or to be more specific, linear combinations of the predictor and the response variables. As for the linear case, the method should handle multicollinearity, underdetermined and noisy systems. Although linear PLS is accepted as an empirical regression method, none of the published nonlinear PLS algorithms have achieved widespread acceptance. This is confirmed from a literature survey where few real applications of the methodology were found. This Thesis investigates two nonlinear PLS methodologies, in particular focusing on their limitations. Based on these studies, two nonlinear PLS algorithms are proposed. In the first of the two existing approaches investigated, the projections are updated by applying an optimization method to reduce the error of the nonlinear inner mapping. This ensures that the error introduced by the nonlinear inner mapping is minimized. However, the procedure is limited as a consequence of problems with the nonlinear optimisation. A new algorithm, Nested PLS (NPLS), is developed to address these issues. In particular, a separate inner PLS is used to update the projections. The NPLS algorithm is shown to outperform existing algorithms for a wide range of regression problems and has the potential to become a more widely accepted nonlinear PLS algorithm than those currently reported in the literature. In the second of the existing approaches, the projections are identified by examining each variable independently, as opposed to minimizing the error of the nonlinear inner mapping directly. Although the approach does not necessary identify the underlying functional relationship, the problems of overfitting and other problems associated with optimization are reduced. Since the underlying functional relationship may not be established accurately, the reliability of the nonlinear inner mapping will be reduced. To address this problem a new algorithm, the Reciprocal Variance PLS (RVPLS), is proposed. Compared with established methodology, RVPLS focus more on finding the underlying structure, thus reducing the difficulty of finding an appropriate inner mapping. RVPLS is shown to perform well for a number of applications, but does not have the wide-ranging performance of Nested PLS.