By virtue of the alternating direction method of multipliers (ADMM)

the multivariate regularized least-squares model fitting problem is decomposed into multiple univariate subproblems that are solvable in parallel. By introducing a tunable step size to accelerate the algorithm

a highly parallel maximally split generalized ADMM (MS-GADMM) is developed for the regularized extreme learning machine (RELM). The convergence condition of the MS-GADMM is established and the computational complexity of the MS-GADMM-based RELM is analyzed. Through experiments on real-world benchmark datasets

the MS-GADMM is compared with a maximally split relaxed ADMM recently presented in the literature. In the GPU implementation experiments

the MS-GADMM has obtained very large GPU acceleration ratios

which demonstrates the high parallelism of the proposed MS-GADMM-based RELM.