abess (Adaptive BEst Subset Selection) library aims to solve general best subset selection, i.e., find a small subset of predictors such that the resulting model is expected to have the highest accuracy. The selection for best subset shows great value in scientific researches and practical application. For example, clinicians wants to know whether a patient is health or not based on the expression level of a few of important genes.
This library implements a generic algorithm framework to find the optimal solution in an extremely fast way [1]. This framework now supports the detection of best subset under: linear regression, (multi-class) classification, censored-response modeling [2], multi-response modeling (a.k.a. multi-tasks learning), etc. It also supports the variants of best subset selection like group best subset selection [3] and nuisance best subset selection [4]. Especially, the time complexity of (group) best subset selection for linear regression is certifiably polynomial [1] [3].
Install abess from R CRAN by running the following command in R:
install.packages("abess")Best subset selection for linear regression on a simulated dataset in R:
library(abess)
sim_dat <- generate.data(n = 300, p = 1000)
abess(x = sim_dat[["x"]], y = sim_dat[["y"]])See more examples analyzed with R in the R markdowns collections.
Install the stable abess Python package from Pypi:
$ pip install abessTo install abess in source, please view chapter: Installation. Import best subset selection solver for linear regression in a Python project:
from abess.linear import abessLmSee more examples analyzed with Python in the tutorials; the notebooks are available here.
To show the power of abess in computation, we assess its timings of the CPU execution (seconds) on synthetic datasets, and compare to state-of-the-art variable selection methods. The variable selection and estimation result are deferred to Tutorial.
We compare abess R package with three widely used R packages: glmnet, ncvreg, L0Learn. Conducting the following commands in shell:
$ Rscript ./R-package/example/timings.Rwe obtain the runtime comparison picture:
We compare abess Python package with scikit-learn on linear and logistic regression. Results are presented in the below figure, and can be reproduce by running the commands in shell:
$ python ./python/example/timings.py
In both R and Python environments, abess reaches a high efficient performance especially in linear regression where it gives the fastest solution.
abess is a free software and its source code are publicly available in Github. The core framework is programmed in C++, and user-friendly R and Python interfaces are offered. You can redistribute it and/or modify it under the terms of the GPL-v3 License. We welcome contributions for abess, especially stretching abess to the other best subset selection problems.
| [1] | (1, 2) Junxian Zhu, Canhong Wen, Jin Zhu, Heping Zhang, and Xueqin Wang (2020). A polynomial algorithm for best-subset selection problem. Proceedings of the National Academy of Sciences, 117(52):33117-33123. |
| [2] | Pölsterl, S (2020). scikit-survival: A Library for Time-to-Event Analysis Built on Top of scikit-learn. J. Mach. Learn. Res., 21(212), 1-6. |
| [3] | (1, 2) Yanhang Zhang, Junxian Zhu, Jin Zhu, and Xueqin Wang (2021). Certifiably Polynomial Algorithm for Best Group Subset Selection. arXiv preprint arXiv:2104.12576. |
| [4] | Qiang Sun and Heping Zhang (2020). Targeted Inference Involving High-Dimensional Data Using Nuisance Penalized Regression, Journal of the American Statistical Association, DOI: 10.1080/01621459.2020.1737079. |