Research Area: Machine Learning
A method for the local and global interpretation of a black-box model on the basis of the well-known generalized additive models is proposed. It can be viewed as an extension or a modification of the algorithm using the neural additive model. The method is based on using an ensemble of gradient boosting machines (GBMs) such that each GBM is learned on a single feature and produces a shape function of the feature. The ensemble is composed as a weighted sum of separate GBMs resulting a weighted sum of shape functions which form the generalized additive model. GBMs are built in parallel using randomized decision trees of depth 1, which provide a very simple architecture. Weights of GBMs as well as features are computed in each iteration of boosting by using the Lasso method and then updated by means of a specific smoothing procedure. In contrast to the neural additive model, the method provides weights of features in the explicit form, and it is simply trained. A lot of numerical experiments with an algorithm implementing the proposed method on synthetic and real datasets demonstrate its efficiency and properties for local and global interpretation.
Keywords:
Additive models
Gradient boosting
Decision trees
Neural additive model
Black-box model
Author(s) Name: Andrei V. Konstantinov, Lev V. Utkin
Journal name: Knowledge-Based Systems
Conferrence name:
Publisher name: Elsevier
DOI: 10.1016/j.knosys.2021.106993
Volume Information: Volume 222
