`gradient_boosting_regression`

Gradient boosting regression supporting continuous and mixed-feature datasets. Builds an additive ensemble of regression trees by repeatedly fitting the current residuals under squared-error loss. Starts from the arithmetic mean of the training targets and then adds a scaled tree prediction at each boosting stage.

The library implements the regressor_protocol defined in the regression_protocols library and learns an additive ensemble of regression trees by repeatedly fitting the current residuals under squared-error loss.

API documentation

Open the ../../apis/library_index.html#gradient_boosting_regression link in a web browser.

Loading

To load this library, load the loader.lgt file:

| ?- logtalk_load(gradient_boosting_regression(loader)).

Testing

To test this library predicates, load the tester.lgt file:

| ?- logtalk_load(gradient_boosting_regression(tester)).

To run the performance benchmark suite, load the tester_performance.lgt file:

| ?- logtalk_load(gradient_boosting_regression(tester_performance)).

Features

Residual Fitting: Fits each regression tree to the residuals of the current additive model under squared-error loss.
Shrinkage: Supports a fixed learning rate to scale stage contributions and reduce overfitting.
Continuous and Mixed Features: Supports continuous attributes and categorical attributes encoded by the underlying regression-tree learner.
Tree Configuration: Exposes the underlying regression-tree depth, minimum-leaf, variance-reduction, and scaling options.
Diagnostics Metadata: Learned regressors record model name, target, training example count, initial prediction, fitted stage count, and effective options, accessible using the shared regression diagnostics predicates.
Model Export: Learned regressors can be exported as predicate clauses or written to a file.
Reference Benchmarks: Includes a dedicated performance suite reporting training time, RMSE, and MAE for representative regression datasets.

Regressor representation

The learned regressor is represented by default as:

gradient_boosting_regressor(InitialPrediction, WeightedTrees, Diagnostics)

The exported predicate clauses therefore use the shape:

Functor(InitialPrediction, WeightedTrees, Diagnostics)

In this representation, WeightedTrees contains weighted_tree(LearningRate, Tree) terms and Diagnostics stores training metadata including the effective options.

Diagnostics syntax

The diagnostics/2 predicate returns a list of metadata terms with the form:

[
    model(gradient_boosting_regression),
    target(Target),
    training_example_count(TrainingExampleCount),
    options(Options),
    initial_prediction(InitialPrediction),
    stage_count(StageCount)
]

Where:

model(gradient_boosting_regression) identifies the learning algorithm that produced the regressor.
target(Target) stores the target attribute name declared by the training dataset.
training_example_count(TrainingExampleCount) stores the number of examples used during training.
options(Options) stores the effective learning options after merging the user options with the library defaults.
initial_prediction(InitialPrediction) stores the constant prediction used to initialize the additive model before fitting any trees.
stage_count(StageCount) stores the number of boosting stages that were actually fitted.

Use the regression_protocols diagnostic/2 and regressor_options/2 helper predicates when you only need a single metadata term or the effective options.

Options

The learn/3 predicate accepts the following options:

number_of_estimators/1: Maximum number of boosting stages to fit. Each stage adds one regression tree to the ensemble. The default is 50. Training can stop before reaching this limit when the residual sum of squares becomes negligible.
learning_rate/1: Shrinkage factor applied to each stage prediction before it is added to the current model. Smaller values usually require more stages but can improve generalization. The default is 0.1.
maximum_depth/1: Maximum depth allowed for each regression tree used as a base learner. Lower values produce weaker, simpler trees; higher values allow each stage to model more complex residual structure. The default is 3.
minimum_samples_leaf/1: Minimum number of training examples required in each leaf of a base learner tree. Increasing this value makes the fitted trees more conservative and can reduce overfitting. The default is 1.
minimum_variance_reduction/1: Minimum reduction in target variance required to accept a split when fitting a base learner tree. Larger values make tree growth stricter by rejecting weak splits. The default is 0.0.
feature_scaling/1: Controls continuous-feature scaling in the underlying regression-tree learner. The accepted values are true and false. The default is false.

gradient_boosting_regression