linear_svm_classifier

Linear support vector machine classifier supporting both binary and multiclass classification. Multiclass classification is implemented using a one-vs-rest scheme with one linear margin model per class, and training uses batch subgradient descent to fit each hinge-loss model.

The library implements the classifier_protocol defined in the classification_protocols library. It provides predicates for learning a classifier from a dataset object, using it to make predictions, and exporting the learned model as a list of predicate clauses or to a file.

Datasets are represented as objects implementing the dataset_protocol protocol from the classification_protocols library. Existing datasets in classification_protocols/test_datasets can be used for binary, multiclass, continuous, categorical, and mixed-feature testing.

API documentation

Open the ../../docs/library_index.html#linear_svm_classifier link in a web browser.

Loading

To load this library, load the loader.lgt file:

| ?- logtalk_load(linear_svm_classifier(loader)).

Testing

To test this library predicates, load the tester.lgt file:

| ?- logtalk_load(linear_svm_classifier(tester)).

To run the performance benchmark suite, load the tester_performance.lgt file:

| ?- logtalk_load(linear_svm_classifier(tester_performance)).

Features

  • Binary and Multiclass Classification: Learns one-vs-rest linear margin models and predicts the class with the highest score.

  • Continuous Features: Standardizes numeric attributes using z-score scaling derived from the training data.

  • Categorical Features: Expands discrete attributes using one-hot encoding based on the declared dataset attribute values and rejects unseen values with a domain error.

  • Missing Values: Encodes missing numeric and categorical values represented using anonymous variables using explicit missing-value indicator features.

  • Unknown values: Prediction requests containing categorical values that are not declared by the dataset raise a domain error.

  • Classifier Export: Learned classifiers can be exported as predicate clauses or written to a file.

  • Pretty Printing: Prints a compact summary of classes, encoders, and learned weight vector sizes.

  • Reference Benchmarks: Includes a dedicated performance suite covering the weather, mixed, iris_small, missing_mixed, and breast_cancer datasets with reported training time and training accuracy.

Options

The learn/3 predicate supports these options:

  • learning_rate/1 - base gradient descent learning rate (default: 0.1)

  • maximum_iterations/1 - maximum number of optimization iterations (default: 1000)

  • tolerance/1 - convergence threshold for the maximum parameter update (default: 1.0e-6)

  • l2_regularization/1 - L2 penalty factor applied to weights (default: 0.01)

Usage

Learning a classifier

| ?- linear_svm_classifier::learn(weather, Classifier).

| ?- linear_svm_classifier::learn(iris_small, Classifier, [learning_rate(0.05), maximum_iterations(1500)]).

Making predictions

| ?- linear_svm_classifier::learn(mixed, Classifier),
     linear_svm_classifier::predict(Classifier, [age-45, income-75000, student-no, credit_rating-fair], Class).

Exporting the classifier

| ?- linear_svm_classifier::learn(weather, Classifier),
     linear_svm_classifier::export_to_clauses(weather, Classifier, classify, Clauses).

| ?- linear_svm_classifier::learn(weather, Classifier),
     linear_svm_classifier::export_to_file(weather, Classifier, classify, 'classifier.pl').

Classifier representation

The learned classifier is represented as a compound term with the form:

linear_svm_classifier(Classes, Encoders, Models, Options)

Where:

  • Classes: list of class labels

  • Encoders: list of continuous scaling descriptors and categorical value lists

  • Models: list of class_model(Class, Bias, Weights) terms

  • Options: merged training options used to learn the model

When exported using export_to_clauses/4 or export_to_file/4, this classifier term is serialized directly as the single argument of the generated predicate clause so that the exported model can be loaded and reused as-is.

References

  1. Cortes, C. and Vapnik, V. (1995). “Support-Vector Networks”.

  2. Bishop, C.M. (2006). “Pattern Recognition and Machine Learning”. Section 7.1.

  3. Hastie, T., Tibshirani, R. and Friedman, J. (2009). “The Elements of Statistical Learning”. Section 12.3.