diff --git a/docs/index.rst b/docs/index.rst index 50b5ad81..735065c5 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -86,7 +86,7 @@ Import your tree ensemble model into Treelite: .. code-block:: python import treelite - model = treelite.Model.load("my_model.model", model_format="xgboost") + model = treelite.frontend.load_xgboost_model("my_model.json") Compute predictions using :doc:`treelite-gtil-api`: diff --git a/docs/knobs/index.rst b/docs/knobs/index.rst new file mode 100644 index 00000000..346812e8 --- /dev/null +++ b/docs/knobs/index.rst @@ -0,0 +1,8 @@ +==================== +Knobs and Parameters +==================== + +.. toctree:: + :maxdepth: 1 + + postprocessor diff --git a/docs/knobs/postprocessor.rst b/docs/knobs/postprocessor.rst new file mode 100644 index 00000000..1a0214c2 --- /dev/null +++ b/docs/knobs/postprocessor.rst @@ -0,0 +1,28 @@ +List of postprocessor functions +=============================== +When predicting with tree ensemble models, we sum the margin scores from individual trees and apply a postprocessor +function to transform the sum into a final prediction. This function is also known as the link function. + +Currently, Treelite supports the following postprocessor functions. + +Element-wise postprocessor functions +------------------------------------ +* ``identity``: The identity function. Do not apply any transformation to the margin score vector. +* ``signed_square``: Apply the function ``f(x) = sign(x) * (x**2)`` element-wise to the margin score vector. +* ``hinge``: Apply the function ``f(x) = (1 if x > 0 else 0)`` element-wise to the margin score vector. +* ``sigmoid``: Apply the sigmoid function ``f(x) = 1/(1+exp(-sigmoid_alpha * x))`` element-wise to the margin score + vector, to transform margin scores into probability scores in the range ``[0, 1]``. The ``sigmoid_alpha`` parameter + can be configured by the user. +* ``exponential``: Apply the exponential function (``exp``) element-wise to the margin score vector. +* ``exponential_standard_ratio``: Apply the function ``f(x) = exp2(-x / ratio_c)`` element-wise to the margin score + vector. The ``ratio_c`` parameter can be configured by the user. +* ``logarithm_one_plus_exp``: Apply the function ``f(x) = log(1 + exp(x))`` element-wise to the margin score vector. + +Row-wise postprocessor functions +-------------------------------- +* ``identity_multiclass``: The identity function. Do not apply any transformation to the margin score vector. +* ``softmax``: Use the softmax function ``f(x) = exp(x) / sum(exp(x))`` to the margin score vector, to transform the + margin scores into probability scores in the range ``[0, 1]``. Adding up the transformed scores for all classes + will yield 1. +* ``multiclass_ova``: Apply the sigmoid function ``f(x) = 1/(1+exp(-sigmoid_alpha * x))`` element-wise to the margin + scores. The ``sigmoid_alpha`` parameter can be configured by the user. diff --git a/docs/serialization/v4.rst b/docs/serialization/v4.rst index 8c03c226..ff69894c 100644 --- a/docs/serialization/v4.rst +++ b/docs/serialization/v4.rst @@ -90,6 +90,7 @@ written to the byte sequence in the exact order they appear in the following lis * Model parameters - ``postprocessor``: an array of ``char``. Stores a human-readable name of the post-processing function that's applied to prediction outputs. + Consult :doc:`/knobs/postprocessor` for the list of available postprocessors. - ``sigmoid_alpha``: single ``float`` scalar. This model parameter is relevant when ``postprocessor="sigmoid"``. - ``ratio_c``: single ``float`` scalar. This model parameter is relevant when ``postprocessor="exponential_standard_ratio"``. - ``base_scores``: an array of ``double``. This vector is expected to have length ``num_target * max(num_class)``. The elements will be laid out in the row-major layout. diff --git a/docs/tutorials/builder.rst b/docs/tutorials/builder.rst index 7ec6f85f..af47d625 100644 --- a/docs/tutorials/builder.rst +++ b/docs/tutorials/builder.rst @@ -416,7 +416,9 @@ indicates a strong vote for the positive class.) To obtain probability scores, we pass the sum of the tree outputs through a **link function** ``sigmoid(x) = 1/(1+exp(-x))``. In the model builder API, -the link function is specified by the ``postprocessor`` argument. Let's look at how the builder object is constructed: +the link function is specified by the ``postprocessor`` argument. (Consult :doc:`/knobs/postprocessor` for the list of +available postprocessors.) +Let's look at how the builder object is constructed: .. code-block:: python diff --git a/docs/tutorials/edit.rst b/docs/tutorials/edit.rst index 467f7198..3b3e4285 100644 --- a/docs/tutorials/edit.rst +++ b/docs/tutorials/edit.rst @@ -1,5 +1,5 @@ Editing tree models (Advanced) ------------------------------- +============================== :ref:`The field accessor API ` allows users to inspect and edit tree model objects after they have been constructed. Here are some examples: @@ -31,7 +31,7 @@ constructed. Here are some examples: Consult :doc:`/serialization/v4` for the list of fields. How to use setter methods -========================= +------------------------- There are lots of gotchas and pitfalls involved when using :py:meth:`~treelite.model.TreeAccessor.set_field` to modify trees. We must start by the following notice: diff --git a/python/treelite/model_builder.py b/python/treelite/model_builder.py index 4575aa2c..a97590e7 100644 --- a/python/treelite/model_builder.py +++ b/python/treelite/model_builder.py @@ -85,7 +85,8 @@ class PostProcessorFunc: Parameters ---------- name: - Name of the postprocessor + Name of the postprocessor. Consult :py:doc:`/knobs/postprocessor` for the list + of available postprocessor functions. sigmoid_alpha: Scaling parameter for sigmoid function ``sigmoid(x) = 1 / (1 + exp(-alpha * x))``. This parameter is applicable only when ``name="sigmoid"`` or ``name="multiclass_ova"``.