Inspect model weights

By default, VW uses a dense representation for model weights. These weights are accessible from Python in a couple of ways.

Using numpy

The model weights are exposed (vowpal_wabbit_next.Workspace.weights()) in such a way that Numpy can provide a view into the memory directly. This means there are no memory copies required when inspecting the weights.

Attention

Only dense weights are supported for this method.

There are 3 dimensions to the VW weight array:

• The feature index

• The model number (VW supports interleaved models), for most configurations there is only 1 though

• The weight + extra parameters used for training

  • The 0th item in this dimension is the weight itself and the other values should be seen as an implementation detail.

If we load up a default model a print the shape of the weights it will confirm this:

import vowpal_wabbit_next as vw

workspace = vw.Workspace(["--noconstant"])

weights = workspace.weights()

print(weights.shape)

(262144, 1, 4)

The number of possible feature indices is equal to the value of --num_bits/-b, which is 18 by default. Hence, 2^18 is 262144. There is only 1 model and there are 4 values for each weight. If we pass a few examples to this learner we can inspect the weights learned for each feature.

text_parser = vw.TextFormatParser(workspace)

workspace.learn_one(text_parser.parse_line("1 | a b c"))
workspace.learn_one(text_parser.parse_line("1 | b c d"))
workspace.learn_one(text_parser.parse_line("0.5 | a c"))

weights[workspace.get_index_for_scalar_feature("a")]

array([[0.19704719, 4.0011263 , 1.        , 0.49992964]], dtype=float32)

This shows that a weight of 0.13712466 has been learned for this feature so far. When the model is simple like this we can actually use it to calculate the prediction. Keep in mind we disabled the constant feature earlier, this was to make this calculation easier.

print(
    "using predict_one = {}",
    workspace.predict_one(text_parser.parse_line("| a:0.7 c:0.6")),
)

manual_prediction = (
    0.7 * weights[workspace.get_index_for_scalar_feature("a")][0][0]
    + 0.6 * weights[workspace.get_index_for_scalar_feature("c")][0][0]
)
print("manual prediction = {}", manual_prediction)

using predict_one = {} 0.313994824886322
manual prediction = {} 0.31399484127759936

Using JSON weights

There is an API (vowpal_wabbit_next.Workspace.json_weights()) which dumps the contents of the model to JSON. This is also useful for debugging, especially since it allows for the feature names to be embedded in the JSON too. Note, this format is still experimental and may change in future.

To ensure that feature names are recorded there are a few rather verbose options that need to be passed.

import vowpal_wabbit_next as vw
import json

workspace = vw.Workspace(record_feature_names=True)

text_parser = vw.TextFormatParser(workspace)

workspace.learn_one(text_parser.parse_line("1 | a b c"))
workspace.learn_one(text_parser.parse_line("1 | b c d"))
workspace.learn_one(text_parser.parse_line("0.5 | a c"))

print(
    json.dumps(json.loads(workspace.json_weights(include_feature_names=True)), indent=4)
)

{
    "weights": [
        {
            "terms": [
                {
                    "name": "d",
                    "namespace": " ",
                    "string_value": null
                }
            ],
            "offset": 0,
            "index": 70771,
            "value": 0.14921310544013977
        },
        {
            "terms": [
                {
                    "name": "a",
                    "namespace": " ",
                    "string_value": null
                }
            ],
            "offset": 0,
            "index": 92594,
            "value": 0.1371246576309204
        },
        {
            "terms": [
                {
                    "name": "Constant",
                    "namespace": "",
                    "string_value": null
                }
            ],
            "offset": 0,
            "index": 116060,
            "value": 0.2089555412530899
        },
        {
            "terms": [
                {
                    "name": "b",
                    "namespace": " ",
                    "string_value": null
                }
            ],
            "offset": 0,
            "index": 163331,
            "value": 0.2274836003780365
        },
        {
            "terms": [
                {
                    "name": "c",
                    "namespace": " ",
                    "string_value": null
                }
            ],
            "offset": 0,
            "index": 185951,
            "value": 0.2089555412530899
        }
    ]
}

In the above we can find “a” and see that its value is the same.