Metadata Routing
scikit-learn Docs · نسخهٔ 5e9184e53cd3 · BSD-3-Clause
Metadata Routing
The Metadata Routing API is experimental, and is not yet implemented for all estimators. Please refer to the :ref:list of supported and unsupported models <metadata_routing_models> for more information. It may change without the usual deprecation cycle. By default this feature is not enabled. You can enable it by setting the `enable_metadata_routing flag to True`::
توجه: >> import sklearn >> sklearn.set_config(enable_metadata_routing=True)
Note that the methods and requirements introduced in this document are only relevant if you want to pass metadata (e.g. `sample_weight) to a method. If you're only passing X and y and no other parameter / metadata to methods such as fit, transform`, etc., then you don't need to set anything.
This guide demonstrates how metadata can be routed and passed between objects in scikit-learn. If you are developing a scikit-learn compatible estimator or meta-estimator, check our related developer guide: sphx_glr_auto_examples_miscellaneous_plot_metadata_routing.py. For a narrative introduction to metadata routing, see this blog post on Scikit-learn's Metadata Routing API <https://blog.probabl.ai/update-on-scikit-learn-metadata-routing-api>__.
Metadata is data that an estimator, scorer, or CV splitter takes into account if the user explicitly passes it as a parameter. For instance, ~cluster.KMeans accepts sample_weight in its fit() method and considers it to calculate its centroids. classes are consumed by some classifiers and groups are used in some splitters, but any data that is passed into an object's methods apart from X and y can be considered as metadata. The most used metadata in scikit-learn is sample_weight. If you are new to it and want to understand when `sample_weight is useful, see this tutorial on Improving models via subsets <https://www.youtube.com/watch?v=REIg5NH2SNc>`__.
Prior to scikit-learn version 1.3, there was no single API for passing metadata even if these were used in conjunction with other objects, e.g. a scorer accepting sample_weight inside a ~model_selection.GridSearchCV.
With the Metadata Routing API, we can transfer metadata to estimators, scorers, and CV splitters using meta-estimators (such as ~pipeline.Pipeline or ~model_selection.GridSearchCV) or functions such as ~model_selection.cross_validate which route data to other objects. In order to pass metadata to a method like `fit or score, the object consuming the metadata, must *request* it. This is done via set_{method}_request() methods, where {method} is substituted by the name of the method that requests the metadata. For instance, estimators that use the metadata in their fit() method would use set_fit_request(), and scorers would use set_score_request(). These methods allow us to specify which metadata to request, for instance set_fit_request(sample_weight=True). For grouped splitters such as ~model_selection.GroupKFold, a groups` parameter is requested by default.
Usage Examples ****** For convenience and a faster start into metadata routing, see auto-requested metadata <metadata_routing_auto_request_user>. For this section, we disable auto-requests (which are enabled by default)::
توجه: >> sklearn.set_config(enable_metadata_auto_requests=False)
The following examples use explicit `set_{method}_request calls and are designed to show some common use-cases. Our goal here is to pass sample_weight and groups through ~model_selection.cross_validate, which routes the metadata to ~linear_model.LogisticRegressionCV and to a custom scorer made with ~metrics.make_scorer, both of which *can* use the metadata in their methods. In these examples we want to individually set whether to use the metadata within the different consumers <consumer>`.
The examples in this section require the following imports and data::
توجه: >> import numpy as np >> from sklearn.metrics import make_scorer, accuracy_score >> from sklearn.linear_model import LogisticRegressionCV, LogisticRegression >> from sklearn.model_selection import cross_validate, GridSearchCV, GroupKFold >> from sklearn.feature_selection import SelectKBest >> from sklearn.pipeline import make_pipeline >> n_samples, n_features = 100, 4 >> rng = np.random.RandomState(42) >> X = rng.rand(n_samples, n_features) >> y = rng.randint(0, 2, size=n_samples) >> my_groups = rng.randint(0, 10, size=n_samples) >> my_weights = rng.rand(n_samples) >> my_other_weights = rng.rand(n_samples)
Weighted scoring and fitting
The splitter used internally in ~linear_model.LogisticRegressionCV, ~model_selection.GroupKFold, requests `groups by default. However, we need to explicitly request sample_weight for it and for our custom scorer by specifying sample_weight=True in ~linear_model.LogisticRegressionCV's set_fit_request() method and in ~metrics.make_scorer's set_score_request() method. Both consumers <consumer> know how to use sample_weight in their fit() or score() methods. We can then pass the metadata in ~model_selection.cross_validate` which will route it to any active consumers::
توجه: >> weighted_acc = make_scorer(accuracy_score).set_score_request(sample_weight=True) >> lr = LogisticRegressionCV(
... alphas=10, ... cv=GroupKFold(), ... scoring=weighted_acc, ... ).set_fit_request(sample_weight=True)
توجه: >> cv_results = cross_validate(
... lr, ... X, ... y, ... params={"sample_weight": my_weights, "groups": my_groups}, ... cv=GroupKFold(), ... scoring=weighted_acc, ... )
Note that in this example, ~model_selection.cross_validate routes `my_weights to both the scorer and ~linear_model.LogisticRegressionCV`.
If we would pass sample_weight in the params of ~model_selection.cross_validate, but not set any object to request it, UnsetMetadataPassedError would be raised, hinting to us that we need to explicitly set where to route it. The same applies if `params={"sample_weights": my_weights, ...} were passed (note the typo, i.e. weights instead of weight), since sample_weights` was not requested by any of its underlying objects.
Weighted scoring and unweighted fitting
When passing metadata such as `sample_weight into a router (meta-estimators or routing function), all sample_weight :term:consumers <consumer> require weights to be either explicitly requested or explicitly not requested (i.e. True or False). Thus, to perform an unweighted fit, we need to configure ~linear_model.LogisticRegressionCV to not request sample weights, so that ~model_selection.cross_validate` does not pass the weights along::
توجه: >> weighted_acc = make_scorer(accuracy_score).set_score_request(sample_weight=True) >> lr = LogisticRegressionCV(
... alphas=10, cv=GroupKFold(), scoring=weighted_acc ... ).set_fit_request(sample_weight=False)
توجه: >> cv_results = cross_validate(
... lr, ... X, ... y, ... cv=GroupKFold(), ... params={"sample_weight": my_weights, "groups": my_groups}, ... scoring=weighted_acc, ... )
If linear_model.LogisticRegressionCV.set_fit_request had not been called, ~model_selection.cross_validate would raise an error because `sample_weight is passed but ~linear_model.LogisticRegressionCV` would not be explicitly configured to recognize the weights.
Unweighted feature selection
Routing metadata is only possible if the object's method knows how to use the metadata, which in most cases means they have it as an explicit parameter. Only then we can set request values for metadata using set_fit_request(sample_weight=True), for instance. This makes the object a consumer <consumer>.
Unlike ~linear_model.LogisticRegressionCV, ~feature_selection.SelectKBest can't consume weights and therefore no request value for `sample_weight on its instance is set and sample_weight` is not routed to it::
توجه: >> weighted_acc = make_scorer(accuracy_score).set_score_request(sample_weight=True) >> lr = LogisticRegressionCV(
... alphas=10, cv=GroupKFold(), scoring=weighted_acc ... ).set_fit_request(sample_weight=True)
توجه: >> sel = SelectKBest(k=2) >> pipe = make_pipeline(sel, lr) >> cv_results = cross_validate(
... pipe, ... X, ... y, ... cv=GroupKFold(), ... params={"sample_weight": my_weights, "groups": my_groups}, ... scoring=weighted_acc, ... )
Different scoring and fitting weights
Despite ~metrics.make_scorer and ~linear_model.LogisticRegressionCV both expecting the key `sample_weight, we can use aliases to pass different weights to different consumers. In this example, we pass scoring_weight to the scorer, and fitting_weight to ~linear_model.LogisticRegressionCV`::
توجه: >> weighted_acc = make_scorer(accuracy_score).set_score_request(
... sample_weight="scoring_weight" ... )
توجه: >> lr = LogisticRegressionCV(
... alphas=10, cv=GroupKFold(), scoring=weighted_acc ... ).set_fit_request(sample_weight="fitting_weight")
توجه: >> cv_results = cross_validate(
... lr, ... X, ... y, ... cv=GroupKFold(), ... params={ ... "scoring_weight": my_weights, ... "fitting_weight": my_other_weights, ... "groups": my_groups, ... }, ... scoring=weighted_acc, ... )
API Interface *****
A consumer is an object (estimator, meta-estimator, scorer, splitter) which accepts and uses some metadata in at least one of its methods (for instance `fit, predict, inverse_transform, transform, score, split). Meta-estimators which only forward the metadata to other objects (child estimators, scorers, or splitters) and don't use the metadata themselves are not consumers. (Meta-)Estimators which route metadata to other objects are routers <router>. A(n) (meta-)estimator can be a consumer and a router at the same time. (Meta-)Estimators and splitters expose a set_{method}_request method for each method which accepts at least one metadata. For instance, if an estimator supports sample_weight in fit and score, it exposes estimator.set_fit_request(sample_weight=value) and estimator.set_score_request(sample_weight=value). Here value` can be:
- `
True: method requests asample_weight`. This means if the metadata is provided,
it will be used, otherwise no error is raised.
- `
False: method does not request asample_weight`. - `
None: router will raise an error ifsample_weight` is passed. This is in almost
all cases the default value when an object is instantiated and ensures the user sets the metadata requests explicitly when a metadata is passed. The only exception are `Group*Fold` splitters.
- `
"param_name": alias forsample_weight` if we want to pass different weights to
different consumers. If aliasing is used the meta-estimator should not forward `"param_name" to the consumer, but sample_weight instead, because the consumer will expect a param called sample_weight. This means the mapping between the metadata required by the object, e.g. sample_weight and the variable name provided by the user, e.g. my_weights` is done at the router level, and not by the consuming object itself.
Metadata are requested in the same way for scorers using `set_score_request`.
If a metadata, e.g. `sample_weight, is passed by the user, the metadata request for all objects which potentially can consume sample_weight should be set by the user, otherwise an error is raised by the router object. For example, the following code raises an error, since it hasn't been explicitly specified whether sample_weight` should be passed to the estimator's scorer or not::
توجه: >> param_grid = {"alpha": [10, 1]} >> lr = LogisticRegression().set_fit_request(sample_weight=True) >> try:
... GridSearchCV( ... estimator=lr, param_grid=param_grid ... ).fit(X, y, sample_weight=my_weights) ... except ValueError as e: ... print(e) [sample_weight] are passed but are not explicitly set as requested or not requested for LogisticRegression.score, which is used within GridSearchCV.fit. Call LogisticRegression.set_score_request({metadata}=True/False) for each metadata you want to request/ignore. See the Metadata Routing User guide <https://scikit-learn.org/stable/metadata_routing.html> for more information.
The issue can be fixed by explicitly setting the request value::
توجه: >> lr = LogisticRegression().set_fit_request(
... sample_weight=True ... ).set_score_request(sample_weight=False)
We reset `enable_metadata_auto_requests=True` (its default value) for the next section::
توجه: >> sklearn.set_config(enable_metadata_auto_requests=True)
Auto-requested metadata *******
Consumers <consumer> auto-request selected metadata by default when metadata routing is enabled.
In scikit-learn, sample_weight is auto-requested on estimator methods that accept it (typically fit and score), on meta-estimators that also consume it themselves, and on scorers whose metric accepts sample_weight (for example those from ~metrics.make_scorer / ~metrics.get_scorer). Validation-set metadata (X_val, y_val and sample_weight_val) is auto-requested where those arguments are consumed (in ~ensemble.HistGradientBoostingClassifier and ~ensemble.HistGradientBoostingRegressor). (groups in Group*Fold is not an auto-request: it is requested by default whether auto-requests are on or off, because those splitters raise if groups is missing.)
Note that auto-requests are subject to change and stability guarantees applied to the rest of the scikit-learn API do not apply here. They may grow or change over time.
For most use cases it is enough to enable metadata routing and leave request values unchanged. Specifically, you can pass your `sample_weight through a router without calling set_fit_request or set_score_request. Use set_{method}_request` only when you need to opt out, set an alias, or handle a consumer that does not yet auto-request the metadata you pass.
You can still override any auto-request (or default request) with `set_{method}_request. Values set that way take precedence. In particular, set_fit_request(sample_weight=False) opts out of receiving sample_weight` even when auto-requests are enabled.
If you pass metadata that no consumer requests (including under auto-requests), the router still raises an error.
For an end-to-end example on routing a validation set (X_val, y_val), see sphx_glr_auto_examples_compose_plot_metadata_routing_validation_set.py.
Developers who implement auto-requests should follow metadata_routing_auto_request.
Finally, we disable the configuration flag for metadata routing::
توجه: >> sklearn.set_config(enable_metadata_routing=False)
Metadata Routing Support Status ******* All consumers (i.e. simple estimators which only consume metadata and don't route them) support metadata routing, meaning they can be used inside meta-estimators which support metadata routing. However, development of support for metadata routing for meta-estimators is in progress, and here is a list of meta-estimators and tools which support and don't yet support metadata routing.
Meta-estimators and functions supporting metadata routing:
sklearn.calibration.CalibratedClassifierCVsklearn.compose.ColumnTransformersklearn.compose.TransformedTargetRegressorsklearn.covariance.GraphicalLassoCVsklearn.ensemble.StackingClassifiersklearn.ensemble.StackingRegressorsklearn.ensemble.VotingClassifiersklearn.ensemble.VotingRegressorsklearn.ensemble.BaggingClassifiersklearn.ensemble.BaggingRegressorsklearn.feature_selection.RFEsklearn.feature_selection.RFECVsklearn.feature_selection.SelectFromModelsklearn.feature_selection.SequentialFeatureSelectorsklearn.impute.IterativeImputersklearn.linear_model.ElasticNetCVsklearn.linear_model.LarsCVsklearn.linear_model.LassoCVsklearn.linear_model.LassoLarsCVsklearn.linear_model.LogisticRegressionCVsklearn.linear_model.MultiTaskElasticNetCVsklearn.linear_model.MultiTaskLassoCVsklearn.linear_model.OrthogonalMatchingPursuitCVsklearn.linear_model.RANSACRegressorsklearn.linear_model.RidgeClassifierCVsklearn.linear_model.RidgeCVsklearn.model_selection.GridSearchCVsklearn.model_selection.HalvingGridSearchCVsklearn.model_selection.HalvingRandomSearchCVsklearn.model_selection.RandomizedSearchCVsklearn.model_selection.permutation_test_scoresklearn.model_selection.cross_validatesklearn.model_selection.cross_val_scoresklearn.model_selection.cross_val_predictsklearn.model_selection.learning_curvesklearn.model_selection.validation_curvesklearn.multiclass.OneVsOneClassifiersklearn.multiclass.OneVsRestClassifiersklearn.multiclass.OutputCodeClassifiersklearn.multioutput.ClassifierChainsklearn.multioutput.MultiOutputClassifiersklearn.multioutput.MultiOutputRegressorsklearn.multioutput.RegressorChainsklearn.preprocessing.TargetEncodersklearn.pipeline.FeatureUnionsklearn.pipeline.Pipelinesklearn.semi_supervised.SelfTrainingClassifier
Meta-estimators and tools not supporting metadata routing yet:
sklearn.ensemble.AdaBoostClassifiersklearn.ensemble.AdaBoostRegressor