Version 0.16
scikit-learn Docs · نسخهٔ 5e9184e53cd3 · BSD-3-Clause
============
Version 0.16
Version 0.16.1
April 14, 2015
Changelog
Bug fixes .........
- Allow input data larger than `
block_size` in
covariance.LedoitWolf by Andreas Müller_.
- Fix a bug in
isotonic.IsotonicRegressiondeduplication that
caused unstable result in calibration.CalibratedClassifierCV by Jan Hendrik Metzen_.
- Fix sorting of labels in
preprocessing.label_binarizeby Michael Heilman.
- Fix several stability and convergence issues in
cross_decomposition.CCA and cross_decomposition.PLSCanonical by Andreas Müller_
- Fix a bug in
cluster.KMeanswhen `precompute_distances=False`
on fortran-ordered data.
- Fix a speed regression in
ensemble.RandomForestClassifier's `predict`
and `predict_proba by Andreas Müller`_.
- Fix a regression where `
utils.shuffleconverted lists and dataframes to arrays, byOlivier Grisel`_
Version 0.16
March 26, 2015
Highlights
- Speed improvements (notably in
cluster.DBSCAN), reduced memory
requirements, bug-fixes and better default settings.
- Multinomial Logistic regression and a path algorithm in
linear_model.LogisticRegressionCV.
- Out-of core learning of PCA via
decomposition.IncrementalPCA.
- Probability calibration of classifiers using
calibration.CalibratedClassifierCV.
cluster.Birchclustering method for large-scale datasets.
- Scalable approximate nearest neighbors search with Locality-sensitive
hashing forests in neighbors.LSHForest.
- Improved error messages and better validation when using malformed input data.
- More robust integration with pandas dataframes.
Changelog
New features ............
- The new
neighbors.LSHForestimplements locality-sensitive hashing
for approximate nearest neighbors search. By Maheshakya Wijewardena<maheshakya>.
- Added
svm.LinearSVR. This class uses the liblinear implementation
of Support Vector Regression which is much faster for large sample sizes than svm.SVR with linear kernel. By Fabian Pedregosa_ and Qiang Luo.
- Incremental fit for
GaussianNB <naive_bayes.GaussianNB>.
- Added `
sample_weightsupport todummy.DummyClassifier` and
dummy.DummyRegressor. By Arnaud Joly_.
- Added the
metrics.label_ranking_average_precision_scoremetrics.
By Arnaud Joly_.
- Add the
metrics.coverage_errormetrics. ByArnaud Joly_.
- Added
linear_model.LogisticRegressionCV. By
Manoj Kumar_, Fabian Pedregosa_, Gael Varoquaux_ and Alexandre Gramfort_.
- Added `
warm_start` constructor parameter to make it possible for any
trained forest model to grow additional trees incrementally. By Laurent Direr<ldirer>.
- Added `
sample_weightsupport toensemble.GradientBoostingClassifier` and
ensemble.GradientBoostingRegressor. By Peter Prettenhofer_.
- Added
decomposition.IncrementalPCA, an implementation of the PCA
algorithm that supports out-of-core learning with a `partial_fit method. By Kyle Kastner`_.
- Averaged SGD for
SGDClassifier <linear_model.SGDClassifier>
and SGDRegressor <linear_model.SGDRegressor> By Danny Sullivan <dsullivan7>.
- Added
cross_val_predict
function which computes cross-validated estimates. By Luis Pedro Coelho_
- Added
linear_model.TheilSenRegressor, a robust
generalized-median-based estimator. By Florian Wilhelm <FlorianWilhelm>.
- Added
metrics.median_absolute_error, a robust metric.
By Gael Varoquaux_ and Florian Wilhelm <FlorianWilhelm>.
- Add
cluster.Birch, an online clustering algorithm. By
Manoj Kumar_, Alexandre Gramfort_ and Joel Nothman_.
- Added shrinkage support to
discriminant_analysis.LinearDiscriminantAnalysis
using two new solvers. By Clemens Brunner <cle1109> and Martin Billinger_.
- Added
kernel_ridge.KernelRidge, an implementation of
kernelized ridge regression. By Mathieu Blondel_ and Jan Hendrik Metzen_.
- All solvers in
linear_model.Ridgenow supportsample_weight.
By Mathieu Blondel_.
- Added
cross_validation.PredefinedSplitcross-validation
for fixed user-provided cross-validation folds. By Thomas Unterthiner <untom>.
- Added
calibration.CalibratedClassifierCV, an approach for
calibrating the predicted probabilities of a classifier. By Alexandre Gramfort_, Jan Hendrik Metzen_, Mathieu Blondel_ and Balazs Kegl <kegl>.
Enhancements ............
- Add option `
return_distanceinhierarchical.ward_tree`
to return distances between nodes for both structured and unstructured versions of the algorithm. By Matteo Visconti di Oleggio Castello_. The same option was added in hierarchical.linkage_tree. By Manoj Kumar_
- Add support for sample weights in scorer objects. Metrics with sample
weight support will automatically benefit from it. By Noel Dawe_ and Vlad Niculae_.
- Added `
newton-cgandlbfgs` solver support in
linear_model.LogisticRegression. By Manoj Kumar_.
- Add `
selection="random"` parameter to implement stochastic coordinate
descent for linear_model.Lasso, linear_model.ElasticNet and related. By Manoj Kumar_.
- Add `
sample_weight` parameter to
metrics.jaccard_similarity_score and metrics.log_loss. By Jatin Shah <jatinshah>.
- Support sparse multilabel indicator representation in
preprocessing.LabelBinarizer and multiclass.OneVsRestClassifier (by Hamzeh Alsalhi <hamsal> with thanks to Rohit Sivaprasad), as well as evaluation metrics (by Joel Nothman_).
- Add `
sample_weightparameter tometrics.jaccard_similarity_score`.
By Jatin Shah.
- Add support for multiclass in
metrics.hinge_loss. Added `labels=None`
as optional parameter. By Saurabh Jha.
- Add `
sample_weightparameter tometrics.hinge_loss`.
By Saurabh Jha.
- Add `
multi_class="multinomial"` option in
linear_model.LogisticRegression to implement a Logistic Regression solver that minimizes the cross-entropy or multinomial loss instead of the default One-vs-Rest setting. Supports lbfgs and newton-cg solvers. By Lars Buitinck_ and Manoj Kumar_. Solver option newton-cg by Simon Wu.
- `
DictVectorizercan now performfit_transform` on an iterable in a
single pass, when giving the option `sort=False. By :user:Dan Blanchard <dan-blanchard>`.
model_selection.GridSearchCVand
model_selection.RandomizedSearchCV can now be configured to work with estimators that may fail and raise errors on individual folds. This option is controlled by the error_score parameter. This does not affect errors raised on re-fit. By Michal Romaniuk <romaniukm>.
- Add `
digitsparameter tometrics.classification_report` to allow
report to show different precision of floating point numbers. By Ian Gilmore <agileminor>.
- Add a quantile prediction strategy to the
dummy.DummyRegressor.
By Aaron Staple <staple>.
- Add `
handle_unknownoption topreprocessing.OneHotEncoder` to
handle unknown categorical features more gracefully during transform. By Manoj Kumar_.
- Added support for sparse input data to decision trees and their ensembles.
By Fares Hedyati_ and Arnaud Joly_.
- Optimized
cluster.AffinityPropagationby reducing the number of
memory allocations of large temporary data-structures. By Antony Lee_.
- Parallelization of the computation of feature importances in random forest.
By Olivier Grisel_ and Arnaud Joly_.
- Add `
n_iter_attribute to estimators that accept amax_iter` attribute
in their constructor. By Manoj Kumar_.
- Added decision function for
multiclass.OneVsOneClassifier
By Raghav RV_ and Kyle Beauchamp <kyleabeauchamp>.
neighbors.kneighbors_graphandradius_neighbors_graph
support non-Euclidean metrics. By Manoj Kumar_
- Parameter `
connectivityincluster.AgglomerativeClustering`
and family now accept callables that return a connectivity matrix. By Manoj Kumar_.
- Sparse support for
metrics.pairwise.paired_distances. ByJoel Nothman_.
cluster.DBSCANnow supports sparse input and sample weights and
has been optimized: the inner loop has been rewritten in Cython and radius neighbors queries are now computed in batch. By Joel Nothman_ and Lars Buitinck_.
- Add `
class_weight` parameter to automatically weight samples by class
frequency for ensemble.RandomForestClassifier, tree.DecisionTreeClassifier, ensemble.ExtraTreesClassifier and tree.ExtraTreeClassifier. By Trevor Stephens_.
grid_search.RandomizedSearchCVnow does sampling without
replacement if all parameters are given as lists. By Andreas Müller_.
- Parallelized calculation of
metrics.pairwise_distancesis now supported
for scipy metrics and custom callables. By Joel Nothman_.
- Allow the fitting and scoring of all clustering algorithms in
pipeline.Pipeline. By Andreas Müller_.
- More robust seeding and improved error messages in
cluster.MeanShift
by Andreas Müller_.
- Make the stopping criterion for
mixture.GMM,
mixture.DPGMM and mixture.VBGMM less dependent on the number of samples by thresholding the average log-likelihood change instead of its sum over all samples. By Hervé Bredin_.
- The outcome of
manifold.spectral_embeddingwas made deterministic
by flipping the sign of eigenvectors. By Hasil Sharma <Hasil-Sharma>.
- Significant performance and memory usage improvements in
preprocessing.PolynomialFeatures. By Eric Martin_.
- Numerical stability improvements for
preprocessing.StandardScaler
and preprocessing.scale. By Nicolas Goix_
svm.SVCfitted on sparse input now implements `decision_function`.
By Rob Zinkov_ and Andreas Müller_.
cross_validation.train_test_splitnow preserves the input type,
instead of converting to numpy arrays.
Documentation improvements ..........................
- Added example of using
pipeline.FeatureUnionfor heterogeneous input.
By Matt Terry <mrterry>
- Documentation on scorers was improved, to highlight the handling of loss
functions. By Matt Pico <MattpSoftware>.
- A discrepancy between liblinear output and scikit-learn's wrappers
is now noted. By Manoj Kumar_.
- Improved documentation generation: examples referring to a class or
function are now shown in a gallery on the class/function's API reference page. By Joel Nothman_.
- More explicit documentation of sample generators and of data
transformation. By Joel Nothman_.
sklearn.neighbors.BallTreeandsklearn.neighbors.KDTree
used to point to empty pages stating that they are aliases of BinaryTree. This has been fixed to show the correct class docs. By Manoj Kumar_.
- Added silhouette plots for analysis of KMeans clustering using
metrics.silhouette_samples and metrics.silhouette_score. See sphx_glr_auto_examples_cluster_plot_kmeans_silhouette_analysis.py
Bug fixes .........
- Metaestimators now support ducktyping for the presence of `
decision_function`,
`predict_proba and other methods. This fixes behavior of grid_search.GridSearchCV, grid_search.RandomizedSearchCV, pipeline.Pipeline, feature_selection.RFE, feature_selection.RFECV when nested. By Joel Nothman`_
- The `
scoring` attribute of grid-search and cross-validation methods is no longer
ignored when a grid_search.GridSearchCV is given as a base estimator or the base estimator doesn't have predict.
- The function
hierarchical.ward_treenow returns the children in
the same order for both the structured and unstructured versions. By Matteo Visconti di Oleggio Castello_.
feature_selection.RFECVnow correctly handles cases when
`step is not equal to 1. By Nikolay Mayorov <nmayorov>`
- The
decomposition.PCAnow undoes whitening in its
`inverse_transform. Also, its components_ now always have unit length. By Michael Eickenberg <eickenberg>`.
- Fix incomplete download of the dataset when
datasets.download_20newsgroups is called. By Manoj Kumar_.
- Various fixes to the Gaussian processes subpackage by Vincent Dubourg
and Jan Hendrik Metzen.
- Calling `
partial_fitwithclass_weight=='auto'` throws an
appropriate error message and suggests a workaround. By Danny Sullivan <dsullivan7>.
RBFSampler <kernel_approximation.RBFSampler>with `gamma=g`
formerly approximated rbf_kernel <metrics.pairwise.rbf_kernel> with `gamma=g/2.; the definition of gamma is now consistent, which may substantially change your results if you use a fixed value. (If you cross-validated over gamma, it probably doesn't matter too much.) By Dougal Sutherland <dougalsutherland>`.
- Pipeline object delegates the `
classes_` attribute to the underlying
estimator. It allows, for instance, to make bagging of a pipeline object. By Arnaud Joly_
neighbors.NearestCentroidnow uses the median as the centroid
when metric is set to `manhattan. It was using the mean before. By Manoj Kumar`_
- Fix numerical stability issues in
linear_model.SGDClassifier
and linear_model.SGDRegressor by clipping large gradients and ensuring that weight decay rescaling is always positive (for large l2 regularization and large learning rate values). By Olivier Grisel_
- When
compute_full_treeis set to "auto", the full tree is
built when n_clusters is high and is early stopped when n_clusters is low, while the behavior should be vice versa in cluster.AgglomerativeClustering (and friends). This has been fixed By Manoj Kumar_
- Fix lazy centering of data in
linear_model.enet_pathand
linear_model.lasso_path. It was centered around one. It has been changed to be centered around the origin. By Manoj Kumar_
- Fix handling of precomputed affinity matrices in
cluster.AgglomerativeClustering when using connectivity constraints. By Cathy Deng <cathydeng>
- Correct `
partial_fithandling ofclass_prior` for
sklearn.naive_bayes.MultinomialNB and sklearn.naive_bayes.BernoulliNB. By Trevor Stephens_.
- Fixed a crash in
metrics.precision_recall_fscore_support
when using unsorted `labels in the multi-label setting. By Andreas Müller`_.
- Avoid skipping the first nearest neighbor in the methods `
radius_neighbors`,
`kneighbors, kneighbors_graph and radius_neighbors_graph in sklearn.neighbors.NearestNeighbors and family, when the query data is not the same as fit data. By Manoj Kumar`_.
- Fix log-density calculation in the
mixture.GMMwith
tied covariance. By Will Dawson_
- Fixed a scaling error in
feature_selection.SelectFdr
where a factor `n_features was missing. By Andrew Tulloch`_
- Fix zero division in
neighbors.KNeighborsRegressorand related
classes when using distance weighting and having identical data points. By Garret-R <https://github.com/Garrett-R>_.
- Fixed round off errors with non positive-definite covariance matrices
in GMM. By Alexis Mignon <AlexisMignon>.
- Fixed an error in the computation of conditional probabilities in
naive_bayes.BernoulliNB. By Hanna Wallach_.
- Make the method `
radius_neighbors` of
neighbors.NearestNeighbors return the samples lying on the boundary for `algorithm='brute'. By Yan Yi`_.
- Flip sign of `
dual_coef_ofsvm.SVC`
to make it consistent with the documentation and `decision_function`. By Artem Sobolev.
- Fixed handling of ties in
isotonic.IsotonicRegression.
We now use the weighted average of targets (secondary method). By Andreas Müller_ and Michael Bommarito <https://bommaritollc.com/>_.
API changes summary
GridSearchCVand
cross_val_score and other meta-estimators don't convert pandas DataFrames into arrays any more, allowing DataFrame specific operations in custom estimators.
multiclass.fit_ovr,multiclass.predict_ovr,
predict_proba_ovr, multiclass.fit_ovo, multiclass.predict_ovo, multiclass.fit_ecoc and multiclass.predict_ecoc are deprecated. Use the underlying estimators instead.
- Nearest neighbors estimators used to take arbitrary keyword arguments
and pass these to their distance metric. This will no longer be supported in scikit-learn 0.18; use the `metric_params` argument instead.
n_jobsparameter of the fit method shifted to the constructor of the
LinearRegression class.
- The `
predict_probamethod ofmulticlass.OneVsRestClassifier`
now returns two probabilities per sample in the multiclass case; this is consistent with other estimators and with the method's documentation, but previous versions accidentally returned only the positive probability. Fixed by Will Lamond and Lars Buitinck_.
- Change default value of precompute in
linear_model.ElasticNetand
linear_model.Lasso to False. Setting precompute to "auto" was found to be slower when n_samples > n_features since the computation of the Gram matrix is computationally expensive and outweighs the benefit of fitting the Gram for just one alpha. `precompute="auto" is now deprecated and will be removed in 0.18 By Manoj Kumar`_.
- Expose `
positiveoption inlinear_model.enet_path` and
linear_model.enet_path which constrains coefficients to be positive. By Manoj Kumar_.
- Users should now supply an explicit `
average` parameter to
sklearn.metrics.f1_score, sklearn.metrics.fbeta_score, sklearn.metrics.recall_score and sklearn.metrics.precision_score when performing multiclass or multilabel (i.e. not binary) classification. By Joel Nothman_.
scoringparameter for cross validation now accepts'f1_micro',
'f1_macro' or 'f1_weighted'. 'f1' is now for binary classification only. Similar changes apply to 'precision' and 'recall'. By Joel Nothman_.
- The `
fit_intercept,normalizeandreturn_models` parameters in
linear_model.enet_path and linear_model.lasso_path have been removed. They were deprecated since 0.14
- From now onwards, all estimators will uniformly raise `
NotFittedError`
when any of the `predict like methods are called before the model is fit. By Raghav RV`_.
- Input data validation was refactored for more consistent input
validation. The `check_arrays function was replaced by check_array and check_X_y. By Andreas Müller`_.
- Allow `
X=Nonein the methodsradius_neighbors,kneighbors`,
`kneighbors_graph and radius_neighbors_graph in sklearn.neighbors.NearestNeighbors and family. If set to None, then for every sample this avoids setting the sample itself as the first nearest neighbor. By Manoj Kumar`_.
- Add parameter `
include_selfinneighbors.kneighbors_graph`
and neighbors.radius_neighbors_graph which has to be explicitly set by the user. If set to True, then the sample itself is considered as the first nearest neighbor.
threshparameter is deprecated in favor of newtolparameter in
GMM, DPGMM and VBGMM. See Enhancements section for details. By Hervé Bredin_.
- Estimators will treat input with dtype object as numeric when possible.
By Andreas Müller_
- Estimators now raise
ValueErrorconsistently when fitted on empty
data (less than 1 sample or less than 1 feature for 2D input). By Olivier Grisel_.
- The `
shuffleoption of.linear_model.SGDClassifier`,
linear_model.SGDRegressor, linear_model.Perceptron, linear_model.PassiveAggressiveClassifier and linear_model.PassiveAggressiveRegressor now defaults to `True`.
cluster.DBSCANnow uses a deterministic initialization. The
random_state parameter is deprecated. By Erich Schubert <kno10>.
Code Contributors
A. Flaxman, Aaron Schumacher, Aaron Staple, abhishek thakur, Akshay, akshayah3, Aldrian Obaja, Alexander Fabisch, Alexandre Gramfort, Alexis Mignon, Anders Aagaard, Andreas Mueller, Andreas van Cranenburgh, Andrew Tulloch, Andrew Walker, Antony Lee, Arnaud Joly, banilo, Barmaley.exe, Ben Davies, Benedikt Koehler, bhsu, Boris Feld, Borja Ayerdi, Boyuan Deng, Brent Pedersen, Brian Wignall, Brooke Osborn, Calvin Giles, Cathy Deng, Celeo, cgohlke, chebee7i, Christian Stade-Schuldt, Christof Angermueller, Chyi-Kwei Yau, CJ Carey, Clemens Brunner, Daiki Aminaka, Dan Blanchard, danfrankj, Danny Sullivan, David Fletcher, Dmitrijs Milajevs, Dougal J. Sutherland, Erich Schubert, Fabian Pedregosa, Florian Wilhelm, floydsoft, Félix-Antoine Fortin, Gael Varoquaux, Garrett-R, Gilles Louppe, gpassino, gwulfs, Hampus Bengtsson, Hamzeh Alsalhi, Hanna Wallach, Harry Mavroforakis, Hasil Sharma, Helder, Herve Bredin, Hsiang-Fu Yu, Hugues SALAMIN, Ian Gilmore, Ilambharathi Kanniah, Imran Haque, isms, Jake VanderPlas, Jan Dlabal, Jan Hendrik Metzen, Jatin Shah, Javier López Peña, jdcaballero, Jean Kossaifi, Jeff Hammerbacher, Joel Nothman, Jonathan Helmus, Joseph, Kaicheng Zhang, Kevin Markham, Kyle Beauchamp, Kyle Kastner, Lagacherie Matthieu, Lars Buitinck, Laurent Direr, leepei, Loic Esteve, Luis Pedro Coelho, Lukas Michelbacher, maheshakya, Manoj Kumar, Manuel, Mario Michael Krell, Martin, Martin Billinger, Martin Ku, Mateusz Susik, Mathieu Blondel, Matt Pico, Matt Terry, Matteo Visconti dOC, Matti Lyra, Max Linke, Mehdi Cherti, Michael Bommarito, Michael Eickenberg, Michal Romaniuk, MLG, mr.Shu, Nelle Varoquaux, Nicola Montecchio, Nicolas, Nikolay Mayorov, Noel Dawe, Okal Billy, Olivier Grisel, Óscar Nájera, Paolo Puggioni, Peter Prettenhofer, Pratap Vardhan, pvnguyen, queqichao, Rafael Carrascosa, Raghav R V, Rahiel Kasim, Randall Mason, Rob Zinkov, Robert Bradshaw, Saket Choudhary, Sam Nicholls, Samuel Charron, Saurabh Jha, sethdandridge, sinhrks, snuderl, Stefan Otte, Stefan van der Walt, Steve Tjoa, swu, Sylvain Zimmer, tejesh95, terrycojones, Thomas Delteil, Thomas Unterthiner, Tomas Kazmar, trevorstephens, tttthomasssss, Tzu-Ming Kuo, ugurcaliskan, ugurthemaster, Vinayak Mehta, Vincent Dubourg, Vjacheslav Murashkin, Vlad Niculae, wadawson, Wei Xue, Will Lamond, Wu Jiang, x0l, Xinfan Meng, Yan Yi, Yu-Chin