pyDVL is a library for data valuation (see our review). You can install with PyPi, contribute to the project or check out the documentation.

As of version 0.5.0, pyDVL provides robust, parallel implementations of:

• Leave One Out
• Data Shapley
  • Exact values with different sampling methods
  • Exact values, for KNN [Jia19aE]
  • Truncated Monte Carlo Shapley [Gho19D]
  • Owen sampling [Okh21M]
• Group testing Shapley [Jia19aE]
• Least Core [Yan21I]
• Data Utility Learning [Wan22I]
• Influence functions (beta / in progress) [Koh17U]
  • Fast Hessian-Vectors with stochastic optimization for large models
  • (Approximate) Maximum Influence Perturbation (upcoming)
  • …

In addition, we provide analyses of the strengths and weaknesses of key methods, as well as detailed examples.

Usage

If you have any scikit-learn compatible supervised model, you can use pyDVL in three easy steps:

1. Create a Dataset object with your train and test splits.
2. Create a Utility object to wrap the Dataset, the model and a scoring function.
3. Use one of the methods defined in the library to compute the values.

   from sklearn.datasets import load_breast_cancer
   from sklearn.linear_model import LogisticRegression
   from pydvl.utils import Dataset, Utility, Scorer
   from pydvl.value import *
   
   data = Dataset.from_sklearn(load_breast_cancer(), train_size=0.7)
   model = LogisticRegression()
   u = Utility(model, data, Scorer("accuracy", default=0.0))
   values = compute_shapley_values(
       u,
       mode=ShapleyMode.TruncatedMontecarlo,
       done=MaxUpdates(100),
       truncation=RelativeTruncation(u, rtol=0.01),
   )
   values.sort(key="value")
   print(f"The lowest 5% valued points are: {values.indices[:int(len(values) * 0.05)]}")
   

Roadmap

We are currently implementing

• Data Banzhaf [Wan22D]
• Beta Shapley [Kwo22B]
• Class-Wise Shapley [Sch22C]

To see what new methods, features and improvements are coming, check out the issues on GitHub.