Scientists use numerical simulators to model real-world phenomena. Unlike purely statistical models, simulators incorporate scientific principles, which help them make better predictions with fewer parameters. However, fine-tuning these parameters to match real-world data is difficult. Simulation-Based inference (SBI) aims to find parameter values that align with prior knowledge and observed data. In other words, SBI aims to perform fully Bayesian parameter inference on simulation-based models, i.e., using only simulated data and without requiring access to the likelihood function.

SBI is an active field of research with new algorithms being developed regularly. The sbi package aims to provide a central resource for practitioners and researchers to gain access to state-of-the-art SBI algorithms, as well as extensive documentation and tutorials.

Install with pip install sbi, or check out the documentation

Inference methods

Neural posterior estimation (amortized and sequential [Pap18F, Lue17F, Gre19A])
Neural likelihood estimation [Pap19S]
Neural ratio estimation [Her20L]
Balanced neural ratio estimation [Del22R]
Neural variational inference (amortized and sequential [Glo22V])
Mixed neural likelihood estimation [Boe22F]
Parallelized MCMC slice sampling
Access to Pyro and PyMC MCMC samplers

Evaluation methods

Extensive plotting tools
Prior and posterior predictive checks
Access to arviz visualizations
Simulation-based calibration [Tal20V]
Classifier-based two-sample test (C2ST, [Lop17R])
Expected coverage [Dei22T, Mil21T]

Roadmap

We are currently implementing or plan to implement:

Flow-matching neural posterior estimation [Lip22F, Wil23F]
Sequential neural posterior score estimation [Sha22S]
Compositional neural score estimation [Gef23C]
Local-C2ST [Lin23L]

Acknowledgments

The sbi package originated at the University of Tübingen in the mackelab, funded by the German Federal Ministry of Education and Research (BMBF) through project ADIMEM (FKZ 01IS18052 A-D), project SiMaLeSAM (FKZ 01IS21055A) and the Tübingen AI Center (FKZ 01IS18039A) (see here for details).

Today, sbi is a community project maintained by a group of people from both the University of Tübingen and the TransferLab. It has a large group of contributors from across Europe—new contributors are welcome.

References

[Pap18F]

Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation, George Papamakarios, Iain Murray.

Apr 2018

Many statistical models can be simulated forwards but have intractable likelihoods. Approximate Bayesian Computation (ABC) methods are used to infer properties of these models from data. Traditionally these methods approximate the posterior over parameters by conditioning on data being inside an $\epsilon$-ball around the observed data, which is only correct in the limit …

[Lue17F]

Flexible statistical inference for mechanistic models of neural dynamics, Jan-Matthis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnenmacher, Jakob H Macke.

2017

Mechanistic models of single-neuron dynamics have been extensively studied in computational neuroscience. However, identifying which models can quantitatively reproduce empirically measured data has been challenging. We propose to overcome this limitation by using likelihood-free inference approaches (also known as Approximate Bayesian Computation, ABC) to perform full Bayesian inference on …

[Gre19A]

Automatic Posterior Transformation for Likelihood-Free Inference, David Greenberg, Marcel Nonnenmacher, Jakob Macke.

May 2019

How can one perform Bayesian inference on stochastic simulators with intractable likelihoods? A recent approach is to learn the posterior from adaptively proposed simulations using neural network-based conditional density estimators. However, existing methods are limited to a narrow range of proposal distributions or require importance weighting that can limit performance in practice. Here we …

[Pap19S]

Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows, George Papamakarios, David C. Sterratt, Iain Murray.

Jan 2019

We present Sequential Neural Likelihood (SNL), a new method for Bayesian inference in simulator models, where the likelihood is intractable but simulating data from the model is possible. SNL trains an autoregressive flow on simulated data in order to learn a model of the likelihood in the region of high posterior density. A sequential training procedure guides simulations and reduces simulation …

[Del22R]

Towards Reliable Simulation-Based Inference with Balanced Neural Ratio Estimation, Arnaud Delaunoy, Joeri Hermans, François Rozet, Antoine Wehenkel, Gilles Louppe.

May 2022

Modern approaches for simulation-based inference build upon deep learning surrogates to enable approximate Bayesian inference with computer simulators. In practice, the estimated posteriors' computational faithfulness is, however, rarely guaranteed. For example, Hermans et al., 2021 have shown that current simulation-based inference algorithms can produce posteriors that are overconfident, hence …

[Wil23F]

Flow Matching for Scalable Simulation-Based Inference, Jonas Bernhard Wildberger, Maximilian Dax, Simon Buchholz, Stephen R. Green, Jakob H. Macke, Bernhard Schölkopf.

Jul 2023

Neural posterior estimation methods based on discrete normalizing flows have become established tools for simulation-based inference (SBI), but scaling them to high-dimensional problems can be challenging. Building on recent advances in generative modeling, we here present flow matching posterior estimation (FMPE), a technique for SBI using continuous normalizing flows. Like diffusion models, and …

[Glo22V]

Variational methods for simulation-based inference, Manuel Glöckler, Michael Deistler, Jakob H. Macke.

2022

We present Sequential Neural Variational Inference (SNVI), an approach to perform Bayesian inference in models with intractable likelihoods. SNVI combines likelihood-estimation (or...

[Boe22F]

Flexible and efficient simulation-based inference for models of decision-making, Jan Boelts, Jan-Matthis Lueckmann, Richard Gao, Jakob H Macke.

Jul 2022

Inferring parameters of computational models that capture experimental data is a central task in cognitive neuroscience. Bayesian statistical inference methods usually require the ability to evaluate the likelihood of the model—however, for many models of interest in cognitive neuroscience, the associated likelihoods cannot be computed efficiently. Simulation-based inference (SBI) offers a …

[Her20L]

Likelihood-free MCMC with Amortized Approximate Ratio Estimators, Joeri Hermans, Volodimir Begy, Gilles Louppe.

Nov 2020

Posterior inference with an intractable likelihood is becoming an increasingly common task in scientific domains which rely on sophisticated computer simulations. Typically, these forward models do not admit tractable densities forcing practitioners to rely on approximations. This work introduces a novel approach to address the intractability of the likelihood and the marginal model. We achieve …

[Tal20V]

Validating Bayesian Inference Algorithms with Simulation-Based Calibration, Sean Talts, Michael Betancourt, Daniel Simpson, Aki Vehtari, Andrew Gelman.

Oct 2020

Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce \emph{simulation-based calibration} (SBC), a general procedure for validating inferences from Bayesian algorithms capable of generating posterior samples. This …

[Lop17R]

Revisiting Classifier Two-Sample Tests, David Lopez-Paz, Maxime Oquab.

Jul 2022

The goal of two-sample tests is to assess whether two samples, $S_P \sim P^n$ and $S_Q \sim Q^m$, are drawn from the same distribution. Perhaps intriguingly, one relatively unexplored method to build two-sample tests is the use of binary classifiers. In particular, construct a dataset by pairing the $n$ examples in $S_P$ with a positive label, and by pairing the $m$ examples in $S_Q$ with a …

[Mil21T]

Truncated Marginal Neural Ratio Estimation, Benjamin K Miller, Alex Cole, Patrick Forré, Gilles Louppe, Christoph Weniger.

2021

Parametric stochastic simulators are ubiquitous in science, often featuring high-dimensional input parameters and/or an intractable likelihood. Performing Bayesian parameter inference in this context can be challenging. We present a neural simulation-based inference algorithm which simultaneously offers simulation efficiency and fast empirical posterior testability, which is unique among modern …

[Dei22T]

Truncated proposals for scalable and hassle-free simulation-based inference, Michael Deistler, Pedro J. Goncalves, Jakob H. Macke.

Dec 2022

Inference methods

Evaluation methods

Roadmap

Acknowledgments

References

In this series →