Paper pills

Paper pills are written for practitioners or researchers who want a peek into recent developments but are short on time. We publish monthly digests as well as occasional highlights of major conferences.

Pills

Augmented Language Models: a survey

A survey of recent advances in augmenting (large) language models with new capabilities such as reasoning, tool use, and more. While the field of LLMs moves at an incredible pace, this article from Meta AI from February 2023 remains a good overview.

A picture of Michael
Large Language Models

Joint Probability Trees

Joint probability trees (JPTs) are a novel formalism for the representation of full-joint distributions over sets of random variables in hybrid domains. The learning algorithm fundamentally builds on the principles well-known from decision tree learning, decomposing the representation into tractable mixture components based on the notion of distribution impurity.

A photo of Dominik
Bayesian ML and Probabilistic Programming

Density estimation using deep generative neural networks

Density estimation is among the fundamental problems in statistics. It is difficult to estimate the density of high-dimensional data due to the “curse of dimensionality”. Roundtrip describes a new general-purpose density estimator based on deep generative neural networks.

A picture of Maternus
Simulation-Based Inference

Physics-informed DeepONets

Physics-informed DeepONets can learn the solution operator of arbitrary partial differential equations, even without any paired input-output training data. This methodology is reported to have the potential to accelerate simulations by up to three orders of magnitude compared to conventional methods.

Samuel
Simulation and AI

Post-Hoc Concept Bottleneck Models

Concept Bottleneck models are a set of intrinsically interpretable models that rely on high level concepts to make predictions. They often do no match the accuracy of unrestricted NNs and have thus rarely been used in practice. A recent paper proposes a way to address these limitations through post-hoc enhancements.

A photo of Fabio
Explainable AI

Measuring the Intrinsic Dimension of Objective Landscapes

This paper proposes a method for measuring the difficulty of a machine learning task by training neural networks using a randomly oriented subspace of the parameter space instead of the full parameter space.

A picture of Anes
Learning in High Dimensions

Learning nonlinear operators: the DeepONet architecture

The universal approximation theorem for operators suggests a neural network architecture for learning continuous nonlinear operators, called DeepONet. It can be used to learn explicit function operators, such as integrals, as well as implicit ones, for example the solution operator of differential equations.

Samuel
Simulation and AI

Spectral likelihood expansions for Bayesian inference

Spectral Likelihood expansion represents the Likelihood using Polynomial Chaos Expansion to speed up access to the posterior. It thus avoids costly MCMC simulations.

A picture of Maternus
Bayesian ML and Probabilistic Programming

A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful

In posterior estimation, it is generally worse to exclude plausible parameters than wrongly including implausible ones. However, extensive tests yield empirical evidence that popular algorithms provide overconfident estimations.

A picture of Maternus
Simulation-Based Inference

Benchmarking Simulation-Based Inference

A much-needed benchmark for methods of simulation-based inference.

A picture of Maternus
Simulation-Based Inference