What we are reading: paper pills
Paper pills are summaries of publications, talks or new software, with context and a short description of the methods, main ideas and key results. They are written for practitioners or researchers who want a peek into recent developments but are short on time. We publish monthly digests as well as highlights of major conferences.

Pills

Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions

An investigation of common problems and pitfalls of the popular MMD technique for comparing distributions on graphs. Additionally, the authors offer a set for practical guidelines to work with MMD.

A picture of Faried
Geometric Deep Learning
Graphs in ML

Energy-Based Learning for Cooperative Games, with Applications to Valuation Problems in Machine Learning

A theoretical framework connecting data valuation scores, game theory and energy based models is presented. Common criteria like Shapley or Banzhaf can be recovered as special cases.

A photo of Fabio
Data Valuation

Resolving Training Biases via Influence-based Data Relabeling

Influence functions are used to correct corrupt labels in a dataset that would significantly decrease a trained model’s performance. Experiments show this to be better than previous approaches that focused on removing such samples from the dataset.

A photo of Fabio
Data Efficiency

Understanding over-squashing and bottlenecks on graphs via curvature

Oversquashing refers to the struggling of graph neural networks with tasks requiring long-distance interactions between nodes. A new notion of edge curvature is shown to be related to this problem.

A picture of Faried
Geometric Deep Learning
Graphs in ML

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

An analytic form for the de-noising process of diffusion neural networks is found in this important work. Monte Carlo sampling with the analytic expression results in a significant improvement of sampling speed (i.e. image generation) which should be relevant for most practicioners using diffusion models.

A photo of Fabio
Diffusion Models

GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation

Diffusion models are used for predicting the stable 3d conformations of a molecule from its 2d graph.

A photo of Fabio
Diffusion Models

Label Encoding for Regression Networks

Regression is reformulated to predicting a binary code of length M with M binary classifiers. With carefully chosen encoding and decoding schemes, the resulting neural networks outperform those trained with standard regression by a large margin - should be very useful for practitioners!

A picture of Michael

Boosting Certified Robustness of L Infinity Distance Nets

Recent architectural improvements for neural network classifiers make them certifiably robust against certain types of adversarial attacks - meaning that they are guaranteed not to change their prediction on small changes to the input. A new type of neuron is shown to have the ability of achieving perfect 100% robust accuracy under certain attack models.

A picture of Miguel
Robustness in ML

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)

A non-intrusive and user-input based method for interpreting decision-making of trained neural networks is introduced. It is cheap and has many applications, e.g. detecting bias in the network, and is implemented in several standard frameworks, but must also be used with some care.

A picture of Michael
Explainable AI

Learning Strides in Convolutional Neural networks

Instead of finding strides through a discrete search, an analogy of the strides hyperparameter for CNNs can be learned during training with gradient descent. The method results in new layer which is based on the discrete Fourier transform, spectral pooling and a “soft mask”.

A picture of Michael