Empirical Inference


2024


no image
GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs

Gao, G., Liu, W., Chen, A., Geiger, A., Schölkopf, B.

The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024 (conference) Accepted

[BibTex]

2024

[BibTex]


no image
Out-of-Variable Generalization for Discriminative Models

Guo, S., Wildberger, J., Schölkopf, B.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding

Pace, A., Yèche, H., Schölkopf, B., Rätsch, G., Tennenholtz, G.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion

Meterez*, A., Joudaki*, A., Orabona, F., Immer, A., Rätsch, G., Daneshmand, H.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks

Spieler, A., Rahaman, N., Martius, G., Schölkopf, B., Levina, A.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Open X-Embodiment Collaboration ( incl. Guist, S., Schneider, J., Schölkopf, B., Büchler, D. ).

IEEE International Conference on Robotics and Automation (ICRA), May 2024 (conference) Accepted

[BibTex]

[BibTex]


no image
Can Large Language Models Infer Causation from Correlation?

Jin, Z., Liu, J., Lyu, Z., Poff, S., Sachan, M., Mihalcea, R., Diab*, M., Schölkopf*, B.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal supervision (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Certified private data release for sparse Lipschitz functions

Donhauser, K., Lokna, J., Sanyal, A., Boedihardjo, M., Hönig, R., Yang, F.

27th International Conference on Artificial Intelligence and Statistics (AISTATS), May 2024 (conference) Accepted

[BibTex]

[BibTex]


Ghost on the Shell: An Expressive Representation of General 3D Shapes
Ghost on the Shell: An Expressive Representation of General 3D Shapes

(Oral)

Liu, Z., Feng, Y., Xiu, Y., Liu, W., Paull, L., Black, M. J., Schölkopf, B.

In Proceedings of the Twelfth International Conference on Learning Representations, The Twelfth International Conference on Learning Representations, May 2024 (inproceedings) Accepted

Abstract
The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and lighting, 2) support physical simulation, and 3) are memory-efficient for modern graphics pipelines. Recent work on reconstructing and statistically modeling 3D shape, however, has critiqued meshes as being topologically inflexible. To capture a wide range of object shapes, any 3D representation must be able to model solid, watertight, shapes as well as thin, open, surfaces. Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling. Inspired by the observation that open surfaces can be seen as islands floating on watertight surfaces, we parameterize open surfaces by defining a manifold signed distance field on watertight templates. With this parameterization, we further develop a grid-based and differentiable representation that parameterizes both watertight and non-watertight meshes of arbitrary topology. Our new representation, called Ghost-on-the-Shell (G-Shell), enables two important applications: differentiable rasterization-based reconstruction from multiview images and generative modelling of non-watertight meshes. We empirically demonstrate that G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks, while also performing effectively for watertight meshes.

Home Code Video Project [BibTex]

Home Code Video Project [BibTex]


no image
Identifying Policy Gradient Subspaces

Schneider, J., Schumacher, P., Guist, S., Chen, L., Häufle, D., Schölkopf, B., Büchler, D.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Some Intriguing Aspects about Lipschitz Continuity of Neural Networks

Khromov*, G., Singh*, S. P.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

Liu, W., Qiu, Z., Feng, Y., Xiu, Y., Xue, Y., Yu, L., Feng, H., Liu, Z., Heo, J., Peng, S., Wen, Y., Black, M. J., Weller, A., Schölkopf, B.

In Proceedings of the Twelfth International Conference on Learning Representations, The Twelfth International Conference on Learning Representations, May 2024 (inproceedings) Accepted

Abstract
Large foundation models are becoming ubiquitous, but training them from scratch is prohibitively expensive. Thus, efficiently adapting these powerful models to downstream tasks is increasingly important. In this paper, we study a principled finetuning paradigm -- Orthogonal Finetuning (OFT) -- for downstream task adaptation. Despite demonstrating good generalizability, OFT still uses a fairly large number of trainable parameters due to the high dimensionality of orthogonal matrices. To address this, we start by examining OFT from an information transmission perspective, and then identify a few key desiderata that enable better parameter-efficiency. Inspired by how the Cooley-Tukey fast Fourier transform algorithm enables efficient information transmission, we propose an efficient orthogonal parameterization using butterfly structures. We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT). By subsuming OFT as a special case, BOFT introduces a generalized orthogonal finetuning framework. Finally, we conduct an extensive empirical study of adapting large vision transformers, large language models, and text-to-image diffusion models to various downstream tasks in vision and language.

Home Code HuggingFace project [BibTex]

Home Code HuggingFace project [BibTex]


no image
Skill or Luck? Return Decomposition via Advantage Functions

Pan, H., Schölkopf, B.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference) Accepted

[BibTex]

[BibTex]


no image
Transformer Fusion with Optimal Transport

Imfeld*, M., Graldi*, J., Giordano*, M., Hofmann, T., Anagnostidis, S., Singh, S. P.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Causal Modeling with Stationary Diffusions

Lorch, L., Krause*, A., Schölkopf*, B.

27th International Conference on Artificial Intelligence and Statistics (AISTATS), May 2024, *equal supervision (conference) Accepted

[BibTex]

[BibTex]


no image
Multi-View Causal Representation Learning with Partial Observability

Yao, D., Xu, D., Lachapelle, S., Magliacane, S., Taslakian, P., Martius, G., von Kügelgen, J., Locatello, F.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
Towards Meta-Pruning via Optimal Transport

Theus, A., Geimer, O., Wicke, F., Hofmann, T., Anagnostidis, S., Singh, S. P.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024 (conference) Accepted

[BibTex]

[BibTex]


no image
Stochastic Gradient Descent for Gaussian Processes Done Right

Lin*, J. A., Padhy*, S., Antorán*, J., Tripp, A., Terenin, A., Szepesvari, C., Hernández-Lobato, J. M., Janz, D.

Proceedings of the Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (conference) Accepted

arXiv [BibTex]

arXiv [BibTex]


no image
PILLAR: How to make semi-private learning more effective

Hu, Y., Pinto, F., Yang, F., Sanyal, A.

2nd IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), April 2024 (conference) Accepted

[BibTex]

[BibTex]

2023


no image
Spuriosity Didn’t Kill the Classifier: Using Invariant Predictions to Harness Spurious Features

Eastwood*, C., Singh*, S., Nicolicioiu, A. L., Vlastelica, M., von Kügelgen, J., Schölkopf, B.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

2023

[BibTex]


no image
CLadder: A Benchmark to Assess Causal Reasoning Capabilities of Language Models

Jin*, Z., Chen*, Y., Leeb*, F., Gresele*, L., Kamal, O., Lyu, Z., Blin, K., Gonzalez, F., Kleiman-Weiner, M., Sachan, M., Schölkopf, B.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *main contributors (conference) Accepted

[BibTex]

[BibTex]


no image
Transitivity Recovering Decompositions: Interpretable and Robust Fine-Grained Relationships

Chaudhuri, A., Mancini, M., Akata, Z., Dutta, A.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023 (conference) Accepted

[BibTex]

[BibTex]


no image
Generalized Bayesian Inference for Scientific Simulators via Amortized Cost Estimation

Gao*, R., Deistler*, M., Macke, J. H.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

[BibTex]


no image
Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Data

Guo*, S., Tóth*, V., Schölkopf, B., Huszár, F.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

[BibTex]


no image
Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent

Lin*, J. A., Antorán*, J., Padhy*, S., Janz, D., Hernández-Lobato, J. M., Terenin, A.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

[BibTex]


no image
Meta-in-context learning in large language models

Coda-Forno, J., Binz, M., Akata, Z., Botvinick, M., Wang, J. X., Schulz, E.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023 (conference) Accepted

[BibTex]

[BibTex]


Controlling Text-to-Image Diffusion by Orthogonal Finetuning
Controlling Text-to-Image Diffusion by Orthogonal Finetuning

Qiu*, Z., Liu*, W., Feng, H., Xue, Y., Feng, Y., Liu, Z., Zhang, D., Weller, A., Schölkopf, B.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

Abstract
Large text-to-image diffusion models have impressive capabilities in generating photorealistic images from text prompts. How to effectively guide or control these powerful models to perform different downstream tasks becomes an important open problem. To tackle this challenge, we introduce a principled finetuning method -- Orthogonal Finetuning (OFT), for adapting text-to-image diffusion models to downstream tasks. Unlike existing methods, OFT can provably preserve hyperspherical energy which characterizes the pairwise neuron relationship on the unit hypersphere. We find that this property is crucial for preserving the semantic generation ability of text-to-image diffusion models. To improve finetuning stability, we further propose Constrained Orthogonal Finetuning (COFT) which imposes an additional radius constraint to the hypersphere. Specifically, we consider two important finetuning text-to-image tasks: subject-driven generation where the goal is to generate subject-specific images given a few images of a subject and a text prompt, and controllable generation where the goal is to enable the model to take in additional control signals. We empirically show that our OFT framework outperforms existing methods in generation quality and convergence speed.

Home Code [BibTex]

Home Code [BibTex]


no image
On the Importance of Step-wise Embeddings for Heterogeneous Clinical Time-Series

Kuznetsova*, R., Pace*, A., Burger*, M., Yèche, H., Rätsch, G.

Proceedings of the 3rd Machine Learning for Health Symposium (ML4H) , 225, pages: 268-291, Proceedings of Machine Learning Research, (Editors: Hegselmann, S.and Parziale, A. and Shanmugam, D. and Tang, S. and Asiedu, M. N. and Chang, S. and Hartvigsen, T. and Singh, H.), PMLR, December 2023, *equal contribution (conference)

link (url) [BibTex]

link (url) [BibTex]


SE(3) Equivariant Augmented Coupling Flows
SE(3) Equivariant Augmented Coupling Flows

Midgley*, L. I., Stimper*, V., Antorán*, J., Mathieu*, E., Schölkopf, B., Hernández-Lobato, J. M.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

Abstract
Coupling normalizing flows allow for fast sampling and density evaluation, making them the tool of choice for probabilistic modeling of physical systems. However, the standard coupling architecture precludes endowing flows that operate on the Cartesian coordinates of atoms with the SE(3) and permutation invariances of physical systems. This work proposes a coupling flow that preserves SE(3) and permutation equivariance by performing coordinate splits along additional augmented dimensions. At each layer, the flow maps atoms’ positions into learned SE(3) invariant bases, where we apply standard flow transformations, such as monotonic rational-quadratic splines, before returning to the original basis. Crucially, our flow preserves fast sampling and density evaluation, and may be used to produce unbiased estimates of expectations with respect to the target distribution via importance sampling. When trained on the DW4, LJ13 and QM9-positional datasets, our flow is competitive with equivariant continuous normalizing flows, while allowing sampling two orders of magnitude faster. Moreover, to the best of our knowledge, we are the first to learn the full Boltzmann distribution of alanine dipeptide by only modeling the Cartesian positions of its atoms. Lastly, we demonstrate that our flow can be trained to approximately sample from the Boltzmann distribution of the DW4 and LJ13 particle systems using only their energy functions.

arXiv [BibTex]

arXiv [BibTex]


no image
In-Context Impersonation Reveals Large Language Models’ Strengths and Biases

Salewski, L., Alaniz, S., Rio-Torto, I., Schulz, E., Akata, Z.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023 (conference) Accepted

[BibTex]

[BibTex]


no image
Neural Harmonics: Bridging Spectral Embedding and Matrix Completion in Self-Supervised Learning

Munkhoeva, M., Oseledets, I.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023 (conference) Accepted

[BibTex]

[BibTex]


no image
Flow Matching for Scalable Simulation-Based Inference

Wildberger*, J., Dax*, M., Buchholz*, S., Green, S. R., Macke, J. H., Schölkopf, B.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

[BibTex]


no image
Learning Linear Causal Representations from Interventions under General Nonlinear Mixing

Buchholz*, S., Rajendran*, G., Rosenfeld, E., Aragam, B., Schölkopf, B., Ravikumar, P.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

[BibTex]


no image
Causal Component Analysis

Liang, W., Kekić, A., von Kügelgen, J., Buchholz, S., Besserve, M., Gresele*, L., Schölkopf*, B.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *shared last author (conference) Accepted

[BibTex]

[BibTex]


no image
A Measure-Theoretic Axiomatisation of Causality

Park, J., Buchholz, S., Schölkopf, B., Muandet, K.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023 (conference) Accepted

[BibTex]

[BibTex]


no image
Causal Modeling with Stationary Diffusions

Lorch, L., Krause*, A., Schölkopf*, B.

Causal Representation Learning Workshop at NeurIPS 2023, December 2023, *equal supervision (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Nonparametric Identifiability of Causal Representations from Unknown Interventions

von Kügelgen, J., Besserve, M., Liang, W., Gresele, L., Kekić, A., Bareinboim, E., Blei, D., Schölkopf, B.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023 (conference) Accepted

[BibTex]

[BibTex]


no image
Meta-learning families of plasticity rules in recurrent spiking networks using simulation-based inference

Confavreux*, B., Ramesh*, P., Goncalves, P. J., Macke, J. H., Vogels, T. P.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

[BibTex]


no image
Can semi-supervised learning use all the data effectively? A lower bound perspective

Tifrea*, A., Yüce*, G., Sanyal, A., Yang, F.

Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (conference) Accepted

[BibTex]

[BibTex]


no image
A taxonomy and review of generalization research in NLP

Hupkes, D., Giulianelli, M., Dankers, V., Artetxe, M., Elazar, Y., Pimentel, T., Christodoulopoulos, C., Lasri, K., Saphra, N., Sinclair, A., Ulmer, D., Schottmann, F., Batsuren, K., Sun, K., Sinha, K., Khalatbari, L., Ryskina, M., Frieske, R., Cotterell, R., Jin, Z.

Nature Machine Intelligence, 5(10):1161-1174, October 2023 (article)

DOI [BibTex]

DOI [BibTex]


no image
Certified private data release for sparse Lipschitz functions

Donhauser, K., Lokna, J., Sanyal, A., Boedihardjo, M., Hönig, R., Yang, F.

TPDP 2023 - Theory and Practice of Differential Privacy, September 2023 (conference)

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
How to make semi-private learning more effective

Pinto, F., Hu, Y., Yang, F., Sanyal, A.

TPDP 2023 - Theory and Practice of Differential Privacy, September 2023 (conference)

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
Getting personal with epigenetics: towards individual-specific epigenomic imputation with machine learning

Hawkins-Hooker, A., Visonà, G., Narendra, T., Rojas-Carulla, M., Schölkopf, B., Schweikert, G.

Nature Communications, 14(1), August 2023 (article)

DOI [BibTex]

DOI [BibTex]


no image
Causal Effect Estimation from Observational and Interventional Data Through Matrix Weighted Linear Estimators

Kladny, K., von Kügelgen, J., Schölkopf, B., Muehlebach, M.

Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI), 216, pages: 1087-1097, Proceedings of Machine Learning Research, (Editors: Evans, Robin J. and Shpitser, Ilya), PMLR, August 2023 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
USIM-DAL: Uncertainty-aware Statistical Image Modeling-based Dense Active Learning for Super-resolution

Rangnekar, V., Upadhyay, U., Akata, Z., Banerjee, B.

Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI), 216, pages: 1707-1717, (Editors: Evans, Robin J. and Shpitser, Ilya), PMLR, August 2023 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Adversarial robustness of amortized Bayesian inference

Glöckler, M., Deistler, M., Macke, J. H.

Proceedings of 40th International Conference on Machine Learning (ICML) , 202, pages: 11493-11524, Proceedings of Machine Learning Research, (Editors: A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato and J. Scarlett), PMLR, July 2023 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
On the Identifiability and Estimation of Causal Location-Scale Noise Models

Immer, A., Schultheiss, C., Vogt, J. E., Schölkopf, B., Bühlmann, P., Marx, A.

Proceedings of the 40th International Conference on Machine Learning (ICML), 202, pages: 14316-14332, Proceedings of Machine Learning Research, (Editors: A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato and J. Scarlett), PMLR, July 2023 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Temporal Label Smoothing for Early Event Prediction

Yèche*, H., Pace*, A., Rätsch, G., Kuznetsova, R.

Proceedings of the 40th International Conference on Machine Learning (ICML), 202, pages: 39913-39938, Proceedings of Machine Learning Research, (Editors: A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato and J. Scarlett), PMLR, July 2023, *equal contribution (conference)

arXiv link (url) [BibTex]

arXiv link (url) [BibTex]


no image
Efficient Semiring-Weighted Earley Parsing

Opedal, A., Zmigrod, R., Vieira, T., Cotterell, R., Eisner, J.

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), 1, pages: 3687-3713, (Editors: Anna Rogers, Jordan L. Boyd-Graber and Naoaki Okazaki), Association for Computational Linguistics, July 2023 (conference)

link (url) [BibTex]

link (url) [BibTex]


no image
Discrete Key-Value Bottleneck

Träuble, F., Goyal, A., Rahaman, N., Mozer, M. C., Kawaguchi, K., Bengio, Y., Schölkopf, B.

Proceedings of the 40th International Conference on Machine Learning (ICML) , 202, pages: 34431-34455, Proceedings of Machine Learning Research, (Editors: A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato and J. Scarlett), PMLR, July 2023 (conference)

link (url) [BibTex]

link (url) [BibTex]