Scalable Trustworthy AI

Creating scalable and trustworthy AI with human guidance

Overview

AI has left the lab. It is reshaping how we live and work. To fully exploit its benefits, we must address critical gaps in trustworthiness. We work on Deploying General AI in the Private World.

AI’s transformative era. AI offers potential to address global challenges. It can increase productivity, support ageing populations, and tackle urgent issues like climate change. Since the introduction of ChatGPT, AI has left the lab. It is now used by billions worldwide and is shaping the economy and society. AI is no longer experimental; it is a real force driving change.

The rise of general-purpose AI. Recent developments have focused on general-purpose AI, often called foundational or frontier models. These are large models trained on vast data to perform a wide range of tasks without task-specific training. They have made impressive progress in language, vision, and multimodal reasoning. Yet they remain limited in reliably solving specific, real-world problems without substantial adaptation.

The adaptation gap. The key challenge we see today is the adaptation of general AI to private, real-world settings. General-purpose capabilities alone are insufficient. To close this gap, we identify three concrete challenges:

By addressing these areas, we aim to reduce AI-related risks and maximise its societal benefits. Effective adaptation of general AI to private contexts will signal the true beginning of an AI-led industrial revolution.

Fortunately, we are not alone in this effort. There are many other research labs around the world that make important contributions on Trustworthy AI. Our group find our uniqueness by striving for working solutions that are widely applicable and can be deployed at a large-scale. We thus name our group Scalable Trustworthy AI. For impact at scale, we commit ourselves to the following principles:

For prospective students: You might be interested in our internal curriculum and guidelines for a PhD program: Principles for a PhD Program.

STAI group is part of the Tübingen AI Center and the University of Tübingen. STAI is also within the ecosystem of International Max Planck Research School for Intelligent Systems (IMPRS-IS) and the ELLIS Society.

logo
logo
logo
logo

Members

Seong Joon Oh

Group Leader

Elisa Nguyen

PhD Student

Arnas Uselis

PhD Student

Stefano Woerner

PhD Student

Ankit Sonthalia

PhD Student

Lennart Bramlage

Collaborating PhD Student

Bora Kargi

MSc Student

Luca Füger

MSc student

Fabian Morelli

MSc Student

Alumni

Elif Akata

PhD Student

Michael Kirchhof

Collaborating PhD Student

Evgenii Kortukov

MSc Student

Johannes Bertram

Research Assistant

Publications

Does Data Scaling Lead to Visual Compositional Generalization?

ICML

Do Deep Neural Network Solutions Form a Star Domain?

ICLR

Intermediate Layer Classifiers for OOD Generalization

ICLR

Decoupled Finetuning for Domain Generalizable Semantic Segmentation

ICLR

Are We Done with Object-Centric Learning?

SCSL @ ICLR

DiCoTTA: Domain-invariant Learning for Continual Test-time Adaptation

arXiv

Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles

SCSL @ ICLR

Playing repeated games with Large Language Models

Nature Human Behaviour

CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modally

arXiv

Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks

NeurIPS D&B (Spotlight)

Overcoming Domain Limitations in Open-vocabulary Segmentation

arXiv

Studying Large Language Model Behaviors Under Realistic Knowledge Conflicts

CoLM

Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AI

arXiv

Scalable Ensemble Diversification for OOD Generalization and Detection

arXiv

Pretrained Visual Uncertainties

arXiv

A Bayesian Perspective On Training Data Attribution

NeurIPS

Exploring Practitioner Perspectives On Training Data Attribution Explanations

NeurIPS XAI in Action Workshop

ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets

NeurIPS

URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates

NeurIPS D&B

Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts

ICCV

Scratching Visual Transformer's Back with Uniform Attention

ICCV

Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs

ICML

URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates

UAI-EAI Best Student Paper

ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCO

ECCV

Dataset Condensation via Efficient Synthetic-Data Parameterization

ICML

Weakly Supervised Semantic Segmentation Using Out-of-Distribution Data

CVPR

Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective

ICLR

Openings

Postdoc Opportunity: Scalable Trustworthy AI - Novel Dataset Development

We are seeking a highly motivated Postdoctoral Researcher to join our team at the University of Tübingen for an exciting two-year project on Scalable Trustworthy AI. The successful candidate will play a pivotal role in developing and collecting novel datasets that capture not only task outputs from human annotators but also valuable annotation byproducts, such as mouse traces, gaze patterns, click history, time to complete the task, and any corrections made during the process. Our goal is to leverage this rich data to better align AI systems with human cognitive mechanisms. Read the Annotation Byproducts paper for further details. This unique opportunity will allow the selected applicant to enhance their research expertise, contribute to cutting-edge advancements in AI, and benefit from Tübingen’s vibrant research ecosystem and extensive international network. The position comes with a competitive postdoc salary and German social benefits. The starting date is flexible, and the selected candidate will be based at the Tübingen AI Center. We encourage candidates with a strong PhD degree in machine learning, natural language processing, computer vision, mathematics, statistics, human-computer interaction, or a related field to apply. To apply, please send your CV and research statement to coallaoh@gmail.com.