Broadly speaking, we work at the intersection of computational neuroscience and machine learning, aka AI4(Neuro)Science. Ultimately, we are interested in reverse engineering the algorithms of the brain, in order to figure out how the brain works and to build better artificial intelligence systems.
Check out group's website for more information, and see our open source code below!
- DeepLabCut: for animal pose estimation
- DLC2action: for action segmentation
- hBehaveMAE: unsupervised action decomposition for hierarchical behavior
Computer Vision and Behavioral Analysis:
- Elucidating the Hierarchical Nature of Behavior with Masked Autoencoders: Stoffl, Bonnetto, d'Ascoli & Mathis ECCV 2024
- HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields: Code for Haozhe Qi, Chen Zhao, Mathieu Salzmann, & Alexander Mathis. CVPR 2024
- WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models: Code for Gabeff, Russwurm, Tuia & Mathis International Journal of Computer Vision 2024 (also oral at CVPR CV4animals 2023)
- Bottom-up conditioned top-down pose estimation (BUCTD): Code for Zhou*, Stoffl*, Mathis and Mathis ICCV 2023. State of the art code for performing 2D pose estimation in crowded scenes.
- End-to-end trainable multi-instance pose estimation with transformers: Code for POET model, Stoffl, Vidal & Mathis arxiv 2021
- AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild, Joska et al. ICRA 2021
- Primer on Motion Capture, Mathis et al. Neuron 2020
Reinforcement learning (mostly for motor skills):
- Acquiring musculoskeletal skills with curriculum-based reinforcement learning: Code for Chiappa*, Tano*, Patel* et al. Neuron 2024
- Winning code for the object manipulation track of the MyoChallenge at NeurIPS 2023, submitted
- Latent Exploration for Reinforcement Learning: Code for Chiappa et al. NeurIPS 2023
- DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body: Code for Chiappa, Marin-Vargas & Mathis NeurIPS 2022
- Winning code for the Baoding ball MyoChallenge at NeurIPS 2022, joint work with Pouget Lab (University of Geneva). Caggiano et al. Proceedings of Machine Learning Research 2022
AI4Science including modeling sensorimotor control:
- Task-driven-proprioception: Code for modeling the proprioceptive system of primates. Marin Vargas* & Bisi* et al. Cell 2024
- ODEformer: symbolic regression of dynamical systems with transformers: Code from d'Ascoli*, Becker*, Mathis, Schwaller & Kilbertus ICLR 2024 (spotlight). Cool code to infer symbolic formulas from data
- DeepDraw: Code for modeling proprioception with task-driven modeling, Sandbrink*, Mamidanna* et al. eLife 2023
🌈 Please reach out, if you want to work with us! We love collaborative, open-source science.
We often collaborate with the group of Mackenzie Mathis, and also recommend checking out their GitHub repository!
