DeepLearning 2.0: Meta-Learning Qualitatively New Components

Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition

Author(s): Dooley, Samuel; Sukthanker, Rhea Sanjay; Dickerson, John P.; White, Colin; Hutter, Frank; Goldblum, Micah
Published in: Advances in Neural Information Processing Systems, Issue 36, 2023, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2210.09943

Multi-objective Tree-structured Parzen Estimator Meets Meta-learning

Author(s): Watanabe, Shuhei; Awad, Noor; Onishi, Masaki; Hutter, Frank
Published in: Workshop on Meta-Learning at the Conference on Neural Information Processing Systems, 2022
Publisher: openreview.net

PED-ANOVA: Efficiently Quantifying Hyperparameter Importance in Arbitrary Subspaces

Author(s): Watanabe, Shuhei; Bansal, Archit; Hutter, Frank
Published in: International Joint Conference on Artificial Intelligence, Issue 32, 2023, ISSN 1045-0823
Publisher: International Joint Conferences on Artificial Intelligence
DOI: 10.24963/ijcai.2023/488

Weight-Entanglement Meets Gradient-Based Neural Architecture Search

Author(s): Sukthanker, Rhea Sanjay; Krishnakumar, Arjun; Safari, Mahmoud; Hutter, Frank
Published in: International Conference on Automated Machine Learning, Issue 4, 2024
Publisher: openreview.net
DOI: 10.48550/arxiv.2312.10440

Quick-Tune-Tool: A Practical Tool and its User Guide for Automatically Finetuning Pretrained Models

Author(s): Rapant, Ivo; Purucker, Lennart; Ferreira, Fabio; Arango, Sebastian Pineda; Kadra, Arlind; Grabocka, Josif; Hutter, Frank
Published in: International Conference on Automated Machine learning (Workshop Track), 2024
Publisher: openreview.net

LMEMs for post-hoc analysis of HPO Benchmarking

Author(s): Geburek, Anton Merlin; Mallik, Neeratyoy; Stoll, Danny; Bouthillier, Xavier; Hutter, Frank
Published in: International Conference on Automated Machine learning (Workshop Track), 2024
Publisher: openreview.net

Warmstarting for Scaling Language Models

Author(s): Mallik, Neeratyoy; Janowski, Maciej; Hog, Johannes; Rakotoarison, Herilalaina; Klein, Aaron; Grabocka, Josif; Hutter, Frank
Published in: Workshop on Adaptive Foundation Models: Evolving AI for Personalized and Efficient Learning at NeurIPS 2024, 2024
Publisher: openreview.net

Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models

Author(s): Bhethanabhotla, Sathya Kamesh; Swelam, Omar; Siems, Julien; Salinas, David; Hutter, Frank
Published in: Workshop on Time Series in the Age of Large Models at NeurIPS 2024, 2024
Publisher: openreview.net

Preserving Principal Subspaces to Reduce Catastrophic Forgetting in Fine-tuning

Author(s): Franke, Jorg K.H.; Hefenbrock, Michael; Hutter, Frank
Published in: Workshop on Mathematical and Empirical Understanding of Foundation Models at ICLR 2024, 2024
Publisher: openreview.net

Probabilistic Transformer: Modelling Ambiguities and Distributions for RNA Folding and Molecule Design

Author(s): Franke, Jörg K. H.; Runge, Frederic; Hutter, Frank
Published in: Advances in Neural Information Processing Systems, Issue 35, 2022, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2205.13927

From Epoch to Sample Size: Developing New Data-driven Priors for Learning Curve Prior-Fitted Networks

Author(s): Viering, Tom Julian; Adriaensen, Steven; Rakotoarison, Herilalaina; Hutter, Frank
Published in: International Conference on Automated Machine learning (Workshop Track), 2024
Publisher: openreview.net

Joint Entropy Search For Maximally-Informed Bayesian Optimization

Author(s): Hvarfner, Carl; Hutter, Frank; Nardi, Luigi
Published in: Advances in Neural Information Processing Systems, Issue 35, 2022, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arXiv.2206.04771

Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimator

Author(s): Watanabe, Shuhei; Awad, Noor; Onishi, Masaki; Hutter, Frank
Published in: International Joint Conference on Artificial Intelligence, Issue 32, 2023, ISSN 1045-0823
Publisher: International Joint Conferences on Artificial Intelligence
DOI: 10.24963/ijcai.2023/487

Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks

Author(s): Adriaensen, Steven; Rakotoarison, Herilalaina; Müller, Samuel; Hutter, Frank
Published in: Advances in neural information processing systems, Issue 36, 2023, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2310.20447

LoRA-DARTS: Low Rank Adaptation for Differentiable Architecture Search

Author(s): Krishnakumar, Arjun; Jha, Abhash Kumar; Moradian, Shakiba; Rapp, Martin; Hutter, Frank
Published in: International Conference on Automated Machine learning (Workshop Track), 2024
Publisher: openreview.net

HPO-RL-Bench: A Zero-Cost Benchmark for HPO in Reinforcement Learning

Author(s): Shala, Gresa; Arango, Sebastian Pineda; Biedenkapp, Andre; Hutter, Frank; Grabocka, Josif
Published in: International Conference on Automated Machine learning (ABCD Track), Issue 4, 2024
Publisher: openreview.net

GraViT-E: Gradient-based Vision Transformer Search with Entangled Weights

Author(s): Sukthanker, Rhea Sanjay; Krishnakumar, Arjun; Patil, Sharat; Hutter, Frank
Published in: Sixth Workshop on Meta-Learning at the Conference on Neural Information Processing Systems, 2022
Publisher: openreview.net

Fast Benchmarking of Asynchronous Multi-Fidelity Optimization on Zero-Cost Benchmarks

Author(s): Watanabe, Shuhei; Mallik, Neeratyoy; Bergman, Edward; Hutter, Frank
Published in: International Conference on Automated Machine Learning, Issue 4, 2024
Publisher: openreview.net
DOI: 10.48550/arXiv.2403.01888

Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars

Author(s): Schrodi, Simon; Stoll, Danny; Ru, Binxin; Sukthanker, Rhea; Brox, Thomas; Hutter, Frank
Published in: Advances in Neural Information Processing Systems, Issue 36, 2023, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2211.01842

HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models

Author(s): Sukthanker, Rhea Sanjay; Zela, Arber; Staffler, Benedikt; Klein, Aaron; Purucker, Lennart; Franke, Jorg K.H.; Hutter, Frank
Published in: Advances in Neural Information Processing Systems, Issue 38, 2024
Publisher: openreview.net

Towards Quantifying the Effect of Datasets for Benchmarking: A Look at Tabular Machine Learning

Author(s): Kohli, Ravin; Feurer, Matthias; Eggensperger, Katharina; Bischl, Bernd; Hutter, Frank
Published in: Data-centric Machine Learning Research (DMLR) Workshop at ICLR (2024), 2024
Publisher: openreview.net

Gray-Box Gaussian Processes for Automated Reinforcement Learning

Author(s): Shala, Gresa; Biedenkapp, André; Hutter, Frank; Grabocka, Josif
Published in: International Conference on Learning Representations, Issue 11, 2023
Publisher: openreview.net

c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization

Author(s): Watanabe, Shuhei; Hutter, Frank
Published in: International Joint Conference on Artificial Intelligence, Issue 32, 2023, ISSN 1045-0823
Publisher: International Joint Conferences on Artificial Intelligence
DOI: 10.24963/ijcai.2023/486

Ensembling Finetuned Language Models for Text Classification

Author(s): Arango, Sebastian Pineda; Janowski, Maciej; Purucker, Lennart; Zela, Arber; Hutter, Frank; Grabocka, Josif
Published in: Workshop on Fine-Tuning in Modern Machine Learning: Principles and Scalability at NeurIPS 2024, 2024
Publisher: openreview.net

Towards Generative RNA Design with Tertiary Interactions

Author(s): Sharat Patil and Runge, Frederic; Franke, Jorg K.H.; Hutter, Frank
Published in: Workshop on Generative and Experimental Perspectives for Biomolecular Design at ICLR 2024, 2024
Publisher: openreview.net

NOSBench-101: Towards Reproducible Neural Optimizer Search

Author(s): Karakasli, Goktug; Adriaensen, Steven; Hutter, Frank
Published in: International Conference on Automated Machine learning (Workshop Track), 2024
Publisher: openreview.net

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

Author(s): Hollmann, Noah; Muller, Samuel; Eggensperger, Katharina; Hutter, Frank
Published in: International Conference on Learning Representations, Issue 11, 2023
Publisher: openreview.net
DOI: 10.48550/arXiv.2207.01848

Beyond Graph-Based Modeling for Hierarchical Neural Architecture Search

Author(s): Birinxhiku, Lum; Stoll, Danny; Schrodi, Simon; Hutter, Frank
Published in: International Conference on Automated Machine learning (Workshop Track), 2024
Publisher: openreview.net

Is Mamba Capable of In-Context Learning?

Author(s): Grazzi, Riccardo; Siems, Julien Niklas; Schrodi, Simon; Brox, Thomas; Hutter, Frank
Published in: Workshop on Mathematical and Empirical Understanding of Foundation Models at ICLR 2024, 2024
Publisher: openreview.net

DeepCAVE: An Interactive Analysis Tool for Automated Machine Learning

Author(s): Sass, René; Bergman, Eddie; Biedenkapp, André; Hutter, Frank; Lindauer, Marius
Published in: Workshop on Adaptive Experimental Design and Active Learning in the Real World at ICML 20222, 2022
Publisher: PMLR
DOI: 10.48550/arxiv.2206.03493

Self-Correcting Bayesian Optimization through Bayesian Active Learning

Author(s): Hvarfner, Carl; Hellsten, Erik; Hutter, Frank; Nardi, Luigi
Published in: Advances in neural information processing systems, Issue 36, 2023, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arXiv.2304.11005

PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning

Author(s): Mallik, Neeratyoy; Bergman, Edward; Hvarfner, Carl; Stoll, Danny; Janowski, Maciej; Lindauer, Marius; Nardi, Luigi; Hutter, Frank
Published in: Advances in Neural Information Processing Systems, Issue 36, 2023, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2306.12370

Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data

Author(s): Helli, Kai; Schnurr, David; Hollmann, Noah; Müller, Samuel; Hutter, Frank
Published in: Advances in Neural Information Processing Systems, Issue 38, 2024
Publisher: openreview.net

On the Importance of Hyperparameters and Data Augmentation for Self-Supervised Learning

Author(s): Wagner, Diane; Ferreira, Fabio; Stoll, Danny; Schirrmeister, Robin Tibor; Müller, Samuel; Hutter, Frank
Published in: First Workshop of Pre-training: Perspectives, Pitfalls, and Paths at ICML 2022, 2022
Publisher: PMLR
DOI: 10.48550/arxiv.2207.07875

Improving Deep Learning Optimization through Constrained Parameter Regularization

Author(s): Franke, Jörg; Hefenbrock, Michael; Koehler, Gregor; Hutter, Frank
Published in: Advances in Neural Information Processing Systems, Issue 38, 2024
Publisher: openreview.net

Large Language Model Compression with Neural Architecture Search

Author(s): Sukthanker, Rhea Sanjay; Staffler, Benedikt; Hutter, Frank; Klein, Aaron
Published in: Workshop on Machine Learning and Compression at NeurIPS 2024, 2024
Publisher: openreview.net

RNA-Protein Interaction Classification via Sequence Embeddings

Author(s): Matus, Dominika; Runge, Frederic; Franke, Jorg K.H.; Gerne, Lars; Uhl, Michael; Hutter, Frank; Backofen, Rolf
Published in: Workshop on Generative and Experimental Perspectives for Biomolecular Design at ICLR 2024, 2024
Publisher: openreview.net

Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How

Author(s): Arango, Sebastian Pineda; Ferreira, Fabio; Kadra, Arlind; Hutter, Frank; Grabocka, Josif
Published in: International Conference on Learning Representations, Issue 11, 2024
Publisher: OpenReview.net
DOI: 10.48550/arxiv.2306.03828

The Tabular Foundation Model TabPFN Outperforms Specialized Time Series Forecasting Models Based on Simple Features

Author(s): Hoo, Shi Bin; Muller, Samuel; Salinas, David; Hutter, Frank
Published in: Workshop on Table Representation Learning at NeurIPS 2024, Issue 3, 2024
Publisher: openreview.net

Zero-Shot AutoML with Pretrained Models

Author(s): Öztürk, Ekrem; Ferreira, Fabio; Jomaa, Hadi S.; Schmidt-Thieme, Lars; Grabocka, Josif; Hutter, Frank
Published in: International Conference on Machine Learning, Issue 39, 2022, ISSN 2640-3498
Publisher: PMLR
DOI: 10.48550/arxiv.2206.08476

AutoRL-Bench 1.0

Author(s): Shala, Gresa; Arango, Sebastian Pineda ; Biedenkapp, André; Hutter, Frank; Grabocka, Josif
Published in: Workshop on Meta-Learning at NeurIPS 2022, Issue 6, 2023
Publisher: openreview.net

NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies

Author(s): Krishnakumar, Arjun; White, Colin; Zela, Arber; Tu, Renbo; Safari, Mahmoud; Hutter, Frank
Published in: Neural Information Processing Systems, Issue 35, 2022, ISSN 1049-5258
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2210.03230

GAMformer: Exploring In-Context Learning for Generalized Additive Models

Author(s): Mueller, Andreas C; Siems, Julien; Nori, Harsha; Salinas, David; Zela, Arber; Caruana, Rich; Hutter, Frank
Published in: Workshop Table Representation Learning at NeurIPS 2024, Issue 3, 2024
Publisher: openreview.net

In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization

Author(s): Rakotoarison, Herilalaina; Adriaensen, Steven; Mallik, Neeratyoy; Garibov, Samir; Bergman, Eddie; Hutter, Frank
Published in: International Conference on Machine Learning, Issue 41, 2024, ISSN 2640-3498
Publisher: PMLR
DOI: 10.48550/arXiv.2404.16795

JAHS-Bench-201: A Foundation For Research On Joint Architecture And Hyperparameter Search

Author(s): Bansal, Archit; Stoll, Danny; Janowski, Maciej; Zela, Arber; Hutter,Frank
Published in: Advances in Neural Information Processing Systems Datasets and Benchmarks Track, Issue 36, 2022, ISSN 1049-5258
Publisher: Curran Associates, Inc.

One-shot World Models Using a Transformer Trained on a Synthetic Prior

Author(s): Ferreira, Fabio; Schlageter, Moreno; Rajan, Raghu; Biedenkapp, Andre; Hutter, Frank
Published in: Workshop Open-World Agents at NeurIPS 2024, 2024
Publisher: openreview.net

New Horizons in Parameter Regularization: A Constraint Approach

Author(s): Franke, Jörg K. H.; Hefenbrock, Michael; Koehler, Gregor; Hutter, Frank
Published in: 15th Annual Workshop on Optimization for Machine Learning, 2023
Publisher: open review.net
DOI: 10.48550/arxiv.2311.09058

Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues

Author(s): Grazzi, Riccardo; Siems, Julien; Franke, Jorg K.H.; Zela, Arber; Hutter, Frank; Pontil, Massimiliano
Published in: Workshop on Mathematics of Modern Machine Learning at NeurIPS 2024, 2024
Publisher: openreview.net

Multi-objective Differentiable Neural Architecture Search

Author(s): Sukthanker, Rhea S.; Zela, Arber; Staffler, Benedikt; Dooley, Samuel; Grabocka, Josif; Hutter,Frank
Published in: Arxiv, 2024
Publisher: Arxiv

KinPFN: Bayesian Approximation of RNA Folding Kinetics using Prior-Data Fitted Networks

Author(s): Dominik Scheuer, Frederic Runge, Jörg K.H. Franke, Michael T. Wolfinger, Christoph Flamm, Frank Hutter
Published in: 2024
Publisher: Cold Spring Harbor Laboratory
DOI: 10.1101/2024.10.15.618378

Partial RNA Design

Author(s): Runge, Frederic; Franke, Jörg K.H.; Fertmann, Daniel; Backofen, Rolf; Hutter, Frank
Published in: Biorxiv, 2024
Publisher: Cold Spring Harbor Laboratory
DOI: 10.1101/2023.12.29.573656

Neural Architecture Search: Insights from 1000 Papers

Author(s): White, Colin; Safari, Mahmoud; Sukthanker, Rhea; Ru, Binxin; Elsken, Thomas; Zela, Arber; Dey, Debadeepta, Hutter, Frank
Published in: Arxiv, 2023
Publisher: arxiv
DOI: 10.48550/arXiv.2301.08727

Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML

Author(s): Weerts, Hilde; Pfisterer, Florian; Feurer, Matthias; Eggensperger, Katharina; Bergman, Edward; Awad, Noor; Vanschoren, Joaquin; Pechenizkiy, Mykola; Bischl, Bernd; Hutter, Frank
Published in: Journal of Artificial Intelligence Research, Issue 79, 2024, ISSN 1076-9757
Publisher: Morgan Kaufmann Publishers, Inc.
DOI: 10.1613/jair.1.14747

Publications

Share this page Share this page on social networks

Download Download the content of the page