Groundbreaking work, congrats to the team!! 🎉 When I started my PhD 3 years ago, our tabular benchmark showed tree-based models miles ahead of neural networks. On the same benchmark, TabPFN v2 now reaches in 10s what CatBoost achieves in 4h of tuning 🤯

Samuel Müller@sammuller.bsky.social · 2y ago

This might be the first time after 10 years that boosted trees are not the best default choice when working with data in tables. Instead a pre-trained neural network is, the new TabPFN, as we just published in Nature 🎉

Can deep learning finally compete with boosted trees on tabular data? 🌲 In our NeurIPS 2024 paper, we introduce RealMLP, a NN with improvements in all areas and meta-learned default parameters. Some insights about RealMLP and other models on large benchmarks (>200 datasets): 🧵