Amazon releases Mitra-v2 tabular foundation model with scaled synthetic pretraining
Mitra-v2: Scaling synthetic pretraining for tabular foundation models
Amazon introduced Mitra-v2, a second-generation tabular foundation model for classification and regression, released with fine-tuned models and inference and fine-tuning code under the Apache-2.0 license on Hugging Face. It is pretrained on synthetic tables with 10 times as many rows and roughly three times as many columns as Mitra-v1, using a new hybrid SCM prior and hybrid optimizer while keeping the Transformer architecture unchanged. On TabArena, a public benchmark of 51 real-world datasets, Mitra-v2 leads the single-model field, beats Mitra-v1 on 32 of the 33 datasets both run natively, and finishes all 51 within the one-hour budget; fine-tuned Mitra-v2 also ranks first on ScoringBench.
Mitra-v2 shows that widening synthetic pretraining data, rather than deepening the network, lifts a tabular foundation model to the top of TabArena's single-model field.
Source:Amazon Science · amazon.science