Most LLM environmental reporting covers only the final pretraining runs. For Olmo 3, we measured every stage across all four variants: 7B and 32B, instruct and reasoning, and found that 82% of the compute went to development, all before the final runs 😱
Ai2@ai2.bsky.social · 3mo agoOur research estimates that in today’s model training efforts, 82% of compute goes into exploratory work. At closed labs, the output of that work stays within those labs. In an open system, models, datasets, & methods are shared, and the value compounds across the field.
