Diminishing returns of data in sparse autoencoders: functional behavior does not guarantee feature reproducibility
Read original on arXiv cs.LG ↗arXiv:2610.20680v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary....
Original headline: "Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility"
Timeline
- October 16, 2026 Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility arXiv cs.LG