Engineering ·
Post says a Meta paper proposes tokenizer-free byte-level models
A post claims Meta's paper 'Breaking the Token Ceiling' trains models on raw bytes with a 256-value vocabulary and applies distillation. According to the post, 1B byte models match or beat two cited 1B models. No link to the original work was provided.