High-performance tokenizer for C0DATA.
Scans a byte buffer for control codes (< 0x20) and emits tokens as offsets into the original buffer. No allocation for data values.
High-performance tokenizer for C0DATA.
Scans a byte buffer for control codes (< 0x20) and emits tokens as offsets into the original buffer. No allocation for data values.