perf(parser): reduce Token size to 8 bytes from 16

[branchseer]

• Replace end: u32 with len: u16. Ends of long tokens (which are rare) are stored in lexer.long_token_ends;
• Pack bools into bitflags;
• Now that end is calculated from start + len, start must be properly set. In some places they were not. This PR fixes them and adds a debug-assertion check in the lexer.

[graphite-app]

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

[codspeed-hq]

CodSpeed Performance Report

Merging #8153 will degrade performances by 13.21%

_{Comparing branchseer:token_eight_bytes (93f4f1a) with main (5234d96)}

Summary

❌ 5 regressions
✅ 24 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	branchseer:token_eight_bytes	Change
❌	lexer[RadixUIAdoptionSection.jsx]	24 µs	26.4 µs	-9.34%
❌	lexer[antd.js]	22.3 ms	25.3 ms	-11.98%
❌	lexer[cal.com.tsx]	5.5 ms	6.3 ms	-12.27%
❌	lexer[checker.ts]	13.2 ms	14.8 ms	-10.59%
❌	lexer[pdf.mjs]	3.6 ms	4.1 ms	-13.21%