End of input counts as one more character read, as it did before 3.8: a rule matching s, eof beats an earlier rule matching s. 3.8 let the earlier rule win, so a lexer with a rule matching the empty string before its eof rule looped forever at end of input (#223)
3.8 (2026-10-01)
New features
Named capture groups: (r as x) binds x in the action to a Sedlexing.submatch, extracted with Sedlexing.Utf8.of_submatch, Latin1.of_submatch, Utf16.of_submatch or lexeme_of_submatch. Both branches of an or-pattern must bind the same names, and as is not allowed under Star, Plus, Opt, Rep, Compl, Sub or Intersect. Submatches are chosen leftmost-greedy within the longest match, using a tagged DFA; a boundary at a fixed offset from the start or end of the lexeme costs nothing at runtime. A submatch is only valid inside its action (#177, #178, #204, #208, #209, #212)
Nested let .. in definitions of [%sedlex.regexp?] (#176)
Unicode 18.0.0 (#205)
Compl, Sub and Intersect accept Rep (c, 1 .. 1) (#201)
Clearer errors for a bare Star, Plus, Opt, Utf8, Latin1 or Ascii and for unknown operators (#196)
Bug fixes
Rule priority with eof: a rule matching s, eof no longer beats an earlier rule matching s (#211)
End of input is read once: Plus eof and Star ('a' | eof) no longer loop forever at end of input, and eof, eof no longer matches (#211)
Empty character sets (Chars "", Compl any, Sub or Intersect of coinciding classes) are a compile error instead of crashing the ppx or producing a rule that never matches (#201)
Rule sets where every rule is nullable, such as a lone "", no longer generate invalid code (#201)
UTF-8 decoding: the surrogate range rejected by Utf8.Helper.check_three ended at U+DF00 instead of U+DFFF (#184)
Sedlexing.sub_lexeme and its Latin1, Utf8 and Utf16 versions raise Invalid_argument when the range is not within the lexeme, instead of returning code points of the rest of the buffer or, for a negative length, an empty string (#219)
Performance
Faster lexeme, sub_lexeme and of_submatch in Sedlexing.Latin1, Sedlexing.Utf8 and Sedlexing.Utf16 (#219)
The memory cells of as bindings are no longer cleared at the start of every token (#220)
Other changes
Require OCaml 4.11, the oldest compiler still in opam-repository
3.7 (2025-10-06)
Update to unicode 17.0.0
3.6 (2025-01-05)
Fixed one of the ranges implementing Implement Corrigendum #1: UTF-8 Shortest Form for 4-bytes long characters (#171)
3.5 (2025-05-29)
Implement Corrigendum #1: UTF-8 Shortest Form
Add utf8 support for string literal (#127)
3.4 (2025-03-28)
Make the library compatibility with ppxlib.0.36 (#166)
3.3 (2024-10-29)
Add support for unicode 16.0.0 (#157)
Add API for retrieving start and stop positions separately (#155)
3.2 (2023-06-28):
Restore compatibility with OCaml 4.08
Use Sedlexing.{Utf8,Utf16}.from_gen to initialize UTF8 (resp. UTF16) lexing buffers from string.
Delay raising Malformed until actually reading the malformed part of the imput. (#140)
Count lines in all cases (#130). Previously, certain functions for initiating the lexical buffer would disable lines counting.
Check and fix invariants from Cset. The codebase was not respecting invariants documented in the Cset module which could break code relying on it. The code generated by sedlex.ppx could be affected.
Do not rely on comments from unicode UCD files
Add API to track position in bytes. Should be opt-in and backward compatible. (#146)