Mica::Core::TextInput

ByteInput for text formats whose input may arrive in an encoding other than UTF-8: the scanner always sees UTF-8, while positions still refer to the original bytes.

A format starts transcoding either up front (after a byte order mark, with setup_transcoded_stream) or mid-stream (after an in-band declaration, with switch_to_transcoder). Invalid code units reach the scanner as Transcoder marker bytes, which are never valid UTF-8; the scanner reports them as Mica::EncodingError (see describe_transcoder_marker) at an offset source_offset_at maps back to the original input.

The including type calls account_transcoded_discard(keep) from its discarding_input hook, and defines source_read_returned, which runs after every read from the source under a transcoder (where to check a deadline).