Hwaro::Services::CharsetHandler
Inherits HTTP::Handler < Reference < Object
HTTP::StaticFileHandler derives Content-Type from the file
extension via MIME.from_extension and writes it without a
charset parameter — so robots.txt / llms.txt / sitemap and
search index responses all advertise no charset, leaving UTF-8
bytes (e.g. Korean LLM-instructions) at the mercy of client-side
heuristics.
The primary fix is Server.register_utf8_mime_types, which teaches the
MIME table the charset up front so StaticFileHandler writes the right
value at any response size. This handler stays as the safety net for
types we don't enumerate there and for handlers that set a bare
text-shaped Content-Type themselves — but note it can only work below
the response output buffer (IO::DEFAULT_BUFFER_SIZE, 32 KiB): past
that the headers are already on the wire when call_next returns.
We only touch types we know are text. Binary types (image/png,
font/woff2, etc.) are left alone — adding a charset there would
be wrong, not just useless. image/svg+xml is treated as text
because it's XML.