Constants
ALLOWED = {Kind::Style => STYLE, Kind::Script => SCRIPT, Kind::Image => IMAGE.merge(SVG), Kind::Media => MEDIA, Kind::Source => (IMAGE.merge(SVG)).merge(MEDIA), Kind::CssRef => (FONT.merge(IMAGE)).merge(STYLE), Kind::Preload => (((STYLE.merge(SCRIPT)).merge(FONT)).merge(IMAGE)).merge(MEDIA)}
ATTR_RE = /\s([a-zA-Z_:][-\w:.]*)(?:\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>`]+)))?/
CACHE_DIR = ".hwaro/external"
CSS_RE = /\/\*[\s\S]*?\*\/|((?:-webkit-)?image-set\((?:[^()]|\([^()]*\))*\))|url\(\s*(?:"([^"]*)"|'([^']*)'|([^)"'\s]*))\s*\)|@import\s*(?:"([^"]*)"|'([^']*)')/i
EXTENSION_ALIASES = {".jpeg" => ".jpg", ".mjs" => ".js"}
Spellings a URL path may use for an allowed extension.
FETCH_DEADLINE = 120.seconds
FONT = {"font/woff2" => ".woff2", "font/woff" => ".woff", "font/ttf" => ".ttf", "font/otf" => ".otf", "application/font-woff2" => ".woff2", "application/font-woff" => ".woff", "application/x-font-ttf" => ".ttf", "application/x-font-woff" => ".woff", "application/x-font-woff2" => ".woff2", "font/sfnt" => ".ttf", "application/font-sfnt" => ".ttf", "application/x-font-otf" => ".otf", "application/x-font-opentype" => ".otf", "font/collection" => ".ttc", "application/vnd.ms-fontobject" => ".eot"}
HTML_RE = /<!--[\s\S]*?-->|<(script|style|textarea)\b(#{Utils::TextUtils::HTML_TAG_ATTRS_QUOTED_VALUES})>([\s\S]*?)<\/\1\s*>|<(link|img|source|video|audio)\b(#{Utils::TextUtils::HTML_TAG_ATTRS_QUOTED_VALUES})>/i
Tag attributes are quote-aware: alt="a > b" does not end the tag.
IMAGE = {"image/png" => ".png", "image/jpeg" => ".jpg", "image/jpg" => ".jpg", "image/pjpeg" => ".jpg", "image/gif" => ".gif", "image/webp" => ".webp", "image/avif" => ".avif", "image/bmp" => ".bmp", "image/x-icon" => ".ico", "image/vnd.microsoft.icon" => ".ico"}
IMAGE_SET_REF_RE = /type\(\s*(?:"[^"]*"|'[^']*')\s*\)|url\(\s*(?:"([^"]*)"|'([^']*)'|([^)"'\s]*))\s*\)|"([^"]*)"|'([^']*)'/
A reference inside image-set(...): url(...) or a bare string.
A type("image/avif") argument is a MIME type, matched first so
it passes through untouched.
INDEX_FILE = "index.json"
LINK_RE = /<link\b#{Utils::TextUtils::HTML_TAG_ATTRS_QUOTED_VALUES}>/i
MAX_BYTES = (20_i64 * 1024) * 1024
MAX_DEPTH = 4
Stylesheet nesting followed from a page: the page's own <link> is
depth 0, what that file imports or references is depth 1, and so on.
Anything deeper keeps its absolute URL.
MEDIA = {"video/mp4" => ".mp4", "video/webm" => ".webm", "video/ogg" => ".ogv", "audio/mpeg" => ".mp3", "audio/ogg" => ".ogg", "audio/wav" => ".wav", "audio/mp4" => ".m4a", "audio/webm" => ".weba", "video/quicktime" => ".mov", "audio/flac" => ".flac", "audio/aac" => ".aac"}
NOT_A_FILE = ["text/html", "application/xhtml+xml", "text/xml", "application/xml", "application/json"]
Content-Types that mean "not the file the URL names".
RETRY_AFTER = 10.minutes
A URL that failed is not retried for this long by the same process
(a hwaro serve session's full rebuilds), so a dead host costs one
timeout, not one per rebuild.
SCRIPT = {"text/javascript" => ".js", "application/javascript" => ".js", "application/x-javascript" => ".js", "text/ecmascript" => ".js", "application/ecmascript" => ".js"}
SELF_LOCATING = {"https://cdn.jsdelivr.net/npm/mathjax@"}
Scripts that load their own siblings relative to their URL and
break when moved (Hwaro's math = "mathjax" tag): left external.
SRI_RE = /\s(?:integrity|crossorigin)(?:\s*=\s*(?:"[^"]*"|'[^']*'|[^\s"'<>`]+))?/i
STYLE = {"text/css" => ".css"}
SVG = {"image/svg+xml" => ".svg"}
SVG can carry script when opened directly, so it is published only
for <img>-like references, with a warning.
URL_ATTRS = {"link" => ["href"] of ::
String, "script" => ["src"] of ::
String, "img" => ["src", "srcset"] of ::
String, "source" => ["src", "srcset"] of ::
String, "video" => ["src", "poster"] of ::
String, "audio" => ["src"] of ::
String}
USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
Google Fonts picks the font format from the User-Agent: an unknown
agent gets TrueType, a modern browser gets woff2.