module

Fetcher::URLValidator

Constants

ALLOWED_SCHEMES = {"http", "https"}
BLOCKED_IPV4_RANGES = [{167772160_u32, 8, "10.0.0.0/8 (RFC 1918 private)"}, {2886729728_u32, 12, "172.16.0.0/12 (RFC 1918 private)"}, {3232235520_u32, 16, "192.168.0.0/16 (RFC 1918 private)"}, {2851995648_u32, 16, "169.254.0.0/16 (link-local)"}, {1681915904_u32, 10, "100.64.0.0/10 (CGNAT, RFC 6598)"}, {3323068416_u32, 15, "198.18.0.0/15 (benchmark, RFC 2544)"}, {3758096384_u32, 4, "224.0.0.0/4 (multicast)"}, {4026531840_u32, 4, "240.0.0.0/4 (reserved)"}, {0_u32, 8, "0.0.0.0/8 (current network)"}, {3221225984_u32, 24, "192.0.2.0/24 (TEST-NET-1, RFC 5737)"}, {3325256704_u32, 24, "198.51.100.0/24 (TEST-NET-2, RFC 5737)"}, {3405803776_u32, 24, "203.0.113.0/24 (TEST-NET-3, RFC 5737)"}]

IPv4 CIDR ranges blocked for SSRF protection. Format: {prefix_as_uint32, prefix_length, description}

DANGEROUS_SCHEMES = {"javascript", "vbscript", "data", "file", "ftp", "jar", "mailto"}
DNS_RESOLVE_PORT = 80

DNS rebinding mitigation: track recently validated hostnames and their IPs

IPV4_OR_HEX = /^[0-9a-fA-F.:]+$/
MAX_URL_LENGTH = 2048
SAFE_SCHEMES = {"http", "https"}

Class methods

check_rebinding(host : String, current_ip : Socket::IPAddress) : Bool
Source
clear_validated
Source
default_service
Source
extract_domain(url : String) : String

Extract host/domain from a URL string. Returns "default" on error.

Source
looks_like_ip?(host : String) : Bool
Source
purge_expired(expiry : Time::Span | Nil = nil) : Nil
Source
register_ip(host : String, ip : Socket::IPAddress) : Nil
Source
resolve_and_validate(url : String) : Bool
Source
safe_scheme?(url : String | Nil) : Bool

Permissive scheme check: returns true for nil/empty, relative URLs (no colon), and http/https URLs. Returns false only for explicitly dangerous schemes (javascript:, data:, vbscript:, file:, etc.).

This is intentionally more permissive than valid?, which requires a full http/https URL with a valid host. safe_scheme? is used for secondary URLs (comment URLs, author URLs, attachment URLs) that may legitimately be relative paths like "/comments/123".

Both methods block dangerous schemes -- safe_scheme? via the DANGEROUS_SCHEMES blacklist, valid? via the ALLOWED_SCHEMES whitelist. The defense-in-depth is: secondary URLs go through safe_scheme? (permissive), primary URLs go through valid? (strict).

Source
safe_url(url : String | Nil) : String
Source
valid?(url : String | Nil) : Bool
Source
valid_redirect?(redirect_url : String) : Bool
Source
validate_connected_ip(host : String, connected_ip : Socket::IPAddress) : Bool
Source
validated_store

Backwards-compatible store. We encapsulate the mutable cache in ValidatedIpStore but keep a class-level default instance so existing call-sites remain functional without signature changes.

Source

Nested types