class

Krikri::DifferentialFuzz::Runner

Inherits Reference < Object

Runs the SAME generated expression through both evaluators and compares them at the JSON-compact string level:

  • hand-rolled side: ExpressionEvaluator#evaluate (the entry point VarSubstitutor#substitute uses for a plain {{ }} span; lenient, "undefined" sentinel).
  • krikri-jinja side: JinjaRenderer#evaluate_value! (the delegation target, raw structured JSON::Any) rendered through the SAME VariableLookup#format_value convention the hand-rolled side uses for its own structured results, so container/bool/float spellings are compared under one convention instead of flooding on format.

Agreement: identical strings, OR both sides parse as numbers and are numerically equal (int-vs-float spellings like "3" vs "3.0" are formatting, not semantics). Both sides raising on the same input is agreement on invalid input. One side raising while the other returns a value, or differing values, is a disagreement - either a real bug or a documented deliberate limit, for the caller to classify.

Constants

FILTER_OPERAND_LENIENCE = [/has no length/, /is not iterable/, /cannot convert .+ to list/, /expects a number/, /No first item/, /No last item/, /sequence was empty/, /unsupported operand type\(s\) for \+/, /dictsort expects/, /is not callable/, /cannot interpret the precision/, /expects a mapping/]

Jinja error messages meaning "a FILTER received an operand/value of the wrong type" - the FilterEngine's per-filter argument leniency (| sum on strings, | list on a scalar, | abs on a string, | length on a bool, | first on an empty sequence, ...), which the hand-rolled side answers leniently. Deliberately out of scope for the strictness fixes (per-filter argument validation is its own project).

KNOWN_DIFFERENCES = [KnownDifference.new("hand-strict-undefined-base-index", ->(o : Outcome) do ( (o.status.one_errored? && ((o.hand_error || "").includes?("None has no element"))) && (o.jinja_value == "undefined") ) && (o.expr.includes?("[")) end), KnownDifference.new("hand-lenient-filter-operand", ->(o : Outcome) do if (o.status.one_errored? && o.hand_value.is_a?(String)) && (error = o.jinja_error).is_a?(String) FILTER_OPERAND_LENIENCE.any? do |pattern| error.matches?(pattern) end else false end end), KnownDifference.new("hand-lenient-nested-or-string-comparison", ->(o : Outcome) do if ( (o.status.one_errored? && o.hand_value.is_a?(String)) && (error = o.jinja_error).is_a?(String) ) && ((error.includes?("cannot compare")) || (error.includes?("has no element"))) if Runner.nested_condition_shape?(o.expr) true else error.matches?(/cannot compare (String and (Int64|Float64)|(Int64|Float64) and String|Bool and String|String and Bool)/) end else false end end), KnownDifference.new("hand-lenient-unimplemented-construct", ->(o : Outcome) do if (o.status.one_errored? && o.hand_value.is_a?(String)) && (error = o.jinja_error).is_a?(String) ( ( ( ( ((/unsupported operand|cannot negate|not all arguments converted/).matches?(error)) || ((/integer division|division by zero/).matches?(error)) ) || ((/unexpected token|expected "/).matches?(error)) ) || ((/' is undefined|is undefined'/).matches?(error)) ) || ((/has no attribute|not JSON-compatible/).matches?(error)) ) || ((/Cast from .+ to String failed/).matches?(error)) else false end end)]

Constructors

new(vars : Hash(String, JSON::Any) = Fixtures.build)
Source

Class methods

nested_condition_shape?(expr : String) : Bool

Whether expr is a nested/compound shape rather than a bare comparison: ternary conditions, and/or value selectors, is tests, not-prefixed conditions, or filter-piped operands. Those route through fallback evaluators (ConditionalEvaluator, the heuristic not (...)/lookup fallbacks) whose comparison handling is still class-lenient.

Source

Instance methods

classify_known_differences=(classify_known_differences : Bool)

When false, triaged known differences are reported as findings too (the unfiltered strict view).

Source
classify_known_differences?

When false, triaged known differences are reported as findings too (the unfiltered strict view).

Source
run(expr : String) : Outcome
Source
vars
Source