class

Obelisk::CoalescingIterator

Inherits Iterator < Enumerable < Reference < Object

Iterator that coalesces consecutive tokens of the same type This improves performance by reducing the number of token objects

NOTE: This iterator is currently DISABLED due to a Crystal compiler bug. When SafeTokenIteratorAdapter instances created via lexer.tokenize() on lexers stored in the Registry Hash are wrapped in CoalescingIterator, calling next() causes an EXC_BREAKPOINT crash.

The bug only occurs when:

  1. Lexer is stored in a Hash (like the Registry)
  2. SafeTokenIteratorAdapter is created via lexer.tokenize() method
  3. The iterator is wrapped in another iterator (like CoalescingIterator)
  4. next() is called on the wrapper

Workaround: Use SafeTokenIteratorAdapter directly (already done by default)

Constructors

new(source : TokenIterator, max_size : Int32 | Nil = nil)
Source

Class methods

wrap(source : TokenIterator, max_size : Int32 | Nil = 4096) : TokenIterator

Static helper to wrap any iterator with coalescing Default max_size of 4KB prevents unbounded growth while still providing significant performance benefits for typical code

NOTE: Currently disabled due to a possible Crystal compiler bug. To enable coalescing for testing/debugging, comment out the source line and uncomment the CoalescingIterator.new line below.

Source

Instance methods

next

Returns the next element in this iterator, or Iterator::Stop::INSTANCE if there are no more elements.

Source