Vocabulary
lexer

Inputs and outputs

end	a string
quot	a quotation with stack effect ( ... token -- ... elt )
seq	a new sequence of objects

Word description
Reads a sequence of tokens until the first occurrence of end. quot is called on each token as it is read, and the results are collected into a new output sequence.

Notes
This word should only be called from parsing words.

See also
parse-tokens, each-token, (parse-until), parse-until

Definition

USING: kernel sequences ;

IN: lexer

: map-tokens
( ... end quot: ( ... token -- ... elt ) -- ... seq )
collector [ each-token ] dip { } like ; inline