parse-tokens ( end -- seq )
Factor handbook » The language » Parsing words » Parsing raw tokens

Prev:

map-tokens ( ... end quot: ( ... token -- ... elt ) -- ... seq )

Vocabulary
lexer

Inputs

end

a string

Outputs

seq	a sequence of strings

Word description
Reads a sequence of tokens until the first occurrence of end. The tokens remain as strings and are not processed in any way. This word is equivalent to map-tokens with an empty quotation.

Notes
This word should only be called from parsing words.

See also
each-token, map-tokens, (parse-until), parse-until

Definition

IN: lexer

: parse-tokens ( end -- seq ) [ ] map-tokens ;