simple-tokenizer vocabulary
Factor handbook > Vocabulary index


Summary
Simple string tokenizer

Meta-data
Tags:text
Authors:Slava Pestov


Words

Ordinary words
WordStack effect
tokenize( input -- ast )


Files