simple-tokenizer vocabulary
Factor documentation > Factor handbook > Vocabulary index


Summary
simple-tokenizer vocabulary

Meta-data
Tags:text
Authors:Slava Pestov


Words

Ordinary words
WordStack effect
tokenize( input -- ast )


Files
resource:basis/simple-tokenizer/simple-tokenizer.factor
resource:basis/simple-tokenizer/simple-tokenizer-docs.factor
resource:basis/simple-tokenizer/simple-tokenizer-tests.factor