simple-tokenizer vocabulary - Factor Documentation

simple-tokenizer vocabulary
Factor documentation > Factor handbook > Vocabulary index

Summary
simple-tokenizer vocabulary

Meta-data

Tags:	text
Authors:	Slava Pestov

Words

Ordinary words

Word	Stack effect
tokenize	( input -- ast )

Files

resource:basis/simple-tokenizer/simple-tokenizer.factor
resource:basis/simple-tokenizer/simple-tokenizer-docs.factor
resource:basis/simple-tokenizer/simple-tokenizer-tests.factor