tokenize-line ( line-context line rules -- line-context' seq )

Vocabulary

xmode.marker

Inputs

line-context	an object
line	an object
rules	an object

Outputs

line-context'	an object
seq	an object

Definition

USING: assocs kernel make namespaces xmode.marker.state ;

IN: xmode.marker

: tokenize-line
    ( line-context line rules -- line-context' seq )
    [
        "MAIN" of -rot init-token-marker
        mark-token-loop mark-remaining unwind-no-line-break
        context get
    ] { } make ;