<spider> ( base -- spider )
Spider

Prev:	Spider tutorial
Next:	run-spider ( spider -- spider )

Vocabulary
spider

Inputs

base	a string or url

Outputs

spider

a spider

Word description
Creates a new web spider with a given base url.

Definition

USING: accessors concurrency.semaphores kernel
spider.unique-deque urls ;

IN: spider

: <spider> ( base -- spider )
    >url spider new over >>base over >>currently-spidering
    swap 0 <unique-deque> [ push-url ] keep >>todo
    <unique-deque> >>nonmatching H{ } clone >>spidered
    1 <semaphore> >>semaphore ;