Summary
Metadata
Words
Ordinary words
Files
Metadata
| Parents: | robots |
Words
Ordinary words
| Word | Stack effect |
 <rules> | ( -- rules ) |
| >robots.txt-url | ( url -- url' ) |
| add-allow | ( rules allow -- rules ) |
| add-disallow | ( rules disallow -- rules ) |
| add-user-agent | ( rules agent -- rules ) |
| derive-urls | ( url seq -- seq' ) |
| get-robots.txt | ( url -- robots.txt ) |
| normalize-robots.txt | ( string -- sitemaps seq ) |
| parse-robots.txt-line | ( rules seq -- rules ) |
| relevant-rules | ( robots -- rules ) |
| robot-rules-quot | ( robots -- quot ) |
Files