Authors: | Doug Coleman |
Class | Superclass | Slots |
robots | tuple | site sitemap rules rules-quot |
rules | tuple | user-agents allows disallows visit-time request-rate crawl-delay unknowns |
robot-identities |
Word | Stack effect |
<robots> | ( site sitemap rules -- robots ) |
parse-robots.txt | ( string -- sitemaps rules-seq ) |
url>robots | ( url -- robots ) |
Word | Stack effect |
robots? | ( object -- ? ) |
rules? | ( object -- ? ) |
Vocabulary | Summary |
robots.private |