User-agent: *
Crawl-Delay: 5
Disallow: /rss
Disallow: /search
Disallow: /nanna
Disallow /record/*/export/*$
Sitemap: https://lawcat.berkeley.edu/sitemap_index.xml.gz
User-agent: Yandex
User-agent: YandexBot
Crawl-Delay: 5
Disallow: /