JurisetBot

JurisetBot fetches official government pages that state employment rules (minimum wage, overtime, termination notice, paid leave) so that Juriset can detect when they change. It identifies itself as JurisetBot/0.1 (+https://juriset.dev/crawler; [email protected]).

How it behaves

• It reads robots.txt before every request and follows it, including Crawl-delay.
• The text of the pages it fetches is processed by an AI model (Anthropic's Claude) to extract the rules, and a second model from a different vendor checks each result against the page before it is published; a person reviews anything they disagree on. It does not fetch pages whose robots.txt carries a Content-Signal of ai-input=no, and no page is used to train a model.
• It waits at least 10 seconds between requests to the same host.
• It fetches each page about once a week (daily in the weeks around January 1, July 1 and October 1) and sends conditional requests, so unchanged pages cost a 304.
• It never tries to get past CAPTCHAs, bot protection or access controls.

What we keep

We keep a copy of each page we use as evidence of what it said when a rule was published, and we do not delete it. Copies are public records of the issuing authority, kept for verification and audit and offered as downloads to people checking a rule; we do not republish them as pages or sell them as a collection. We fetch only pages on a list that was checked against the live pages before they were added, and nothing beyond it.

Contact

To ask us to fetch less often, to stop, or to use a different official source, email [email protected]. To block it, add User-agent: JurisetBot with Disallow: / to your robots.txt.

More about Juriset: what it is and is not, the terms and the privacy policy.