Documentation
Three ways to use FoxTrust. All of them give the same customer verdict: a risk from 0 to 100, a level and the reasons, each with its code, when it was last seen and how much it adds.
API
Look up one address per request over HTTPS. Keys, free-tier limits, answer fields and error codes.
Signed snapshots
The whole data set as MaxMind DB files, a delta every hour and a full file every day. How to download them, verify their signatures and read them with any MMDB reader.
/verify
A forward-auth service that runs next to your reverse proxy (nginx, Traefik or Caddy), reads the signed snapshots and applies your policy to each request: allow, challenge with a proof-of-work page instead of a captcha, or block. Its setup guide is being written for this site.
Analytics. Set FOXTRUST_ACCOUNT_KEY to a key from your account, and /verify sends counts of its decisions once a minute. Your account then shows them under Analytics: requests over time by action, the rules that fired, the challenge funnel, countries and networks, levels and reason codes.
- Reports carry no visitor address, path or header.
- Without a key nothing is sent.
- Unsent minutes wait up to 24 hours if FoxTrust cannot be reached, and decisions never wait for them.
/statusshows the reporting state.
Logs. /verify writes visitor addresses to its log as their /24 or /48 network.
Search crawlers
Search crawlers cannot solve a challenge, so a policy that challenges them takes your site out of search results. Search crawler addresses carry the category search_crawler: addresses in the ranges that search engines publish for their crawlers, and crawler addresses that FoxTrust confirms by reverse DNS for a search engine that publishes no list. It adds nothing to the risk; it only names the address. Put a rule that allows it before every other rule, whether you write your policy for /verify or apply the API or the snapshots yourself:
rules:
- name: search-crawlers
when:
categories: [search_crawler]
action: allow
# your other rules follow
Search crawlers in the address ranges that search engines publish for their crawlers, and crawler addresses that FoxTrust confirms by reverse DNS, are never challenged or blocked by a policy that puts the search_crawler rule first. This does not cover other crawler addresses, a list its search engine stopped updating, or search engines that FoxTrust does not check. policy check and the /status page of /verify warn when the rule is missing or comes after a rule on hosting, cloud or a level.
AI crawlers and agents
AI companies publish the addresses of their bots. FoxTrust marks those addresses by what the company says the bot is for. Like search_crawler, these categories add nothing to the risk; they only name the address:
ai_training: a crawler that collects content to train AI models;ai_search: a crawler or fetcher that serves AI search and answers;ai_user: an agent that fetches a page when a person asks it to.
The example policy has no rule for them, so they are treated like any other address until you decide. The first rule that matches an address decides, so the order of the rules matters. An address can carry more than one of these categories when its company uses it for several purposes. Keep the search crawler rule first in every recipe.
Refuse training, strictly. An address that is used for both training and AI search is blocked:
rules:
- name: search-crawlers
when:
categories: [search_crawler]
action: allow
- name: no-ai-training
when:
categories: [ai_training]
action: block
- name: ai-search
when:
categories: [ai_search]
action: allow
# your other rules follow
Prefer AI search. An address that is used for both is allowed:
rules:
- name: search-crawlers
when:
categories: [search_crawler]
action: allow
- name: ai-search
when:
categories: [ai_search]
action: allow
- name: no-ai-training
when:
categories: [ai_training]
action: block
# your other rules follow
To make agents acting for a person prove they are welcome, add a rule that challenges ai_user.
What these categories cannot do:
- Search engines also use their search crawlers for AI products, and those addresses carry
search_crawler, notai_training. Training cannot be refused by address without refusing search; robots.txt is the place for that. - Some companies publish one list for all of their bots. Their addresses carry all three categories, so refusing training also refuses their AI search and their agents.
- Agents in
ai_userfetch on a person's request, may ignore robots.txt, and can be pointed at any page by anyone with an account. - The categories cover only the addresses the companies declare. A company that also crawls from addresses it does not declare is not covered, and a request that only claims to be an AI bot in its user agent gets no category.
Ads and other special crawlers
A search engine also runs crawlers that are not search crawlers: they check the landing pages of ads, fetch pages for the ads shown on them, fetch for its APIs, and scan links for abuse. Their addresses carry the category ads_crawler, from the ranges the search engine publishes for them, confirmed by reverse DNS where the addresses have reverse names. It adds nothing to the risk and is not search_crawler: these crawlers ignore the rules that robots.txt gives to every crawler (User-agent: *) and follow only rules that name them, and the abuse scanner ignores robots.txt altogether.
The example policy has no rule for them. If you advertise and a challenge on your landing pages would get your ads disapproved, allow them after the search crawler rule:
rules:
- name: search-crawlers
when:
categories: [search_crawler]
action: allow
- name: ads-crawlers
when:
categories: [ads_crawler]
action: allow
# your other rules follow
Bots FoxTrust does not mark
Some bots that you may want to let through carry no category, because no published list identifies them by address:
-
Link previews. Chat and social apps fetch a page to show a preview when someone shares a link. None of them publishes addresses that belong only to its preview fetcher, so their addresses look like any other server. Keep the public pages people share free of challenges, or allow them by path.
-
Uptime monitoring. You know which monitoring service you use, and most publish the addresses of their probes. Allow your health-check path, or let the probes through at your proxy before the request reaches FoxTrust:
rules: - name: health-check when: path: "/health" action: allow # your other rules follow -
A search crawler whose operator forbids republishing its list is not marked today.
-
One search engine publishes a single list for its search crawler and its AI answers. Its addresses carry
ai_search, notsearch_crawler, because the list cannot tell the two apart and the addresses are shared cloud servers.
Your own address
My IP shows the address you are browsing from and what FoxTrust's data says about it, with each reason explained. For scripts, https://foxtrust.dev/ip answers with the address as plain text and https://foxtrust.dev/ip.json with {"ip": "…", "family": 4}; use curl -4 or curl -6 to choose the family. They need no key, give no verdict and take no parameters, and allow at most 60 requests a minute from one address.