This page collects the raw data behind our tests so you can check our numbers or reuse them. Every file is dated and described, and nothing is estimated.
Datasets
| File | What it contains | Date |
|---|---|---|
| resultats.csv | One row per site and bot: allowed or blocked, whether the bot is named in robots.txt, llms.txt present | 29 Oct 2026 |
| stats.csv | Totals per bot over the 86 analysable sites | 29 Oct 2026 |
| sites.txt | The 100 domains tested | 29 Oct 2026 |
| logs.csv | Requests per AI bot in our own server log, with IP-list verification where an official list exists | 29 Oct 2026 |
| audit.py | The script used for the robots.txt audit | 29 Oct 2026 |
| citations.csv | Citation test results (10 fixed questions) | Empty: not run yet |
Column names in the CSV files are in French (domaine, statut, bot) because the scripts were written that way. Values: autorise means allowed, bloque means blocked, non_recuperable means robots.txt could not be read.
Reading the server-log file
The log covers a single day because the site launched on 29 October 2026. "IP verified" counts requests whose IP address falls inside the list the vendor publishes (OpenAI and Perplexity publish such lists). For bots without a list we checked, the row says "user-agent only": the label in the request could be spoofed.
The citation test
The citation test uses ten fixed questions, asked in each engine, with the cited domains recorded by hand. The file citations.csv is empty on purpose until the tests have been run. Guides show "Citation tests: not run yet" until then.
How to cite this data
Please link to this page and mention the test date. See the audit write-up for method and limits, and the methodology for how we label evidence.