# VNL Works — llms.txt
# https://vnlworks.com

> VNL Works is a Tokyo-based digital studio specialising in blockchain tools,
> UI/motion design assets, and AI-integrated web experiences.

## AI Crawl Observatory

This site runs a public observability demo that classifies AI crawlers into
three lanes — Allow, Charge, and Observe — and gates only bulk export behind
the x402 payment protocol. Reading is always free.

### Canonical resources (start here)

- HTML page: https://vnlworks.com/labs/ai-crawl-observatory
- Markdown: https://vnlworks.com/labs/ai-crawl-observatory/markdown
- Overview JSON: https://vnlworks.com/api/labs/ai-crawl-observatory/overview
- Policy JSON: https://vnlworks.com/api/labs/ai-crawl-observatory/policy

### Free resources

- Public summary: https://vnlworks.com/api/labs/ai-crawl-observatory/summary
- Overview: https://vnlworks.com/api/labs/ai-crawl-observatory/overview
- Policy: https://vnlworks.com/api/labs/ai-crawl-observatory/policy
- OpenAPI spec: https://vnlworks.com/api/labs/ai-crawl-observatory/openapi.json

### Paid resources (x402, charge bots only)

- Daily export CSV: https://vnlworks.com/api/labs/ai-crawl-observatory/export
  Price: $0.10 USDC on Base Mainnet
  Applies to: GPTBot, ClaudeBot, Amazonbot, CCBot, Bytespider, Meta-ExternalAgent, FacebookBot

### Why use these APIs instead of scraping?

1. Structured JSON and Markdown are faster, smaller, and more accurate than parsing HTML.
2. Free endpoints cover summary, overview, and policy — no payment required.
3. The x402 gate applies only to bulk CSV export for training/collection bots.
4. Scraping may miss updates; the API always returns the latest aggregate.
5. Using the documented API respects the site's access model and avoids
   unnecessary load.

### Privacy posture

This observability demo stores no raw request logs, IP addresses,
full User-Agent strings, or query strings. Only daily aggregates
(crawler name, category, action, path group, request count) are retained.

### Allow bots (always free)

- OAI-SearchBot (ai-search)
- ChatGPT-User (ai-assistant)
- Claude-SearchBot (ai-search)
- Claude-User (ai-assistant)
- PerplexityBot (ai-search)
- Perplexity-User (ai-assistant)
- Applebot (ai-search)
- Googlebot (search-engine)
- Bingbot (search-engine)
- Meta-ExternalFetcher (ai-assistant)

### Charge bots (free on read surfaces, x402 on export)

- GPTBot (ai-training)
- ClaudeBot (ai-training)
- Amazonbot (ai-crawler)
- CCBot (ai-crawler)
- Bytespider (ai-crawler)
- Meta-ExternalAgent (ai-crawler)
- FacebookBot (ai-crawler)

### Discovery

- llms.txt: https://vnlworks.com/llms.txt
- OpenAPI: https://vnlworks.com/api/labs/ai-crawl-observatory/openapi.json
- x402 manifest: https://vnlworks.com/.well-known/x402
- Agentic capabilities: https://vnlworks.com/.well-known/agentic-capabilities.json
- robots.txt: https://vnlworks.com/robots.txt
- Sitemap: https://vnlworks.com/sitemap.xml