# TRACE

> Weekly benchmark of web scraping APIs (context.dev, Exa, Firecrawl, Keenable). Every Friday at 00:15 UTC a new fresh sample of about 2,500 never-tested web pages goes to all four providers with caching turned off. A provider-blind automated judge grades each response. Published by context.dev, which is also one of the providers tested.

Latest sample: 2026-11-07 UTC; run ID: scraping-bench-v6:2026-11-07; protocol: scraping-bench-v6; status: completed; scored for page access; 2527 pages.

## Pages accessed (95% confidence interval)
- 1. context.dev: 94.4% ±0.9
- 2 (tied). Firecrawl: 92.9% ±1.0
- 2 (tied). Keenable: 92.3% ±1.0
- 4. Exa: 84.9% ±1.3

## Key findings
- context.dev accessed the most pages, 94.4%. It led Firecrawl by 1.6 points across 2,527 shared pages, a statistically significant gap.
- context.dev leads in news. Keenable leads in jobs. The top two are statistically tied in 7 categories.
- Exa answers fastest at the median, 1.17 s. context.dev has the shortest slow tail: 95% of its requests finish within 10.8 s.
- 53 pages defeated every provider. Most came from jobs, Wikipedia links, and research pages.

## Latency (all requests, caching off)
- context.dev: median 2.06 s, p90 5.96 s, p95 10.76 s, p99 26.07 s, timeouts 0.0%
- Exa: median 1.17 s, p90 7.62 s, p95 22.62 s, p99 27.11 s, timeouts 0.0%
- Firecrawl: median 1.75 s, p90 10.35 s, p95 11.37 s, p99 27.09 s, timeouts 2.0%
- Keenable: median 1.32 s, p90 11.98 s, p95 19.00 s, p99 30.00 s, timeouts 2.3%

## By category
- News: context.dev 93.0%, Firecrawl 83.7%, Keenable 83.3%, Exa 84.7%; context.dev leads
- Commerce: context.dev 99.3%, Firecrawl 98.3%, Keenable 99.0%, Exa 79.0%; statistically tied
- Jobs: context.dev 90.7%, Firecrawl 89.0%, Keenable 95.7%, Exa 80.0%; Keenable leads
- Finance: context.dev 100.0%, Firecrawl 99.3%, Keenable 99.3%, Exa 99.3%; statistically tied
- Research: context.dev 87.0%, Firecrawl 89.3%, Keenable 80.3%, Exa 81.3%; statistically tied
- Legal & government: context.dev 96.7%, Firecrawl 95.7%, Keenable 92.3%, Exa 70.0%; statistically tied
- Software & docs: context.dev 98.4%, Firecrawl 100.0%, Keenable 100.0%, Exa 97.6%; statistically tied
- Community: context.dev 94.7%, Firecrawl 91.3%, Keenable 94.7%, Exa 94.0%; statistically tied
- Wikipedia links: context.dev 90.0%, Firecrawl 89.0%, Keenable 86.3%, Exa 78.0%; statistically tied

## Cost per 1,000 accessed pages (published prices)
- context.dev: $0.79
- Exa: $1.18 (assumes failed requests are billed)
- Firecrawl: $1.07 (assumes failed requests are billed)
- Keenable: $4.00

## Questions
### Which web scraping API has the highest success rate?
In the Nov 7, 2026 sample, context.dev accessed 94.4% of 2,527 pages, followed by Firecrawl at 92.9%, Keenable at 92.3%, Exa at 84.9%. Its lead over the next provider is statistically significant.

### Which web scraping API is fastest?
Exa had the fastest median response, 1.17 seconds. context.dev had the shortest slow tail: 95% of its requests finished within 10.8 seconds. Times are measured with caching turned off.

### Does the benchmark use cached pages?
No. Every request asks for a fresh fetch, and we check each response for the provider's own cache marker. Responses served from cache are left out of the scores and latency.

### How often is the benchmark updated?
A new sample of about 2,500 never-tested pages is drawn every Friday at 00:15 UTC and sent to all four providers. Its scores fill in as collection progresses.

### What does a success mean?
Each sample states its scoring rule. Under page access, a page succeeds when actual page data loaded rather than a block, login screen, error, or empty shell. Under useful content, the response must also contain substantive content from the requested page, a stricter test, so compare scores only between samples that use the same rule.

### Can I reproduce the results?
Yes. Every page-level record is downloadable, and the collector, judge prompt, and request settings are open source.

## Links
- [Full report](https://trace.context.dev/)
- [Run summaries API](https://trace.context.dev/api/report)
- [Page-level records for 2026-11-07](https://trace.context.dev/api/download?run=scraping-bench-v6%3A2026-11-07)
- [Source code](https://github.com/context-dot-dev/trace-benchmark)