As a counterpoint, I asked Claude to write a script to fetch Claude usage and expose it as a Prometheus metric. As no public API exists, Claude suggested I grab the request from the Network tab. I copied it as cURL, and attempted to run it, and was denied with a 403 from CF.
I forgot the script open, polling for about 20 minutes, and suddenly it started working.
So even sending all the same headers as Firefox, but with cURL, CF seemed to detect automated access, and then eventually allowed it through anyway after it saw I was only polling once a minute. I found this rather impressive. Are they using subtle timings? Does cURL have an easy-to-spot fingerprint outside of its headers?
Reminded me of this attack, where they can detect when a script is running under "curl | sh" and serve alternate code versus when it is read in the browser: https://news.ycombinator.com/item?id=17636032
> Does cURL have an easy-to-spot fingerprint outside of its headers?
If it's a https URL: Yes, the TLS handshake. There are curl builds[1] which try (and succeed) to imitate the TLS handshake (and settings for HTTP/2) of a normal browser, though.
I forgot the script open, polling for about 20 minutes, and suddenly it started working.
So even sending all the same headers as Firefox, but with cURL, CF seemed to detect automated access, and then eventually allowed it through anyway after it saw I was only polling once a minute. I found this rather impressive. Are they using subtle timings? Does cURL have an easy-to-spot fingerprint outside of its headers?
Reminded me of this attack, where they can detect when a script is running under "curl | sh" and serve alternate code versus when it is read in the browser: https://news.ycombinator.com/item?id=17636032