Something really important thats going under the radar is TLS fingerprinting [1].
Multiple servers are using this now, including some requests to subdomains on google.com, googleapis.com, CloudFlare and others. I keep reporting this [2][3], and no one seems to care. If a server blacklists your client, whether its cURL or Go "net/http", you can no longer request to that server using that client. Period. Any HTTP client that wants to be robust, should be thinking about this.
This kind of flexibility is a non-goal of crypto/tls. We have a TLS stack with one of the best security track records because we implement an opinionated subset of the specification, amongst other things. Moreover, fingerprint evasion is a cat-and-mouse game we can't sustain in the six months Go release cycle.
That doesn't mean I don't care! I was just talking with a friend about this the other day, and I suggested it should be possible to make a small, easily maintained patch that focuses on chasing the fingerprint of one well-known browser. He implemented https://github.com/hellais/utls-light in that spirit, which looks like a viable solution to me.
Anyway, I think matching TLS fingerprints to HTTP User-Agent strings is a valid abuse prevention technique. Rejecting any non-browser fingerprint is bad, and websites should get pushback for that, but I am skeptical that's something they can reliably do without breaking any time Chrome flips a field study. TLS is not _that_ rusted shut.
I feel like this answers one particular possible endgoal rather than a problem.
The problem in this case being, the user not being able to specify and guarantee a cipher list.
If there's a good reason for not allowing it, it should be spelled out for dummies(like me). And left at that. If there's not a good reason, perhaps it should be considered.
Both the bug report and reply here instead argue about TLS fingerprinting, which while was brought up in the bug report, is not explicitly what's being reported as the problem.
The cipher list being out of the hands of the user is following a modern best practice in security API design: remove configuration at both protocol level (where possible), and from the end user, as they present constant sources of security problems.
For TLS in particular, to provide a safe, known secure set of defaults that the language can change with shifting guidance as cipher suites rotate in and out or are blacklisted for problems is necessary to meet the goals of usability without sacrificing on security. Adding user configuration is introducing a footgun, a place where the user can break the security properties of the system. In addition to not being free to implement, maintaining that configuration option, and then explaining how exposing it isn't a vulnerability everytime someone misconfigures their system and breaks TLS is undesirable and has the potential to give golang's crypto packages a bad name.
Note that nowhere was there a hard no, we'll never implement this, but so far, the use case for needing it is fingerprint bypassing, and that doesn't outweigh the general concerns.
If server blacklists your TLS fingerprint, you can no longer use that server with that client. Only workaround would be a client that can configure its own fingerprint. What you're saying is akin to "let remove the clients ability to set HTTP headers". IE its just nonsense. If server rejects every request because of clients TLS configuration, and client has no agency over that configuration, then the client has no usability.
> Note that nowhere was there a hard no, we'll never implement this
I would say the maintainer closing the issue, is a pretty clear signal for that:
The issue was closed for bypassing fingerprinting, a use case that the maintainers feel is not a valid precursor to this feature. Again, that isn’t a no to the feature under any circumstance, just a statement that this use case is insufficient.
More generally, if a server wants to fingerprint you, there are a variety of additional fields and changes that will flow from that as it’s an arms race, one that this libraries authors are not inclined to keep adding features and configuration for them. There exist other libraries for this purpose, some linked in the discussion. So no, it’s not like saying you can’t set your own HTTP headers at all, more akin to saying that if your needs fall outside the 99% use case you may need to go elsewhere.
> The issue was closed for bypassing fingerprinting, a use case that the maintainers feel is not a valid precursor to this feature. Again, that isn’t a no to the feature under any circumstance, just a statement that this use case is insufficient.
I implore you to open a new issue then, under the guise of your valid use case. I am sure that it will stay open, and that it will not be closed as a duplicate /s
Multiple servers are using this now, including some requests to subdomains on google.com, googleapis.com, CloudFlare and others. I keep reporting this [2][3], and no one seems to care. If a server blacklists your client, whether its cURL or Go "net/http", you can no longer request to that server using that client. Period. Any HTTP client that wants to be robust, should be thinking about this.
1. https://wikipedia.org/wiki/Device_fingerprint#Sources_of_ide...
2. https://github.com/golang/go/issues/48207
3. https://github.com/curl/curl/issues/8119