curl-impersonate and curl_cffi: TLS Parity Limits

An HTTP client can be made to send a handshake that closely resembles a browser's, and the resemblance stops at the first surface that needs a document. BotBrowser provides a declared profile that keeps the surfaces a page can read consistent with each other, including the JavaScript-visible ones, and it does not turn an HTTP client into a browser. This article measures three clients on one local listener: the curl binary, a Python client that impersonates Chrome, and Chrome itself. The difference between the last two is the useful part, because it is small enough to be missed.

TL;DR

• Three clients sent three different handshakes to the same listener: curl t13i4906h2, Node t13i521000, Chrome t13d1517h2.
• A client told to impersonate Chrome produced t13d1516h2, which matches Chrome on cipher count and differs by one extension.
• The extension difference is the interesting part: a near match that omits a single extension is still a distinguishable value.
• The i against d in those prefixes is the domain name flag; connecting by address and connecting by name are two different fingerprints.
• curl on this machine links SecureTransport rather than OpenSSL, so its cipher and extension sets come from a different TLS library than the one most guides assume.
• An HTTP client has no DOM, no canvas and no WebGL, so every signal derived from those surfaces is simply absent rather than mismatched.
• Setting a user agent header changes what the server reads and does not change what the handshake says.
• Matching the connection layer and matching the page layer are two separate jobs with two separate toolboxes.

Contents

• Three clients on one listener, and what impersonation matches
• The library underneath decides the value
• The layer an HTTP client does not have
• What a near match costs, and the practical conclusion

Three clients on one listener, and what impersonation matches

The measurement is deliberately small. A TCP listener on the loopback interface accepts a connection, records the first record, and closes; the handshake never completes, so no server certificate is involved and no trust decision interferes. Each client is pointed at that address in turn.

curl:   bytes=301  alpn=h2,http/1.1 ciphers=49 exts=6  versions=0x0304,0x0303,0x0302,0x0301
        ja4=t13i4906h2_0d8feac7bc37_7395dae3b2f3
node:   bytes=1566 alpn=(none)     ciphers=52 exts=10 versions=0x0304,0x0303
        ja4=t13i521000_b262b3658495_8e6e362c5eac
chrome: bytes=1917 alpn=h2,http/1.1 ciphers=15 exts=17 versions=0x0304,0x0303
        ja4=t13d1517h2_8daaf6152771_cb7bf5808d99

Read the three values side by side and the shape of the problem appears immediately. They differ in three independent positions: the cipher count, the extension count and the protocol tag. A client that fixes only the first two still announces itself in the third, and a client that fixes none of them is trivially separable.

Node's entry says 521000 for its counts, and the last four characters are 1000 rather than a protocol tag. That happens because a plain Node TLS connection offers no protocols at all, so the tag falls back to zeroes while the counts continue. An HTTP client that never negotiates HTTP/2 therefore has a fingerprint that states so, without any detection logic being involved.

The i in the first two values against the d in Chrome's is the domain name flag. curl and Node were pointed at a bare address in this run and sent no server name; Chrome was pointed at a name. That single character is not a subtle judgement about intent, it is a statement about what the client put in the message. Pointing curl and the impersonating client at a hostname instead of an address flipped their flags to d and raised each extension count by one, so the flag and the count are two readings of the same decision. Node did not follow: it stayed at t13i521000 with ten extensions until servername was passed explicitly, at which point it became t13d521100 with eleven. The library decides whether the name travels, and the caller decides whether the library is told.

curl-impersonate and curl_cffi both work by replacing or reconfiguring the TLS and HTTP/2 layer so the bytes resemble a chosen browser. The claim is narrow and worth testing rather than assuming, so the same listener was used with a Python client asked to impersonate Chrome:

default: bytes=517  ciphers=28 exts=12 alpn=h2,http/1.1
         ja4=t13d2812h2_257f3020b3a2_cbb9361b6bf9
chrome:  bytes=1815 ciphers=15 exts=16 alpn=h2,http/1.1
         ja4=t13d1516h2_8daaf6152771_d8a2da3f94cd

The second line is the result that matters. Asked to impersonate Chrome, the client produced fifteen ciphers and sixteen extensions, and its hashed cipher set 8daaf6152771 is identical to the real browser's. On the cipher axis the impersonation is exact.

The extension count is not. Chrome sent seventeen and the impersonating client sent sixteen, so the prefixes read t13d1516h2 against t13d1517h2. One digit apart, and that digit is exactly the kind of difference a value is stored for.

This is not a criticism of either project. A browser ships extensions that a client library has no reason to send, and reproducing the full list would mean tracking a moving target for no functional gain. The point is narrower: a near match is not a match, and a comparison that only checks the first hash part will report success where the prefix already disagrees.

The library underneath decides the value

The curl numbers at the top of this article look unusual for a reason that has nothing to do with impersonation. The local build reports its TLS backend:

curl 8.7.1 (x86_64-apple-darwin24.0) libcurl/8.7.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.64.0
curl_has_http2=true

SecureTransport, not OpenSSL. A curl built against SecureTransport draws its cipher list and extension set from that library, so its fingerprint is a property of the build rather than of curl the program. Two machines running the same curl version with different backends produce different values, and neither is wrong.

That has a practical consequence for anyone comparing fingerprints across a fleet. Recording "curl" as the client is not enough information, because it does not name the TLS library, and the library is what the value describes. The same reasoning applies to the impersonating clients: the version of the underlying library matters more than the version of the wrapper.

A second frame shows the same separation at the HTTP/2 level. The connection opens with a settings frame, and the entries in that frame are part of what a peer sees before any request exists:

chrome_like: bytes=24 HEADER_TABLE_SIZE=65536 ENABLE_PUSH=0 INITIAL_WINDOW_SIZE=6291456 MAX_HEADER_LIST_SIZE=262144
node_like:   bytes=30 HEADER_TABLE_SIZE=4096 ENABLE_PUSH=0 INITIAL_WINDOW_SIZE=65535 MAX_FRAME_SIZE=16384 MAX_CONCURRENT_STREAMS=4294967295
curl_like:   bytes=36 HEADER_TABLE_SIZE=4096 ENABLE_PUSH=0 MAX_CONCURRENT_STREAMS=100 INITIAL_WINDOW_SIZE=65535 MAX_FRAME_SIZE=16384 MAX_HEADER_LIST_SIZE=16384

Three clients, three frame lengths, and the difference is not decorative. The browser declares a large initial window and a large header list; the Node client declares no header list limit at all and leaves the window at its default; the curl-shaped client declares a concurrent stream cap the other two omit. A peer that reads the frame learns which of three architectures it is talking to before the request line is parsed.

The frame also shows why "match the browser" is harder than matching a cipher list. Cipher order is a fixed property of a library build; the settings frame is a set of tuning choices a client makes for its own workload. Making both look like a browser means adopting the browser's tuning as well, including the parts that exist to serve a rendering engine this client does not have.

Impersonation libraries do address this, and the measurement above shows they get close. What they cannot change is that the tuning exists in the first place: a client whose only job is to fetch bytes has different natural settings from a client whose job is to render a page, and the frame records the difference even when the code has been adjusted.

A client that reuses one connection makes this visible in a second place. Session resumption and connection reuse both do something useful for throughput, and both mean the second request does not repeat the handshake. A peer counting handshakes therefore sees one fingerprint for ten requests rather than ten, while a client that opens a fresh connection per request shows ten. Neither behaviour is a deception; they are two normal designs, and the counts differ because the designs differ.

The layer an HTTP client does not have

Everything above is about the connection. The page layer is a different set of values and an HTTP client does not have any of them, in the same sense that a text editor does not have a screen resolution.

page_layer: ua=header-only  languages=en-US,en  platform=MacIntel
            hardwareConcurrency=16  deviceMemory=null
            canvas=true  webgl=true  timezone=America/Toronto
            screen=1280x720  dpr=1  webdriver=true

Those values came from evaluating a document in a browser. None of them can be produced by a client that parses bytes, because each one is computed by a JavaScript engine or a rendering system that the client does not run. There is no partial credit here and no clever workaround: a header can be set to any string, and the string is not the value.

The last entry is worth separating from the rest. webdriver=true is reported by the browser because it was launched under automation, and an HTTP client has no equivalent field to report at all. A comparison that treats "field absent" as "field clean" reads the absence backwards, which is why a fingerprint report has to say which surfaces were evaluated rather than only which values were found.

Put the two layers together and the cost of a near match becomes concrete. A client that reaches t13d1516h2 has spent real effort on the connection layer and still differs from the browser it imitates by one extension. It has also gained nothing on the page layer, where it remains a client with no document.

That combination is the worst of both positions in one specific sense: it invites a comparison it cannot win. The gap is one extension (t13d1516h2 against t13d1517h2) and the hashed cipher set is identical, so the disagreement is visible in the prefix and invisible in the hash. A plain client that makes no attempt at resemblance is obviously not a browser, and tools treat it accordingly. A client that closely resembles a browser on the connection layer sets an expectation that the page layer then fails to meet.

plain_client      -> clearly not a browser on both layers
impersonated      -> close on connection, absent on page
browser           -> both layers present and consistent

The useful question is therefore not "how close can the handshake get" but "which layer does this job actually need". A request for JSON behind a simple check needs the first row. A request for a rendered list behind a login needs the third. Choosing the second row is a decision to pay for a resemblance that only half arrives.

There is a third reason the two layers stay separate in practice, and it is about what a comparison can even be stated as. A connection-layer claim has the form "this handshake offered these parameters", which a single recorded message can support. A page-layer claim has the form "this environment reported these values", which needs an evaluation inside a document and a record of what that evaluation could reach. The two claims have different evidence requirements, so a report that merges them into one score has to throw away the field that says which one it is making.

That matters most when a mismatch appears. If a client matches on the connection layer and the page layer is absent, the report should say "page layer not evaluated" rather than "page layer clean". The first statement is true and actionable; the second is a claim about a measurement nobody took. The same discipline applies in reverse: a browser session whose handshake was never captured cannot answer a question about the connection, no matter how thorough its page-side report is.

Cost follows the same split. A connection-layer match is paid for once, at build time, by choosing a library and a version and keeping them pinned, which is why a recorded value such as t13d1516h2 reads as finished the moment it is written down. A page-layer match is paid for continuously, because the values a document reports depend on the renderer, the fonts, the screen and the profile in use, and any of those can drift between runs. Teams frequently budget for the first and are surprised by the second, which is why a pinned TLS library feels finished while a consistent page environment never quite does.

A last operational note belongs here rather than in a tutorial, because it is the part that breaks silently. When a client library is upgraded, the cipher list and extension set move with it, and the recorded fingerprint stops matching for a reason that has nothing to do with configuration. When a browser is upgraded underneath a pinned automation setup, the page-layer values can move the same way. Both are version events, both are visible in a diff if the value is stored next to the version, and both are invisible if the value is stored alone.

What a near match costs, and the practical conclusion

Measure before assuming parity: capture your own client's ClientHello and compare the extension count, not only one hash. The impersonating client measured here matched Chrome's cipher set exactly and differed by one extension, and the prefix states that difference plainly. Name the TLS library as well as the program, because SecureTransport and OpenSSL builds of the same curl version produce different values. And keep the two layers separate in any comparison: the connection layer can be approximated by a client with no DOM, and the page layer cannot be approximated at all by one. For how the connection-layer value is constructed in the first place, read JA3, JA4 and the HTTP/2 settings frame; for which detection rules need a wire value, read edge-layer rules a page script cannot evaluate. A rendered page needs a browser, and browser setup for scraping covers that path.

The boundary is worth stating without decoration. A declared profile can keep the values a page exposes consistent with each other; it does not make a handshake match another client, and it does not give an HTTP client a document to render. Matching the connection and matching the page are separate results obtained from separate tools, and a job usually needs one of them rather than both.

A short note on what the measurements here do not say, because a reader can reasonably ask. They do not say that a near match is worthless: for a JSON endpoint behind a simple score, the connection layer is most of what is examined, and a client that gets the ciphers right is in a different position from one that does not. They also do not say that a browser is required for every rendered page, since a page that ships its data in an initial state object can be read by a client with no renderer at all.

What they do say is that the two layers are not a gradient. There is no setting that moves a client halfway from "no DOM" to "renders a page", and there is no header that moves a browser's handshake from t13d1517h2 to t13d1516h2. A job that needs both layers needs a browser; a job that needs the connection layer can often be served by a client that never loads a document. Deciding which of those two jobs is in front of you, and saying so in the record, is the whole discipline.

The measurement setup itself is worth one paragraph, because a reader who wants to check these numbers needs the same conditions. The listener has to accept the connection and close without completing a handshake; if it presents a certificate, the client may send a different second message and the comparison moves. The capture has to happen on the client side of any proxy, or the recorded bytes belong to whatever terminated TLS rather than to the client being studied. And the parse has to drop the grease values before counting, because a client that draws a fresh placeholder on each connection appears to grow and shrink its cipher list between runs. None of those steps is difficult, and each of them, skipped, produces a number that changes for a reason unrelated to the client.

Sources

• curl-impersonate
• curl_cffi
• RFC 8446: TLS 1.3
• BotBrowser: automation consistency