How I reverse engineer undocumented web protocols

most web applications look simple from the outside

you click a button, a request leaves the browser, a response comes back, and the interface changes

but some applications hide a much stranger system underneath: persistent WebSockets, binary frames, compressed payloads, encrypted messages, generated field names, undocumented state machines, and sometimes a WebAssembly module doing the interesting work

i have spent a lot of time inside systems like this, especially while working with WhatsApp Web

the first mistake i made was trying to understand everything at once i would open a directory full of minified JavaScript, search for an interesting word, jump through fifteen modules, and end the day knowing less than when i started

the workflow that eventually worked was much smaller:

choose one observable behavior
capture a known-good run
find the boundary where that behavior enters or leaves the client
preserve the raw data before interpreting it
change one variable and compare
decode one layer at a time
reproduce the smallest useful part

this post is about that workflow

a sealed paper packet being opened layer by layer

start with a behavior, not a protocol

"understand the protocol" is not a useful first goal

a real client may contain hundreds of message types and years of compatibility code there is no obvious beginning, and most of it will not matter to the thing you want to build

i start with one sentence that describes something i can see:

when an incoming call arrives, the browser sends a preaccept message

or:

when a file is uploaded, the client receives a direct path and media metadata

that sentence gives me an input, an output, and an event i can reproduce

it also creates a stopping condition if i can explain that one transition and reproduce it, the investigation has produced something useful even if the rest of the protocol is still unknown

for my WhatsApp call experiment, the first goal was not "implement WhatsApp calling" it was simply to make one incoming call move from ringing to accepted

that narrower question still led through binary nodes, encrypted keys, WASM callbacks, relay traffic, and audio devices, but it gave every detour a test: does this help explain why the call is not being accepted?

record the known-good client first

before writing a parser or copying code, i watch the official client perform the behavior successfully

the browser is the best protocol documentation available because it already knows the current endpoint, handshake, message format, feature flags, and ordering rules

i usually record several views of the same run:

browser network activity
WebSocket frames
console output
relevant JavaScript and WASM assets
timestamps for the action i performed
visible state changes in the interface

the timestamp matters more than it sounds a busy client may send presence, receipts, synchronization messages, heartbeats, and telemetry around the same time without a rough timeline, it is easy to mistake background traffic for the action being investigated

i keep the first capture untouched it becomes the reference i can return to after my scripts and assumptions have made everything confusing

find the narrowest useful boundary

there are several layers between a button and the wire:

interface
  -> application action
  -> protocol object
  -> serializer
  -> encryption or compression
  -> transport frame
  -> socket

starting at the interface usually produces too much framework code

starting at the socket often produces an opaque encrypted blob

the useful boundary is normally somewhere in the middle: the last point where the data still has names, or the first point where incoming bytes become a structured object

some productive boundaries are:

the function immediately before WebSocket.send
a protobuf or binary-node encoder
the callback that receives a decoded stanza
the JavaScript wrapper around an exported WASM function
an event dispatcher that maps message tags to handlers

in whatsmeow, a map of node names to handlers was effectively a protocol table of contents once i found it, i stopped searching the entire repository and followed only the path for the event i cared about

save the bytes before naming them

it is tempting to immediately convert a captured frame into JSON, guess a schema, and throw the original away

i keep the raw bytes

captures/
  001-idle.bin
  002-action.bin
  003-response.bin
  notes.md

the raw capture protects the investigation from my own parser a parser can silently truncate a field, normalize a value, discard ordering, or confidently label something that was only a guess

the first pass is deliberately boring:

file captures/002-action.bin
wc -c captures/002-action.bin
xxd -g 1 captures/002-action.bin | head
strings -n 8 captures/002-action.bin | head

i am looking for basic clues:

does the frame have a repeated header?
does its length appear inside the first few bytes?
is any part readable?
does the first byte behave like a flags field?
does the payload have the signature of gzip, zlib, protobuf, CBOR, or a custom format?
is the same prefix present in every message?

at this stage unknown byte at offset 3 is a better label than an invented field name

change one variable

one capture tells me what happened comparisons tell me which bytes mean something

i repeat the same action while changing one controlled value:

send the same request with a different recipient
upload two files with the same type but different sizes
repeat an action after reconnecting
accept once and reject once
change a short text value to another value of the same length

then i compare the artifacts

cmp -l captures/accept.bin captures/reject.bin | head

some changes will be noise: timestamps, counters, random identifiers, nonces, authentication tags, or encrypted payloads a field that changes does not automatically represent the input i changed

the goal is not to label every difference it is to form a small hypothesis that can survive another capture

my notes often look like this:

offset 0..2    stable, probably framing
offset 3       changes with payload size
offset 8..15   changes every run, possibly id or timestamp
body           high entropy after handshake

the word possibly is important reverse engineering gets expensive when guesses quietly turn into facts

separate the protocol into layers

an undocumented protocol rarely has one encoding it is usually a stack

for example, the data may be:

WebSocket frame
  -> client framing
  -> encrypted payload
  -> compressed bytes
  -> binary tree
  -> application message

if i try to parse the binary tree before removing the client framing, every offset is wrong if i try to search ciphertext for message names, nothing appears

i write down the layers i know and the layers i only suspect:

[known] WebSocket binary frame
[known] one-byte flags field
[suspected] three-byte payload length
[unknown] encrypted section
[known] decoded node after existing client handler

then i attack the first unknown boundary with known data on both sides

this is why runtime instrumentation is so useful if i can log the bytes entering a decoder and the object leaving it, i do not need to solve encryption, framing, and application semantics simultaneously

search for strings, not function names

minified function names are temporary error messages and protocol strings are often much more stable

instead of beginning with a name like sendCallOffer, i search downloaded assets for things visible at runtime:

an error message from the console
a WebSocket tag
a log category
an endpoint fragment
a state name
an uncommon object key

rg -n "failed to parse offer|change_call_state|preaccept" files/

one good string can land close to the code that builds a message or crosses a native boundary from there i work outward through callers, imports, tables, and adjacent constants

when the source is bundled, i do not try to beautify the entire application i extract or annotate only the modules on the path i am following

the surrounding code often reveals more than the matching line:

which values are validated before the call
which fields are optional
what happens on failure
whether the function expects bytes, an object, or a handle
which callback receives the result

instrument the runtime

static source shows what can happen runtime traces show what did happen

i add logging as close to the useful boundary as possible and log both shape and timing

function inspect(label, value) {
  const now = performance.now().toFixed(2)
  console.log(`[${now}] ${label}`, value)
  return value
}

wrapping a serializer, dispatcher, or WASM import can reveal the order of operations without requiring a complete understanding of the surrounding application

the safest hooks are passive they copy arguments and return values without changing them mutating a buffer while logging it can create a new bug that looks exactly like a protocol error

for binary values, i log the length and save a copy rather than dumping thousands of decimal bytes into the console

function copyBytes(value) {
  const view = value instanceof Uint8Array ? value : new Uint8Array(value)
  return view.slice()
}

this becomes especially important around shared WASM memory where a view may point to memory that is reused immediately after the call if i keep the view instead of copying it, the evidence can change underneath me

treat WebAssembly as a boundary, not a wall

when the interesting code lives in WASM, my first instinct is not to decompile the whole module

i begin with its edges:

JavaScript imports supplied to the module
exported functions called by JavaScript
arguments written into linear memory
callbacks from WASM into JavaScript
strings printed through logging imports

the wrapper has to translate browser objects into values the module understands, so it often exposes the hidden API more clearly than the compiled code does

during my WhatsApp call experiment, the WASM module was only one part of the problem the JavaScript around it still had to provide signaling callbacks, timers, network I/O, audio devices, and memory helpers

recreating those imports one by one was more useful than understanding every function inside the binary

a missing callback also gives a strong signal if the module progresses until it calls send and then fails, the initialization path probably worked the next boundary to investigate is that callback, not the entire module

build the smallest decoder first

my first decoder is not a reusable library it is an executable notebook for one capture

it should be able to say:

frame length: 184 bytes
flags: encrypted
payload: 180 bytes
decoded node tag: call
attributes: 3
children: 1

i preserve unknown data and fail loudly on unexpected structure

type Field = {
  offset: number
  length: number
  meaning?: string
  raw: Uint8Array
}

throwing away an unknown field because the current sample does not use it makes the parser look cleaner, but it removes exactly the evidence needed for the next sample

only after the decoder works across several captures do i turn repeated observations into names and types

use the client as an oracle

the official client can answer small questions even when there is no documentation

if i give a decoder a carefully modified payload, its error may reveal:

which layer rejected the data
the expected message type
whether a length is wrong
whether a required field is missing
which state transition was not allowed

this does not mean randomly replaying corrupted traffic against a production service most experiments can happen locally at a parser or module boundary using saved inputs

the useful pattern is:

begin with a valid artifact
change one thing
feed it into the narrowest local component available
record the exact result
restore the original before the next test

error messages are observations, not specifications a parser rejecting value 4 does not prove that only values 1 through 3 are valid, but it gives the next experiment somewhere to start

state is part of the protocol

one of the most frustrating failures is constructing a message that looks byte-for-byte correct and still gets rejected

the missing field may not be in the message at all it may be earlier in the session

modern web protocols depend on state:

a feature flag enabled during initialization
a key installed during pairing
a counter advanced by previous frames
an identifier returned by an earlier request
an expected ordering between offer, preaccept, and accept
cached device or routing information

that is why replaying a single captured frame rarely reproduces a feature

i draw the smallest state machine i can justify from observations:

disconnected -> connected -> initialized -> ready
                                      |
                                      v
                              action requested
                                      |
                         acknowledged or rejected

then i annotate each transition with the message or callback that caused it this prevents me from treating a valid message in the wrong state as an encoding problem

reproduce before redesigning

once i understand enough of the path, i reproduce the known-good sequence as literally as possible

i do not begin by creating a beautiful abstraction or improving the protocol the first implementation should keep the ugly ordering, strange constants, and redundant calls until i know which parts are actually optional

my usual milestones are:

load the same assets
initialize the same minimum state
decode one real incoming event
construct one valid outgoing response
receive the expected acknowledgment
replace captured values with generated values one at a time

each milestone produces evidence if the implementation stops working after replacing a captured identifier, i know where to look

after the sequence is stable, i can separate transport, codec, state, and application logic into cleaner components

keep an evidence log

reverse engineering creates many convincing stories most of them are at least partly wrong

i keep three categories in my notes:

observed
- byte 0 is 0x80 in all six encrypted frames
- callback A runs before callback B in accepted calls

inferred
- byte 0 probably contains encryption flags
- callback A may install relay information

unknown
- whether the counter is per-device or per-connection
- whether field 7 is required for group calls

when new evidence contradicts an inference, i can change it without rewriting history

i also keep scripts and captures together with a short reproduction file months later, the hardest detail to recover is often not the code but which browser version, account state, feature flag, and exact sequence produced the capture

common dead ends

reading everything

large clients contain experiments, dead paths, compatibility branches, and features unrelated to the investigation follow one behavior end to end before expanding the map

trusting names too much

a function called sendMessage may enqueue work rather than touch the network a field called id may be a device id, request id, stanza id, or database key confirm names with data flow

assuming every difference matters

randomness and session state create noise repeat the same experiment more than once before associating a changed byte with your input

rebuilding crypto from memory

small differences in key derivation, padding, nonce construction, or authentication order produce identical-looking failures use test vectors from a working client and verify every intermediate value

ignoring failure paths

success code shows the intended route error handling reveals validation rules, retry behavior, alternate states, and assumptions the happy path hides

abstracting too early

an elegant wrapper around an incorrect model is harder to debug than one explicit script with logs at every boundary

the toolkit is smaller than it looks

i use sophisticated tools when necessary, but most investigations begin with ordinary ones:

browser DevTools for network frames, initiators, and runtime breakpoints
curl for repeatable HTTP requests
rg for searching downloaded source and strings
xxd, file, and strings for first-pass binary inspection
a small Node.js or Go script for capture and decoding
a formatter for isolating minified modules
Wireshark when the transport layer itself matters
WASM inspection tools when the JavaScript boundary is not enough

the important skill is not knowing every tool it is choosing the layer that can answer the current question with the least uncertainty

the workflow i use now

when i open a new undocumented web protocol, this is the checklist i follow:

write one observable goal
make the official client perform it successfully
save the traffic, assets, logs, and timeline
locate a boundary with structured data on at least one side
preserve the raw bytes
repeat the action while changing one variable
separate transport, framing, encoding, crypto, semantics, and state
search stable strings and error messages in the source
instrument the smallest useful runtime boundary
write a decoder that preserves unknowns
reproduce the known-good sequence literally
replace captured values one at a time
record observations separately from inferences
automate the experiment before the client updates

the result is rarely a complete specification

usually it is something more practical: a reliable map of one path through a changing system, enough to explain what happened and enough to build the next experiment

*How I reverse engineer undocumented web protocols