In this section

Network Forensics of Web and TLS Traffic: HTTP Logs, Certificates, QUIC and Decryption

Module 0

Introduction

Most of what crosses a network boundary today is web traffic, and most web traffic is encrypted. An investigator therefore spends much of the time reading sessions whose content cannot be seen, and needs to know exactly what can still be read from them, and when the content can be recovered after all.

Where a web session can be read, along its pathws46its own key log, if it keeps oneLAN sensorHTTP in fullTLS metadataFirewalltranslates to 198.51.100.10Web serverevery requestone client address

Encryption moves the content out of the sensor's reach. It does not move the metadata, and it does not reach the server, which reads everything it serves.

This lesson reads a few minutes of the lab's web tier: ws45 and ws46 browsing, downloading and uploading through the firewall to a set of web servers, some over plain HTTP and most over TLS, with the LAN sensor's Zeek and Suricata logs, a capture, one server's private key, one browser's key log and the web server's own access log.

It covers a page load, a redirect chain to a download, uploads, TLS metadata and certificates, QUIC, decryption and the server's view. Module 6 teaches each in depth.

1

Reading a Page Load

A request, a response, and what the sensor keeps

ws45 opened www.example.net over plain HTTP. Zeek's HTTP log has one line per request and response:

$ zeek-cut id.orig_h method host uri status_code resp_mime_types \
  < zeek-lan/http.log | head -5
10.20.30.45	GET	www.example.net	/	200	text/html
10.20.30.45	GET	www.example.net	/css/site.css	200	text/plain
10.20.30.45	GET	www.example.net	/js/app.js	200	text/plain
10.20.30.45	GET	www.example.net	/img/logo.png	200	image/png
10.20.30.45	GET	www.example.net	/favicon.ico	404	text/html

A page is rarely one request. The browser fetched the HTML, then the style sheet, the script and the logo the page refers to, and asked for a favicon that did not exist, which returned 404. Each line carries the client, the method, the host the browser asked for, the path, the status code and the type of what came back.

Two of those fields deserve care from the start. The host is what the client put in its request, not where the connection actually went; that is the destination address in the connection record, and the two can differ, which is how one address serves many sites.

The type in the last column is the type Zeek found by reading the response's first bytes, which is not necessarily the type the server declared. In this page load they agree. In the next section they do not.

The same log also records the Referer, the page the browser was on when it made each request, and the User-Agent, the string the client says it is. Every request after the first here names http://www.example.net/ as its Referer, which is how the requests group into one page view.

The User-Agent is a claim made by the client: the lab's ws45 is curl sending a browser's string, and the log cannot tell the difference. Module 6's first lesson reads a session field by field and keeps those two facts separate, what the log observed and what the client asserted.

For plain HTTP the sensor holds everything: requests, responses, headers and the files themselves, which Zeek's file analysis reads as they pass. That completeness is what encryption removes, so it is worth seeing once before reading what remains.

2

A Redirect Chain to a Download

Following the Referer back, and reading the file

ws46 clicked a link. Three requests followed within the same few milliseconds:

One click, three serversws4610.20.30.46go.example.com203.0.113.91share.example.org203.0.113.92files.example.org203.0.113.92GET /r/q3302, Location: share.example.orgGET /s/7f3a/ (Referer: go.example.com)302, Location: files.example.orgGET /d/7f3a/q3-invoice.pdf200, "application/pdf", 214,152 bytes of PE

ws46 follows a link: two redirects, each sending it to the next server, and a download at the end. Every hop names the page that sent it there.

The sensor's HTTP log holds the same three requests, with the Referer each one carried:

$ zeek-cut host uri status_code referrer < zeek-lan/http.log \
  | grep -e go.example -e share.example -e files.example
go.example.com	/r/q3	302	-
share.example.org	/s/7f3a/	302	http://go.example.com/r/q3
files.example.org	/d/7f3a/q3-invoice.pdf	200	http://share.example.org/s/7f3a/

The first request has no Referer: it started from outside the browser, a link in a mail or a document. go.example.com answered 302, a redirect, and ws46 went to share.example.org, carrying go.example.com as its Referer.

That answered 302 too, and the third request fetched /d/7f3a/q3-invoice.pdf from files.example.org. Read in reverse, the Referer chain leads from the download back to the link that started it, which is the question an investigation usually asks first.

The chain also shows something the final request alone does not: share.example.org and files.example.org are different names on the same address, 203.0.113.92, so a block on the download's name would have missed the server behind it, and a search of the connection log by address finds both of them at once.

The download's name says PDF, and so did the server. Suricata's HTTP record keeps the Content-Type header the server sent:

$ jq -r 'select(.event_type=="http" and .http.hostname=="files.example.org")
  | .http.http_content_type' suricata/eve.json
application/pdf

Zeek's file analysis read the content itself. Its documentation describes the logged type as the one from the strongest file magic signature match against the start of the file:

$ zeek-cut mime_type seen_bytes analyzers < zeek-lan/files.log \
  | grep dosexec
application/x-dosexec	214152	PE

The server said PDF; the content says Windows executable, and Zeek ran its PE analyzer on it because of what it found.

The reasonable mistake

Trusting the name and the declared type

The sensor shows ws46 downloading q3-invoice.pdf, and the server declared it application/pdf. An analyst reads it as a document. Zeek's file analysis read the first bytes and logged application/x-dosexec, and its PE analyzer ran: 214152 bytes of a Windows executable.

Instead: Read the type the content shows, from the file log or the bytes, beside the type the server declared and the name in the URL. When they disagree, the content wins.

A Windows executable named as an invoice, declared as a PDF and reached through two redirects is the shape of a malicious download.

None of the three facts alone proves it, and the lab's servers are only examples; but the content, the declaration and the name disagree, and the content is the one that cannot be chosen by whoever set up the page. Module 6 reads redirects, downloads and the file evidence the sensor extracts from them in its second lesson.

3

Uploads: What Left

Request bodies are evidence too

Downloads are what most monitoring is built around. Uploads are where data leaves. ws46 sent two POST requests to upload.example.net over plain HTTP, and Zeek's file analysis found a file in each request body:

$ zeek-cut id.orig_h filename mime_type seen_bytes < zeek-lan/files.log \
  | grep -v -e '	-	'
10.20.30.46	meeting-notes.txt	text/plain	61440
10.20.30.46	scan-0412.pdf	-	99064

The filenames come from the multipart form the client sent, and the sizes from the bytes the sensor actually saw. The second upload, scan-0412.pdf, was cut off before it finished, and Zeek's HTTP log records no response for it: the sensor saw a request body and no answer.

The web server logged three uploads, not two:

$ grep POST web-access.log | awk '{print $1, $3, $4, $5, $7, $9, $11}'
198.51.100.10 "upload.example.net" "POST /api/upload 201 62074 -
198.51.100.10 "upload.example.net" "POST /api/upload 201 2400554 TLSv1.3
198.51.100.10 "upload.example.net" "POST /api/upload 400 196931 -

The middle line is the one the sensor could not read. It came over TLS 1.3, as the server's log records, and it was a request of about 2.4 MB, ws45 uploading a file over HTTPS.

The sensor saw a TLS session to upload.example.net and its byte counts; it saw no filename, no form and no file. The server saw all of it, because it decrypted the request to handle it.

That pattern recurs throughout encrypted traffic. The upload over HTTPS is visible to the sensor as a size and a direction: a large number of bytes from the client to the server, in one session, to a name associated with uploads.

That alone is often enough to raise the question, and the server's log, or the client, answers it. Module 6's third lesson reads request bodies, multipart forms and the size signatures that remain when the bodies are encrypted.

The status column adds one more fact: the server answered the third upload, the cut-off scan, with 400, a rejected request, after receiving part of it. The sensor recorded no response at all, so the two logs disagree about how that exchange ended, and Module 6's uploads lesson reads why.

Note the source address in that log: 198.51.100.10 on every line, the firewall's outside address. The server knows exactly what was uploaded and not which workstation uploaded it; section 7 returns to that.

4

What TLS Still Shows

The name, the version, the cipher and the certificate

Most of the window is TLS. The sensor cannot read the content of those sessions, but the handshake that sets each one up is largely in the clear, and Zeek's TLS log records what it showed:

$ zeek-cut id.orig_h server_name version cipher sni_matches_cert \
  < zeek-lan/ssl.log | grep -v '	-	-	'
10.20.30.45	www.example.net	TLSv13	TLS_AES_256_GCM_SHA384	-
10.20.30.45	upload.example.net	TLSv13	TLS_AES_256_GCM_SHA384	-
10.20.30.45	hr.example.net	TLSv12	TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256	T
10.20.30.46	hr.example.net	TLSv12	TLS_RSA_WITH_AES_128_GCM_SHA256	T
10.20.30.46	cdn-update.example.org	TLSv12	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	T
10.20.30.45	portal.example.com	TLSv12	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	F
10.20.30.46	portal.example.com	TLSv12	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	F
10.20.30.46	www.example.net	TLSv13	TLS_AES_256_GCM_SHA384	-

None of that needed the session's content. Three things in every handshake stay readable:

What a TLS session still shows the sensor

The name asked for

The server name the client sends in the clear

Version and cipher

Which decide, among other things, whether decryption is possible

The certificate, in TLS 1.2

Who issued it, and whether it names the server asked for

Each line names the server the client asked for, the protocol version and the cipher suite the server chose. The last column is Zeek's check of the certificate against the name: its documentation sets it true if the name the client sent matches the certificate, false if it does not, and leaves it unset when there is no name to compare.

For the TLS 1.3 sessions that column is empty and no certificate appears in Zeek's certificate log. In TLS 1.3 the certificate travels after encryption has started, so a passive sensor does not see it; Module 6's certificates lesson reads why. For the TLS 1.2 sessions it does, and two of them stand out. Both sessions to portal.example.com came back F. The capture shows which certificate the server sent:

$ tshark -r lan.pcap -Y 'tcp.stream==13 && tls.handshake.type==11' \
  -T fields -e tls.handshake.extensions_server_name -e x509ce.dNSName
tshark -r lan.pcap -Y 'tcp.stream==13 && tls.handshake.type==1' \
  -T fields -e tls.handshake.extensions_server_name
	*.example.net,example.net
portal.example.com

The client asked for portal.example.com and the server answered with a certificate for *.example.net and example.net. A browser would refuse it. A client that does not check, such as the sync agent behind those requests, connects anyway, and the mismatch is the clue that either the client is misconfigured or something is answering in the real server's place.

The certificate log shows one more irregularity. cdn-update.example.org presented a certificate whose subject and issuer are the same, a self-signed certificate that no public authority vouches for. ws46's update agent accepted it. Neither finding needs the content of a single request: the handshake alone is enough to question both servers.

That makes the TLS log one of the most useful a sensor keeps, and one of the cheapest. It has one line per session rather than one per packet, it survives encryption, and its server names can be searched the same way a DNS log can.

A name that a threat feed lists, a server that presents a self-signed certificate, a certificate that does not match the name asked for: each is a search across the TLS log, and each works on traffic nobody can decrypt.

5

QUIC: The Web Over UDP

A different transport, the same questions

Three of the window's sessions to www.example.net did not use TCP at all:

$ zeek-cut id.orig_h id.resp_p server_name client_protocol history \
  < zeek-lan/quic.log
10.20.30.46	443	www.example.net	h3	ISXIXIXIXIXIX
10.20.30.45	443	www.example.net	h3	ISXishxoIHOOooOoo
10.20.30.45	443	www.example.net	h3	ISXishxoIHOOoOo

QUIC carries HTTP/3 over UDP port 443, with TLS 1.3 built into the transport. The sensor's QUIC log records the client, the server name from the client's first packet, the application protocol the client offered, h3, and a history of the packets it saw. Zeek's TLS log lists the same three sessions too, each marked as not established.

QUIC matters to an investigator for two reasons. First, a sensor or firewall that looks only at TCP misses it entirely, and much browser traffic to large sites now uses it whenever both sides support it.

Second, its records look different: there is no TCP connection to reassemble, the connection is identified by connection IDs rather than addresses and ports alone, and the version of TShark the lab builds lists no HTTP/3 header decoding, so even a decrypted QUIC session yields less than a decrypted TLS one.

The history column is a compact record of the packets.

Zeek's documentation gives the letters: I for an Initial packet, S for a hello, H for a handshake packet, O for short-header packets counted logarithmically, capitals for what the client sent and lower case for the server. ws45's two sessions show both sides and data flowing. ws46's shows only capitals, the client sending Initial packets again and again with nothing coming back, and 4.6 seconds later ws46 opened an ordinary TLS 1.3 session over TCP to the same server.

A client giving up on QUIC and falling back to TCP leaves exactly that pair of records.

For now the habit to take away is to count UDP 443 as web traffic. A question like "what did this host reach over the web" is answered by the TLS log and the QUIC log together, not by the TLS log alone. Module 6's lesson on HTTP/2 and QUIC reads the stream structure of both, and what the sensor keeps of each.

6

When the Content Can Be Recovered

Keys, and which keys

Encrypted content can sometimes be read after the fact, but only with the right secret. The lab supplies two kinds: the HR server's private key, and a key log written by ws45's clients, a file of the session secrets each one negotiated.

What decrypts a session

The server's RSA key

Only sessions that used RSA key exchange

A client's key log

Any session that client made while logging, TLS 1.2 or 1.3

Neither

Metadata only: name, sizes, timing

The server key first. TShark takes it in its RSA keys table, and it decrypts one of the two sessions to hr.example.net:

$ tshark -r lan.pcap -o 'uat:rsa_keys:"keys/hr.example.net.key",""' \
  -Y 'tcp.stream in {10,11} && http' -T fields \
  -e tcp.stream -e http.request.method -e http.response.code
11	GET	
11		200

Stream 11, ws46's session, decrypts: a GET, and the server's 200. Stream 10, ws45's session to the same server with the same certificate, does not.

The cipher suites in the TLS log above explain it. ws46's session used TLS_RSA_WITH_AES_128_GCM_SHA256, RSA key exchange, in which the client encrypts the session secret to the server's key, so the key recovers it. ws45's used TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, in which the two sides agree a secret by an ephemeral exchange that the key only signs; the secret was never sent, and no key can recover it.

Every TLS 1.3 session works the second way.

The key log works from the other end. It holds the secrets of every session the logging client made, whatever the cipher or version, so with ws45's file TShark reads ws45's HTTPS page load, its HTTPS upload and its session to hr.example.net. It reads nothing of ws46's sessions, because ws46 kept no log.

Decryption is therefore a question about what was recorded at the time: which server keys exist and were used with RSA exchange, and which clients were writing their secrets. After the event, nothing else helps.

That is why the question belongs in preparation rather than in the investigation: an organization that wants to read its own encrypted traffic later has to decide now which servers keep which keys and which clients, if any, log their secrets, and it has to protect those secrets as carefully as the data they unlock.

Module 6's decryption lesson reads both methods in full, with what TShark and Zeek each do with the secrets, and the risk a key log file carries for the host that keeps it.

7

The Server's Own Log

Every request, and nobody's address

The web server sees what the sensor cannot: it terminates the encryption, so its access log records every request, HTTPS included. These are two TLS requests the sensor could read nothing of:

$ grep -e cdn-update -e portal web-access.log \
  | awk -F'"' '{print substr($1, 1, 14) $2, $4, $6}'
198.51.100.10 cdn-update.example.org GET /v3/manifest.json HTTP/1.1 UpdateAgent/4.2
198.51.100.10 portal.example.com GET / HTTP/1.1 PortalSync/1.0

The update agent's manifest request and the portal sync agent's request, with their paths, user agents and protocol versions, all from sessions the sensor saw only as handshakes.

That is the strength of the server's log. Its weakness is in the first column of every line: 198.51.100.10, the firewall's address. Every one of the window's twenty requests came from the same address, because every workstation goes out through the same translation.

Joining the two is the method. The server's log says what was requested, at what second and over which protocol; the sensor's log says which internal host had a session to that server at that second. Matched on server name, time and size, they name the host and the content together, which neither does alone.

The window makes the join easy, because each server here had one or two sessions; on a busy server the second and the byte count do the work, and the TLS log's session times and the access log's request times have to be read on the same clock, so check both clocks before trusting a join made to the second.

For a server the organization runs itself, the access log is usually available; for anything on the internet it is not,

The access log also records what the sensor would need decryption to find: the HTTP version each request used, and, for the TLS requests, the TLS version the server negotiated.

Those columns confirm the sensor's reading of the same sessions from the other end, which is worth doing whenever both logs exist, because a sensor that dropped packets and a server whose clock drifted both show up as disagreements between them.

8

What Module 6 Builds On It

Web and encrypted traffic in depth

Module 6 takes each part of this lesson in turn: reading an HTTP session, redirects and downloads, uploads and request bodies, TLS handshakes and metadata, certificates, HTTP/2 and QUIC, and decryption and its prerequisites.

Its exercise traces a web session from a mail link through two redirects to a download and on to uploads to a server with a self-signed certificate, with a key log for one browser only, and asks for the evidence to be split into what was observed and what is inferred.

The lab is the same: the same capture, the same Zeek and Suricata logs, the same keys and the same server log, read more deeply. Each lesson adds one kind of reading to the ones this lesson started, and the exercise uses all of them at once.

Module 6, in order

016.1 to 6.3An HTTP session; redirects and downloads; uploads and request bodies
026.4 to 6.5TLS handshakes and metadata; certificates
036.6 to 6.7HTTP/2 and QUIC; decryption and its prerequisites
046.8Exercise: trace a web session

Nothing has to be captured again: the capture, logs, keys and server log this lesson read are the ones the module reads.

The distinction the exercise asks for is the one this lesson has drawn throughout. Observed: ws46 requested a file named q3-invoice.pdf, the server declared it a PDF, and its first bytes were a Windows executable. Inferred: that ws46's user was deceived, or that the file ran.

The first kind goes in a report as fact. The second needs evidence from the host. A network investigator who keeps the two apart hands the endpoint team a precise question, which file and which second, instead of a conclusion they then have to unpick before they can test it.

The habit to carry into it: for every web session, ask first whether the content is readable. If it is, read it. If it is not, read the name, the version, the cipher, the certificate, the sizes and the timing, and look for the log or the key that holds the rest.

Do

Practice

What does the sensor show?
the lab web tier

The prepared evidence below is the sensor's HTTP log and TLS log for the window. Find the download whose content and name disagree, the sessions answered with a certificate for another name, and the one session the HR server's key can decrypt. Each answer comes with its reasoning, so check it against the rows rather than only the choice.

If you can, open the developer tools in a browser, load a site, and read the request list: the redirects, the Referer on each request and the protocol column, which shows h2 or h3 for most large sites. The same fields are what the sensor records.

Plain HTTP gives the sensor everything; TLS leaves the server name, version, cipher and, in TLS 1.2, the certificate, which is often enough to question a server; QUIC is web traffic on UDP; content comes back only with a server key used with RSA exchange or a client key log written at the time; and the server's own log reads every request but names only the address it came from.

The next lesson reads the traffic that comes before an attack: scanning, and what it showed the scanner.

Versions this lesson relies on: Wireshark and TShark 4.6.9; Zeek 9.0.1 (LTS); Suricata 8.0.8.