Reading width
Wide uses the full column for everything, text, diagrams, code, and exercises. Narrow keeps the standard reading width.
Text size
Scales the body text. Headings and code blocks keep their size.
In this section
Network Security Telemetry for Investigators: Zeek, Suricata, Flow, Firewall and Proxy Logs
Introduction
Full packet capture is the exception. Most networks keep logs instead: a firewall's decisions, a resolver's queries, a proxy's requests, flow records exported from routers, and on better-instrumented networks Zeek's protocol logs and Suricata's events. Each was written by a device at one point on the path, and each summarizes the packets that passed that point in its own units and its own vocabulary.
Investigating from logs means knowing what each one is, where it sat, and what it left out. It also means knowing which of them an organization actually keeps, and for how long, because a source that would have answered the question is of no use at all if it was never switched on or was rotated away last month.
Five positions, six records: the resolver and the firewall write their own logs; the sensor feeds three tools from one copy of the packets.
This lesson reads two minutes of the lab network that every one of those sources recorded at once. It follows one direct HTTPS connection through six separate records, counts one refused attempt five different ways, and watches a proxied connection lose its client on the way out.
Module 4 teaches each source in depth: Zeek, Suricata, NetFlow and IPFIX, firewall and NAT logs, proxy, DNS and VPN logs, and the flow logs of the three large clouds.
Six Records of One Connection
Starting with the nameAt 06:28:06, ws45 connected directly to cdn-sync.example.org over HTTPS. The first record of it is the resolver's, written before any connection existed:
$ grep -E 'cdn-sync.example.org( from 10.20.30.45| to | is )' resolver.log \
| head -3 | cut -c17-
dnsmasq[18213]: query[A] cdn-sync.example.org from 10.20.30.45
dnsmasq[18213]: forwarded cdn-sync.example.org to 203.0.113.53
dnsmasq[18213]: reply cdn-sync.example.org is 203.0.113.150
The resolver logs the query, the client that asked, the upstream it forwarded to and the answer. That one log line pair names the workstation, the domain and the address the workstation was about to contact. Nothing else in this lesson names the domain and the workstation together as directly; the connection records that follow carry addresses, and the name has to be joined back to them from here.
The next record is the firewall's, written when the first packet of the connection reached it:
$ grep 'SPT=55794 ' firewall.log | tr ' ' '\n' \
| grep -E '^(ALLOW-NEW|IN=|SRC=|DST=|SPT=|DPT=)' | paste -sd ' '
ALLOW-NEW IN=lan0 SRC=10.20.30.45 DST=203.0.113.150 SPT=55794 DPT=443
ALLOW-NEW is the log prefix the lab firewall's rule writes for a new outbound connection it allowed. The line carries the interface the packet arrived on, and the addresses and ports as the firewall saw them on the way in, from the LAN, before any translation. A firewall log is the most common network record there is, and in many small organizations it is the only one.
The firewall's connection table adds what the log line does not, the translation:
[1794292086.079433] [UPDATE] tcp ESTABLISHED
original src=10.20.30.45 dst=203.0.113.150 sport=55794 dport=443
reply src=203.0.113.150 dst=198.51.100.10 sport=443 dport=55794
The original direction is ws45 to the server; the reply direction is the server to 198.51.100.10, the firewall's public address. That pair is how the firewall remembered which inside host owned the outside connection, and it is exactly the join section 2 of lesson 0.1 made from packets.
Recorded here, it answers an abuse complaint without any capture at all, provided someone kept the connection-table events. Most firewalls can log them, and few are configured to by default; Module 4 reads what the lab firewall was set to record and what a default configuration would have lost.
Three records in, the connection is already attributed: the resolver names the workstation and the domain, the firewall confirms the connection left, and its table ties the public address back to the workstation. None of it needed a sensor, which is the point. Many investigations have to work exactly this way.
What the Sensor Tools Add
Zeek, Suricata and a flow record from one copy of the packetsA sensor reading the LAN feeds three tools from the same packets, and each writes a different record. Zeek writes a connection log line and a protocol log for what it recognized:
conn.log uid=C6hGqh3VNHxL8I3ej9 service=ssl conn_state=RSTR history=ShADadfr
duration=0.005970 orig_bytes=730 resp_bytes=2117
ssl.log uid=C6hGqh3VNHxL8I3ej9 server_name=cdn-sync.example.org version=TLSv13
The two lines share a uid, Zeek's identifier for the connection, which is how its logs join: conn.log has the connection's shape and size, ssl.log has what the TLS handshake revealed. The server name, cdn-sync.example.org, is in the clear in the handshake even though everything after it is encrypted, and Zeek recorded it.
The conn_state and history say the connection completed and ended with a reset. The history letters are the handshake, data in both directions, and the closing packets, read the same way lesson 0.1 read them for the lossy capture.
The uid is also how Zeek's other logs refer to the connection. A file transferred over it, a certificate it presented, a weird event it triggered, all carry the same uid, so one identifier pulls everything Zeek knew about the connection together.
Suricata, reading the same packets against a rule set, raised an alert because a lab rule matches TLS to that server name:
$ jq 'select(.src_port==55794 and .event_type=="alert")
| {timestamp, dest_ip, dest_port, sig: .alert.signature}' suricata/eve.json
{
"timestamp": "2026-11-10T06:28:06.082772+0000",
"dest_ip": "203.0.113.150",
"dest_port": 443,
"sig": "ND LAB TLS to cdn-sync.example.org"
}
An alert is a rule's verdict, not a record of everything: it exists because someone wrote a rule that matched. Suricata also writes flow, protocol and file events without any rule firing, which Module 4 reads. The flow exporter writes the least:
$ nfdump -r flows-lan/nfcapd.202611100600 -q \
-o 'fmt:%ts %pr %sap -> %dap %pkt %byt' 'port 55794' | tr -s ' '
2026-11-10 06:28:06.079 TCP 10.20.30.45:55794 -> 203.0.113.150:443 8 1154
2026-11-10 06:28:06.079 TCP 203.0.113.150:443 -> 10.20.30.45:55794 6 2425
Two records, one per direction, with packet and byte counts and nothing else. Flow records are cheap enough to keep for months on a large network, which is why they are often the only record that reaches back to the start of an intrusion.
They never say what was said, only how much and between whom. And because each direction is its own record, a question about one connection means finding both records and pairing them on addresses, ports and time, which is the first thing Module 4 practices with nfdump.
Three tools reading the same packets produced three records with almost nothing in common except the addresses. That is not redundancy. Each answers questions the others cannot, and their agreement on the addresses and the time is what lets them be joined.
Same Connection, Different Numbers
Units and clocksThe sources disagree about the size of the connection, and every one of them is right:
The same connection, three byte counts
730: payload only, what the application sent
1154: IP packets, headers included
1266: whole frames, Ethernet headers too
Zeek's orig_bytes counts payload, the bytes the application sent inside TCP. Its orig_ip_bytes and the flow record count IP packets including their headers, and agree exactly at 1154. Suricata's flow event counts whole frames, Ethernet headers included, which is why its figure is exactly fourteen bytes per packet larger. Three correct numbers for one connection, because each tool chose a different unit.
That matters whenever numbers are compared across sources: a byte count from a flow record and a byte count from Zeek's payload field are not the same measurement, and an apparent discrepancy between them is usually a unit, not a finding. The units are documented for each tool, and Module 4 sets them out field by field.
Time differs too. The resolver and firewall logs here write whole seconds in local syslog format; Zeek writes seconds since 1970 with microseconds; Suricata writes ISO 8601 with a zone offset; the flow record writes milliseconds. Joining them means converting every timestamp to one form and allowing for the coarsest resolution among them, so a firewall line at 06:28:06 matches a Zeek record anywhere within that second.
Clock agreement is the other half. Every source here ran on the lab's synchronized clock, so their times agree within the resolution of each format.
On a real network the firewall, the resolver and the sensor may each be seconds or minutes apart, and a join that assumes otherwise will pair the wrong records. Module 2 measured a sensor's offset from matched packets; the same measurement, done once per source, is what makes a multi-source timeline trustworthy.
The deeper lesson is that every log is a summary chosen by its author. A firewall summarizes a connection as its first packet and a decision. A flow exporter summarizes it as counts. Zeek summarizes it as a state machine and the protocol fields it recognized. Reading a log well starts with knowing what its author chose to keep.
One Attempt, Counted Five Ways
Units again, with a refused connectionAt 06:28:17, ws45 tried to reach the same server on TCP port 445, the SMB file-sharing port, and the firewall dropped it. Every source recorded the attempt, and no two agree on how many there were:
ws45 tried TCP 445 to the internet; the firewall dropped every SYN. Each source counts a different unit, and the connection table, which only tracks what it lets through, counts nothing.
The firewall wrote one line per SYN it dropped: the first attempt and five retransmissions, as the workstation's TCP stack retried a connection that never got an answer. Suricata alerted on each of the same six SYNs.
Zeek, which models connections, recorded them as attempts with state S0, an attempt with no reply, and split them into two connections because the retries spread past its inactivity window. The flow exporter summed all six packets into one record. The connection table recorded nothing, because the firewall dropped every packet before it was tracked.
The reasonable mistake
Adding up records as if they were events
An analyst counts 6 firewall denies and 6 Suricata alerts for ws45 on port 445 and reports twelve blocked SMB connections. There was one attempt: the workstation's SYN and its retransmissions, all to the same port from the same source port.
Instead: Count events in one stated unit and say which: one connection attempt from source port 50604, retried until it gave up, every packet dropped at the firewall. Then each source's count becomes corroboration rather than a total.
The habit is to count in a stated unit and treat the other sources as corroboration. Six firewall lines, six alerts, two Zeek records and one flow record are four independent confirmations of one fact: ws45 tried SMB to the internet once and was blocked.
Module 4's opening lesson counts a refused connection the same way across its sources, and its exercise asks for every answer in a stated unit for exactly this reason.
Counted correctly, the attempt is still significant. Outbound SMB to the internet has no business purpose on most networks, and an attempt from a workstation that has just fetched a file from an unfamiliar domain is worth following up on the host. The firewall stopped it; the question of what made it remains.
The absence in the connection table is instructive too. A source that only records what it allows will never show what it blocked, so a search of that table for the attempt finds nothing, and the nothing means the source could not have recorded it.
What a Proxy Hides
The client disappears at the first hopAt 06:28:10, ws46 reached the same server name through the company's web proxy. The proxy's own log and the firewall's line for the outbound connection:
$ grep -E '^CONNECT.*06:28:10' proxy.log | cut -c11-86
grep 'SRC=10.20.0.8 DST=203.0.113.150' firewall.log | tr ' ' '\n' \
| grep -E '^(ALLOW-NEW|SRC=|DST=|DPT=)' | paste -sd ' '
Nov 10 06:28:10.097 [18202]: Connect (file descriptor 2): 10.20.30.46
Nov 10 06:28:10.098 [18202]: Request (file descriptor 2): CONNECT cdn-sync.e
Nov 10 06:28:10.100 [18202]: Established connection to host "cdn-sync.exampl
ALLOW-NEW SRC=10.20.0.8 DST=203.0.113.150 DPT=443
The proxy logged the client, 10.20.30.46, and the request, CONNECT cdn-sync.example.org:443. The firewall, one hop further out, logged the connection as coming from 10.20.0.8, the proxy itself. The resolver, too, logged the proxy as the client asking for the name. Beyond the proxy, ws46 is gone from every record.
Suricata also fired its TLS rule only for ws45. Through the proxy, the TLS session is carried inside the CONNECT tunnel between ws46 and the proxy, and the rule written for a direct TLS connection to the server did not match it.
One rule, one alert, and the same activity by a different route produced none. Module 4 reads why, and Module 11 writes detections that do not depend on a route.
The firewall line is not wrong, and neither is the resolver's. Both recorded exactly what reached them. The connection that left the network really did come from the proxy; the proxy made it on ws46's behalf.
Each record is accurate about its own position, and the investigator's job is to chain them: proxy log to name the client, firewall line to confirm the connection left, and the timing and destination to join the two. Here the proxy's CONNECT at 06:28:10.098 and the firewall's ALLOW-NEW from the proxy in the same second, to the address the name resolves to, make the join without ambiguity.
On a busy proxy carrying hundreds of connections a second, the join needs the proxy's own record of the outbound connection, which some proxies log and some do not.
Proxies are deliberate choke points, and their logs are among the most valuable records a network keeps for exactly this reason: they are the only place the client and the destination name appear together.
A proxy log that was not kept is a gap no other record can fill. When an organization's proxy logs are retained for a week and its firewall logs for a year, every proxied connection older than a week can be traced to the proxy and no further.
The same pattern repeats with every intermediary: a load balancer hides the client from the server behind it, a VPN concentrator hides the remote user's home address from the inside network, and a NAT gateway in a cloud network hides the instance from the internet. Each keeps its own log, and each log is the only link across it.
Choosing the Source for the Question
Each answers some questions wellWith six sources available, the work is choosing which to read for each question, then corroborating from another position:
Choosing a source for a question
The resolver's query log: it names the client
The firewall and its connection table
Zeek and Suricata: TLS server name, versions, fingerprints
Flow records: cheap to keep, no content
Which host asked for a name is the resolver's question, and only the resolver's. Whether traffic left the network, and under what address, is the firewall's and its connection table's.
What was negotiated on an encrypted connection, the server name and the TLS version and the client's fingerprint, is the sensor tools'. How much moved between two addresses over months is the flow records', because nothing else is kept that long.
Each answer is stronger when a second source in a different position agrees. The resolver says ws45 asked for cdn-sync.example.org and got 203.0.113.150; the firewall says ws45 connected to 203.0.113.150 a moment later; Zeek says the TLS handshake asked for cdn-sync.example.org.
Three positions, one story, and no packet capture needed. Each of the three could be challenged alone; together they are hard to argue with, because a challenger would have to explain three separate devices recording the same false story.
Corroboration from a different position is worth more than corroboration from the same one. Zeek and Suricata reading the same sensor's packets will agree because they read the same copy, and a fault at that sensor affects both. The resolver and the firewall are separate devices on separate hosts, so their agreement is independent in a way two tools on one sensor cannot be.
Choosing well also saves time. A question about which workstation looked up a domain goes straight to the resolver log, not to months of flow records; a question about volume over a quarter goes to the flows, not to a packet capture that covers a day. Knowing what each source is good for is most of the skill of working from logs.
The converse also holds: a question no available source can answer should be stated as such. If the only record is a flow export, the server name of an HTTPS connection is not known, and the finding says the destination address and stops there.
What Module 4 Builds On It
Every source, in depthModule 4 takes the sources one at a time, on the same lab window this lesson read. It opens with telemetry as evidence, the window in every source at once, then reads Zeek's logs field by field, Suricata's EVE events and rules, NetFlow and IPFIX with nfdump, firewall and NAT logs, the proxy, DNS and VPN logs, and the flow logs of AWS, Azure and Google Cloud from their documentation.
In order:
Module 4, in order
Its exercise is a second incident recorded by every source, with six questions to answer: for each, choose the source, answer from it, corroborate from another position, and say what remains unknown. That is the work of most network investigations in organizations without full capture.
The habit to carry into it is to read a log as a summary someone designed. Ask where its author sat, what it counts, in what unit and at what time resolution, and what it was configured to leave out. Those answers decide whether a log can carry a finding, and how to join it to the next one.
The module also reads the formats an organization is likely to meet in practice beyond the lab: cloud flow logs, which differ by provider in fields, intervals and what they omit, and the VPN server's log and status file, which are often the only record of who was connected remotely and from where.
Every lab source in the module is real software, configured as an organization might configure it, so the gaps the lessons find are the gaps a real deployment would have.
Where a setting that matters is off by default, the module says so and shows the record it would have added. That turns each lesson into a check an investigator can run against their own organization's logging before the next incident, which is when the gaps are cheapest to close.
Practice
Which record answers which question? the lab logsThe prepared evidence below is the lab window in three views: each source's record of the refused SMB attempt, each source's view of the proxied connection, and what each source adds to the direct one. Count the attempt in one unit, find the client behind the proxy, and find the server name.
If you run a home router or firewall that keeps logs, find one connection you made today in its log and list what the line records and what it does not. Then check whether the router keeps the translation it made, because without it the line names only the router.
Logs are summaries made by devices at points on the path, each in its own units and time format. One connection can be read across a resolver, a firewall and its connection table, Zeek, Suricata and a flow export, each adding something; one attempt is counted differently by each, so counts are stated in one unit; and a proxy removes the client from every record beyond it except its own.
The next lesson reads the record that comes before almost every connection: the name lookup.
Versions this lesson relies on: Zeek 9.0.1 (LTS); Suricata 8.0.8; nfdump 1.7.3.