How to Analyze 5G PCAPs in Wireshark: Step-by-Step Guide

Why most 5G PCAP analysis goes wrong

Most engineers I see open a 5G PCAP, type ngap in the filter, scroll for thirty seconds, and give up. The problem is not Wireshark. It is that 5G traffic is split across at least three planes you have to correlate manually: the radio side (RRC/NAS over MAC PDUs), the N2 control plane (NGAP carrying NAS), and the SBA mesh (HTTP/2 between AMF, AUSF, UDM, SMF). If you do not line them up by time and identifier, nothing makes sense.

This guide assumes Wireshark 4.4 or later (the nas-5gs, ngap, and http2 dissectors have all stabilized) and a capture taken at the AMF N2 interface plus a separate gNB-DU side capture for the air interface, merged with mergecap -w merged.pcap n2.pcap du.pcap.

Setting up Wireshark for 5G

Before you open anything, configure these:

• Edit > Preferences > Protocols > NGAP: set the SCTP port to 38412 (default).
• Protocols > HTTP2: enable "Reassemble HTTP/2 bodies" and set TCP ports 7777, 8080, 8443 if your core uses non-standard SBI ports.
• Protocols > NAS-5GS: enable "Dissect NAS-5GS message in NGAP" — without this, the NAS PDU shows as bytes.
• Protocols > PDCP-NR: if you are analyzing radio captures, set the security keys (RRCEnc, RRCInt, UPEnc) under the PDCP-NR keys table, otherwise everything after Security Mode Complete is encrypted.

Decryption keys

For RRC/NAS decryption you need the K_gNB or K_AMF derived keys. In a lab, dump them from the AMF or the simulator. Without them, you can still see the procedure outline but not the IE contents after Security Mode Command.

Walkthrough: a real registration capture

The sample capture below is a clean Initial Registration of a UE in standalone mode. I will use frame numbers consistent with a typical capture order.

Step 1: Filter to the procedure

Start with the filter:

ngap or nas-5gs or (http2 and http2.headers.path contains "namf")

This gives you N2 plus the SBA calls AMF makes during registration. Add && sctp.assoc_index == X once you identify the right SCTP association so you do not pull in other UEs.

Step 2: RRC Setup and Initial UE Message

The first interesting frames are on the air-interface trace:

In the N2 trace, frame 18 is the NGAP InitialUEMessage carrying the same NAS PDU. Confirm the RAN UE NGAP ID is allocated here — note its value, you will track it through the entire procedure.

Filter: ngap.RAN_UE_NGAP_ID == 1 (or whatever the value is).

Step 3: Authentication

NGAP DownlinkNASTransport carries the NAS Authentication Request. The 5G-AKA RAND and AUTN are visible inside the NAS-5GS dissection. The corresponding SBA call is from AMF to AUSF — filter:

http2.headers.path contains "ueauthentications"

You should see a POST to /nausf-auth/v1/ue-authentications and a 201 Created with the 5G-AKA challenge. The AMF then forwards the RAND/AUTN over NGAP to the UE. Cross-check that the RAND in the HTTP/2 body matches the RAND in the NAS message — if it does not, something is rewriting it (test PLMN, edge case).

Step 4: Security Mode Command

After the UE returns Authentication Response, the AMF sends Security Mode Command. From this point the NAS is integrity-protected and (usually) ciphered. If you have the K_AMF, paste it into the NAS-5GS preferences key table; otherwise the next message (Security Mode Complete) will show as a ciphered NAS PDU.

Filter to confirm the algorithms negotiated:

nas-5gs.mm.message_type == 0x5d

Look at selected_nas_security_algorithms — typically 5G-EA2/5G-IA2 (AES-128).

Step 5: SBA storm during registration

The AMF triggers a flurry of SBA calls. Useful filters:

• http2.headers.path contains "nudm-uecm" — UE Context Management registration with UDM
• http2.headers.path contains "nudm-sdm" — Subscription Data Management get
• http2.headers.path contains "npcf-am-policy-control" — AM policy

Use Statistics > Flow Graph > Limit to display filter to draw the message-sequence chart. This is the fastest way to spot a missing response — you will see a dangling arrow.

Step 6: Registration Accept and complete

The AMF sends NGAP DownlinkNASTransport carrying NAS Registration Accept (5GMM message type 0x42), with allocated 5G-GUTI, allowed NSSAI, TAI list. The UE responds with Registration Complete. The procedure ends with InitialContextSetupResponse from the gNB.

Useful Wireshark filters cheat sheet

ngap.procedureCode == 15            # InitialContextSetup
ngap.procedureCode == 14            # PDU Session Resource Setup
nas-5gs.mm.message_type == 0x41     # Registration Request
nas-5gs.mm.message_type == 0x42     # Registration Accept
nas-5gs.sm.message_type == 0xc1     # PDU Session Establishment Request
http2.headers.method == "POST" && http2.headers.path contains "smContexts"

Common pitfalls

> If your NGAP dissector shows "Malformed Packet", check the SCTP PPID. Some vendors use PPID 60 (NGAP) but encapsulate inside a non-standard SCTP stream.

• Mixed 4G/5G captures: ensure S1AP and NGAP SCTP ports do not overlap.
• HTTP/2 over TLS: you need the SSLKEYLOGFILE from the NF to decrypt. Without it, only the connection setup is visible.
• Time skew between gNB and AMF traces: always sync with NTP before capturing, or the message-flow graph will lie.

When the dissector just gives you bytes

Sooner or later nas-5gs shows you a NAS PDU as a hex blob and no field tree. Three

reasons account for nearly all of it: the message is ciphered and you have no key, the

dissector is not being invoked because the NGAP preference is off, or the message is one

your Wireshark build does not know. Only the third one requires reading the bytes yourself,

and the header is short enough to do by eye.

The first octet is the Extended Protocol Discriminator, and it tells you which half of

TS 24.501 you are in:

The second octet is the security header type, and this is the one that decides how to

read everything after it:

So a 5GMM message that starts 7e 00 41 is a plain Registration Request — mobility

management, unprotected, message type 0x41. One that starts 7e 02 is ciphered, and the

0x41 you are looking for is inside, six octets further on, and unreadable without the key.

That is the difference between "the capture is broken" and "the capture is working exactly

as designed".

The message types worth memorising are the ones that appear in every registration:

Session management messages sit behind EPD 2e and carry two extra octets before the

message type — the PDU session identity and the procedure transaction identity — so the

type of a PDU Session Establishment Request (0xc1) is the fourth octet, not the third.

Miss that and every 5GSM message looks malformed.

There is one shortcut worth knowing before you count octets at all. If the security header

type is non-zero and you have the key, put it in the NAS-5GS key table rather than decoding

by hand — Wireshark will do the whole tree. Read the bytes when the dissector cannot,

not when it merely has not been told to.

Practise on a capture that is already broken

Every filter above works on a clean trace. The skill that gets paid for is the other one:

opening a capture where the procedure does not complete and finding the octet that explains

why. That is hard to practise, because the captures you have at work are the ones you

already understand.

Three of our packet-trace labs are free with an account, and each is a real failing capture

with the frames laid out the way Wireshark shows them:

• Registration failure — a registration that

gets as far as the AMF and comes back rejected. The cause value is in the trace; the

reason it was sent is not, and that is the exercise.

• RACH failure — before there is any NAS to read at all.
• IMS SIP 401 loop — a registration that keeps being

challenged and never completes.

And when a capture shows a session establishing cleanly while nothing forwards,

the fault is in the rules rather than the exchange —

PFCP session debugging reads them back.

If you want the full set alongside the procedures they come from, the

5G Foundation track walks the registration path end to

end, and the Core & E2E specialist track covers

the SBA side — the nudm-uecm, nudm-sdm and policy calls in Step 5 above — in the

detail a filter cheat sheet cannot.

Takeaway: A 5G PCAP only tells the truth when the radio, N2, and SBA traces are

time-aligned and the security keys are loaded — everything else is guessing.