This is a small guide on how to collect PCAPs for network traffic from/to a virtual machine running on VMware ESXi. Scripts have been provided by VMware GSS and can be found here and used at your own risk.
(I get it, who would want to use Python scripts downloaded from a random website and run them on your ESXi, feel free to look into the code after download, or else request them from VMware)
The scripts can help to collect on multiple PortIDs, divide the PCAPs into smaller files, and rotate the logs for each x minutes and it can help to monitor the capture that is ongoing.
If you don’t want to use the scripts, then there is also a more manual way to run the collection here.
Finding the VM port IDs
Connect to the ESXi host with SSH and list the running VMs.
[root@dc1esxedge1-02:~] esxcli network vm list World ID Name Num Ports Networks -------- ---------------- --------- -------- 68018705 nsxedgenode1 3 dvportgroup-914135, dvportgroup-914135
Find the VM that you want to collect from, let’s take WorldID 68018705 and check the switch port IDs for it
[root@esxedge1-02:~] esxcli network vm port list -w 68018092 Port ID: 67108942 vSwitch: DSVNSX Portgroup: dvportgroup-914135 DVPort ID: 109 MAC Address: 00:50:56:ab:82:59 IP Address: 0.0.0.0 Team Uplink: vmnic2 Uplink Port ID: 2214592556 Active Filters: Port ID: 67108943 vSwitch: DSVNSX Portgroup: dvportgroup-914135 DVPort ID: 108 MAC Address: 00:50:56:ab:06:b0 IP Address: 0.0.0.0 Team Uplink: vmnic2 Uplink Port ID: 2214592556 Active Filters: Port ID: 67108944 vSwitch: DSVNSX Portgroup: dvportgroup-914134 DVPort ID: 98 MAC Address: 00:50:56:ab:93:ab IP Address: 0.0.0.0 Team Uplink: vmnic1 Uplink Port ID: 2214592558 Active Filters:
In this case, we note down the following port IDs and save them for later
67108942 67108943 67108945 67108946
Start the collection with scripts
Upload the Python script to the /tmp/ on the ESXi host. You also need a folder on the local datastore of the ESXi host where the pcap logs can be stored.
Prepare and run the first script where we provide the PortIDs and where to store the PCAPs
[root@esxedge1-02:~] python /tmp/prepare_pktcap.py -p 67108942 -p 67108943 -p 67108945 -p67108946 -u -d /vmfs/volu mes/prd.dc1esxedge1-02/capture/42434546 -o /tmp/rotating_cap.sh -G 15m -r 3600 The current vmci heap is 99% free and so far there have been 0 allocation failures [root@esxedge1-02:~]
Next, we will need to start the rotating log script, this will start the PCAPs collection threads and rotate the logs each 15 min as we defined in the prepare_pktcap script. The script should be left running in the SSH connection. if you stop it, the collection will also stop.
[root@esxedge1-02:~] /tmp/rotating_cap.py Dump: 294272, broken : 0, drop: 0, file err: 0. Dump: 175296, broken : 0, drop: 0, file err: 0. Dump: 371584, broken : 0, drop: 0, file err: 0. Dump: 317248, broken : 0, drop: 0, file err: 0. Dump: 371648, broken : 0, drop: 0, file err: 0. Dump: 317312, broken : 0, drop: 0, file err: 0.
Monitor the collection
The last script is for doing the monitoring of the sessions. You will need to open a new SSH session to the host to run this script.
[root@esxedge1-02:~] /tmp/pktcap_sessions.py -l The vmci heap is 58% free session portID devName 579 67108942 580 67108942 581 67108945 582 67108946 583 67108943 584 2214592556 vmnic2 585 67108946 586 2214592556 vmnic2 587 67108943 588 67108945 589 2214592558 vmnic1 590 2214592558 vmnic1
From the output, we can see that it collects the PortIDs that we have defined, but it also collects PCAP for the VMNics. This is valuable to us if we need to compare traffic on what is going in on the host and to the VM and visa versa.
Looking at the output that is stored on the datastore
[root@esxedge1-02:/vmfs/volumes/5fd89aca-1f6f344a-f65f-043f72c0064a/capture/42434546] ls dc1esxedge1-02_2023-11-29T07_42_11_p67108942_d0_sna.pcap .... dc1esxedge1-02_2023-11-29T07_42_11_p67108945_d0_sna.pcap.log dc1esxedge1-02_2023-11-29T07_42_11_pvmnic2_d0_sna.pcap_rot.log dc1esxedge1-02_2023-11-29T07_42_11_p67108945_d0_sna.pcap_rot.log dc1esxedge1-02_2023-11-29T07_42_11_pvmnic2_d1_sna.pcap dc1esxedge1-02_2023-11-29T07_42_11_p67108945_d1_sna.pcap dc1esxedge1-02_2023-11-29T07_42_11_pvmnic2_d1_sna.pcap.log dc1esxedge1-02_2023-11-29T07_42_11_p67108945_d1_sna.pcap.log dc1esxedge1-02_2023-11-29T07_42_11_pvmnic2_d1_sna.pcap_rot.log dc1esxedge1-02_2023-11-29T07_42_11_p67108945_d1_sna.pcap_rot.log killfile
We can see the PCAPs for each of the PortIDs and VMnics, and it will rotate every 15 minutes.
Stopping the collection
You might have noticed the “killfile” from above. Remote this file and the collection will stop.
[root@esxedge1-02:~] rm -rf /vmfs/volumes/5fd89aca-1f6f344a-f65f-043f72c0064a/capture/42434546/killfile
The scripts are handy because they rotate the logs and have a way to monitor and kill the collections. This way we don’t have to manually kill processes on the ESXi host.
When the collection is done, you can copy out the logs for further analysis, I found the Filezilla SFTP client to be the fastest way of copying out the data.
If out want to merge the PCAPs afterward, on MacOS, you can do it with
mergecap -w merged.pcap *.pcap
If you find that you can’t start the rotating logs script it might be because you have tried to start it before and it somehow stall. You can find the process IDs for it and kill it manually.
[root@esxedge1-02:~] ps -Tcjstv | grep -i rotating_cap [root@esxedge1-02:~] kill