Inter-VM Shared Memory (ivshmem) Ethernet

Overview

This application demonstrates how to use IVSHMEM Ethernet to communicate with another “cell” in the Jailhouse hypervisor. Currently only the qemu_cortex_a53 board is supported.

Building Jailhouse Linux for QEMU

Clone Jailhouse yocto project. At the time of writing, the “next” branch has some fixes that are not yet on the “master” branch:

git clone https://github.com/siemens/jailhouse-images.git
cd jailhouse-images
git checkout origin/next

Open the menu, select “QEMU ARM64 virtual target” then “Save & Build” (this will take a while):

./kas-container menu
Edit “start-qemu.sh”:
  • change -cpu cortex-a57 -> -cpu cortex-a53 under the arm64|aarch64 case

  • Enable SSH access by appending ,hostfwd=tcp::2222-:22 to -netdev user,id=net in the QEMU process arguments (2222 can be replaced with any unused port)

Start QEMU:

./start-qemu.sh arm64

This should boot Linux and drop into a shell.

Build the Zephyr sample

# From the root of the zephyr repository
west build -b qemu_cortex_a53 samples/drivers/ethernet/eth_ivshmem

Running the sample

Copy the generated zephyr.bin to the Jailhouse Linux root cell:

scp -P 2222 path/to/zephyr.bin root@localhost:/root

Jailhouse has a prebuilt Zephyr cell configuration that works for this sample “qemu-arm64-zephyr-demo”.

Back in Jailhouse Linux shell, create the Zephyr cell:

jailhouse enable /etc/jailhouse/qemu-arm64.cell
jailhouse console
jailhouse cell create /etc/jailhouse/qemu-arm64-zephyr-demo.cell

You may see a quirk here where the Linux shell stops taking input… If you get this, open a second shell via SSH:

ssh -p 2222 root@localhost

Load and start the Zephyr cell:

jailhouse cell load qemu-arm64-zephyr-demo zephyr.bin --address 0x70000000
jailhouse cell start qemu-arm64-zephyr-demo

Now you can run Zephyr and Linux shell commands to communicate between the cells.

Ping Linux from Zephyr

*** Booting Zephyr OS build v3.3.0-475-g45b9e84c6013 ***
uart:~$ net ping 192.168.19.1
PING 192.168.19.1
28 bytes from 192.168.19.1 to 192.168.19.2: icmp_seq=1 ttl=64 time=5.06 ms
28 bytes from 192.168.19.1 to 192.168.19.2: icmp_seq=2 ttl=64 time=7.99 ms
28 bytes from 192.168.19.1 to 192.168.19.2: icmp_seq=3 ttl=64 time=1.77 ms

Ping Zephyr from Linux

root@demo:~# ping -c 3 192.168.19.2
PING 192.168.19.2 (192.168.19.2) 56(84) bytes of data.
64 bytes from 192.168.19.2: icmp_seq=1 ttl=64 time=0.646 ms
64 bytes from 192.168.19.2: icmp_seq=2 ttl=64 time=1.45 ms
64 bytes from 192.168.19.2: icmp_seq=3 ttl=64 time=1.28 ms

--- 192.168.19.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2006ms
rtt min/avg/max/mdev = 0.646/1.124/1.450/0.345 ms

Run zperf / iPerf

zperf: Network Traffic Generator / iPerf can be used to perform network throughput measurements.

In Zephyr:

# Start zperf TCP server
zperf tcp download 5001

In Linux:

# Install iPerf 2.0.5
apt install wget
wget https://iperf.fr/download/ubuntu/iperf_2.0.5+dfsg1-2_arm64.deb
apt install ./iperf_2.0.5+dfsg1-2_arm64.deb
rm iperf_2.0.5+dfsg1-2_arm64.deb

# Connect iPerf TCP client
iperf -l 1K -V -c 192.168.19.2 -p 5001

Zephyr output:

TCP server started on port 5001
New TCP session started.
TCP session ended
 Duration:              10.01 s
 rate:                  57.72 Mbps

Linux output:

------------------------------------------------------------
Client connecting to 192.168.19.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.19.1 port 58430 connected with 192.168.19.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  72.2 MBytes  60.6 Mbits/sec