I have recently started using Linux for virtualization on my desktop, and encountered terrible network performance between the host and (FreeBSD) guest.
Here are some representative numbers, obtained using iperf:
# Host IP: 192.168.3.254 # Guest IP: 192.168.3.1 # Host -> guest # $ guest ~> iperf -s # $ host ~> iperf -c 192.168.3.1 [ ID] Interval Transfer Bandwidth [ 3] 0.0-11.7 sec 3.00 MBytes 2.14 Mbits/sec # Guest to host # $ host ~> iperf -s # $ guest ~> iperf -c 192.168.3.254 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.63 GBytes 1.14 Gbits/sec
This is a truly abysmal result. Remember, these numbers are for transfer rates between a host and a virtualized guest -- we are not even hitting my network.
What could the problem be?
$ virsh domiflist net.anserinae.installation03 Interface Type Source Model MAC ------------------------------------------------------- vnet0 network default rtl8139 52:54:00:cd:58:29
Wait what? An
if_rl.c on FreeBSD:
* The RealTek 8139 PCI NIC redefines the meaning of 'low end.' This is * probably the worst PCI ethernet controller ever made, with the possible * exception of the FEAST chip made by SMC. The 8139 supports bus-master * DMA, but it has a terrible interface that nullifies any performance * gains that bus-master DMA usually offers. [snip] * It's impossible given this rotten design to really achieve decent * performance at 100Mbps, unless you happen to have a 400Mhz PII or * some equally overmuscled CPU to drive it.
Let's use something slightly less terrible for our guest's NIC:
$ virsh edit net.anserinae.installation03
<interface type='bridge'> [random output] <model type='rtl8139'/> </inteface>
<interface type='bridge'> [random output] <model type='virtio'/> </inteface>
Restart the virtual machine:
$ virsh reset net.anserinae.installation03
...and retry the performance tests again:
# Host -> guest [ ID] Interval Transfer Bandwidth [ 3] 0.0-11.7 sec 52.9 MBytes 45.4 Gbits/sec # Guest -> host [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 59.7 GBytes 51.3 Gbits/sec
Performance is, conservatively speaking, improved by three orders of magnitude.
I haven't tested
rtl8139 performance on any other guest operating systems, but my gut feeling here is to say that the
libvirt maintainers could have chosen a slightly less garbage NIC as the default option for newly created virtual machines.