Skip to content

Instantly share code, notes, and snippets.

@hasezoey

hasezoey/23.3.2019 libvirt GPU Passthrought result.md

Last active September 22, 2019 10:46
Show Gist options
  • Save hasezoey/5b66158ad7f81fced4f9ddaea991c72d to your computer and use it in GitHub Desktop.
Save hasezoey/5b66158ad7f81fced4f9ddaea991c72d to your computer and use it in GitHub Desktop.
Download ZIP
what i learned in libvirt, and some wokrarounds
Raw
23.3.2019 libvirt GPU Passthrought result.md

Result of GPU Passthrought (sate 23.3.2019)

System Infomation

CPU: Intel i7-7700K (8) @ 4.50 GHz
GPU 1 (Host): NVIDIA GeForce GTX 1080 (didnt want to risk breaking it)
GPU 2 (Guest): NVIDIA GeForce GTX 950
RAM: 32GB Corsair (giving 16GB to Guest)
System: Linux Mint 19.1 Tessa x86_64 Cinnamon
Kernel: 4.15.*
Looking-Glass: git commit (current HEAD -> master, origin/master, origin/HEAD) d1c0d2b5f80cea156c18b956024b84b17736d62a
using libvirt & qemu PPA: jacob/virtualisation (because Mints (/Ubuntus?) were to old)
libvirt-bin (apt): 4.0.0-1ubuntu8.8
qemu-system-common (apt): 1:2.12+dfsg-0~18.04~ppa0
Windows iso: Win10_1803_German_x64

Final Result

a working ("instable") VM with GPU Passthrought, Looking-glass and pulseaudio-sound

How to get there

There were many problems, and still are, but i could workaround some:

Requirements

Ubuntu 18.04 or Linux Mint 19, 19.1
(packages) libvirt-bin, virt-manager, ovmf
2 graphic cards (or APU & GPU)
Intel VT-x support (or AMD equivalent)
A VM with Windows 7, 8 or 10 installed with UEFI

Get GPU Passthrought working in Ubuntu & libvirt

first edit /etc/initramfs-tools/modules
to include (above any video driver)

softdep nvidia pre: vfio-pci
vfio
vfio_iommu_type1
vfio_virqfd
options vfio_pci ids=gpu:id,gpuaudio:id
vfio_pci ids=gpu:id,gpuaudio:id
vfio_pci

replace gpu:id with your gpu pci id (it will look like 10de:1402)
replace gpuaudio:id with your gpu-audio pci id (it will look like 10de:0fba)
-> to get those ID's execute (the id's are in [])

for d in /sys/kernel/iommu_groups/*/devices/*; do
    n=${d#*/iommu_groups/*}; n=${n%%/*};
    printf 'IOMMU Group %s ' "$n";
    lspci -nns "${d##*/}"; 
done;

second step is to add those 2 pci modules in libvirt as pci devices (this is easy with virt-manager)
and you have GPU Passthrought, then you can install the drivers
but NVIDIA will probably give you error 43 (Denying to start in a virtual maschine)

NVIDIA's Error 43

NVIDIA will give you this error because they want to prevent consumer-grade GPU's to be using in VM's
to fix this we must hide kvm (and an emulated cpu)

todo this, execute virsh edit yourvm
and change <domain type='kvm'> to <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> because we need to add to the qemu commandline
and then after </devices> add this

<qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off'/>
</qemu:commandline>

this will passthrought the real CPU's info and hides KVM

Pulseaudio Passthrought

to have Audio while using Looking-glass you have to have some kind of audio device, physical or passthrought
to have it working you need to change some things:

  1. in /etc/libvirt/qemu.conf change #user = "root" to user = "your_current_user_here" and the same with #group = "root" to user = "your_current_users_group_here" and #dynamic_ownership = 1 to dynamic_ownership = 1
    -> in my case: #user = "root" to #user = "hasezoey" and #group = "root" to user = "hasezoey"
  2. give your current user the libvirt and the kvm group
  3. change your sound device to ich6 or ich9 (i used ich9)
  4. add in your ~/.pulse/default.pa (if you dont have it mkdir ~/.pulse && cp /etc/pulse/default.pa ~/.pulse/default.pa) to the end of the file (its the safest method) load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1 auth-anonymous=1
  5. add in /etc/apparmor.d/abstractions/libvirt-qemu /run/user/1000/pulse/native rw, (really important otherwise qemu will be blocked to use it)
  6. execute virsh edit yourvm and add the following under <qemu:arg value='host,kvm=off'/>
<qemu:env name='QEMU_AUDIO_DRV' value='pa'/> <!--set the audio driver to pulse audio-->
<qemu:env name='QEMU_AUDIO_DAC_FIXED_FREQ' value='96000'/> <!--setting the hz rate-->
<qemu:env name='QEMU_AUDIO_ADC_FIXED_FREQ' value='96000'/> <!--2' setting the hz rate-->
<qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/> <!--how to connect to pulseaudio-->

-> 96000hz is the highest value Windows(10) supports, and i experienced not much crackling with this value 7. in the Windows VM: open "Sound" and change the Rate to "16 bits 96000 hz" -> and finished

Looking-Glass setup

  1. downlaod the looking-glass git-repo
  2. cd into the repo and then into mkdir client/build && cd client/build
  3. execute apt-get install cmake libsdl2-dev libsdl2-ttf-dev nettle-dev libspice-protocol-dev libfontconfig1-dev libx11-dev fonts-freefont-ttf libconfig-dev to get all packages
  4. execute cmake ../ to build a makefile
  5. execute make to build
  6. the client is build
  7. execute virsh edit yourvm and add the following before </device>
<shmem name='looking-glass'>
  <model type='ivshmem-plain'/>
  <size unit='M'>32</size> <!--reade more for the value here https://looking-glass.hostfission.com/quickstart/linux/libvirt -->
</shmem>

  1. edit /etc/apparmor.d/abstractions/libvirt-qemu and add /dev/shm/looking-glass rw,, like on PulseAudio

  2. you can startup your vm

  3. in your vm you will find a "Standart RAM-Controller" in SystemDevices in the "Device-Manager" install the driver "virtio-win10-prewhql-0.1-161.zip" from here https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/upstream-virtio/ and extract it, then install it

  4. then download the looking-glass-server exe from here https://github.com/gnif/LookingGlass/releases You will need to have your GPU Driver installed and working

  5. execute it, and it should run

Update 11.5.2019

after moving drive from SATA to VirtIO the "IO Lockup" problem is gone and more reliable -> with kernel 5.0.*

Final Values and Configs

Final VM Config:

config from virsh edit:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win10-uefi</name>
  <uuid>64d0a368-c184-42fb-be9b-dc954a400e9b</uuid>
  <title>Windows 10 - UEFI</title>
  <memory unit='KiB'>16842752</memory>
  <currentMemory unit='KiB'>16842752</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-2.12'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10-uefi_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
  </features>
  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none' io='native'/>
      <source file='/projects/VM/win10-uefi.qcow2'/>
      <target dev='sda' bus='sata'/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='sdb' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/projects/VM/Destiny-Disk.img'/>
      <target dev='sdc' bus='sata'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='usb' index='0' model='piix3-uhci'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:49:04:0c'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
    </graphics>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='1.1'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='1.2'/>
    </redirdev>
    <hub type='usb'>
      <address type='usb' bus='0' port='1'/>
    </hub>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </memballoon>
    <shmem name='looking-glass'>
      <model type='ivshmem-plain'/>
      <size unit='M'>32</size>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/>
    </shmem>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off'/>
    <qemu:arg value='-smp'/>
    <qemu:arg value='cores=4,threads=1,sockets=1'/>
    <qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
    <qemu:env name='QEMU_AUDIO_DAC_FIXED_FREQ' value='96000'/>
    <qemu:env name='QEMU_AUDIO_ADC_FIXED_FREQ' value='96000'/>
    <qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/>
  </qemu:commandline>
</domain>

Final AppArmor

/etc/apparmor.d/abstractions/libvirt-qemu

# Last Modified: Wed Sep 3 21:52:03 2014

  #include <abstractions/base>
  #include <abstractions/consoles>
  #include <abstractions/nameservice>

  # required for reading disk images
  capability dac_override,
  capability dac_read_search,
  capability chown,

  # needed to drop privileges
  capability setgid,
  capability setuid,

  # for 9p
  capability fsetid,
  capability fowner,

  network inet stream,
  network inet6 stream,

  ptrace (readby, tracedby) peer=/usr/sbin/libvirtd,

  signal (receive) peer=/usr/sbin/libvirtd,

  /dev/net/tun rw,
  /dev/kvm rw,
  /dev/ptmx rw,
  /dev/kqemu rw,
# custom line below
  /dev/shm/looking-glass rw,
  /run/user/1000/pulse/native rw,
  
  @{PROC}/*/status r,
  # When qemu is signaled to terminate, it will read cmdline of signaling
  # process for reporting purposes. Allowing read access to a process
  # cmdline may leak sensitive information embedded in the cmdline.
  @{PROC}/@{pid}/cmdline r,
  # Per man(5) proc, the kernel enforces that a thread may
  # only modify its comm value or those in its thread group.
  owner @{PROC}/@{pid}/task/@{tid}/comm rw,
  @{PROC}/sys/kernel/cap_last_cap r,
  owner @{PROC}/*/auxv r,
  @{PROC}/sys/vm/overcommit_memory r,

  # For hostdev access. The actual devices will be added dynamically
  /sys/bus/usb/devices/ r,
  /sys/devices/**/usb[0-9]*/** r,
  # libusb needs udev data about usb devices (~equal to content of lsusb -v)
  /run/udev/data/c16[6,7]* r,
  /run/udev/data/c18[0,8,9]* r,
  /run/udev/data/+usb* r,

  # WARNING: this gives the guest direct access to host hardware and specific
  # portions of shared memory. This is required for sound using ALSA with kvm,
  # but may constitute a security risk. If your environment does not require
  # the use of sound in your VMs, feel free to comment out or prepend 'deny' to
  # the rules for files in /dev.
  /{dev,run}/shm r,
  /{dev,run}/shmpulse-shm* r,
  /{dev,run}/shmpulse-shm* rwk,
  /dev/snd/* rw,
  capability ipc_lock,
  # spice
  owner /{dev,run}/shm/spice.* rw,
  # 'kill' is not required for sound and is a security risk. Do not enable
  # unless you absolutely need it.
  deny capability kill,

  # Uncomment the following if you need access to /dev/fb*
  #/dev/fb* rw,

  /etc/pulse/client.conf r,
  @{HOME}/.pulse-cookie rwk,
  owner /root/.pulse-cookie rwk,
  owner /root/.pulse/ rw,
  owner /root/.pulse/* rw,
  /usr/share/alsa/** r,
  owner /tmp/pulse-*/ rw,
  owner /tmp/pulse-*/* rw,
  /var/lib/dbus/machine-id r,

  # access to firmware's etc
  /usr/share/kvm/** r,
  /usr/share/qemu/** r,
  /usr/share/qemu-kvm/** r,
  /usr/share/bochs/** r,
  /usr/share/openbios/** r,
  /usr/share/openhackware/** r,
  /usr/share/proll/** r,
  /usr/share/vgabios/** r,
  /usr/share/seabios/** r,
  /usr/share/misc/sgabios.bin r,
  /usr/share/ovmf/** r,
  /usr/share/OVMF/** r,
  /usr/share/AAVMF/** r,
  /usr/share/qemu-efi/** r,
  /usr/share/slof/** r,

  # pki for libvirt-vnc and libvirt-spice (LP: #901272, #1690140)
  /etc/pki/CA/ r,
  /etc/pki/CA/* r,
  /etc/pki/libvirt{,-spice,-vnc}/ r,
  /etc/pki/libvirt{,-spice,-vnc}/** r,

  # the various binaries
  /usr/bin/kvm rmix,
  /usr/bin/qemu rmix,
  /usr/bin/qemu-kvm rmix,
  /usr/bin/qemu-system-aarch64 rmix,
  /usr/bin/qemu-system-alpha rmix,
  /usr/bin/qemu-system-arm rmix,
  /usr/bin/qemu-system-cris rmix,
  /usr/bin/qemu-system-i386 rmix,
  /usr/bin/qemu-system-lm32 rmix,
  /usr/bin/qemu-system-m68k rmix,
  /usr/bin/qemu-system-microblaze rmix,
  /usr/bin/qemu-system-microblazeel rmix,
  /usr/bin/qemu-system-mips rmix,
  /usr/bin/qemu-system-mips64 rmix,
  /usr/bin/qemu-system-mips64el rmix,
  /usr/bin/qemu-system-mipsel rmix,
  /usr/bin/qemu-system-moxie rmix,
  /usr/bin/qemu-system-or32 rmix,
  /usr/bin/qemu-system-ppc rmix,
  /usr/bin/qemu-system-ppc64 rmix,
  /usr/bin/qemu-system-ppcemb rmix,
  /usr/bin/qemu-system-s390x rmix,
  /usr/bin/qemu-system-sh4 rmix,
  /usr/bin/qemu-system-sh4eb rmix,
  /usr/bin/qemu-system-sparc rmix,
  /usr/bin/qemu-system-sparc64 rmix,
  /usr/bin/qemu-system-tricore rmix,
  /usr/bin/qemu-system-unicore32 rmix,
  /usr/bin/qemu-system-x86_64 rmix,
  /usr/bin/qemu-system-xtensa rmix,
  /usr/bin/qemu-system-xtensaeb rmix,
  /usr/bin/qemu-aarch64 rmix,
  /usr/bin/qemu-alpha rmix,
  /usr/bin/qemu-arm rmix,
  /usr/bin/qemu-armeb rmix,
  /usr/bin/qemu-cris rmix,
  /usr/bin/qemu-i386 rmix,
  /usr/bin/qemu-m68k rmix,
  /usr/bin/qemu-microblaze rmix,
  /usr/bin/qemu-microblazeel rmix,
  /usr/bin/qemu-mips rmix,
  /usr/bin/qemu-mips64 rmix,
  /usr/bin/qemu-mips64el rmix,
  /usr/bin/qemu-mipsel rmix,
  /usr/bin/qemu-mipsn32 rmix,
  /usr/bin/qemu-mipsn32el rmix,
  /usr/bin/qemu-or32 rmix,
  /usr/bin/qemu-ppc rmix,
  /usr/bin/qemu-ppc64 rmix,
  /usr/bin/qemu-ppc64abi32 rmix,
  /usr/bin/qemu-ppc64le rmix,
  /usr/bin/qemu-s390x rmix,
  /usr/bin/qemu-sh4 rmix,
  /usr/bin/qemu-sh4eb rmix,
  /usr/bin/qemu-sparc rmix,
  /usr/bin/qemu-sparc32plus rmix,
  /usr/bin/qemu-sparc64 rmix,
  /usr/bin/qemu-unicore32 rmix,
  /usr/bin/qemu-x86_64 rmix,
  # for Debian/Ubuntu qemu-block-extra / RPMs qemu-block-* (LP: #1554761)
  /usr/{lib,lib64}/qemu/*.so mr,
  /usr/lib/@{multiarch}/qemu/*.so mr,

  # swtpm
  /{usr/,}bin/swtpm rmix,
  /usr/{lib,lib64}/libswtpm_libtpms.so mr,
  /usr/lib/@{multiarch}/libswtpm_libtpms.so mr,

  # for save and resume
  /{usr/,}bin/dash rmix,
  /{usr/,}bin/dd rmix,
  /{usr/,}bin/cat rmix,

  # for restore
  /{usr/,}bin/bash rmix,

  # for usb access
  /dev/bus/usb/ r,
  /etc/udev/udev.conf r,
  /sys/bus/ r,
  /sys/class/ r,

  # for rbd
  /etc/ceph/ceph.conf r,

  # Various functions will need to enumerate /tmp (e.g. ceph), allow the base
  # dir and a few known functions like samba support.
  # We want to avoid to give blanket rw permission to everything under /tmp,
  # users are expected to add site specific addons for more uncommon cases.
  # Qemu processes usually all run as the same users, so the "owner"
  # restriction prevents access to other services files, but not across
  # different instances.
  # This is a tradeoff between usability and security - if paths would be more
  # predictable that would be preferred - at least for write rules we would
  # want more unique paths per rule.
  /{,var/}tmp/ r,
  owner /{,var/}tmp/**/ r,

  # Various functions will need to enumerate /tmp (e.g. ceph), allow the base
  # dir and a few known functions like samba support.
  # We want to avoid to give blanket rw permission to everything under /tmp,
  # users are expected to add site specific addons for more uncommon cases.
  # Qemu processes usually all run as the same users, so the "owner"
  # restriction prevents access to other services files, but not across
  # different instances.
  # This is a tradeoff between usability and security - if paths would be more
  # predictable that would be preferred - at least for write rules we would
  # want more unique paths per rule.
  /{,var/}tmp/ r,
  owner /{,var/}tmp/**/ r,

  # for file-posix getting limits since 9103f1ce
  /sys/devices/**/block/*/queue/max_segments r,

  # for ppc device-tree access
  @{PROC}/device-tree/ r,
  @{PROC}/device-tree/** r,
  /sys/firmware/devicetree/** r,

  # allow connect with openGraphicsFD to work
  unix (send, receive) type=stream addr=none peer=(label=/usr/sbin/libvirtd),

  # allow connect with openGraphicsFD to work
  unix (send, receive) type=stream addr=none peer=(label=/usr/sbin/libvirtd),

  # allow access to charm-specific ceph config (LP: #1403648).
  # No more silencing spurious denials as it can more critically hide other issues (LP: #1719579)
  # Also allow the optional asok key that might be enabled by the charm (LP: #1779674)
  /var/lib/charm/*/ceph.conf r,
  /run/ceph/rbd-client-*.asok rw,

  # kvm.powerpc executes/accesses this
  /{usr/,}bin/uname rmix,
  /{usr/,}sbin/ppc64_cpu rmix,
  /{usr/,}bin/grep rmix,
  /sys/devices/system/cpu/subcores_per_core r,
  /sys/devices/system/cpu/cpu*/online r,

  # for gathering information about available host resources
  /sys/devices/system/cpu/ r,
  /sys/devices/system/node/ r,
  /sys/devices/system/node/node[0-9]*/meminfo r,
  /sys/module/vhost/parameters/max_mem_regions r,

  # silence refusals to open lttng files (see LP: #1432644)
  deny /dev/shm/lttng-ust-wait-* r,
  deny /run/shm/lttng-ust-wait-* r,

  # for vfio hotplug on systems without static vfio (LP: #1775777)
  /dev/vfio/vfio rw,

  # required for sasl GSSAPI plugin
  /etc/gss/mech.d/ r,
  /etc/gss/mech.d/* r,

  # Site-specific additions and overrides. See local/README for details.
  #include <local/abstractions/libvirt-qemu>

Final initramfs-tools

/etc/initramfs-tools/modules

# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax:  module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
#
# Examples:
#
# raid1
# sd_mod
softdep nvidia pre: vfio-pci
vfio
vfio_iommu_type1
vfio_virqfd
options vfio_pci ids=10de:1402,10de:0fba
vfio_pci ids=10de:1402,10de:0fba
vfio_pci
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment