Just a quick note on how to use paravirtualized NixOS VMs on a Debian Xen with the original NixOS kernel using pygrub. Most people seem to run NixOS VMs on a NixOS Xen (and can hence use the HV's kernel) or use full virtualization (looking at you, Proxmox) with a standalone grub in the VM. It took me quite some time to figure this out on Debian so I thought I'd share.

Objectives:

• Have a fully contained NixOS (grub config, kernel and all) that can be run on any Xen HV with pygrub
• Use NixOS's grub config to generate kernel and initrd updates
• Have Xen's pygrub parse NixOS's grub config to allow kernel selection and boot NixOS
• Have everything stable and automated, no manual steps after the initial setup

Here's what I came up with:

We start with an install-only HVM xen config to be able to boot from the NixOS ISO image. The relevant Xen config parts are:

type = "hvm"
device_model_version = "qemu-xen"
bios = "seabios"
disk = [
        'phy:/dev/vgNIXOS/nixos-boot,xvda,rw',
        'phy:/dev/vgNIXOS/nixos-root,xvdb,rw',
        'file:/path/to/latest-nixos-minimal-x86_64-linux.iso,hdc:cdrom,r',
       ]
boot = "dc"

The boot LV can be as small as 500 MiB, the root LV should be at least 5 GiB. Now, start this domU with the -c parameter to get a console. It'll boot into the NixOS installer. Once that's ready, we prepare the NixOS install:

mkfs.ext4 /dev/xvda -L NIXBOOT
mkfs.ext4 /dev/xvdb -L NIXROOT

mount /dev/disk/by-label/NIXROOT /mnt
mkdir -p /mnt/boot
mount /dev/disk/by-label/NIXBOOT /mnt/boot

nixos-generate-config --root /mnt

Now comes the actual NixOS config. In hardware-configuration.nix, we need to specify the Xen modules and the block devices:

boot.initrd.availableKernelModules = [ "ata_piix" "sr_mod" "xen_blkfront" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ ];
boot.extraModulePackages = [ ];

fileSystems."/" =
  { device = "/dev/xvdb";
    fsType = "ext4";
  };

fileSystems."/boot" =
  { device = "/dev/xvda";
    fsType = "ext4";
  };

and in configuration.nix we configure the boot loader:

boot.loader.grub = {
  enable = true;
  device = "nodev";           # don't actually install grub
  configurationLimit = 3;
  fsIdentifier = "provided";  # disable uuid identifiers
  extraConfig = ''
    set root=(hd0)
  '';                         # force kernel + initrd load from boot partition
};

The combination of device = "nodev", fsIdentifier = "provided" and extraConfig makes sure that NixOS generates a grub config that allows pygrub to boot the correct kernel and initrd. Now start the NixOS installation:

nixos-install

Once this has finished, you can poweroff the domU. We're now switching to a paravirtualized domU for optimal performance. Create a new Xen config file with the relevant parts being:

bootloader = 'pygrub'
disk = [
        'phy:/dev/vgNIXOS/nixos-boot,xvda,rw',
        'phy:/dev/vgNIXOS/nixos-root,xvdb,rw',
       ]

This should now nicely boot into your NixOS system. Have fun!