I just had to run this in Windows as an administrator: bcdedit /set {bootmgr} path \EFI\ubuntu\grubx64.efi