Bug 1024022

Summary: regression in btrfs on crypto root
Product: [openSUSE] openSUSE Distribution Reporter: Olaf Hering <ohering>
Component: BasesystemAssignee: Daniel Molkentin <daniel>
Status: RESOLVED WONTFIX QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: ohering, trenn
Version: Leap 42.1   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: Add all crypto modules to initrd if LUKS fs needs to be mounted in initrd

Description Olaf Hering 2017-02-07 12:18:09 UTC
I have an encrypted partition which contains a btrfs. This btrfs contains several subvolumes with 13.1/13.2/42.1/TW/SLE12. At some point each one was able to boot into its root filesystem.

But currently this fails with SLE12SP1 and 42.1 when initrd is asking for the passphrase. Kernel reports:

device-mapper: table: 254:0: crypt: Error allocating crypto tfm

Right now its only possible to boot into 13.2 and Tumbleweed.
Comment 1 Olaf Hering 2017-02-07 13:07:39 UTC
lsmod in initrd, tw vs. sle12. sle12 has 43 modules, while tw has just 87.

--- /dev/shm/tw
+++ /dev/shm/sl12
@@ -1,87 +1,143 @@
-kernel/sound/soundcore.ko
-kernel/sound/pci/hda/snd-hda-intel.ko
-kernel/sound/pci/hda/snd-hda-codec.ko
-kernel/sound/pci/hda/snd-hda-codec-realtek.ko
-kernel/sound/pci/hda/snd-hda-codec-generic.ko
-kernel/sound/hda/snd-hda-core.ko
-kernel/sound/core/snd.ko
-kernel/sound/core/snd-timer.ko
-kernel/sound/core/snd-pcm.ko
-kernel/sound/core/snd-hwdep.ko
-kernel/net/wireless/cfg80211.ko
-kernel/net/rfkill/rfkill.ko
-kernel/net/packet/af_packet.ko
-kernel/net/mac80211/mac80211.ko
-kernel/net/llc/llc.ko
-kernel/net/bridge/bridge.ko
-kernel/net/802/stp.ko
+kernel/net/ceph/libceph.ko
 kernel/lib/raid6/raid6_pq.ko
-kernel/fs/fuse/fuse.ko
+kernel/lib/libcrc32c.ko
+kernel/fs/configfs/configfs.ko
 kernel/fs/btrfs/btrfs.ko
-kernel/drivers/watchdog/iTCO_wdt.ko
-kernel/drivers/watchdog/iTCO_vendor_support.ko
-kernel/drivers/video/fbdev/core/sysimgblt.ko
-kernel/drivers/video/fbdev/core/sysfillrect.ko
-kernel/drivers/video/fbdev/core/syscopyarea.ko
-kernel/drivers/video/fbdev/core/fb_sys_fops.ko
+kernel/fs/autofs4/autofs4.ko
+kernel/drivers/virtio/virtio_ring.ko
+kernel/drivers/virtio/virtio.ko
 kernel/drivers/usb/storage/usb-storage.ko
 kernel/drivers/usb/storage/uas.ko
+kernel/drivers/usb/host/xhci-hcd.ko
 kernel/drivers/usb/host/uhci-hcd.ko
+kernel/drivers/usb/host/ohci-pci.ko
+kernel/drivers/usb/host/ohci-hcd.ko
+kernel/drivers/usb/host/ehci-platform.ko
 kernel/drivers/usb/host/ehci-pci.ko
 kernel/drivers/usb/host/ehci-hcd.ko
 kernel/drivers/usb/core/usbcore.ko
+kernel/drivers/usb/common/usb-common.ko
+kernel/drivers/uio/uio.ko
+kernel/drivers/target/target_core_mod.ko
+kernel/drivers/target/loopback/tcm_loop.ko
+kernel/drivers/scsi/ufs/ufshcd.ko
 kernel/drivers/scsi/sr_mod.ko
 kernel/drivers/scsi/sg.ko
+kernel/drivers/scsi/sd_mod.ko
+kernel/drivers/scsi/scsi_transport_srp.ko
+kernel/drivers/scsi/scsi_transport_sas.ko
+kernel/drivers/scsi/scsi_transport_iscsi.ko
+kernel/drivers/scsi/scsi_transport_fc.ko
+kernel/drivers/scsi/scsi_tgt.ko
+kernel/drivers/scsi/scsi_mod.ko
+kernel/drivers/scsi/scsi_debug.ko
+kernel/drivers/scsi/ppa.ko
+kernel/drivers/scsi/osd/osd.ko
+kernel/drivers/scsi/osd/libosd.ko
+kernel/drivers/scsi/libiscsi.ko
+kernel/drivers/scsi/libfc/libfc.ko
+kernel/drivers/scsi/imm.ko
+kernel/drivers/scsi/fcoe/libfcoe.ko
+kernel/drivers/scsi/fcoe/fcoe.ko
+kernel/drivers/scsi/eata.ko
 kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko
 kernel/drivers/scsi/device_handler/scsi_dh_emc.ko
 kernel/drivers/scsi/device_handler/scsi_dh_alua.ko
-kernel/drivers/pci/hotplug/shpchp.ko
-kernel/drivers/parport/parport_pc.ko
+kernel/drivers/scsi/device_handler/scsi_dh.ko
+kernel/drivers/scsi/bnx2fc/bnx2fc.ko
+kernel/drivers/pcmcia/pcmcia_core.ko
 kernel/drivers/parport/parport.ko
-kernel/drivers/net/wireless/ath/ath5k/ath5k.ko
-kernel/drivers/net/wireless/ath/ath.ko
-kernel/drivers/net/fjes/fjes.ko
-kernel/drivers/net/ethernet/marvell/sky2.ko
-kernel/drivers/mfd/mfd-core.ko
-kernel/drivers/mfd/lpc_ich.ko
+kernel/drivers/net/ethernet/broadcom/cnic.ko
+kernel/drivers/mtd/ssfdc.ko
+kernel/drivers/mtd/rfd_ftl.ko
+kernel/drivers/mtd/mtdswap.ko
+kernel/drivers/mtd/mtdblock_ro.ko
+kernel/drivers/mtd/mtdblock.ko
+kernel/drivers/mtd/mtd_blkdevs.ko
+kernel/drivers/mtd/mtd.ko
+kernel/drivers/mtd/inftl.ko
+kernel/drivers/mtd/ftl.ko
+kernel/drivers/mmc/host/tifm_sd.ko
+kernel/drivers/mmc/host/sdhci.ko
+kernel/drivers/mmc/host/sdhci-pltfm.ko
+kernel/drivers/mmc/core/mmc_core.ko
+kernel/drivers/misc/tifm_core.ko
+kernel/drivers/memstick/core/mspro_block.ko
+kernel/drivers/memstick/core/ms_block.ko
+kernel/drivers/memstick/core/memstick.ko
+kernel/drivers/md/persistent-data/dm-persistent-data.ko
+kernel/drivers/md/dm-zero.ko
+kernel/drivers/md/dm-verity.ko
+kernel/drivers/md/dm-thin-pool.ko
+kernel/drivers/md/dm-snapshot.ko
+kernel/drivers/md/dm-service-time.ko
+kernel/drivers/md/dm-round-robin.ko
 kernel/drivers/md/dm-region-hash.ko
+kernel/drivers/md/dm-queue-length.ko
 kernel/drivers/md/dm-multipath.ko
 kernel/drivers/md/dm-mod.ko
 kernel/drivers/md/dm-mirror.ko
 kernel/drivers/md/dm-log.ko
+kernel/drivers/md/dm-log-userspace.ko
+kernel/drivers/md/dm-delay.ko
 kernel/drivers/md/dm-crypt.ko
-kernel/drivers/input/serio/serio_raw.ko
-kernel/drivers/input/misc/pcspkr.ko
-kernel/drivers/input/joydev.ko
-kernel/drivers/iio/light/acpi-als.ko
-kernel/drivers/iio/industrialio.ko
-kernel/drivers/iio/buffer/kfifo_buf.ko
-kernel/drivers/i2c/busses/i2c-i801.ko
+kernel/drivers/md/dm-cache.ko
+kernel/drivers/md/dm-cache-cleaner.ko
+kernel/drivers/md/dm-bufio.ko
+kernel/drivers/md/dm-bio-prison.ko
+kernel/drivers/md/bcache/bcache.ko
+kernel/drivers/infiniband/ulp/srp/ib_srp.ko
+kernel/drivers/infiniband/core/ib_sa.ko
+kernel/drivers/infiniband/core/ib_mad.ko
+kernel/drivers/infiniband/core/ib_core.ko
+kernel/drivers/infiniband/core/ib_cm.ko
+kernel/drivers/infiniband/core/ib_addr.ko
 kernel/drivers/i2c/algos/i2c-algo-bit.ko
-kernel/drivers/hwmon/coretemp.ko
+kernel/drivers/hid/hid-generic.ko
 kernel/drivers/gpu/drm/i915/i915.ko
 kernel/drivers/gpu/drm/drm_kms_helper.ko
 kernel/drivers/gpu/drm/drm.ko
-kernel/drivers/cpufreq/acpi-cpufreq.ko
-kernel/drivers/char/tpm/tpm_tis_core.ko
-kernel/drivers/char/tpm/tpm_tis.ko
-kernel/drivers/char/tpm/tpm.ko
-kernel/drivers/char/ppdev.ko
+kernel/drivers/crypto/padlock-aes.ko
 kernel/drivers/cdrom/cdrom.ko
+kernel/drivers/block/rbd.ko
+kernel/drivers/block/pktcdvd.ko
+kernel/drivers/block/osdblk.ko
+kernel/drivers/block/nbd.ko
+kernel/drivers/block/aoe/aoe.ko
+kernel/drivers/ata/pata_acpi.ko
+kernel/drivers/ata/libata.ko
+kernel/drivers/ata/libahci.ko
 kernel/drivers/ata/ata_piix.ko
 kernel/drivers/ata/ata_generic.ko
+kernel/drivers/ata/ahci.ko
 kernel/drivers/acpi/video.ko
-kernel/drivers/acpi/thermal.ko
-kernel/drivers/acpi/fan.ko
 kernel/drivers/acpi/button.ko
-kernel/drivers/acpi/battery.ko
-kernel/drivers/acpi/ac.ko
 kernel/crypto/xor.ko
-kernel/crypto/crypto_simd.ko
+kernel/crypto/twofish_common.ko
+kernel/crypto/tcrypt.ko
+kernel/crypto/lrw.ko
+kernel/crypto/gf128mul.ko
+kernel/crypto/drbg.ko
+kernel/crypto/crypto_user.ko
+kernel/crypto/crypto_null.ko
 kernel/crypto/cryptd.ko
+kernel/crypto/cast_common.ko
+kernel/crypto/blowfish_common.ko
+kernel/crypto/async_tx/async_xor.ko
+kernel/crypto/async_tx/async_tx.ko
+kernel/crypto/async_tx/async_raid6_recov.ko
+kernel/crypto/async_tx/async_pq.ko
+kernel/crypto/async_tx/async_memcpy.ko
 kernel/crypto/arc4.ko
+kernel/crypto/ansi_cprng.ko
 kernel/crypto/algif_skcipher.ko
+kernel/crypto/algif_hash.ko
 kernel/crypto/af_alg.ko
-kernel/arch/x86/kernel/msr.ko
+kernel/arch/x86/crypto/sha256-ssse3.ko
+kernel/arch/x86/crypto/sha1-ssse3.ko
 kernel/arch/x86/crypto/glue_helper.ko
+kernel/arch/x86/crypto/crct10dif-pclmul.ko
+kernel/arch/x86/crypto/crc32c-intel.ko
+kernel/arch/x86/crypto/aesni-intel.ko
 kernel/arch/x86/crypto/aes-x86_64.ko
+kernel/arch/x86/crypto/ablk_helper.ko
Comment 2 Olaf Hering 2017-02-07 13:09:32 UTC
Appearently last boot into 42.1/sle12 was 2016-08-19.
Comment 3 Olaf Hering 2017-02-07 13:16:48 UTC
# cryptsetup luksDump /dev/sda8 
LUKS header information for /dev/sda8

Version:        1
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha1
...
Comment 4 Olaf Hering 2017-02-07 15:25:51 UTC
Appearently the xts module is not included. Which looks like a dract bug.
Comment 5 Olaf Hering 2017-02-07 17:11:25 UTC
Appearently the reason is that 'kernel=`uname -r`', which obviously breaks if the initrd is created in a chroot.
Comment 6 Olaf Hering 2017-02-07 17:39:09 UTC
Likeley the bug is a plain mkinitrd calls dracut with '--logfile <file> --force kernelversion'. kernelversion is assigned to outfile, instead of kernel. Then it goes downhill. Once that is fixed the initrd contains the correct list of modules.
Comment 7 Olaf Hering 2017-02-08 09:28:03 UTC
Another bug is the handling of crypt, or rather the lack of knowledge about crypt.

Somewhere dracut does a instmods dm_crypt =crypto. The last arg is supposed to copy the entry directory. But then it processes each module, finds that xts has aliases, none of these aliases is in the host_aliases array, and throws it away.
Comment 8 Daniel Molkentin 2017-02-17 11:34:11 UTC
Some of the assessments were red herrings:

1. instmods dm_crypt =crypto does not include xts because the call is host_only. This implies that in order to get included, the module must be in the running kernel. As I don't have the module either with LUKSCryto and Ciphermod: xts-plain64 , I assume it's at least optional, correct? i.e. it will not prevent booting the system.

If this is correct, then we'll have to add xts to the install initrd.

2. I am not sure about https://bugzilla.suse.com/show_bug.cgi?id=1024022#c6. Can you paste the fix you applied?
Comment 9 Thomas Renninger 2017-02-17 18:10:50 UTC
Created attachment 714644 [details]
Add all crypto modules to initrd if LUKS fs needs to be mounted in initrd

As crypto dracut module only is included if LUKS fs is found/needed, it should be ok to blindly add all crypto modules to initrd then.

Does this (partly) help?
Comment 10 Daniel Molkentin 2017-02-28 15:59:46 UTC
ohering: Ping?
Comment 11 Olaf Hering 2017-03-03 10:09:32 UTC
This patch may help for the "=dir" case. But is it really correct?
What is "=dir" supposed to do? If its really "copy the entire directory unconditional", why does it even look at the current host?
Also this host_only thing looks bogus. If it is known that a certain feature is required, why would it matter if a given module has aliases or not?
Comment 12 Tomáš Chvátal 2018-04-13 15:25:03 UTC
This is automated batch bugzilla cleanup.

The openSUSE 42.1 changed to end-of-life (EOL [1]) status. As such
it is no longer maintained, which means that it will not receive any
further security or bug fix updates.
As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
openSUSE, or you can still observe it under openSUSE Leap 15.0, please
feel free to reopen this bug against that version (see the "Version"
component in the bug fields), or alternatively open
a new ticket.

Thank you for reporting this bug and we are sorry it could not be fixed
during the lifetime of the release.

[1] https://en.opensuse.org/Lifetime
Comment 13 Olaf Hering 2019-06-13 10:16:53 UTC
This happens to work with newer kernels because CONFIG_CRYPT_XTS is compiled into the kernel.

/usr/lib/dracut/modules.d/90crypt/module-setup.sh:installkernel may still need some update because it has apparently no knowledge about the underlying crypto configuration. Instead of copying the entrire '=crypto' directory, it should gain some knowledge what driver is required for the block device.