Bug 987358

Summary: ucode-intel update in TW snapshot 0629 breaks system start-up
Product: [openSUSE] openSUSE Tumbleweed Reporter: Robby Engelmann <robby.engelmann>
Component: BasesystemAssignee: Cristian Rodríguez <crrodriguez>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Major    
Priority: P1 - Urgent CC: crrodriguez, dimstar, dmueller, forgotten__FQzTs08vU, jeffm, mmarek, robby.engelmann, tiwai, trenn
Version: Current   
Target Milestone: ---   
Hardware: x86-64   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: hwinfo
dracut output
dmidecode

Description Robby Engelmann 2016-07-01 12:53:56 UTC
I picked the snapshot from openqa and updated the system using zypper dup 
(from  http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-oss-i586-x86_64-Snapshot20160629).
I did not see any fails during the upgrade. On the next restart, grub started 
normally. Then the screen with "loading kernel..." appears and the startup 
stops at "loading initrd...". Nothing happens from there.

Updating all other packages except for ucode-intel leads to a working system.
Comment 1 Robby Engelmann 2016-07-01 12:58:52 UTC
Created attachment 682820 [details]
hwinfo
Comment 2 Robby Engelmann 2016-07-01 14:25:58 UTC
some more infos: I am using a crypt-lvm setup on skylake.
Zaitor mentioned that he already updated two intel machines without issue. He did not use crypt-lvm setup on both.
Comment 3 Bernhard Wiedemann 2016-07-01 16:00:24 UTC
This is an autogenerated message for OBS integration:
This bug (987358) was mentioned in
https://build.opensuse.org/request/show/405914 Factory / ucode-intel
Comment 4 Cristian Rodríguez 2016-07-01 18:23:32 UTC
Well.. unless it is a problem in the kernel side that loads the microcrode..or a problem in dracut that creates a microcode blob that makes the kernel choke..there is nothing we can do about it other than reverting it. it is closed source..
Comment 5 Cristian Rodríguez 2016-07-02 06:15:20 UTC
BTW.. I updated skylake box with CPU Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
With no problem whatsoever.. what CPU on which motherboard breaks ? Is the BIOS of the machine up2date as well ?
Comment 6 Robby Engelmann 2016-07-02 06:29:29 UTC
I bought a Tuxedo InfinityBook v2 in April this year. Thus, I would think the BIOS is up-to-date. I do not know how to check this.
Did anybody try to update it on an crypt-lvm setup machine?
And will add the dracut output from the update here.
Comment 7 Robby Engelmann 2016-07-02 06:30:01 UTC
Created attachment 682889 [details]
dracut output
Comment 8 Cristian Rodríguez 2016-07-02 06:54:31 UTC
(In reply to Robby Engelmann from comment #6)
> I bought a Tuxedo InfinityBook v2 in April this year. Thus, I would think
> the BIOS is up-to-date. I do not know how to check this.
> Did anybody try to update it on an crypt-lvm setup machine?
> And will add the dracut output from the update here.


Tuxedo should supply firmware updates.. 

crypt-lvm is not involved in your issue..your system fails to load the initrd in the first place..
Comment 9 Robby Engelmann 2016-07-04 12:39:54 UTC
Is there a way to figure out what goes wrong here? As far as I know I am the only one hit by this bug.
I would expect that the bios is up to date on a newly bought notebook.
Comment 10 Robby Engelmann 2016-07-04 12:42:27 UTC
the BIOS is quite fresh:
BIOS Information
	Vendor: American Megatrends Inc.
	Version: 5.11
	Release Date: 04/08/2016
Comment 11 Robby Engelmann 2016-07-04 12:43:01 UTC
Created attachment 682937 [details]
dmidecode
Comment 12 Cristian Rodríguez 2016-07-11 17:27:51 UTC
Hi Michal.. do you know how to debug a hang during the initrd loading or the early microcode stuff ?
Comment 13 Takashi Iwai 2016-07-12 10:50:29 UTC
(In reply to Cristian Rodríguez from comment #12)
> Hi Michal.. do you know how to debug a hang during the initrd loading or the
> early microcode stuff ?

Try to pass dis_ucode_ldr boot option.  This will disable the early microcode loading.

If this helps, it implies that the ucode is the culprit.  You can try to downgrade the ucode-intel package.
Comment 14 Michal Marek 2016-07-12 11:34:22 UTC
Try booting with ignore_loglevel on the kernel command line.
Comment 15 Robby Engelmann 2016-07-12 14:30:14 UTC
(In reply to Takashi Iwai from comment #13)
> (In reply to Cristian Rodríguez from comment #12)
> > Hi Michal.. do you know how to debug a hang during the initrd loading or the
> > early microcode stuff ?
> 
> Try to pass dis_ucode_ldr boot option.  This will disable the early
> microcode loading.
> 
> If this helps, it implies that the ucode is the culprit.  You can try to
> downgrade the ucode-intel package.

I updated to the snapshot containing the updated ucode-intel package --> boot hang
I updated to the snapshot containing the updated ucode-intel package but this time updated all packages except for ucode-intel --> no boot hang

--> thus, it should be clear that the ucode package was the culprit.
Comment 16 Thomas Renninger 2016-07-21 13:17:44 UTC
There is a new version: 20160714
which should fix the issue.

I pushed it into Base:System and submitted it against openSUSE:Factory.
Please give it a try.

I already close this one fixed, please re-open if you see issues.
Comment 17 Robby Engelmann 2016-07-21 13:20:44 UTC
two questions:

1. I only need to pick the rpm and install it via rpm -i ?

2. how can I recover the system, if it still does not work (without reinstalling my whole system)

then I will give it a try....
Comment 18 Takashi Iwai 2016-07-21 13:40:20 UTC
(In reply to Robby Engelmann from comment #17)
> two questions:
> 
> 1. I only need to pick the rpm and install it via rpm -i ?

Usually yes.  In doubt, rebuild initrd manually (e.g. call mkinitrd).

> 2. how can I recover the system, if it still does not work (without
> reinstalling my whole system)

You can skip ucode loading by passing dis_ucode_ldr boot option, as mentioned in comment 13.

> then I will give it a try....
Comment 19 Bernhard Wiedemann 2016-07-21 14:00:47 UTC
This is an autogenerated message for OBS integration:
This bug (987358) was mentioned in
https://build.opensuse.org/request/show/412561 Factory / ucode-intel
Comment 20 Robby Engelmann 2016-07-22 04:58:08 UTC
works!

[    0.000000] microcode: microcode updated early to revision 0x9e, date = 2016-06-22
Comment 22 Swamp Workflow Management 2016-09-05 10:09:15 UTC
openSUSE-RU-2016:2235-1: An update that has two recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 958562,987358
CVE References: 
Sources used:
openSUSE Leap 42.1 (src):    ucode-intel-20160714-5.1