Bug 847158

So what are you asking, whether microcode_ctl has update for your cpu?
Do you know, by chance, whether there actually is such an update? What
does 'cat /proc/cpuinfo' say?

I know there are microcode updates for this CPU -- when I boot with dracut, they are loaded. They are not loaded at all when using mkinitrd.

BTW, I would have responded sooner but the status wasn't set to NEEDINFO.

from /proc/cpuinfo (I have two of these listed...):


model name      : Intel(R) Pentium(R) CPU B950 @ 2.10GHz
stepping        : 7
microcode       : 0x1b

Does 

echo 1 > /sys/devices/system/cpu/microcode/reload

as root load it?

The problem is not limited to Intel CPUs.
I am observing it with an AMD Phenom II and openSUSE 13.1

dmesg after startup contains:
[    0.863592] microcode: CPU0: patch_level=0x010000c6
[    0.863638] microcode: CPU1: patch_level=0x010000c6
[    0.863691] microcode: CPU2: patch_level=0x010000c6
[    0.863741] microcode: CPU3: patch_level=0x010000c6
[    0.863813] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba


later, after executing 
echo 1 > /sys/devices/system/cpu/microcode/reload
as root the microcode update happens:
[18651.820578] microcode: CPU0: new patch_level=0x010000db
[18651.820606] microcode: CPU1: new patch_level=0x010000db
[18651.820643] microcode: CPU2: new patch_level=0x010000db
[18651.820665] microcode: CPU3: new patch_level=0x010000db


With openSUSE 12.3 the microcode update at this system happened correctly and automatically during startup.

Ok, so this could be caused by mkinitrd not including the microcode so
that when the microcode loader is builtin (which is the case on oS13.1).
Michal, are we adding the microcode blobs to the initrd as described
here:

Documentation/x86/early-microcode.txt

Btw, this could be the same issue as BNC#761728. Dirk, Jon, can you guys
try the suggestion in comment #9 there to verify this fixes it?

Thanks.

The original initrd does not contain the microcode or the directory structure 
kernel/x86/microcode/ as described in BNC#761728 according to lsinitrd or as seen from unpacking the initrd.

I have done the procedure as described in 
https://bugzilla.novell.com/show_bug.cgi?id=847158#c9 
to create a single cpio initrd.

The microcode was still not updated during boot with these modified initrds I tried.

It is not absolutely clear to me how the microcode file(s) in 
kernel/x86/microcode should be named, so maybe this is the problem.

I tried AuthenticAMD.bin (as mentioned in https://www.kernel.org/doc/Documentation/x86/early-microcode.txt) and the original names from
/lib/firmware/amd-ucode/
microcode_amd.bin
microcode_amd_fam15h.bin

These microcode files are from the package
microcode_ctl-1.17-142.5.1.x86_64

I just noticed:
rpm -V microcode_ctl
Unsatisfied dependencies for microcode_ctl-1.17-142.5.1.x86_64:
        microcode_ctl is obsoleted by (installed) kernel-desktop-3.11.6-4.1.x86_64

It was not uninstalled during zypper dup from 12.3 to 13.1

rpm -ql kernel-desktop-3.11.6-4.1.x86_64
does not show file names which "look like" microcode updates.

The package kernel-firmware-20130714git-2.1.1.noarch
contains 2 signatures for amd microcode files, but not the binaries:
/lib/firmware/amd-ucode/microcode_amd.bin.asc
/lib/firmware/amd-ucode/microcode_amd_fam15h.bin.asc



BTW: *maybe* bnc#850794 is also related to the missing microcode update.

Hmm, that's strange. Thomas, any ideas?

Btw, Dirk, you can get the .bin files from here, in the meantime: https://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/amd-ucode

I have a valid microcode file which loads successfully with
echo 1 > /sys/devices/system/cpu/microcode/reload

Therefore I do not think the version of the microcode file is the problem, even if seems to be a leftover of 12.3

I read arch/x86/kernel/microcode_amd_early.c, and according to this the  microcode must be prepended to the initrd in a file 
kernel/x86/microcode/AuthenticAMD.bin
"Microcode patch container file is prepended to the initrd in cpio format."

I will try to produce an initrd according to the procedure described in 
Documentation/x86/early-microcode.txt

Still confusing is that there are 2 microcode files for AMD and lots for Intel. At least for AMD it is easy to choose the correct one.

The procedure described in Documentation/x86/early-microcode.txt works.

dmesg|grep microc
[    0.147631] microcode: CPU1: new patch_level=0x010000db
[    0.162593] microcode: CPU2: new patch_level=0x010000db
[    0.177585] microcode: CPU3: new patch_level=0x010000db
[    0.853640] microcode: updated early to new patch_level=0x010000db
[    0.863500] microcode: CPU0: patch_level=0x010000db
[    0.863545] microcode: CPU1: patch_level=0x010000db
[    0.863596] microcode: CPU2: patch_level=0x010000db
[    0.863646] microcode: CPU3: patch_level=0x010000db
[    0.863716] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba

Specifically I used:
cp /lib/firmware/amd-ucode/microcode_amd.bin kernel/x86/microcode/AuthenticAMD.bin

with /lib/firmware/amd-ucode/microcode_amd.bin from the "obsolete" microcode_ctl package.

(In reply to comment #11)
> I have a valid microcode file which loads successfully with
> echo 1 > /sys/devices/system/cpu/microcode/reload
> 
> Therefore I do not think the version of the microcode file is the problem, even
> if seems to be a leftover of 12.3

Who said the version is the problem?

> I read arch/x86/kernel/microcode_amd_early.c, and according to
> this the microcode must be prepended to the initrd in a file
> kernel/x86/microcode/AuthenticAMD.bin "Microcode patch container file
> is prepended to the initrd in cpio format."

That's what the last 'cat' command in
Documentation/x86/early-microcode.txt does.

> I will try to produce an initrd according to the procedure described
> in Documentation/x86/early-microcode.txt
>
> Still confusing is that there are 2 microcode files for AMD and lots
> for Intel At least for AMD it is easy to choose the correct one.

The Intel microcode is also supposed to be one file. The split patches
in the microcode_ctl package are not the ones you can directly use for
early microcode loading - this file is talking about the microcode blobs
you get from the CPU vendors directly, at least in the Intel case.

In the AMD case, you can simply cat all the bin files together.

(In reply to comment #11)
> The procedure described in Documentation/x86/early-microcode.txt works.
> 
> dmesg|grep microc
> [    0.147631] microcode: CPU1: new patch_level=0x010000db
> [    0.162593] microcode: CPU2: new patch_level=0x010000db
> [    0.177585] microcode: CPU3: new patch_level=0x010000db
> [    0.853640] microcode: updated early to new patch_level=0x010000db
> [    0.863500] microcode: CPU0: patch_level=0x010000db
> [    0.863545] microcode: CPU1: patch_level=0x010000db
> [    0.863596] microcode: CPU2: patch_level=0x010000db
> [    0.863646] microcode: CPU3: patch_level=0x010000db
> [    0.863716] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> 
> Specifically I used:
> cp /lib/firmware/amd-ucode/microcode_amd.bin
> kernel/x86/microcode/AuthenticAMD.bin
> 
> with /lib/firmware/amd-ucode/microcode_amd.bin from the "obsolete"
> microcode_ctl package.

Good, thanks for confirming this.

@Michal, can we add the microcode to the initrd creation process for oS?
Actually for SLE12 too, while we're at it.

Thanks.

Some additional observations:

From bnc#851398 I learned that for 13.1 ucode-amd and ucode-intel packages exist.

I can confirm that the update with zypper dup from 12.3 did not replace the microcode_ctl from 12.3 with the new ucode-* packages of 13.1.

I manually uninstalled microcode_ctl and installed ucode-amd. 

With the microcode from ucode-amd  
(/lib/firmware/amd-ucode/microcode_amd.bin)
microcode update works on my system (of course). 
I have only tried "late" updating by
echo 1 > /sys/devices/system/cpu/microcode/reload
with this version, but I am quite sure if properly included in the initrd early update will work, too.

To me the packaging of ucode files and their signatures is a little bit strange:
$ rpm -qf /lib/firmware/amd-ucode/microcode_amd.bin
ucode-amd-20130714git-2.1.1.noarch

$ rpm -qf /lib/firmware/amd-ucode/microcode_amd.bin.asc
kernel-firmware-20130714git-2.1.1.noarch

Is there some deeper meaning that the binaries and their signatures are not in the same package?

> @Michal, can we add the microcode to the initrd creation process for oS?
dracut does this already. At least for Intel the "early microcode update feature is implemented", for AMD it needs to be double checked. Iirc I at least stumbled over some kernel code, that enables early microcode loading for AMD.

> Is there some deeper meaning that the binaries and their signatures are not in
> the same package?
Yes. I asked Intel and AMD to submit their microcodes to the kernel-firmware project/git repo for the future. Like that they do not have to open bugs every some months for all distros they would like to see their microcodes included, but can simply submit their stuff to where it belongs: kernel-firmware git repo. And all distros can pick it up and add things to their supported products with all the other up-to-date firmware in there.

While AMD did adjust their license and is submitting their microcodes to kernel-firmware now, Intel does and will not, due to license issues.

It looks like the microcode updating mechanism works in general, but we have a higher level problem at packaging/installer level?
> Unsatisfied dependencies for microcode_ctl-1.17-142.5.1.x86_64:
>        microcode_ctl is obsoleted by (installed)
This looks weird? Did you do a fresh 13.1 (RC1 or earlier?) installation and then update (zypper dup) to GM?
Then this may simply be an issue that you installed a devel version when later the microcode_ctl package got split.
Or was this a 12.3 to 13.1 update?
Hm, I remember a bug where zypper/rpm did not detect correctly whether firmware package has to be installed. Maybe you installed at that time?
Anyway, if we want to root cause this, it makes sense to install a fresh 13.1 GM to make sure there are no "devel release only" problems we are hunting.

> > @Michal, can we add the microcode to the initrd creation process for oS?
> dracut does this already.
That means this is not implemented in 13.1, but only in factory yet.
But that should not matter, this is a brand new feature this bug is not about.
Let's discuss and make sure general microcode loading via services is working correctly for 13.1 here.

> Anyway, if we want to root cause this, it makes sense to install a fresh 13.1
> GM to make sure there are no "devel release only" problems we are hunting.

I've installed a fresh 13.1 (used DVD x86_64 and KDE Live x86_64) and ucode-intel is not installed.

(In reply to comment #14)
> > @Michal, can we add the microcode to the initrd creation process for oS?
> dracut does this already. At least for Intel the "early microcode update
> feature is implemented", for AMD it needs to be double checked. Iirc I at least
> stumbled over some kernel code, that enables early microcode loading for AMD.
in comment #11 I have written that the "early microcode update" works for AMD CPUs with standard 13.1 if only the initrd correctly contains the microcode.



> It looks like the microcode updating mechanism works in general, but we have a
> higher level problem at packaging/installer level?
I agree.

> > Unsatisfied dependencies for microcode_ctl-1.17-142.5.1.x86_64:
> >        microcode_ctl is obsoleted by (installed)
> This looks weird? Did you do a fresh 13.1 (RC1 or earlier?) installation and
> then update (zypper dup) to GM?
> Then this may simply be an issue that you installed a devel version when later
> the microcode_ctl package got split.
> Or was this a 12.3 to 13.1 update?
I observed it at 2 systems.
One was updated with zypper dup from a fully updated 12.3 to 13.1RC2, the other one from a fully updated 12.3 to 13.1GM.


> Hm, I remember a bug where zypper/rpm did not detect correctly whether firmware
> package has to be installed. Maybe you installed at that time?
> Anyway, if we want to root cause this, it makes sense to install a fresh 13.1
> GM to make sure there are no "devel release only" problems we are hunting.
comment #16 also confirms that the ucode-* packages are not installed in a clean install from DVD.

There are now several tickets open related to microcode loading and the information is distributed between these tickets:
bnc#851398
bnc#761728

Summing these up it seems:
- early microcode loading is working (tested myself with AMD systems) when the ucode is prepended to the initrd.
- "late" microcode loading is working (tested myself with AMD systems) - if the ucode files are available
BUT:
- mkinitrd does not prepend the ucodes into the initrd - could be fixed by an addon script to mkinitrd, the steps to do it are not complicated.
- the ucode-* packages are not installed by default and the old microcode_ctl package is not removed - could maybe be fixed by some dependency or pattern update?

(In reply to comment #17)
> Summing these up it seems:
> - early microcode loading is working (tested myself with AMD systems) when the
> ucode is prepended to the initrd.
> - "late" microcode loading is working (tested myself with AMD systems) - if the
> ucode files are available

I have additional results from another system with an AMD E-350 CPU.
In this case "late" microcode update works:
[    1.301598] microcode: CPU0: patch_level=0x05000028
[    1.301616] microcode: CPU1: patch_level=0x05000028
[    1.301780] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba

# echo 1 > /sys/devices/system/cpu/microcode/reload

[  737.932099] microcode: CPU0: new patch_level=0x05000029
[  737.940576] microcode: CPU1: new patch_level=0x05000029

But early microcode update with a handcrafted initrd (like in comment #11) causes a reboot (everything tested with plain 13.1).

So there are still more problems in this area, and just adding the ucodes to the initrd could cause boot problems for users on certain systems.

(In reply to comment #18)
> I have additional results from another system with an AMD E-350 CPU.
> In this case "late" microcode update works:
> [    1.301598] microcode: CPU0: patch_level=0x05000028
> [    1.301616] microcode: CPU1: patch_level=0x05000028
> [    1.301780] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> 
> # echo 1 > /sys/devices/system/cpu/microcode/reload
> 
> [  737.932099] microcode: CPU0: new patch_level=0x05000029
> [  737.940576] microcode: CPU1: new patch_level=0x05000029
> 
> But early microcode update with a handcrafted initrd (like in comment #11)
> causes a reboot (everything tested with plain 13.1).
> 
> So there are still more problems in this area, and just adding the ucodes to
> the initrd could cause boot problems for users on certain systems.

bnc#852007 opened to track this specific issue with AMD E-350 separately from the initrd creation or general microcode loading via services as mentioned in comment #15

*** Bug 851398 has been marked as a duplicate of this bug. ***

The bug is that this line in ucode-intel.spec is not recognized:
Supplements:    modalias(x86cpu:vendor:0000:family:*:model:*:feature:*)

Not sure whether:
1) "x86cpu" modalias does not work with rpm in general.
2) This is due to the fact that microcode driver is not build as a module
   (from which Supplements modalias is derived), but is built into the kernel
   with 13.1

A quick look at find-supplements.ksyms in rpm package tells me that modules.alias is heavily used, but I am not familiar with this code.
For me calling:
/usr/lib/rpm/find-supplements.ksyms
blocks here:
   for module in $(grep -E '/lib/modules/.+\.ko$'); do
-> grep -E '/lib/modules/.+\.ko$'

Adding Michael: Can you help with making this Supplements line work, please:
Supplements:    modalias(x86cpu:vendor:0000:family:*:model:*:feature:*)

This is modinfo from 12.3 (where microcode driver was still built as a module):
modinfo microcode
filename:       /lib/modules/3.4.63-2.44-desktop/kernel/arch/x86/kernel/microcode.ko
...
alias:          x86cpu:vendor:0002:family:*:model:*:feature:*
alias:          x86cpu:vendor:0000:family:*:model:*:feature:*

On 13.1 you get:
modinfo microcode
ERROR: modinfo: could not find module microcode

Not sure how things are implemented in rpm, it looks as if there is a general issue, that Supplements tag must match driver module alias(es)?
Ideally modalias(pci...) is checked against lspci
and
modalias(x86cpu...) is checked against /proc/cpuinfo
etc.
then things would not depend on kernel modules, but on HW.

> Ideally modalias(pci...) is checked against lspci
And best there is an interface in module-init-tools which asks the kernel whether a specific modalias fits the machine to avoid duplicated code, something like:
modprobe --alias "x86cpu:vendor:0002:family:*:model:*:feature:*"
returning true or false, depending on whether the machine matches or not.
Not sure what exactly needs to be touched to implement this.

How's that an rpm problem? Please use zypp-maintainers@forge.provo.novell.com
for zypp questions.

But by chance I know the answer anyway: you must only use one ':' in the supplements and use URL encoding for the other ones. Thus change the supplements to:

Supplements: modalias(x86cpu:vendor%3A0000%3Afamily%3A*%3Amodel%3A*%3Afeature%3A*)

(This assumes that

find /sys -name modalias -print0 | xargs -0 cat | grep x86cpu

outputs some matching entry)

As for the "obsoleted by installed kernel-desktop" error: that's probably because the kernel is marked as "multiversion" package, i.e. it gets installed with 'rpm -i' instead of 'rpm -U'. 'rpm -i' does not erase obsoleted packages.
(That's not entirely true for new rpm/zypp versions, which *will* consider obsoletes and deinstall packages, though.)

Supplements:
modalias(x86cpu:vendor%3A0000%3Afamily%3A*%3Amodel%3A*%3Afeature%3A*)

-> I submitted a fix for factory and will give it a test as soon as factory built and can be installed.

eosc submitpac
created request id 208267

I wonder how this can be fixed in openSUSE:13.1 automatically (install ucode-intel package even it has not been at distro install time)?
Is this possible at all?
As 13.1 is out the fix will not show up in GM, but only as maintenance update and I doubt zypper will re-evaluate when doing zypper dup or zypper update, that the package now has to be installed additionally?

(infoprovider set to zypp-maintainers)

(In reply to comment #24)
> I wonder how this can be fixed in openSUSE:13.1 automatically (install
> ucode-intel package even it has not been at distro install time)?
please do not forget the ucode-amd package :-)

> As 13.1 is out the fix will not show up in GM, but only as maintenance update
> and I doubt zypper will re-evaluate when doing zypper dup or zypper update,
> that the package now has to be installed additionally?
Maybe for 13.1 just a hint could be added to the release note that users should or have to install ucode-intel/ucode-amd packages (and possibly uninstall microcode_ctl if they upgraded).

Apart from this - at the moment there is also no trigger to actually load the ucode patches into the CPU when the system is up.
Something would need to do:
echo 1 > /sys/devices/system/cpu/microcode/reload

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/208545 Factory / msr-tools

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/208646 Factory / ucode-intel

(In reply to comment #24)
> I wonder how this can be fixed in openSUSE:13.1 automatically (install
> ucode-intel package even it has not been at distro install time)?
> Is this possible at all?
> As 13.1 is out the fix will not show up in GM, but only as maintenance update
> and I doubt zypper will re-evaluate when doing zypper dup or zypper update,
> that the package now has to be installed additionally?

'zypper inr' will -amongst others- re-evaluate the modalias deps. 
Also 'zypper dup'. 
'zypper up' won't.

2 issues with both: ucode-intel and ucode-amd:
  - Bad Supplements: modalias string (compare with comment #23 and #24)
  - Because microcode driver is built-in  (or would get loaded early because
    of new cpux86 autoload feature), the microcode has to be added to the initrd

I have put fixed ucode-intel and ucode-amd packages here:
ftp.suse.com/pub/people/trenn/13.1_microcode_updates
Installing should trigger mkinitrd.
When initrd is created there is an additional line now:
Microcode: ...
telling the user whether a suitable microcode update has been found and added to initrd. If one is found, rebooting must result in a microcode update (check dmesg or /var/log/messages).

For AMD I gave it a try and run into strang issues (but on a 13.1 Milestone 4 system):
  - AMD microcode got added to initrd correctly, but reboot did not result in
    an update. Triggering microcode reload manually:
    echo 1 >/sys/devices/system/cpu/microcode/reload
    resulted in updating only some of the 16 cores:
microcode: CPU0: new patch_level=0x0600063d
microcode: CPU1: new patch_level=0x0600063d
microcode: CPU4: new patch_level=0x0600063d
microcode: CPU5: new patch_level=0x0600063d
microcode: CPU8: new patch_level=0x0600063d
microcode: CPU9: new patch_level=0x0600063d
microcode: CPU12: new patch_level=0x0600063d
microcode: CPU13: new patch_level=0x0600063d
Strange, I guess/hope this is a leftover of old 3.11.0-rc7-1.g99e1318-default kernel.

Would be great if others can give the packages a try.

In reply to comment #30)
> For AMD I gave it a try and run into strang issues (but on a 13.1 Milestone 4
> system):
>   - AMD microcode got added to initrd correctly, but reboot did not result in
>     an update.

That's a different bug: bnc#852007 and you're on CC there too. I've
uploaded a fix today for the upstream kernel and it would be great if
you could test it and let me know if it works on your box.

> Triggering microcode reload manually:
>     echo 1 >/sys/devices/system/cpu/microcode/reload
>     resulted in updating only some of the 16 cores:
> microcode: CPU0: new patch_level=0x0600063d
> microcode: CPU1: new patch_level=0x0600063d
> microcode: CPU4: new patch_level=0x0600063d
> microcode: CPU5: new patch_level=0x0600063d
> microcode: CPU8: new patch_level=0x0600063d
> microcode: CPU9: new patch_level=0x0600063d
> microcode: CPU12: new patch_level=0x0600063d
> microcode: CPU13: new patch_level=0x0600063d
> Strange, I guess/hope this is a leftover of old 3.11.0-rc7-1.g99e1318-default
> kernel.

No, this is because Bulldozer has compute units which have 2 cores but
need to update the microcode on only one core of the compute unit to
have the updated microcode on both cores.

You can verify this by doing

$ grep microcode /proc/cpuinfo

It should give you the same microcode version on each core.

Thanks.

(In reply to comment #30)
> I have put fixed ucode-intel and ucode-amd packages here:
> ftp.suse.com/pub/people/trenn/13.1_microcode_updates
> Installing should trigger mkinitrd.
> When initrd is created there is an additional line now:
> Microcode: ...
> telling the user whether a suitable microcode update has been found and added
> to initrd. If one is found, rebooting must result in a microcode update (check
> dmesg or /var/log/messages).

I've tested your package and microcode is updated after install but is not updated after reboot.
/var/log/messages after package install:

2013-12-03T09:37:18.869222+01:00 bestia kernel: [ 4084.183167] microcode: CPU0 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:37:18.869232+01:00 bestia kernel: [ 4084.183628] microcode: CPU0 updated to revision 0x19, date = 2013-06-13
2013-12-03T09:37:18.869234+01:00 bestia kernel: [ 4084.183641] microcode: CPU1 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:37:18.869234+01:00 bestia kernel: [ 4084.183872] microcode: CPU1 updated to revision 0x19, date = 2013-06-13
2013-12-03T09:37:18.869237+01:00 bestia kernel: [ 4084.183891] microcode: CPU2 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:37:18.869750+01:00 bestia kernel: [ 4084.184134] microcode: CPU2 updated to revision 0x19, date = 2013-06-13
2013-12-03T09:37:18.869762+01:00 bestia kernel: [ 4084.184165] microcode: CPU3 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:37:18.869763+01:00 bestia kernel: [ 4084.184411] microcode: CPU3 updated to revision 0x19, date = 2013-06-13

/var/log/messages after reboot:

2013-12-03T09:38:08.315985+01:00 bestia kernel: [    0.296788] microcode: CPU0 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:38:08.315985+01:00 bestia kernel: [    0.296793] microcode: CPU1 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:38:08.315986+01:00 bestia kernel: [    0.296798] microcode: CPU2 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:38:08.315986+01:00 bestia kernel: [    0.296800] microcode: CPU3 sig=0x306a9, pf=0x2, revision=0x15
2013-12-03T09:38:08.315986+01:00 bestia kernel: [    0.296822] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba

The reason why the microcode is loaded on install is due to this script, supplied by the rpm, which executes at that time:

test -f /sys/devices/system/cpu/microcode/reload && /bin/echo 1 > /sys/devices/system/cpu/microcode/reload || exit 0


It seems to me that the package (or some other, higher-level package) should just provide a systemd service which does the same thing any time a file in the appropriate directories changes.

That would be a trivial, reliable solution to this problem.

(In reply to comment #30)
> ...
> I have put fixed ucode-intel and ucode-amd packages here:
> ftp.suse.com/pub/people/trenn/13.1_microcode_updates
> Installing should trigger mkinitrd.
> When initrd is created there is an additional line now:
> Microcode: ...
> telling the user whether a suitable microcode update has been found and added
> to initrd. If one is found, rebooting must result in a microcode update (check
> dmesg or /var/log/messages).
> 
> For AMD I gave it a try and run into strang issues (but on a 13.1 Milestone 4
> system):
>   - AMD microcode got added to initrd correctly, but reboot did not result in
>     an update. Triggering microcode reload manually:
>     echo 1 >/sys/devices/system/cpu/microcode/reload
>     resulted in updating only some of the 16 cores:
> 
> Would be great if others can give the packages a try.

I have tested your ucode-amd package.
mkinitrd is run after installation and gives this output:
...
Microcode: Adding AMD microcode microcode_amd.bin
Microcode: Adding AMD microcode microcode_amd.bin
...

it seems that the body of the script
lib/mkinitrd/scripts/setup-amd_microcode.sh
is duplicated?

# lsinitrd initrd-3.11.6-4-desktop | grep ucode
lib/firmware/amd-ucode
lib/firmware/amd-ucode/microcode_amd.bin.asc
lib/firmware/amd-ucode/microcode_amd.bin

shows that the microcode is included in the initrd, but I think the path/filename is not correct and therefore it is not found at boot time.

According to https://www.kernel.org/doc/Documentation/x86/early-microcode.txt
the name should be 
kernel/x86/microcode/AuthenticAMD.bin
and this is also the path which appears in 
/usr/src/linux/arch/x86/kernel/microcode_amd_early.c

Besides it seems that the ucode file should be in a separate cpio prepended to the standard initrd, see comment #11 which shows an example of successful early update with AMD CPU.

But please also mind comment #18 and related bnc#852007 where successful creation of an initrd including the ucode update almost resulted in an unbootable system - luckily I had a backup initrd.

BTW: is there a kernel parameter which inhibits early microcode update which can be used to boot the system in case the early update makes the system unbootable and all available initrds contain the microcode?

(In reply to comment #34)
> According to https://www.kernel.org/doc/Documentation/x86/early-microcode.txt
> the name should be 
> kernel/x86/microcode/AuthenticAMD.bin
> and this is also the path which appears in 
> /usr/src/linux/arch/x86/kernel/microcode_amd_early.c

That is correct.

> Besides it seems that the ucode file should be in a separate cpio
> prepended to the standard initrd, see comment #11 which shows an
> example of successful early update with AMD CPU.
>
> But please also mind comment #18 and related bnc#852007 where
> successful creation of an initrd including the ucode update almost
> resulted in an unbootable system - luckily I had a backup initrd.

Right, I'm still working on those - it looks like 32-bit was completely
b0rked from the beginning and it was rushed in without any serious
testing.

> BTW: is there a kernel parameter which inhibits early microcode update
> which can be used to boot the system in case the early update makes
> the system unbootable and all available initrds contain the microcode?

Unfortunately, no. The thing is, if the loading would have worked from
the beginning, we wouldnt've needed a kernel parameter in the first
place. :)

What you can do for now is drop the ucode from the initrd so that it
cannot be found and loading fails.

(In reply to comment #35)
> (In reply to comment #34)
> ...
> Right, I'm still working on those - it looks like 32-bit was completely
> b0rked from the beginning and it was rushed in without any serious
> testing.
It is probably difficult to test the possible combinations. E.g. if the HW vendor includes the microcode update already in the BIOS/UEFI then this machine can not be used for these tests anymore.

> > BTW: is there a kernel parameter which inhibits early microcode update
> > which can be used to boot the system in case the early update makes
> > the system unbootable and all available initrds contain the microcode?
> 
> Unfortunately, no. The thing is, if the loading would have worked from
> the beginning, we wouldnt've needed a kernel parameter in the first
> place. :)
> 
> What you can do for now is drop the ucode from the initrd so that it
> cannot be found and loading fails.
yes, currently with openSUSE 13.1 and the official mkinitrd it is easy to have the ucode *NOT* in the initrd. :-)
But if mkinitrd gets this functionality and automatically all initrds of the system are updated this could be a problem.

Regarding comment #33: I am now just using 
/etc/init.d/after.local
to check if the ucode was not yet updated and then trigger the update.

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/209606 13.1 / ucode-intel

For Intel, this bug should be resolved now.
If I haven't done something wrong while testing (again, first I tried on the system I developed and did not remove all my changes before installing the fixed package) then:
  - ucode-intel will get installed automatically when the changes hit 13.1:
       -'zypper inr' will -amongst others- re-evaluate the modalias deps. 
       - Also 'zypper dup'
    Also the Intel firmware should get added automatically via mkinitrd plugins
    and loaded on next reboot.

  - ucode-amd will get installed automatically (Supplements, modalias fix),
    but firmware will not be added to mkinitrd automatically.
    The solution Dirk suggested, adding:
    if [ -w /sys/devices/system/cpu/microcode/reload ];then
        echo 1 >/sys/devices/system/cpu/microcode/reload
    fi
    to /etc/init.d/after.local
    People would have to add this manually.
    I tried some time, but this kernel-firmware package takes quite some time
    to build and finally I couldn't see why my mkinitrd plugin works for Intel,
    but not for AMD.

I have added above to devel projects (Base:System/ucode-intel and Kernel:HEAD/kernel-firmware) and will try to get things into openSUSE:Factory and openSUSE:13.1 asap.

Setting resolved fixed.

I guess the manual part for AMD (/etc/init.d/after.local) should get documented somewhere. Maybe here (as soon as fixed packages got accepted):
http://en.opensuse.org/openSUSE:Most_annoying_bugs_13.1

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/210022 Factory / ucode-intel
https://build.opensuse.org/request/show/210023 13.1 / ucode-intel

I've installed ucode-intel-20130906-15.1.x86_64.rpm from Base:System/ucode-intel but I've the same problem, after reboot microcode is not updated.
For my system the bug is not resolved, sorry.

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/210038 Factory / kernel-firmware

Ah, I think I know what it was:
The udev mkinitrd boot script has to run first. I now added a dependency to this one.
This would explain why it worked for me, but not for you and in AMD case. It's more or less random what is run first.

It's currently buidling, you'll find a build to test (ucode-intel) here soon:
http://download.opensuse.org/repositories/Base:/System/openSUSE_13.1/x86_64/
Best double check via:
rpm -qp --changelog ucode-intel*.rpm |less
for this line:
-------------------------------------------------------------------
Tue Dec 10 12:43:57 UTC 2013 - trenn@suse.de

- Loading firmware needs udev to be running

Installed ucode-intel-20130906-18.1.x86_64.rpm and now it works, after reboot:

[    0.296872] microcode: CPU0 sig=0x306a9, pf=0x2, revision=0x15
[    0.296877] microcode: CPU1 sig=0x306a9, pf=0x2, revision=0x15
[    0.296881] microcode: CPU2 sig=0x306a9, pf=0x2, revision=0x15
[    0.296884] microcode: CPU3 sig=0x306a9, pf=0x2, revision=0x15
[    0.296904] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    1.913796] microcode: CPU0 sig=0x306a9, pf=0x2, revision=0x15
[    1.914450] microcode: CPU0 updated to revision 0x19, date = 2013-06-13
[    1.914468] microcode: CPU1 sig=0x306a9, pf=0x2, revision=0x15
[    1.914794] microcode: CPU1 updated to revision 0x19, date = 2013-06-13
[    1.914841] microcode: CPU2 sig=0x306a9, pf=0x2, revision=0x15
[    1.915164] microcode: CPU2 updated to revision 0x19, date = 2013-06-13
[    1.915198] microcode: CPU3 sig=0x306a9, pf=0x2, revision=0x15
[    1.915522] microcode: CPU3 updated to revision 0x19, date = 2013-06-13


Thanks for your work!

(In reply to comment #38)
>   - ucode-amd will get installed automatically (Supplements, modalias fix),
>     but firmware will not be added to mkinitrd automatically.
>     The solution Dirk suggested, adding:
>     if [ -w /sys/devices/system/cpu/microcode/reload ];then
>         echo 1 >/sys/devices/system/cpu/microcode/reload
>     fi
>     to /etc/init.d/after.local
>     People would have to add this manually.

Ok, this is not really good. Why can't the installation procedure edit
files in /etc/init.d/? People won't do this manually, believe me. If
we're not going to load the ucode early we at least need to automate the
application later.

> Ok, this is not really good.
Keep calm and use the force...

Seriously: AMD and Intel should work exactly the same way. Now as I understand that udev must go first, I can and will do the same for ucode-amd.

All which is then needed from users is a:
zypper dup
or
zypper inr  (install-new-recommends)
to get ucode-intel or ucode-amd installed. Then mkinitrd will be built automatically at package installation time and the microcode will be loaded at next reboot (in fact a reload is already triggered at installation time as well before rebooting).

I will now also commit the ucode-amd needed changes to Base:System/kernel-firmware and will trigger submit requests against openSUSE:Factory and openSUSE:13.1

Stay tuned...

(In reply to comment #45)
> > Ok, this is not really good.
> Keep calm and use the force...

I do, my young padawan - don't you notice it? :-)

> to get ucode-intel or ucode-amd installed. Then mkinitrd will be built
> automatically at package installation time and the microcode will be
> loaded at next reboot

Question is, will the amd ucode get loaded during every boot? We said we don't
want to do early loading but will a script do

	echo 1 > .../reload

during every boot? And will this script get added automagically, using
the force?

> (in fact a reload is already triggered at installation time as well
> before rebooting).

One reload is enough.

Thanks.

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/210512 Factory / kernel-firmware

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/210769 13.1 / ucode-intel

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/210772 13.1 / ucode-intel
https://build.opensuse.org/request/show/210777 13.1 / kernel-firmware

This is an autogenerated message for OBS integration:
This bug (847158) was mentioned in
https://build.opensuse.org/request/show/211706 13.1 / kernel-firmware

I had to fix the kernel-firmware builds, as it did not build at all. I also submityted a fixed one to Kernel:HEAD

(mkinitrd just does not work in the build system)

openSUSE-RU-2013:1940-1: An update that has one recommended fix can now be installed.

Category: recommended (low)
Bug References: 847158
CVE References: 
Sources used:
openSUSE 13.1 (src):    ucode-intel-20130906-6.2

openSUSE-RU-2013:1948-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 847158
CVE References: 
Sources used:
openSUSE 13.1 (src):    kernel-firmware-20130714git-2.5.1

Should be fixed, please re-open if there still are issues.