Bug 908454

works on x86_64
# get_kernel_version /boot/vmlinuz
3.16.6-2-desktop

is get_kernel_version architecture-specific?

I would not say so. I think it is related to vmlinuz != zImage, or presence of a vmlinux.gz

running get_kernel_version /boot/zImage-3.14.14-cubox-i on my arm board prints nothing.
Running it instead on /boot/vmlinux-3.14.14-cubox-i.gz works as expected
(I dont have a vmlinuz installed for my kernel

If I remember right, adding that .gz was required to trick mkinitrd on 13.1, but that's not required anymore in 13.2. Its useless and I'll drop it completely as soon as I can.

So I installed kernel-default package and performed same tests:
get_kernel_version zImage-3.16.7-2-default : no output
get_kernel_version vmlinux-3.16.7-2-default.gz : works fine
There is no vmlinuz ......

get_kernel_version is architecture-specific.  It only supports uncompressed or gzip compressed kernels or x86 format kernels.

> get_kernel_version is architecture-specific.  It only supports uncompressed
> or gzip compressed kernels or x86 format kernels.

Are you okay to close it?

*** Bug 928673 has been marked as a duplicate of this bug. ***

we should drop get_kernel_version from aaa_base. mkinitrd can use e.g. file to determine the kernel version or parse the file name which has a standardized form for package installed kernels.

reassigning to dracut maintainer for comment

On aarch64:

$ file -L /boot/Image
/boot/Image: MS-DOS executable, MZ for MS-DOS
$ get_kernel_version /boot/Image
4.0.0-1-default

Please wait until dracut version 041 finally hits factory.
Hopefully after fixing this one:
[Bug #927719] Tumbleweed Snapshot blocked: no multipath support in 20150416 (likely dracut issue)

I have some more patches to queue up for dracut Base:System, but I need to have the update accepted first.

Hoopsie, I mixed this up with install_kernel from dracut package.
I wonder why I got assigned to this (aaa_base) bug.
Ruediger is maintaining aaa_base.
Werner is maintaining file.

Summary:
We have following steps, bugs to fix (in this order):
1) file has to support all architectures (including aarch64, others?)
2) All users of get_kernel_version have to be replaced to use "file" instead,
   for example install_kernel in dracut
3) We want to get rid of get_kernel_version in aaa_base.
   Hm, or get_kernel_version in aaa_base uses file and formats the output
   in the same way it did before?

-> I like to re-assign this one. Question is to whom?

Re-assigning to Werner for now (kernel version support for aarch64 zImage?), adding Ruediger for comments.

Hmm ... without the attached Image and without the version of the file package on ARM I can say exactly nothing.  Btw: Do we support zImage format at all?

In file-5.22 we have

  # Linux ARM compressed kernel image
  # From: Kevin Cernekee <cernekee@gmail.com>
  36      lelong  0x016f2818      Linux kernel ARM boot executable zImage (little-endian)
  36      belong  0x016f2818      Linux kernel ARM boot executable zImage (big-endian)

for reference: the original author of get_kernel_version is steffen.

get_kernel_version is perfectly simple in operation:
- if file starts with (037,0213,0236) decompress a chunk
- in the first 4k search for "Linux version " and get what's behind

can't be too hard to extend this for zImage, but I'd need to see such a file

Andreas: I thought this is about /sbin/installkernel from dracut.
There is:
case "$(uname -m)" in
    s390|s390x)
        BOOTFILE=image
        ;;
    ppc|ppc64)
        BOOTFILE=vmlinux
        ;;
    *)
        BOOTFILE=vmlinuz
        ;;
esac

If you need something specific for aarch64, please let me know or best send a tiny patch which shouldn't be hard to do, but needs ARM HW at hand...

> Hmm ... without the attached Image and without the version of the file package
> on ARM I can say exactly nothing.

The ARM zImage file belongs to the kernel-default package which you can
download from the 13.2 ARM repo (it's 6.5MB wasted space in bugzilla).
http://download.opensuse.org/ports/armv7hl/distribution/13.2/repo/oss/suse/armv7hl/kernel-default-3.16.7-11.1.armv7hl.rpm

That also has the file package with the version in its name, which is
file-5.19-3.8.1.armv7hl

> Btw: Do we support zImage format at all?

Can't say.

I like the idea of using file, because that already has all the magic
so why replicate it.

The bottom line is that mkinitrd doesn't work with no arguments (and a useless error msg):

# mkinitrd 
No kernel found in /boot or bad modules dir in /lib/modules

Currently this is because get_kernel_version fails on ARM zImage kernels. However I don't care about get_kernel_version, I care about mkinitrd, so changing mkinitrd to establish kernel version is a nother option.

Using file may not be so simple:

# file /boot/zImage-3.19.3-1-default
/boot/zImage-3.19.3-1-default: Linux kernel ARM boot executable zImage (little-endian)
# file -v
file-5.19

Perhaps that was the reason for the existence of get_kernel_version.
Any reason not to use just the file name (or link name, don't de-reference), as Ludwig suggested?

file only tells the type, not the version we are after.
for reference:
http://morgue.openinkpot.org/wiki/Documentation/ZImageFormat

in short:
objdump -EL -b binary -D -m armv5t zImage-3.16.7-11-default | grep 8b1f | head -2
    4878:	00088b1f 	andeq	r8, r8, pc, lsl fp
    5220:	8b1ffd06 	blhi	0x804640

4878 hex = 18552 dec

dd if=zImage-3.16.7-11-default of=piggy.gz bs=1 skip=18552

# get_kernel_version piggy.gz 
3.16.7-11-default

no idea yet how to get a simple implementation for this without all the ELF magic

(In reply to Volker Kuhlmann from comment #16)

> # file /boot/zImage-3.19.3-1-default
> /boot/zImage-3.19.3-1-default: Linux kernel ARM boot executable zImage (little-endian)
> # file -v
> file-5.19

How does this fit with this?

(In reply to Andreas Schwab from comment #8)

> On aarch64:
> 
> $ file -L /boot/Image
> /boot/Image: MS-DOS executable, MZ for MS-DOS
> $ get_kernel_version /boot/Image
> 4.0.0-1-default

I'd like to know *which* version of file do you have used for your result?
Also, where can I find this Image

> I'd like to know *which* version of file do you have used for your result?

It's all latest Factory.

> Also, where can I find this Image

On seattle.arch.

> parse the file name

The file name can easily get lost or be inaccurate.

(In reply to Andreas Schwab from comment #19)

It seems that the kernel developers on aarch64 are too lazy or not
interested on how to detect their kernel images

 rpm2cpio kernel-default-4.0.1-1.1.ge3a374a.armv7hl.rpm | \
  cpio -i --to-stdout ./boot/zImage-4.0.1-1.ge3a374a-default | file -
 /dev/stdin: Linux kernel ARM boot executable zImage (little-endian)

 rpm2cpio kernel-default-4.0.1-1.1.ge3a374a.armv6hl.rpm | \
   cpio -i --to-stdout ./boot/zImage-4.0.1-1.ge3a374a-default | file -
 /dev/stdin: Linux kernel ARM boot executable zImage (little-endian)

 pm2cpio kernel-default-3.19.4-1.1.g74c332b.aarch64.rpm | \
   cpio -i --to-stdout ./boot/Image-3.19.4-1.g74c332b-default | file -
 /dev/stdin: MS-DOS executable, MZ for MS-DOS
 rpm2cpio kernel-default-3.19.4-1.1.g74c332b.aarch64.rpm | \
 cpio -i --to-stdout ./boot/Image-3.19.4-1.g74c332b-default | head -c 2
 MZ

... it is well known that MZ is the start point of most MS-DOS executables

From /usr/share/misc/magic

 # Linux ARM compressed kernel image
 # From: Kevin Cernekee <cernekee@gmail.com>
 36      lelong  0x016f2818      Linux kernel ARM boot executable zImage (little-endian)
 36      belong  0x016f2818      Linux kernel ARM boot executable zImage (big-endian)

 rpm2cpio kernel-default-4.0.1-1.1.ge3a374a.armv6hl.rpm | \
   cpio -i --to-stdout ./boot/zImage-4.0.1-1.ge3a374a-default | \
   head -c 40 | od -t x1
 0000000 00 00 a0 e1 00 00 a0 e1 00 00 a0 e1 00 00 a0 e1
 *
 0000040 03 00 00 ea 18 28 6f 01

 rpm2cpio kernel-default-3.19.4-1.1.g74c332b.aarch64.rpm | \
 cpio -i --to-stdout ./boot/Image-3.19.4-1.g74c332b-default | head -c 40 | \
 od -t x1

 0000000 4d 5a 00 91 ff 03 00 14 00 00 08 00 00 00 00 00
 0000020 00 00 9e 00 00 00 00 00 00 00 00 00 00 00 00 00
 0000040 00 00 00 00 00 00 00 00
 0000050

 Hex 0x4d 0x5a *is* MZ

It's a standard EFI image.

(In reply to Andreas Schwab from comment #21)

In other words: it is a MS-DOS exe ... hey isn't that ``cool''.

On AArch64 the EFI header gets generated without meta information that can tell us what the contents are.

On x86 the PE header includes metadata that allows tools like 'file' to identify it as a Linux kernel binary which we depend on to identify kernel binaries.

Andrew, do you think we could have the same on AArch64?

(In reply to Alexander Graf from comment #23)
> On AArch64 the EFI header gets generated without meta information that can
> tell us what the contents are.
> 
> On x86 the PE header includes metadata that allows tools like 'file' to
> identify it as a Linux kernel binary which we depend on to identify kernel
> binaries.
> 
> Andrew, do you think we could have the same on AArch64?

It is possible to have this on AArch64. Can you provide details on what exactly you are looking for, is it a specific line or something else? Could you point to an example for x86?

Looking a bit deeper, that particular header seems to be the x86 specific legacy boot loader:

        # Part 2 of the header, from the old setup.S
            
                .ascii  "HdrS"          # header signature
                .word   0x020d          # header version number (>= 0x0105)
                                        # or else old loadlin-1.5 will fail)

Sorry, pressed enter too quickly.

This particular header seems to only exist on x86 systems. On PowerPC for example, the kernel only gets detected as ELF binary as well (OF boots ELF rather than PE natively).

So overall, I think the output of "file" is fully correct. If ARM people want to, they can also add an HdrS chunk to their header, but I don't think it's necessary.

(In reply to Alexander Graf from comment #26)

In other words, the program file is not able to detect the kernel on AArch64 nor on PowerPC64 as it is not designed that way.

Don't know how the small get_kernel_version(.c) program is able to detect the version of such a plain MZ-DOS or ELF kernel image binary without such a header, but it has the same problem as file has: no magics is no informations.

Yet get_kernel_version is able to do its job.

(In reply to Andreas Schwab from comment #28)

> Yet get_kernel_version is able to do its job.

Hmm ... then I don't know why the this bug has the subject

     get_kernel_version does not support zImage

...?

Someone needs to fix that then.

(In reply to Andreas Schwab from comment #30)
> Someone needs to fix that then.

Maybe you should've opened a new bug for your AArch64 issue in the first place.

Werner, one issue was zImage on 32-bit arm, the other issue was Image on 64-bit aarch64. Different users, different systems, similar problems, HTE.

Ok, let's please keep separate issues separate

First off, "file" reports the file type for a specific binary. This is "PE" for AArch64, ELF for PowerPC, "special ARM zImage" for 32bit ARM and "magic x86 Linux kernel" for x86 systems. It behaves as designed and is fully correct.

The "get_kernel_version" tool basically does a simple grep through the binary to look for a string "Linux version". Whatever follows that string is deemed the kernel version. It's a hack, but it's a working hack as long as the image is uncompressed.

Now if you take a look at the script in /sbin/mkinitrd:

  # rpm -qf /sbin/mkinitrd
  dracut-037-32.1.aarch64

then you'll see that this script calls get_kernel_version on our zImage

        if [ "$kernel_image" != "$initrd_image" -a \
             -n "$kernel_version" -a \
             -d "/lib/modules/$kernel_version" ]; then
                kernel_images="$kernel_images $boot_dir/$kernel_image"
                initrd_images="$initrd_images $boot_dir/$initrd_image"
        fi

However, since the zImage is (as its name suggests), compressed, we have a really hard time finding the kernel version inside that file. Before dracut, we had a hack in mkinitrd that would fall back to the vmlinux.gz file instead so that we get_kernel_version can do its job:

  https://github.com/openSUSE/mkinitrd/commit/3e9cb61ee500fcc732e2299c99fcdf3e1813547f

This code must have gotten lost in transition to dracut. So let's ask Thomas :).

get_kernel_version can uncompress the file, it just needs to find the signature.

(In reply to Andreas Faerber from comment #31)

There is nothing to fix as if you have a look into the code of get_kernel_version  you will see a brut force method which search for the string

  "Linux version "

for both little and big endian case in the binary which is ... if required ... uncompressed with

  "/bin/gzip -dc %s 2>/dev/null"

... with the file command you can not do this, as file was not written to dump symbols from an elf binary nor symbols from a compressed elf binary.

I've done an

  strace get_kernel_version /boot/vmlinux | & grep ^read | wc -l

on PowerPC64 and this reports  351 reads with each of 32768 bytes to find
the final "Linux version " ... this has nothing todo with magic headers of magic marks for elf binaries. The PowerPC64 kernel probed with the file command shows

  ELF 64-bit MSB shared object, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, BuildID[sha1]=0x88bcd91b88298c500c6127aae9a4756811267a47, stripped

> ... with the file command you can not do this

That's why we have get_kernel_version.

(In reply to Andreas Schwab from comment #33)

Correct, but a zImage is not in gzip format and the MZ-DOS header fools get_kernel_version

zImage contains a gzip image.

Created attachment 633904 [details]
The list of magics found by file 5.22

(In reply to Andreas Schwab from comment #37)

Even if the image is gziped, it seems to be without magic header:

  typeset -i n=0
  while ((n++ < 65536)); do
    tail -c +$n < Image | file -k - | grep -v '/dev/stdin: data'
  done  > list
  grep -i gzip list

shows nothing about gzip

I've done a deeper search and found at the lines

80885:  gzip compressed data, reserved method, ASCII, has comment, encrypted, last modified: Fri Aug 29 06:40:32 2014, from Unix
81597:  gzip compressed data, reserved method, ASCII, has comment, encrypted, last modified: Fri Aug 29 07:35:28 2014, from Unix

but both

  tail -c +80885 Image | file -bz -
  tail -c +81597 Image | file -bz -

do show

  ERROR: zlib: invalid stored block lengths

Worksforme.

$ tail -c +19065 boot/zImage-4.0.1-1-default | zcat > zImage

gzip: stdin: decompression OK, trailing garbage ignored
$ get_kernel_version zImage
4.0.1-1-default

(In reply to Andreas Schwab from comment #40)

> ...Worksforme.
> 
> $ tail -c +19065 boot/zImage-4.0.1-1-default | zcat > zImage
> 
> gzip: stdin: decompression OK, trailing garbage ignored
> $ get_kernel_version zImage
> 4.0.1-1-default

Where can I find the rpm of this aarch64 kernel?

osc ls -b Kernel:stable kernel-default ARM/aarch64
_buildenv
_statistics
kernel-default-3.19.4-1.1.g74c332b.aarch64.rpm
kernel-default-3.19.4-1.1.g74c332b.nosrc.rpm
kernel-default-base-3.19.4-1.1.g74c332b.aarch64.rpm
kernel-default-base-debuginfo-3.19.4-1.1.g74c332b.aarch64.rpm
kernel-default-debuginfo-3.19.4-1.1.g74c332b.aarch64.rpm
kernel-default-debugsource-3.19.4-1.1.g74c332b.aarch64.rpm
kernel-default-devel-3.19.4-1.1.g74c332b.aarch64.rpm

It's all Factory.

So a zImage is an ELF binary that also contains a gziped chunk of kernel image from which the kernel version should be easily extractable.
file sees the ELF binary and stops (not unreasonably so).
Why does get_kernel_version fail then on so many architectures on a zImage?
Basically, it should scan the ELF file for a gzip header, decompress, proceed as it does for uncompresses kernel images on the decompressed data. Right?

> It's all Factory.
Hm, I added a fix to dracut based on the mkinitrd solution Alex Graf pointed me to. Thanks!
While the bug is against 13.2, can it be closed if only factory may be relevant?

If really needed I can add it to the next 13.2 dracut maintenance update which may take a while.

Hm, I close the bug fixed already.