Bug 562232

Yes, you don't have enough memory to do this with a kernel image that contains all debugging symbols and a very large number of kernel modules.

Either turn off kernel debugging, or don't build all of those kernel modules, or get some more memory for your system.

Anyway, this isn't a kernel error, nothing we can do about it, sorry.

I do care about module-init-tools bugs, though :)

Can you post the output of 'ulimit -a'? If you set some strict limit on virtual memory, then depmod can't do much about it. If that's not the case, please attach the output of
# strace -o strace /sbin/depmod 2.6.31.5-0.1-flavioweb

Here is 'ulimit -a' output:

ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3430
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) 379268
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3430
virtual memory          (kbytes, -v) 729680
file locks                      (-x) unlimited

If you need also 'strace' output tell me, because i must recompile the kernel with "debug_kernel" before...

So you have a limit of about 712MB on virtual memory. How much space do the modules with debuginfo take up? Try
du -sh /lib/modules/2.6.31.5-0.1-flavioweb
If it approaches the 712MB limit, then you need to increase it (depmod doesn't _allocate_ much memory, but it needs to mmap() the *.ko files, which counts as virtual memory usage).

du -sh /lib/modules/2.6.31.5-0.1-flavioweb
90M     /lib/modules/2.6.31.5-0.1-flavioweb

It's possible that "debug info" on modules take up 622 MB ???

...I have installed also the original modules "default", that have "debug info":

du -sh /lib/modules/2.6.31.5-0.1-default/
910M    /lib/modules/2.6.31.5-0.1-default/

Sig!

(In reply to comment #6)
> du -sh /lib/modules/2.6.31.5-0.1-flavioweb
> 90M     /lib/modules/2.6.31.5-0.1-flavioweb

Now I'm confused. Is this the same kernel that you encountered the bug with or did you rebuild w/o debuginfo in the meantime?


> It's possible that "debug info" on modules take up 622 MB ???

That's indeed possible yes (the 11.2 kernel-default-debuginfo package takes about 850MB). I suggest you just lift the virtual memory limit, depmod won't eat much memory, but it needs to mmap() all the *.ko files.

(In reply to comment #7)
> (In reply to comment #6)
> > du -sh /lib/modules/2.6.31.5-0.1-flavioweb
> > 90M     /lib/modules/2.6.31.5-0.1-flavioweb
> 
> Now I'm confused. Is this the same kernel that you encountered the bug with or
> did you rebuild w/o debuginfo in the meantime?
> 

It's the same kernel w/o debuginfo.

> I suggest you just lift the virtual memory limit, depmod won't
> eat much memory, but it needs to mmap() all the *.ko files.

So i need a bigger swap partition on disk, or there are some other things to do for lift the virtual memory limit?

Just a curiosity: why installing precompiled kernel and modules from rpms works fine, and not work compiling from source?

(In reply to comment #8)
> (In reply to comment #7)
> > I suggest you just lift the virtual memory limit, depmod won't
> > eat much memory, but it needs to mmap() all the *.ko files.
> 
> So i need a bigger swap partition on disk, or there are some other things to do
> for lift the virtual memory limit?

No, as I wrote, depmod doesn't allocate much memory, but the file mappings count as virtual memory.


> Just a curiosity: why installing precompiled kernel and modules from rpms works
> fine, and not work compiling from source?

Packages have the debuginfo split off into $package-debuginfo.rpm.

also have a look at this thread:

http://marc.info/?l=linux-kernel&m=119389053128382&w=2

seems that depmod issue has still not really been adressed!?

i`m curious why any bugreport about this has been closed or set to status "resolved", as it seems this has not really been resolved.

a utility which calculates module dependencies should not consume that large amounts of ram. it`s stupid behaviour, imho.

closing this bugreport will result in another user reporting the same isuse again. 

furthermore, i think throwing ram at bad software behaviour is NO solution. 
have seen that too often, especially with java programmers. ("oh - it does not work? add another 4gb, ram is cheap !)

>Anyway, this isn't a kernel error, nothing we can do about it, sorry.
yes, this is no kernel issue, but it`s a kernel build issue. under what should category/component should this be filed instead?

Care to read the previous comments? depmod (at least in the openSUSE module-init-tools package) allocates a few megabytes of "ram". But it needs to mmap the *.ko files, which adds to the virtual memory usage. Hence if you have a low virtual memory limit (ulimit -v) and compile a kernel with CONFIG_DEBUG_INFO=y, it will easily hit the limit.

yes, i have seen that comment, but i don`t understand why the whole bunch of .ko files (actually several thousands if you compile with most module options enabled) needs to be mmapped into virtual memory at the same time. why is that necessary and why to hog the virtual memory subsystem and put that much pressure on it? isn´t there a way to handle this more efficient/elegant?

if a recent kernel depmod needs ~800mb vsz, that`s a LOT if you take into consideration that a typical pc box may have 1-2gb physical ram and the same amount of swap (or less).

One last try: If you mmap a file on disk, it does not affect your ram or swap.

(In reply to comment #11)
> Care to read the previous comments? depmod (at least in the openSUSE
> module-init-tools package) allocates a few megabytes of "ram". But it needs to
> mmap the *.ko files, which adds to the virtual memory usage. Hence if you have
> a low virtual memory limit (ulimit -v) and compile a kernel with
> CONFIG_DEBUG_INFO=y, it will easily hit the limit.

I understand to add some virtual memory, but i still confused:
if i have a 512mb swap partition and set ulimit -v to "1000000", the -additional- memory where is -stored-?
-Virtual memory- and -swap partition- are the same?
Or i can have a 512mb of swap and 1gb of -virtual memory-?
In my sistem i can have 512mb ram + 512mb of swap, and set -virtual memory- to 1,5gb?

And again: the -debug info- installed with the "kernel-debug-info.rpm" are not "depmodded" and the .ko files "mmapped()"?
Why install a "precomiled kernel-debug-info rpm" works, and compiling the same kernel from source cause the "out of memory"?
What's the difference?

IMHO "add memory" is not a solution but a "work around"...
A more efficient depmod is a solution.

Michal did not suggest adding memory, he suggested changing your limit for virtual memory. They're not the same thing and virtual memory is not just another term for "swap" despite how MacOS and Windows have abused the term.

Virtual memory is the address space associated with a program. Each program has it's own 32/64-bit address space, of which most of it is potentially usable. Much of it never is. It can be used by the program instructions, dynamic memory (like malloc), shared libraries, and mmap. Shared libraries can take up a considerable amount of your address space - but there is usually only a single copy in memory that is shared by all programs that need it. So for an easy example, every program running on your system right now probably uses libc. On my system, that is about 1.4M. I have about 200 processes running on my system, but that library will only ever take a maximum of 1.4 MB of memory.

UNIX systems have a facility called mmap, which gives programmers the ability map files into the address space of the program and access them as if they were loaded from disk into a chunk of memory. The difference is that only the portions that are actually accessed ever get read in from disk. Shared libraries use this and only ever load the parts of the library that are actually needed. This happens automatically when the memory is accessed. So for my libc example where I said it loads a maximum of 1.4 MB, it's actually less than that. There are portions of the library that are needed for compatibility but are hardly ever actually used. That code never gets loaded from disk.

For kernel modules, depmod really only needs the module headers and the symbol table. The debuginfo isn't used and is never loaded. That's why Michal is saying that while the modules on disk take up over 800 MB, depmod only ever allocates a few MB. That's all it's actually using and that's all that has ever been read in from disk.

BUT for sake of argument, lets say it did read everything in, since this seems to be the core of the misunderstanding here. If the system ran low on memory, the memory used by the modules wouldn't get pushed out to swap. Swap is used when the kernel doesn't have any other way to restore the contents of the memory when it's needed again. Since the kernel modules are already on disk, in your file system, it can just reuse the memory without writing anything out and re-read them from the file system when it's needed again. Swap would never be touched.

Thank you very much Jeff for the exaustive explanation.
Today i learned something new.
I move up the virtual memory limit, and then try again to compile the kernel.

i`d also like to thank you jeff for the real good explanation! never read such good one before!