Bug 565126

Summary: zypper eats all memory
Product: [openSUSE] openSUSE 11.2 Reporter: Michal Čihař <mcihar>
Component: libzyppAssignee: E-mail List <zypp-maintainers>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Major    
Priority: P5 - None CC: ma
Version: Final   
Target Milestone: ---   
Hardware: x86-64   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: zypper logs

Description Michal Čihař 2009-12-16 12:36:55 UTC
Created attachment 332910 [details]
zypper logs

I today did zypper dup and it ended up zypper eating all memory till I killed it. The problem seems to be that it had problems with locking rpm database and it filled logs with:

# error: cannot get exclusive lock on /var/lib/rpm/Packages

Output on terminal was:

# zypper dup
Retrieving repository 'home:Nijel' metadata [done]
Building repository 'home:Nijel' cache [done]
Retrieving repository 'Packman Repository' metadata [done]
Building repository 'Packman Repository' cache [done]
Retrieving repository 'openSUSE:Tools' metadata [done]
Building repository 'openSUSE:Tools' cache [done]
Loading repository data...
Reading installed packages...
Computing distribution upgrade...

The following packages are going to be upgraded:
  gstreamer-0_10-buzztard gstreamer-0_10-plugins-bad gstreamer-0_10-plugins-bad-lang libavcodec52 libavformat52 
  libavutil50 libgstbasevideo-0_10-0 libgstphotography-0_10-0 libgstsignalprocessor-0_10-0 liblash1 libpostproc51 
  libswscale0 libxine1 libxine1-codecs libxine1-gnome-vfs libxine1-pulse ntfs-3g pam-modules pam-modules-32bit 
  python-ZSI rarian scrollkeeper sox wammu yast2-network 

25 packages to upgrade.
Overall download size: 9.0 MiB. After the operation, additional 168.0 KiB will be used.
Continue? [y/n/?] (y): 
Retrieving package pam-modules-32bit-11.2-4.2.1.x86_64 (1/25), 38.0 KiB (87.0 KiB unpacked)
Retrieving delta: ./rpm/x86_64/pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm, 11.0 KiB
Retrieving: pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm [done]
Applying delta: ./pam-modules-32bit-11.2-4.1_4.2.1.x86_64.delta.rpm [done]
Installing: pam-modules-32bit-11.2-4.2.1 [done]
Retrieving package pam-modules-11.2-4.2.1.x86_64 (2/25), 111.0 KiB (462.0 KiB unpacked)
Retrieving delta: ./rpm/x86_64/pam-modules-11.2-4.1_4.2.1.x86_64.delta.rpm, 22.0 KiB
Retrieving: pam-modules-11.2-4.1_4.2.1.x86_64.delta.rpm [done]
Applying delta: ./pam-modules-11.2-4.1_4.2.1.x86_64.delta.rpm [done]
Installing: pam-modules-11.2-4.2.1 [94%]^C
OK OK! Exiting immediately...

I'm attaching zypper logs.
Comment 1 Michal Čihař 2009-12-16 12:49:01 UTC
Just a side note: the rpm reached memory usage almost 1G before it terminated and zypper memory usage reached 2.6G before I killed it.
Comment 2 Michael Andres 2009-12-16 17:11:57 UTC
This seems to be an rpm problem. Unfortunately the log does not tell anything:

  2009-12-16 11:51:18 rpm pid 30978 was launched
  2009-12-16 12:03:02 Pid 30978 successfully completed

No messages in between. Zypp stores the rpm error messages to report them back to the application. If rpm produces messages in an endless loop, then zypper will grow as well.
Comment 3 Michael Schröder 2009-12-16 17:14:20 UTC
Hmm, needinfo for me? I don't see how I can provide any info.

Doesn't libzypp put the rpm output in some (tmp)file?
Comment 4 Michael Andres 2009-12-17 11:56:25 UTC
No. Rpm usually does not loop endlessly.
Comment 5 Michael Schröder 2010-03-11 18:20:59 UTC
I still don't see how I can help you here.
Comment 6 Michael Andres 2010-03-11 18:46:48 UTC
Closing until it re-occurs and is reproducible.
Comment 7 Michal Čihař 2010-03-12 11:15:33 UTC
I thought the issue is quite clear - zypper stores all rpm output in memory and when rpm produces a lot of output, memory is not enough.
Comment 8 Michael Schröder 2010-03-12 11:28:00 UTC
Maybe, but rpm should not produce lots of output. And as you can't tell us what the output was there's not much to do from my (rpm) side.

ma, maybe zypper should discard the read data after 1 MB or so.
Comment 9 Michal Čihař 2010-03-12 13:07:55 UTC
I'm able to tell quite exactly what the output was and it can be seen from attached logs. It is there  about 3,000,000 times:

error: cannot get exclusive lock on /var/lib/rpm/Packages

It's clear that it was some error or race condition, however zypper should behave more clever in this situation and should at least tell something to user. Not just silently consume all memory waiting for OOM killer to come.
Comment 10 Michael Schröder 2010-03-12 13:16:21 UTC
Yes, but it's also a bug in 11.2's rpm. Seems to be a misapplied hunk of a patch or a copy/paste error on my side. Fixing...
Comment 11 Michael Andres 2010-03-26 14:13:22 UTC
libzypp-6.31.2 will collect at most 10000 lines of output.