Bug 1132650

Hi Per, 

Please, collect the YaST logs executing `save_y2logs` and attach them here. They provide useful information to ease to understand the problem.

Thanks.

See attachment#803074 [details]

I don't think there is much to see, all I observed was a loop. I did an strace too, no calls.  Easy to reproduce though.

how much ram does the machine have? Maybe we need to increase the minimal requirements

(In reply to Ludwig Nussel from comment #3)
> how much ram does the machine have? 

From the logs, it seems to have 1Gb

> 111:[    0.000000] Memory: 978108K/1038904K available (10256K kernel code, 1469K rwdata, 3476K rodata, 2092K init, 9888K bss, 60796K reserved, 0K cma-reserved)

(In reply to Per Jessen from comment #2)
> See attachment#803074 [details]
> 
> I don't think there is much to see, 

As you can see in the comment above, always there is something to see ;)

> Easy to reproduce though.

I disagree. A couple of times, I tried to reproduce some reported bugs following the exact steps given by the reporter without success. Believe me, logs usually help. That's the reason why they are there :)

Thanks.

I can reproduce this bug with 1Gb Ram. It is working with 2Gb.
 
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                      
 3700 root      20   0 1176132 461052  68512 R 27.33 23.22   0:45.70 y2start                                                                      
 3745 root      20   0  104756  31404  11328 S 0.000 1.582   0:00.18 ag_udev_persist                                                              
 4272 root      20   0   46684  18840   4328 S 0.000 0.949   0:00.37 ag_anyxml                                                                    
    1 root      20   0   91872  17768   9024 S 0.000 0.895   0:06.59 init                                                                         
 1889 root      20   0   91872  10584   1840 S 0.000 0.533   0:00.00 init           

Ludwig do you know the minimal requirements ?
I have tested openSUSE-Leap-15.1-DVD-x86_64-Build451.2-Media.iso

Hardware requirements are constantly increasing. Much of that is the consequence of all kinds of subsystems requiring more and more resources, and YaST is always at the very end of the food chain; it has to make do with however much RAM is left. We've seen this many times before.

I am not sure exactly how much we can do about this.

Please be aware that for the installation, much of the system that is booted from DVD or from a USB stick is actually unpacked into a RAM disk which of course increases RAM usage. If you use a NET ISO, this increases because the installation medium does not provide many files, so they have to be fetched over the network and put into temporary storage on the RAM disk as well.

If you tried such a NET ISO, you might be more successful with a full ISO that contains more files directly, so it needs less temporary storage on the RAM disk.

Other than that, it might really be time to increase the minimum memory requirements. Sorry.

As mentioned, YaST can only partly be held responsible for that; the kernel and all kinds of subsystems already consume a lot of RAM before YaST even starts.

(In reply to David Diaz from comment #5)
> (In reply to Per Jessen from comment #2)
> > See attachment#803074 [details]
> > 
> > I don't think there is much to see, 
> 
> As you can see in the comment above, always there is something to see ;)
> 
> > Easy to reproduce though.
> 
> I disagree. A couple of times, I tried to reproduce some reported bugs

I meant it was easy to reproduce here.  Happened at least twice until I realised it was running out of memory and needed swap.

(In reply to Ludwig Nussel from comment #3)
> how much ram does the machine have? Maybe we need to increase the minimal
> requirements

As others have already added, it has 1Gb.

(In reply to Stefan Hundhammer from comment #7)
> Hardware requirements are constantly increasing. Much of that is the
> consequence of all kinds of subsystems requiring more and more resources,
> and YaST is always at the very end of the food chain; it has to make do with
> however much RAM is left. We've seen this many times before.

Nonetheless, once installed, openSUSE runs fine in 512M on my ARM boards, or on a RaspBerry Pi. 

> I am not sure exactly how much we can do about this.

I guess it's not possible (or desirable) to add any swap space found? 
linuxrc has an "addswap" parameter, so it sounds like it is possible at least.

You have to activate any pre-existing swap space manually; that's what that linuxrc options does. We don't always just activate any swap space that we find because you might want to re-partition the disk.

Maybe (not sure) it is activated if your available RAM is below the minimum requirements, but those requirements right now are 1 GB, so that wouldn't apply in your case.

Docs (I guess you already found them):

  https://en.opensuse.org/SDB:Linuxrc

I think the main problem is here that we do not really inform the user that his machine does not have enough memory. In that case YaST is running in an unstable state. Of course it is quite hard to define this because memory allocation depends an many factors (e.g. repo sizes).
So how to go on here
- Simply increasing the memory requirements
- Evaluate if we can decrease memory usage "somehow".
Jiri, what do you think ?

There is one thing not said explicitly in this bugreport: The RAM requirement depends on the install media. In this case, it was installed via HTTP, which - as any other medias which cannot be mounted - just increases the RAM requirement by the size of the installation system.

If the same system was installed from DVD, USB stick, NFS, Samba - it could be successful.

That said: Maybe we should increase the requirement depending on the installation media (and possibly also on whether NCurses or Qt is being used)?


Of course, if we identify who is consuming too much memory, we should ask the respective developer to address that too. But there is not always possible to point to just one component.

(In reply to Jiri Srain from comment #13)
> There is one thing not said explicitly in this bugreport: The RAM
> requirement depends on the install media. In this case, it was installed via
> HTTP, which - as any other medias which cannot be mounted - just increases
> the RAM requirement by the size of the installation system.
> 
> If the same system was installed from DVD, USB stick, NFS, Samba - it could
> be successful.
> 
> That said: Maybe we should increase the requirement depending on the
> installation media (and possibly also on whether NCurses or Qt is being
> used)?
> 
> 
> Of course, if we identify who is consuming too much memory, we should ask
> the respective developer to address that too. But there is not always
> possible to point to just one component.

Thanks Jiri. So the main problem is that we do not know how much memory we need because this depends on the install media. The memory will be used while parsing the package information. Would it be possible that libzypp is throwing an exception while parsing this information if there is no more memory available ? Then YaST could then inform the user at least.
Assigning to libzypp team. Perhaps they have a solution for it :-)

@Schubi, please help me. Where do you see or suspect a loop or error condition zypp could have reported? I can't spot the problem in the log.

(In reply to Michael Andres from comment #15)
> @Schubi, please help me. Where do you see or suspect a loop or error
> condition zypp could have reported? I can't spot the problem in the log.

I posted some details earlier on factory list:

https://lists.opensuse.org/opensuse-factory/2019-03/msg00326.html

(In reply to Michael Andres from comment #15)
> @Schubi, please help me. Where do you see or suspect a loop or error
> condition zypp could have reported? I can't spot the problem in the log.

I have reproduced the "error", but without any "loop". The process has hanged while parsing the repos. It has not hanged when I have increased the memory.

(In reply to Stefan Schubert from comment #17)
> (In reply to Michael Andres from comment #15)
> > @Schubi, please help me. Where do you see or suspect a loop or error
> > condition zypp could have reported? I can't spot the problem in the log.
> 
> I have reproduced the "error", but without any "loop". The process has
> hanged while parsing the repos. It has not hanged when I have increased the
> memory.

What I observed was a yast process running at high cpu% (less than 80), but not doing much (I straced it, no calls).  As soon as I added swap, it started working.

(In reply to Stefan Schubert from comment #17)
> (In reply to Michael Andres from comment #15)
> > @Schubi, please help me. Where do you see or suspect a loop or error
> > condition zypp could have reported? I can't spot the problem in the log.
> 
> I have reproduced the "error", but without any "loop". The process has
> hanged while parsing the repos. It has not hanged when I have increased the
> memory.

If you're able to reproduce it, could you please add the logs.

Created attachment 803825 [details]
y2log from Fujitsu Siemens Lifebook S7110 with 1GiB of mem

I saw the same behavior as described in comment #0 on Fujitsu Siemens Lifebook S7110 with 1GiB of memory. I tried Leap 15.0, 15.1 and Tumbleweed with HTTP installation without success. Only on the last attempt I took y2log snapshot while installer was still looping. I'm attaching the corresponding log.

So I guess all boils down to just increase LOW_MEMORY_MIB in https://github.com/yast/yast-packager/blob/acc6dbfd155d827a2c8f02f42df2fec2888b6572/src/lib/y2packager/clients/inst_productsources.rb#L97 to 1.5GB. Maybe it would make sense to put that setting into control.xml to make it easier to find.

Created attachment 804126 [details]
y2log

It is still reproduce able.

Created attachment 804128 [details]
y2log with Y2DEBUG=1

more logging

(In reply to Stefan Schubert from comment #22)
> 
> It is still reproduce able.

But how should libyzpp help here? All logs (c#20-#23) show libyzpp being able to load all reops, iterate the repos and even able to run the resolver, without indication of any memory related problem. 

All logs end with
> [Pkg] modules/Packages.rb:824 Pkg Builtin called: ResolvableProperties

This is AFAIK the place where yast-pkg-bindings try to build up YCPLists with resolvable attributes. 

AFAIR there've been reports in the past that those lists are too big (as they may contain almost all resolvable attributes and not just what the caller actually needs). 

Sorry, but currently I see no hint on how libyzpp could help here. I'd suggest to figure out first what the yast-pkg-bindings are expected to deliver back to the ruby code. Maybe it's simply too much.

Back to YAST.

(In reply to Michael Andres from comment #24)
> This is AFAIK the place where yast-pkg-bindings try to build up YCPLists
> with resolvable attributes. 
> 
> AFAIR there've been reports in the past that those lists are too big (as
> they may contain almost all resolvable attributes and not just what the
> caller actually needs). 
> 
Ok, that's a point for "investigation". Added to Trello.

Fixed in yast2-packager-4.1.41 (https://github.com/yast/yast-packager/pull/434)

SLE15-SP1 submission: https://build.suse.de/request/show/192319

(This possibly also affects the SLE15-SP1 installer when using too many modules/extensions.)

The code change makes me nervous for Leap. There is no buffer for mistakes left. So I'd rather prefer increasing the requirement as explained in comment#21 for Leap. The code is different from SLE there anyways. For Leap it would only change the wording of the dialog.

(In reply to Ludwig Nussel from comment #28)
> The code change makes me nervous for Leap. There is no buffer for mistakes
> left. So I'd rather prefer increasing the requirement as explained in
> comment#21 for Leap. The code is different from SLE there anyways. For Leap
> it would only change the wording of the dialog.

Understood. I'll reject for SUSE:SLE-15-SP1:GA, we should manage it as installer update instead.

OK, I have reverted the fix, instead I updated the memory limit for displaying a warning popup as suggested by Ludwig.

https://github.com/yast/yast-packager/pull/435

This change affects only Leap, that code is used only in the Leap installer (https://github.com/yast/skelcd-control-openSUSE/blob/openSUSE-15_1/control/control.openSUSE.xml#L908), the SLE installer is not affected.

This is an autogenerated message for OBS integration:
This bug (1132650) was mentioned in
https://build.opensuse.org/request/show/701993 15.1 / yast2-packager

*** Bug 1134758 has been marked as a duplicate of this bug. ***

*** Bug 1129619 has been marked as a duplicate of this bug. ***

*** Bug 1158097 has been marked as a duplicate of this bug. ***

SUSE-RU-2020:0313-1: An update that has four recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1132650,1157202,1158247,1159120
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP5 (src):    yast2-pkg-bindings-devel-doc-3.2.8-3.3.1
SUSE Linux Enterprise Server Installer 12-SP5 (src):    yast2-pkg-bindings-3.2.8-3.3.1
SUSE Linux Enterprise Server 12-SP5 (src):    yast2-pkg-bindings-3.2.8-3.3.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.

This is an autogenerated message for OBS integration:
This bug (1132650) was mentioned in
https://build.opensuse.org/request/show/844234 Factory / yast2-registration