Bug 1097297

Summary: Repository change after failed upgrade to 15.0
Product: [openSUSE] openSUSE Distribution Reporter: Forgotten User SyiDIkBHGJ <forgotten_SyiDIkBHGJ>
Component: YaST2Assignee: YaST Team <yast-internal>
Status: RESOLVED FIXED QA Contact: Jiri Srain <jsrain>
Severity: Normal    
Priority: P2 - High CC: ancor, astieger, dgonzalez, forgotten_SyiDIkBHGJ, jreidinger, lslezak
Version: Leap 42.3   
Target Milestone: ---   
Hardware: Other   
OS: Other   
URL: https://trello.com/c/AL4wp6NJ
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Forgotten User SyiDIkBHGJ 2018-06-12 20:46:02 UTC
I tried to upgrade from 42.3 to 15.0 which fails due to bug #1094963. After the failed upgrade attempt the Yast2 repositories were silently changed to 42.2. No 42.3 repositories are listed anymore. This triggers an avalanche of updates which turn out to be downgrades.
Comment 1 Andreas Stieger 2018-06-13 06:33:27 UTC
(In reply to Peter Szarafinski from comment #0)
> After the failed upgrade attempt the Yast2 repositories were silently changed
> to 42.2.

Uh how is that supposed to have happened?

You write nothing of how exactly you performed the upgrade, and what exactly your observations are
Comment 2 Forgotten User SyiDIkBHGJ 2018-06-13 17:36:00 UTC
Installed OS is 42.3

1. I tried to upgrade from 42.3 to 15.0 via USB (DVD image)
2. The upgrade failed due to bug #1094963
3. After starting the installed 42.3 system, the software updater notified me that there is an update for zypper.
4. After the zypper update, the software updater notified me of several hundred updates.
5. Installing these updates results in downgrading many programs, the most obvious ones for me were Firefox and LibreOffice.
6. When checking the repositories in Yast2, I discovered that they have been changed from 42.3 to 42.2. No 42.3 repositories are listed anymore. I received no information about the change of repositories.

This behaviour can be reproduced. When changing the repositories in Yast2 manually to 42.3 and then trying again to install the upgrade from 42.3 to 15.0, the result is the same.

I am not sure if this behaviour is related to the abvove mentioned bug #1094963.
Comment 3 Josef Reidinger 2018-06-19 14:02:24 UTC
Well, yast2 has feature to restore failed system-upgrade at https://github.com/yast/yast-installation/blob/master/startup/YaST2.call#L278

And it is created during upgrade at https://github.com/yast/yast-update/blob/master/src/modules/Update.rb#L784

My suspicion is that there is both backups from 42.2 update and 42.3 update.

Peter - can you pleace check content of /var/adm/backup/system-upgrade and paste here output of 

ls -l /var/adm/backup/system-upgrade

so we can check if it can happen? Thanks
Comment 4 Josef Reidinger 2018-06-19 14:07:50 UTC
another theory is that it restore old backup before new one can be created due to previously mentioned bug. But both can easily show content of that directory. I think in both case we should clean system-upgrade when we start new upgrade.
Comment 5 Forgotten User SyiDIkBHGJ 2018-06-19 19:41:39 UTC
> Peter - can you pleace check content of /var/adm/backup/system-upgrade and
> paste here output of 
> 
> ls -l /var/adm/backup/system-upgrade

[QUOTE]
insgesamt 39780
-rwxr--r-- 1 root root      324 27. Jul 2017  restore-sw_mgmt.sh
-rw------- 1 root root 40723047 27. Jul 2017  sw_mgmt.tar.gz
[/QUOTE]
Comment 6 Josef Reidinger 2018-06-21 07:54:59 UTC
So here is back. Can you peter please upload that sw_mgmt.tar.gz file so we can inspect content? if it is leap 42.2 then we at least know where it appear and we probably have to inject immediatelly after mount of partition removal of this backup directory, so we are sure that old backup is not used, even if we not create new backup yet.
Comment 7 Forgotten User SyiDIkBHGJ 2018-06-21 18:54:15 UTC
You may download the file at https://www.dropbox.com/s/91015e4qb5rx8t2/sw_mgmt.tar.gz?dl=0 (it is too big for bugzilla). Thanks for your support!
Comment 8 Josef Reidinger 2018-08-20 13:20:22 UTC
Thanks, it confirm my findings that it use old backup from 42.2, so this is source of old repositories. We should remove this backup immediatelly after mounting directory to avoid its misuse as backup for new upgrade.

Thanks for help with finding root of issue.
Comment 9 Josef Reidinger 2018-08-20 13:36:20 UTC
I localize that it failed when mounted "/" but then failed during try to load other mount points and before we clean old backups.

Lada or Ancor: do you have idea how to avoid this window for this kind of error? We are not sure that content of /mnt is complete ( as we do not mount everything ), but on other hand automatic update backup can be already available, we are just not sure that it is correct one. Maybe some check of timestamp??? Like restoring only backup that is no more then 1 day old?
Comment 10 Ancor Gonzalez Sosa 2018-08-22 13:38:57 UTC
Introducing a mechanism to prevent the situation doesn't look like rocket science. Adding some extra info to the backup when we create it (so it's easy to discard it if that information doesn't match) or checking the timestamp (no modification needed when creating the backup but maybe weaker). I would leave that detail to whoever implement it. But it's definitely doable.
Comment 11 Ladislav Slezák 2018-08-30 06:27:24 UTC
Yes, checking a timestamp looks as an easy solution. However, ignoring backups older than 1 day is a bit fragile. You might start the upgrade, leave the machine and continue the next day. I'm doing this with my testing VMs quite often.


We should rather compare the age of the backup with the "uptime" value (either run /usr/bin/uptime or read /proc/uptime). If the backup is older than the uptime then it was not created by the current YaST run and we should ignore it.
Comment 12 Josef Reidinger 2018-08-30 07:53:10 UTC
OK, I think it is clear now what needs to be done. So lets add it to our queue for fixing.
Comment 13 David Diaz 2018-09-20 08:53:17 UTC
In order to keep compatibility with those devices without RTC like RPi, we ended discarding the idea to rely on boot time and/or the "age" of backup files and using a different approach based on copy the /etc/os-release file to check if the available backup should be restored.

Fix will be available in SLE-15 SP1

PRs:
 - https://github.com/yast/yast-update/pull/111
 - https://github.com/yast/yast-installation/pull/742

SRs:
 - https://build.opensuse.org/request/show/636358
 - https://build.opensuse.org/request/show/636838