Bug 292986

Summary: libzypp: Old sources are not migrated after update
Product: [openSUSE] openSUSE 10.3 Reporter: Jan Blunck <jblunck>
Component: libzyppAssignee: Duncan Mac-Vicar <dmacvicar>
Status: RESOLVED FIXED QA Contact: Klaus Kämpf <kkaempf>
Severity: Blocker    
Priority: P5 - None CC: casualprogrammer, coolo, gp, kkaempf, locilka, ma, mvidner, schubi
Version: Alpha 6   
Target Milestone: ---   
Hardware: x86-64   
OS: Other   
Whiteboard:
Found By: Development Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Jan Blunck 2007-07-19 08:46:58 UTC
I updated the system via zypper update -t package. The update succeed but afterward zypper doesn't work anymore.

weierstrass:/var/log # zypper sl
No repositories defined. Use 'zypper addrepo' command to add one or more repositories.
weierstrass:/var/log # 

The repository was dist:/install/stable-x86 from Thu Jul 19 11:02:37 CEST 2007.
Comment 1 Klaus Kämpf 2007-07-19 08:52:55 UTC
Please provide /var/log/zypper.log
Comment 2 Jan Blunck 2007-07-19 08:53:32 UTC
The logfile is at http://w3.suse.de/~jblunck/bug292986/zypper.log.bz2.
Comment 3 Jan Blunck 2007-07-19 08:54:34 UTC
Done.
Comment 4 Jan Kupec 2007-07-19 10:59:53 UTC
(In reply to comment #2 from Jan Blunck)
> The logfile is at http://w3.suse.de/~jblunck/bug292986/zypper.log.bz2.
getting permission denied to the file
Comment 5 Gerald Pfeifer 2007-07-20 00:30:20 UTC
I also have been seeing this on two out of two machines.  Did someone
actually manage a successfull update?

Anyway, if I have ever seen a blocker, this is one.  Bumping severity.
Comment 6 Gerald Pfeifer 2007-07-20 00:40:23 UTC
http://w3.suse.de/~gp/BZ/292986-zypper.log.bz2
Comment 7 Stanislav Visnovsky 2007-07-20 11:59:36 UTC
Just to be sure:
1) this is a clean Alpha6 install
2) after the installation, you did 'zypper update -t package'

Correct?
Comment 8 Gerald Pfeifer 2007-07-21 11:00:39 UTC
In my case (comment #5 and comment #6) I did updates from STABLE
and afterwards all installation sources were gone.
Comment 9 Stanislav Visnovsky 2007-07-23 09:17:43 UTC
Gerald, what was the starting point before you update from STABLE?
Comment 10 Martin Vidner 2007-07-23 15:03:37 UTC
*** Bug 293080 has been marked as a duplicate of this bug. ***
Comment 11 Gerald Pfeifer 2007-07-23 15:06:42 UTC
I'm not sure I understand the question correctly.  I had been
tracking STABLE on a twice-a-week basis, and the update was from
STABLE as of a week or so ago to STABLE as of when I reported the
bug.
Comment 12 Stephan Kulow 2007-07-24 08:14:33 UTC
I wonder what's so complicated about this bug. Install a alpha5, add some repos and do an update to alpha6, try zypper sl -> no repos. What else do you need?
Comment 13 Stanislav Visnovsky 2007-07-24 08:34:49 UTC
Thanks for explaining the scenario. So, here are the comments:

1) new libzypp uses different setup of repo definition
2) on distro upgrade, the source manipulation/transfer from the old version to the new one is typically handled by YaST - so just updating a package is not enough to migrate the repos.

The behavior until now was to disable all existing repos, but they stayed in the system.
Comment 14 Stephan Kulow 2007-07-24 09:15:17 UTC
so zypper can't update itself? Great ;(
Comment 15 Stanislav Visnovsky 2007-07-24 09:31:13 UTC
This is not zypper problem. Either libzypp (the package might migrate the repo setup in post-install script) or YaST installation.
Comment 16 Stephan Kulow 2007-07-24 09:33:06 UTC
Yeah, I meant to say libzypp can't update itself.
Comment 17 Klaus Kämpf 2007-07-25 12:15:32 UTC
We are talking about keeping repositories across distribution upgrade 'boundaries'. Specifically the Code10 -> Code11 (openSUSE 10.3 is a pre-Code11)
boundary.
(Upgrading STABLE from pre-Alpha6 to post-Alpha5 is crossing this boundary for package management, so expect to leave something behind)

I fail to see why we want to have Code10-related repositories on a Code11 system.

Comment 18 Casual J. Programmer 2007-07-25 12:40:02 UTC
While comment #17 may be valid, it overlooks, that after the update in question you are left with 0 ( zero ) repositories.

I don't think it would pose a major problem to set the repositories right. Apart from that also the DVD as well as all non SuSE repositories get eliminated. ( At least that's what happened in my case. )

While most of the people reading this, know where to find the information ( by now ), yet I doubt that the average user will be able to cope.
Comment 19 Klaus Kämpf 2007-07-25 13:04:07 UTC
Well, the zero repository problem in Alpha6 is known and covered by another bug report.

After installation of (or upgrade to) OpenSUSE 10.3, the following repositories should be available
- the 10.3 installation repository (from where the system was installed/upgraded)
- the 10.3 upgrade repository

My point is, that all other previously known repositories are probably not compatible with 10.3 anyways.
Comment 20 Casual J. Programmer 2007-07-25 13:20:02 UTC
Agreed, 

but wouldn't it suffice to set the other repositories to "off" and mark them as "needs to be checked", so the user can then decide what to do with them, rather than eliminating the information completely ?

There are quite some repositories ( Packman, http://software.opensuse.org/ ) that provide software running under 10.3 although labeled 10.2 or older.
Comment 21 Klaus Kämpf 2007-07-25 13:33:20 UTC
Converting the old repo data to the new format isn't that easy and I'd like to save the effort now.

I agree that it would be nice to have, but doubt that this qualifies this bug as a blocker ;-)

Btw, how many 3rd party packages will still be able to run on 10.3 (or later) ?
There are some considerable changes between 10.2 and 10.3 e.g. in library versions in the core system.
Comment 22 Stephan Kulow 2007-07-25 13:40:14 UTC
Yes, it is a blocker because you're loosing your data.

While my repos may not fit 10.3 _now_, I may either want to wait for them to fit or edit them. But having to readd them _from_ scratch is definitely a no-go.
Comment 23 Sid Boyce 2007-07-25 13:55:25 UTC
Adding the following lines to /etc/zypp/repos.d/* solved it.
enabled=1
autorefresh=1
Comment 24 Klaus Kämpf 2007-07-25 14:00:51 UTC
Actually no, since these settings only enable the new (10.3-based) repositories.

Your old ones (from the 10.2 installation) still need to be converted according to comment #22
Comment 25 Duncan Mac-Vicar 2007-07-31 15:08:32 UTC
I just commited to libzypp/tools a tool to migrate old sources to the new repositories.

It is just a test cool, more to get the code needed to do it. Now the question is where, when, how should this migration be done.

What is missing in the tool:

- It accepts a root parameter for the path where the old sources are, but saves them in the current root prefix.
- It migrates them using aliases migrated_XXX and disables them on purpose (this could be configurable).

Klaus?
Comment 26 Klaus Kämpf 2007-07-31 15:16:29 UTC
Thats probably more for Coolo since he had some requirements for repository migration ...


IMHO, it would be sufficient to run the migration from within YaST. However, this would rule out the brave who update from factory. So maybe the %post script of libzypp is a better place ? OTOH, we dont want to drag the migration script around for too long.
Comment 27 Stephan Kulow 2007-07-31 16:31:53 UTC
I would do it in the %post if there is no obvious problem with it. 
Comment 29 Stanislav Visnovsky 2007-08-01 07:06:29 UTC
As the RPM is installed in chroot already, if we use %post (I don't see a problem with that), no need to change the root for finding the configuration.
Comment 30 Stanislav Visnovsky 2007-08-07 12:02:00 UTC
Duncan, any chance to get this fixed soon?
Comment 32 Duncan Mac-Vicar 2007-08-08 14:44:06 UTC
Fixed in 3.3.15
Comment 33 Stanislav Visnovsky 2007-08-16 07:36:56 UTC
Downloading package libzypp-3.14.0-6.i586, 2.5 M (7.7 M unpacked)
* Installing: libzypp-3.14.0-6 [100%]
Invalid object
* Installing: libzypp-3.14.0-6 [100%]
(with --nodeps)Invalid object
* Installing: libzypp-3.14.0-6 [100%]Installation of [S3:1][package]libzypp-3.14.0-6.i586 failed:
(with --nodeps --force) Invalid object: Subprocess failed. Error: RPM failed: Migrating repo:
--------------------------------------
- alias       : migrated_0
Error adding repository: Software management is already running.
--------------------------------------
- alias       : migrated_0
terminate called after throwing an instance of 'zypp::ZYppFactoryException'
  what():  Software management is already running.
/var/tmp/rpm-tmp.63887: line 2:  6583 Aborted                 /usr/lib/zypp/zypp-migrate-sources
error: %post(libzypp-3.14.0-6.i586) scriptlet failed, exit status 134


(A)bort, (R)etry, (I)gnore?   

Using zypp in post-install script of rpm is not a good idea.
Comment 35 Stanislav Visnovsky 2007-08-17 07:43:02 UTC
*** Bug 301197 has been marked as a duplicate of this bug. ***
Comment 37 Stanislav Visnovsky 2007-08-23 09:32:03 UTC
Martin, please, test if the current fix works in YaST/zypper.
Comment 38 Lukas Ocilka 2007-08-23 11:00:31 UTC
Martin, see also bug 297136
Comment 39 Martin Vidner 2007-08-23 13:37:00 UTC
Which current fix? libzypp-3.18.2-4 and zypper-0.8.15 don't work:

totoro:~ # rpm -e --justdb libzypp --nodeps
totoro:~ # LANG=C zypper in libzypp
* Reading repository 'openSUSE-10.3-Updates' cache
* Reading repository 'http://cml.suse.cz/netboot/find/openSUSE-10.3-Beta1-DVD-i386' cache
* Reading repository 'FACTORY - Main Repository (OSS)' cache
* Reading installed packages [100%]


The following NEW package is going to be installed:
  libzypp

Overall download size: 2.5 M. After the operation, additional 7.7 M will be used.
Continue? [y/n]: y
Downloading package libzypp-3.18.2-4.i586, 2.5 M (7.7 M unpacked)
* Installing: libzypp-3.18.2-4 [100%]Installation of [S3:1][package]libzypp-3.18.2-4.i586 failed:
(with --nodeps --force) Invalid object: Subprocess failed. Error: RPM failed: Migrating repo:
--------------------------------------
- alias       : migrated_0
Error adding repository: Software management is already running.
--------------------------------------
- alias       : migrated_0
terminate called after throwing an instance of 'zypp::ZYppFactoryException'
  what():  Software management is already running.
/var/tmp/rpm-tmp.7040: line 2: 15904 Aborted                 /usr/lib/zypp/zypp-migrate-sources
error: %post(libzypp-3.18.2-4.i586) scriptlet failed, exit status 134


(A)bort, (R)etry, (I)gnore?  
Comment 40 Martin Vidner 2007-08-23 15:35:51 UTC
Well now I have to wait until Factory rebuilds the fixed zypp and its users.

(Actually it should not have to be necessary because the changes were only in the patch level, but the users of libzypp wrongly depend on the complete library version. To be continued...)
Comment 42 Stanislav Visnovsky 2007-08-28 08:25:09 UTC
I've tried to verify the fix:

I've downloaded libzypp-3.18.4, libzypp-devel, zypper and dependant yast2-* packages, called createrepo and added the resulting repo-md repository.

mammoth:~ # zypper up -t package
* Reading repository 'http://cml.suse.cz/netboot/find/openSUSE-10.3-Beta1-DVD-i386' cache
* Reading repository 'openSUSE-10.3-Updates' cache
* Reading repository 'test-migrate' cache
* Reading repository 'factory' cache
* Reading installed packages [100%]


The following packages are going to be upgraded:
  zypper yast2-pkg-bindings yast2-ncurses yast2-qt libzypp yast2-perl-bindings
libzypp-devel

Overall download size: 10.8 M. After the operation, additional 116.8 K will be used.
Continue? [y/n]: y
Downloading package libzypp-3.18.4-6.i586, 2.5 M (7.7 M unpacked)
* Installing: libzypp-3.18.4-6 [100%]Installation of [S3:1][package]libzypp-3.18.4-6.i586 failed:
(with --nodeps --force) Invalid object: Subprocess failed. Error: RPM failed: terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_S_construct NULL not valid
/var/tmp/rpm-tmp.438: line 2: 19753 Aborted                 /usr/lib/zypp/zypp-migrate-sources
error: %post(libzypp-3.18.4-6.i586) scriptlet failed, exit status 134


(A)bort, (R)etry, (I)gnore?         
Comment 43 Stanislav Visnovsky 2007-08-28 08:26:55 UTC
This system is Factory already, so I've just copied source file from SLES10 SP1 machine. This also means the cache in /var/lib/zypp/cache does not exist at all.
Comment 44 Martin Vidner 2007-08-28 10:26:37 UTC
if ( string(getenv("YAST_IS_RUNNING")) == "instsys" ) ...
"The getenv() function returns a pointer to the value in the environment, or NULL if there is no match."
Comment 45 Duncan Mac-Vicar 2007-08-29 09:23:01 UTC
new fix in svn. Will close when submitted.
Comment 46 Michael Andres 2007-08-29 16:50:52 UTC
fixed in libzypp-3.20.0