Bug 1061208

Summary: zypper segfaults upon refresh
Product: [openSUSE] openSUSE Distribution Reporter: Richard Weinberger <richard>
Component: libzyppAssignee: Patrick Finie <maninredd>
Status: RESOLVED NORESPONSE QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: maninredd, meissner, richard
Version: Leap 15.1Flags: ma: needinfo? (maninredd)
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Bug Depends on:    
Bug Blocks: 1146027    
Attachments: zypper core
zypper log file
zypper debug log
zypper log
Dmesg when executing zypper
condensed zypper log
updatetestcase WSL
updatetestcase WSL 2
updatetestcase WSL 3

Description Richard Weinberger 2017-10-01 19:28:58 UTC
Created attachment 742668 [details]
zypper core

After adding the mozilla repo zypper crashes every time upon refresh.

e.g.
zypper> ref
Repository 'go' is up to date.                                                                                                                                                                                                               
Repository 'libdvdcss repository' is up to date.                                                                                                                                                                                             
Repository 'Packman Repository' is up to date.                                                                                                                                                                                               
Retrieving repository 'mozilla' metadata -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------[/]Segmentation fault (core dumped)

Core file is attached.
Comment 1 Michael Andres 2017-10-02 08:59:04 UTC
Please attach the zypper logfile /var/log/zypper.log (or an older and compressed /var/log/zypper.log-YYYYMMDD.xz) that shows the reported behavior. To see the execution dates and zypper commands included in a logfile you can install the zypper-log package (available since zypper-1.6.11) and execute:

    zypper-log 

To extract the log of a specific command use 

   zypper-log  pid

See the manual page zypper-log(8) for details on how to read older (rotated) zypper-log files.
Comment 2 Richard Weinberger 2017-10-02 09:16:06 UTC
Created attachment 742705 [details]
zypper log file
Comment 3 Michael Andres 2017-10-02 09:49:24 UTC
@Richard: If the crash is reproducible, please execute (as root)
> ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla
(or any other zypper command that crashes) and attach the logfile /tmp/zypp.1061208.log. 

As the crash happens inside libcurl (libssl/libcrypto) it would be interesting to now if
> curl https://download.opensuse.org/repositories/mozilla/openSUSE_Leap_42.3/repodata/repomd.xml
succeeds?
Comment 4 Richard Weinberger 2017-10-02 10:50:18 UTC
(In reply to Michael Andres from comment #3)
> @Richard: If the crash is reproducible, please execute (as root)
> > ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla
> (or any other zypper command that crashes) and attach the logfile
> /tmp/zypp.1061208.log. 
> 
> As the crash happens inside libcurl (libssl/libcrypto) it would be
> interesting to now if
> > curl https://download.opensuse.org/repositories/mozilla/openSUSE_Leap_42.3/repodata/repomd.xml
> succeeds?

Yes, this succeeds. I tried that already yesterday before filing this bug against zypper.
Comment 5 Michael Andres 2017-10-02 11:11:36 UTC
(In reply to Michael Andres from comment #3)
> @Richard: If the crash is reproducible, please execute (as root)
> > ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla
> (or any other zypper command that crashes) and attach the logfile
> /tmp/zypp.1061208.log. 

Are you able to such a log? As I can't reproduce this, a log with CURL debug output could reveal some ore details.
Comment 6 Richard Weinberger 2017-10-02 11:20:09 UTC
(In reply to Michael Andres from comment #5)
> (In reply to Michael Andres from comment #3)
> > @Richard: If the crash is reproducible, please execute (as root)
> > > ZYPP_LOGFILE=/tmp/zypp.1061208.log ZYPP_MEDIA_CURL_DEBUG=2 zypper ref -f mozilla
> > (or any other zypper command that crashes) and attach the logfile
> > /tmp/zypp.1061208.log. 
> 
> Are you able to such a log? As I can't reproduce this, a log with CURL debug
> output could reveal some ore details.

Sorry, I missed that question. Log is attached.
I can reproduce, also after reboots (thought about a memory corruption at first).
Comment 7 Richard Weinberger 2017-10-02 11:21:51 UTC
Created attachment 742726 [details]
zypper debug log
Comment 8 Michael Andres 2017-10-02 11:58:53 UTC
> MediaCurl.cc(doGetDoesFileExist):1195 URL: https://download.opensuse.org/repositories/mozilla/openSUSE_Leap_42.3/repodata/repomd.xml
> MediaCurl.cc(log_curl):111 *   Trying 195.135.221.134...
> MediaCurl.cc(log_curl):111 * TCP_NODELAY set
> MediaCurl.cc(log_curl):111 * Connected to download.opensuse.org (195.135.221.134) port 443 (#0)
> ZYppFactory.cc(sigsegvHandler):55 Error: signal 11
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (0) /lib64/libc.so.6 : +0x90e5a [0x7f873f0c6e5a]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (1) /lib64/libcrypto.so.1.0.0 : +0x120869 [0x7f873e067869]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (2) /lib64/libcrypto.so.1.0.0 : lh_insert+0x42 [0x7f873e067b42]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (3) /lib64/libcrypto.so.1.0.0 : OBJ_NAME_add+0x6f [0x7f873dfb7eef]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (4) /usr/lib64/libssl.so.1.1 : +0x2beb4 [0x7f873b3c7eb4]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (5) /lib64/libpthread.so.0 : +0x6c13 [0x7f873beb4c13]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (6) /usr/lib64/libcrypto.so.1.1 : CRYPTO_THREAD_run_once+0x9 [0x7f873b0c6c49]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (7) /usr/lib64/libssl.so.1.1 : OPENSSL_init_ssl+0x73 [0x7f873b3c7fc3]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (8) /usr/lib64/libssl.so.1.1 : SSL_CTX_new+0x20 [0x7f873b3cb040]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (9) /usr/lib64/libcurl.so.4 : +0x5c531 [0x7f873e83f531]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (10) /usr/lib64/libcurl.so.4 : +0x5f833 [0x7f873e842833]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (11) /usr/lib64/libcurl.so.4 : +0x60bcb [0x7f873e843bcb]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (12) /usr/lib64/libcurl.so.4 : +0x14562 [0x7f873e7f7562]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (13) /usr/lib64/libcurl.so.4 : +0x26341 [0x7f873e809341]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (14) /usr/lib64/libcurl.so.4 : +0x3a53e [0x7f873e81d53e]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (15) /usr/lib64/libcurl.so.4 : curl_multi_perform+0x81 [0x7f873e81e1c1]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (16) /usr/lib64/libcurl.so.4 : curl_easy_perform+0x102 [0x7f873e814ca2]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (17) /usr/lib64/libzypp.so.1600 : zypp::media::MediaCurl::doGetDoesFileExist(zypp::filesystem::Pathname const&)
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (18) /usr/lib64/libzypp.so.1600 : zypp::media::MediaCurl::getDoesFileExist(zypp::filesystem::Pathname const&) c
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (19) /usr/lib64/libzypp.so.1600 : zypp::media::MediaHandler::doesFileExist(zypp::filesystem::Pathname const&) c
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (20) /usr/lib64/libzypp.so.1600 : zypp::media::MediaAccess::doesFileExist(zypp::filesystem::Pathname const&) co
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (21) /usr/lib64/libzypp.so.1600 : zypp::media::MediaManager::doesFileExist(unsigned int, zypp::filesystem::Path
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (22) /usr/lib64/libzypp.so.1600 : +0x29d61e [0x7f874041661e]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (23) /usr/lib64/libzypp.so.1600 : zypp::MediaSetAccess::provide(boost::function<void (unsigned int, zypp::files
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (24) /usr/lib64/libzypp.so.1600 : zypp::MediaSetAccess::doesFileExist(zypp::filesystem::Pathname const&, unsign
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (25) /usr/lib64/libzypp.so.1600 : zypp::RepoManager::Impl::probe(zypp::Url const&, zypp::filesystem::Pathname c
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (26) /usr/lib64/libzypp.so.1600 : zypp::RepoManager::Impl::refreshMetadata(zypp::RepoInfo const&, zypp::RepoMan
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (27) zypper() [0x4abf53]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (28) zypper : refresh_repo(Zypper&, zypp::RepoInfo const&)+0x247 [0x4af607]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (29) zypper : refresh_repos(Zypper&)+0x6d3 [0x4aff03]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (30) zypper : Zypper::doCommand()+0x8edb [0x475cab]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (31) zypper : Zypper::safeDoCommand()+0x7d [0x48308d]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (32) zypper : Zypper::main(int, char**)+0x44 [0x43e894]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (33) zypper : main+0x3ad [0x43dabd]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (34) /lib64/libc.so.6 : __libc_start_main+0xf5 [0x7f873f0566e5]
> ZYppFactory.cc(sigsegvHandler):55 [bt]: (35) zypper : _start+0x29 [0x43f8e9]

@Marcus It seems to crash pretty early during OPENSSL_init_ssl. Not even CApath etc. is reported. Any idea what could cause this?
Comment 9 Marcus Meissner 2017-10-02 12:52:34 UTC
i think seperate versions of openssl are mixed together.

rpm -qif /lib64/libcrypto.so.1.0.0
rpm -qif /usr/lib64/libssl.so.1.1
rpm -qif /usr/lib64/libcrypto.so.1.1

(There should only be 1 libcrypto.so.1 loaded, so there are two installed ... one is from Leap, the other one is from somewhere else)
Comment 10 Richard Weinberger 2017-10-02 13:12:48 UTC
(In reply to Marcus Meissner from comment #9)
> i think seperate versions of openssl are mixed together.
> 
> rpm -qif /lib64/libcrypto.so.1.0.0
> rpm -qif /usr/lib64/libssl.so.1.1
> rpm -qif /usr/lib64/libcrypto.so.1.1
> 
> (There should only be 1 libcrypto.so.1 loaded, so there are two installed
> ... one is from Leap, the other one is from somewhere else)

You are right, one version came from repositories/security:/tls/openSUSE_Leap_42.3/.
This repo is present in my zypper config since I needed a package from there.
dup --from security_tls fixed the issue.

Feel free to close as invalid. ;-\
Comment 11 Michael Andres 2017-10-02 13:27:39 UTC
.
Comment 12 Patrick Finie 2019-09-19 06:45:13 UTC
Created attachment 818810 [details]
zypper log
Comment 13 Patrick Finie 2019-09-19 06:49:50 UTC
Created attachment 818811 [details]
Dmesg when executing zypper

(In reply to Michael Andres from comment #11)
> .

Actualy i think i need to put in my three fifty here. 

I have been having a situation upgrading Leap 15.0 to Leap 15.1 using the method zypper up > # cp -Rv /etc/zypp/repos.d /etc/zypp/repos.d.Old  ># sed -i 's/15.0/15.1/g' /etc/zypp/repos.d/* >  # zypper --gpg-auto-import-keys ref (SEGFAULT)

so i go and force the refresh and the repos refresh fine.  errors are identical to the above.

Then i go to do a dup and during the download i get the signal 11 error also mentioned above.

So on one machine i forced the upgrade using a brute force script seen here : https://forums.opensuse.org/showthread.php/535230-zypper-segfaulting/page5

This worked in upgrading but now leads to 15.1 having the same issue as above.

Now to a different and third machine,  My server.

instead of doing a update before changing over the repositories i did this method.  
# cp -Rv /etc/zypp/repos.d /etc/zypp/repos.d.Old  ># sed -i 's/15.0/15.1/g' /etc/zypp/repos.d/* >  # zypper --gpg-auto-import-keys ref > zypper dup

dealt with all the package that needed to be changed over and rebooted.  

went to use zypper and segfault.

went to check if i needed to update some packages after just having executed the zypper dup.... 950 megabytes needed updating.

so i go and take a look at this post and what packages are in the list to update and force a few to update including the entire yast stack as going into yast to see what is up with the Openssl as i have 1.0.0 and 1.1.? from the main OSS repo.

Yast crashes saying i am missing a package (hence why i did the second pass of zypper dup and found the 950 mb worth of packages.)

So i start installing the yast package one by one when i run across this.


zypper in yast2-pkg-bindings yast2-x11
Loading repository data...
Reading installed packages...
Resolving package dependencies...

Problem: yast2-pkg-bindings-4.1.2-lp151.1.1.x86_64 requires libzypp.so.1709()(64bit), but this requirement cannot be provided
  not installable providers: libzypp-17.11.4-lp151.1.1.x86_64[download.opensuse.org-oss]
                   libzypp-17.11.4-lp151.1.1.x86_64[repo-oss]
 Solution 1: Following actions will be done:
  downgrade of libzypp-17.12.0-lp150.2.13.1.x86_64 to libzypp-17.11.4-lp151.1.1.x86_64
  downgrade of libsolv-tools-0.7.5-lp150.7.1.x86_64 to libsolv-tools-0.7.3-lp151.1.1.x86_64
  deinstallation of libyui-ncurses-pkg8-2.48.5.2-lp150.7.1.x86_64
  deinstallation of libyui-qt-pkg8-2.45.15.2-lp150.7.1.x86_64
  downgrade of zypper-1.14.28-lp150.2.13.1.x86_64 to zypper-1.14.27-lp151.1.2.x86_64
 Solution 2: do not install yast2-pkg-bindings-4.1.2-lp151.1.1.x86_64
 Solution 3: break yast2-pkg-bindings-4.1.2-lp151.1.1.x86_64 by ignoring some of its dependencies

Choose from above solutions by number or cancel [1/2/3/c/d/?] (c): 1
Resolving dependencies...
Resolving package dependencies...

The following NEW package is going to be installed:
  libyui-ncurses-pkg9

The following 2 packages are going to be REMOVED:
  libyui-ncurses-pkg8 libyui-qt-pkg8

The following 2 packages are going to be upgraded:
  yast2-pkg-bindings yast2-x11

The following 3 packages are going to be downgraded:
  libsolv-tools libzypp zypper



So it seems doing a Zypper Dup is not upgrading zypper to the latest 15.1 release as it thinks the older 15.0 one is newer thus is causing all sorts of headaches.  This would explain why when i had the same issue on one of my other machines, doing the upgrade path to fix it via the DVD image resolved this issue.
Comment 14 Patrick Finie 2019-09-19 06:51:04 UTC
Created attachment 818812 [details]
condensed zypper log

Have a copy of the zypper log
Comment 15 Michael Andres 2019-09-19 12:38:58 UTC
Patrick: Thanks. Your issue is different than the one reported here, but it's something that worries us most ATM. It's seems to be a duplicate of bug#1146027, but for now I'd like to continue here.


Let's stay with your server:

(In reply to Patrick Finie from comment #13)
> Now to a different and third machine,  My server.
> 
> instead of doing a update before changing over the repositories i did this
> method.  
> # cp -Rv /etc/zypp/repos.d /etc/zypp/repos.d.Old  ># sed -i 's/15.0/15.1/g'
> /etc/zypp/repos.d/* >  # zypper --gpg-auto-import-keys ref > zypper dup
> 
> dealt with all the package that needed to be changed over and rebooted.
> 
> went to use zypper and segfault.

I'm most interested in the 'zypper dup' before you rebooted: 

- 'zypper dup' creates a resolver testcase at /var/log/updateTestcase-YYYY-MM-DD-hh_mm_ss (the date the dup command was executed). Please pack the output directory and attach it to THIS bugreport.

(In case the testcase exceeds the 10MB limit for bugzilla uploads, you can use 'split' to divide the file into smaller pieces)

- If the zypper.log showing the refresh command and the dup are still available (maybe backuped) please attach them as well.
Comment 16 Patrick Finie 2019-09-19 20:08:41 UTC
(In reply to Michael Andres from comment #15)
> Patrick: Thanks. Your issue is different than the one reported here, but
> it's something that worries us most ATM. It's seems to be a duplicate of
> bug#1146027, but for now I'd like to continue here.
> 
> 
> Let's stay with your server:
> 
> (In reply to Patrick Finie from comment #13)
> > Now to a different and third machine,  My server.
> > 
> > instead of doing a update before changing over the repositories i did this
> > method.  
> > # cp -Rv /etc/zypp/repos.d /etc/zypp/repos.d.Old  ># sed -i 's/15.0/15.1/g'
> > /etc/zypp/repos.d/* >  # zypper --gpg-auto-import-keys ref > zypper dup
> > 
> > dealt with all the package that needed to be changed over and rebooted.
> > 
> > went to use zypper and segfault.
> 
> I'm most interested in the 'zypper dup' before you rebooted: 
> 
> - 'zypper dup' creates a resolver testcase at
> /var/log/updateTestcase-YYYY-MM-DD-hh_mm_ss (the date the dup command was
> executed). Please pack the output directory and attach it to THIS bugreport.
> 
> (In case the testcase exceeds the 10MB limit for bugzilla uploads, you can
> use 'split' to divide the file into smaller pieces)
> 
> - If the zypper.log showing the refresh command and the dup are still
> available (maybe backuped) please attach them as well.


I executed several attempts at zypper dup post reboot that may have overwritten the directory you want.  I was also unsucessful in splitting the oldest one using the compression method i used.  Still a 27mb lzma2 .7z file.

i will try and upload it to a FTP i have access to but alas that has been undergoing some connection issues related to bitmitigate.

Will post a url as soon as i can.

The main issue looks to me that doing the in place zypper dup... well..

libzypp-17.12.0-lp150.2.13.1.x86_64 to libzypp-17.11.4-lp151.1.1.x86_64  take this for example.  zypper is seeing libzypp-17.12.0 is newer than libzyp-17.11.4 and isnt migrating over.  I had similar issues with zypper thinking 42.3 is newer than 15.0 when migrating from 42.3 to 15.0


 same goes with libsolv-tools-0.7.5-lp150.7.1.x86_64 v libsolv-tools-0.7.3-lp151.1.1.x86_64.


Not sure why on the other machines doing an update on 15.0 to the latest in the 15.0 repos then migrating to 15.1 would cause the segfault issue.
Comment 17 Michael Andres 2019-09-23 09:49:30 UTC
(In reply to Patrick Finie from comment #16)
> The main issue looks to me that doing the in place zypper dup... well..
> 
> libzypp-17.12.0-lp150.2.13.1.x86_64 to libzypp-17.11.4-lp151.1.1.x86_64 
> take this for example.  zypper is seeing libzypp-17.12.0 is newer than
> libzyp-17.11.4 and isnt migrating over.  I had similar issues with zypper
> thinking 42.3 is newer than 15.0 when migrating from 42.3 to 15.0

This should not happen unless you have the 'lp150' repositories enabled during "dup" which is not recommended:


"dup"s job is to arrange all installed packages to originate from one of the enabled repositories. The result should be similar to a fresh installation from these repositories.

So this is not a kind of 'update to newer package versions'. It's more like translating the systems content defined by the currently installed packages into its counterpart within the new repositories.

That's why 'dup' is by default allowed to downgrade, change architecture (opt. to change vendor) - everything that's needed to apply the new package set. All based on the assumption, that the new set of repos ships a consistent set of packages.

Installed packages which have no counterpart within the new repositories are considered to be orphaned (or dropped in the new distribution). They may be removed in order to perform the "dup" job.


So if you keep the old 'lp150' repos enabled you basically say that it's ok to keep some old packages installed.
Comment 18 Patrick Finie 2019-09-23 18:45:29 UTC
(In reply to Michael Andres from comment #17)
> (In reply to Patrick Finie from comment #16)
> > The main issue looks to me that doing the in place zypper dup... well..
> > 
> > libzypp-17.12.0-lp150.2.13.1.x86_64 to libzypp-17.11.4-lp151.1.1.x86_64 
> > take this for example.  zypper is seeing libzypp-17.12.0 is newer than
> > libzyp-17.11.4 and isnt migrating over.  I had similar issues with zypper
> > thinking 42.3 is newer than 15.0 when migrating from 42.3 to 15.0
> 
> This should not happen unless you have the 'lp150' repositories enabled
> during "dup" which is not recommended:
> 
> 
> "dup"s job is to arrange all installed packages to originate from one of the
> enabled repositories. The result should be similar to a fresh installation
> from these repositories.
> 
> So this is not a kind of 'update to newer package versions'. It's more like
> translating the systems content defined by the currently installed packages
> into its counterpart within the new repositories.
> 
> That's why 'dup' is by default allowed to downgrade, change architecture
> (opt. to change vendor) - everything that's needed to apply the new package
> set. All based on the assumption, that the new set of repos ships a
> consistent set of packages.
> 
> Installed packages which have no counterpart within the new repositories are
> considered to be orphaned (or dropped in the new distribution). They may be
> removed in order to perform the "dup" job.
> 
> 
> So if you keep the old 'lp150' repos enabled you basically say that it's ok
> to keep some old packages installed.

Using the SED command to change the 15.0 repos to 15.1 mitigates any possibility of there being a 15.0 repo unless it was forced otherwise.  

In the case of the first two machines the segfault happened after updating 15.0 > sed command to change from 15.0 repos to 15.1  >  zypper ref.

The latter for the zypper ref is because in the past i have had some mission critcal repos (Oracle Virtualbox for example) not change their naming over.  So the refresh allows me to know which ones i need to deal with.

The issue here i see is something i had when migrating from 42.3 to 15.0.  If the lp42.3 is in front of the version number zypper will think 42.3 is bigger than 15.0 and thus it must be newer.

In this instance the version of libzypp is a higher number in 15.0 than the 15.1 which may be where the zypper dup issue stems.
Comment 19 Michael Andres 2019-09-24 12:20:15 UTC
(In reply to Patrick Finie from comment #18)
> In this instance the version of libzypp is a higher number in 15.0 than the
> 15.1 which may be where the zypper dup issue stems.

As I tried to explain: 'dup' will not look at the installed packages version. 

The 'dup' job requires an installed libzypp to be replaced by a version provided by the enabled repos. If this can not be done, a conflict is raised. 

(You want the packages to come from the new distro, even if the distro decided to downgrade some package)


If your repos provided lp15.1 and no lp15.0 packages, a 15.0 libzypp should not have stayed in the system (unless no lp15.1 provided a libzypp).

As it nevertheless happened, the update testcase could have revealed why. It contains enough information about the repos and systems content to be able to replay 'dup's decisions.



! As there is a small chance that the old libzypp was seen due to a broken rpm database index, you should run 'rpm --rebuilddb' (as root). Just to make sure your database is ok.
Comment 20 Michael Andres 2019-09-26 13:36:30 UTC
Patrick:
The SEGV backtrace visible in your logs is most probably caused bug#1146027. This issue will be fixed in libzypp-17.14.1. I'll close the bug again, restoring the original INVALID resolution.

If you again experience an incomplete dup, please open a bug for it and attach the zypper.log and the updateTestcase (see comment #15). Thanks.
Comment 21 Patrick Finie 2019-11-07 02:32:13 UTC
Created attachment 823558 [details]
updatetestcase WSL

(In reply to Michael Andres from comment #20)
> Patrick:
> The SEGV backtrace visible in your logs is most probably caused bug#1146027.
> This issue will be fixed in libzypp-17.14.1. I'll close the bug again,
> restoring the original INVALID resolution.
> 
> If you again experience an incomplete dup, please open a bug for it and
> attach the zypper.log and the updateTestcase (see comment #15). Thanks.

I did a zypper dup on a WSL instance and ran into the same issue with 15.0 and 15.1.  What is the best means of uploading the 25 megabyte updatetestcase directory?  Seems the option is greyed out on most of the archive managers i use.

*several hours later with a lot of yelling and swearing and preying to Geeko*

Looks like it got it.
Comment 22 Patrick Finie 2019-11-07 02:33:17 UTC
Created attachment 823559 [details]
updatetestcase WSL 2

here is the second one.
Comment 23 Patrick Finie 2019-11-07 02:34:11 UTC
Created attachment 823560 [details]
updatetestcase WSL 3

and the third.
Comment 24 Patrick Finie 2019-11-07 02:34:23 UTC
Reopening.
Comment 25 Michael Andres 2019-11-07 14:36:13 UTC
(In reply to Patrick Finie from comment #21)
> updatetestcase directory?  Seems the option is greyed out on most of the
> archive managers i use.

You mean some split-archive option?

There's a generic 'split' command. If you have some `big.tar.bz2`, then
>  split -b 10M --additional-suffix=-big.tar.bz2 big.tar.bz2
will split it into 10MB parts named
>  xaa-big.tar.bz2
>  xab-big.tar.bz2
>  xac-big.tar.bz2
>  ...

Later 'cat' can be used to restore the original file
> cat x??-big.tar.bz2 >big.tar.bz2
Comment 26 Michael Andres 2019-11-07 15:37:16 UTC
May it be that updateTestcase-2019-11-07-01-46-39 is _not_ from the zypper dup, that brought the system into the inconsistent state? 


In the testcase I see 348 packages being installed, but just 28 are lp150. The rest is already lp151. 

   libsolv-tools-0.7.5-lp150.7.1.x86_64
   libzypp-17.12.0-lp150.2.13.1.x86_64
   zypper-1.14.28-lp150.2.13.1.x86_64

The zypp stack is still lp150, but running the dup, the resolver would try to update them as expected:

>!> install libsolv-tools-0.7.6-lp151.2.3.2.x86_64[repo-update]
>!> install libzypp-17.15.0-lp151.2.3.2.x86_64[repo-update]
>!> install zypper-1.14.30-lp151.2.3.1.x86_64[repo-update]


If you install `zypper-log`, the command will print you a summary of the commands found in a zypper.log file (or a rotated one, see the man page). This may help to figure out what was executed before 2019-11-07-01-46-39.
Comment 27 Patrick Finie 2019-12-21 05:55:52 UTC
(In reply to Michael Andres from comment #26)
> May it be that updateTestcase-2019-11-07-01-46-39 is _not_ from the zypper
> dup, that brought the system into the inconsistent state? 
> 
> 
> In the testcase I see 348 packages being installed, but just 28 are lp150.
> The rest is already lp151. 
> 
>    libsolv-tools-0.7.5-lp150.7.1.x86_64
>    libzypp-17.12.0-lp150.2.13.1.x86_64
>    zypper-1.14.28-lp150.2.13.1.x86_64
> 
> The zypp stack is still lp150, but running the dup, the resolver would try
> to update them as expected:
> 
> >!> install libsolv-tools-0.7.6-lp151.2.3.2.x86_64[repo-update]
> >!> install libzypp-17.15.0-lp151.2.3.2.x86_64[repo-update]
> >!> install zypper-1.14.30-lp151.2.3.1.x86_64[repo-update]
> 
> 
> If you install `zypper-log`, the command will print you a summary of the
> commands found in a zypper.log file (or a rotated one, see the man page).
> This may help to figure out what was executed before 2019-11-07-01-46-39.



Testing with my last 15.0 system (i think).  Doing a dup without making sure the 15.0 packages are up to date.  If it rears it's ugly head there i will do that.
Comment 28 Patrick Finie 2020-07-19 20:02:24 UTC
There are now better workaround out.  Going to 15.2 doesnt produce this issue, if there is a zypper issue then dnf is a backup.  Detailed a procedure that eliminates most of the risk involved with this old bug and thus it is being closed.  Never was resolved but not enough information was produced and by the time responses where made the systems where already brute forced to upgrade.