Bug 328486

glibc/glibc-2.3.3-amd64-string.diff

Index: sysdeps/x86_64/bcopy.S
===================================================================
--- /dev/null
+++ sysdeps/x86_64/bcopy.S
@@ -0,0 +1,13 @@
+# $Header: /K8_Projects/Glibc/amd64bcopy.S 2     2/10/04 11:48 Emenezes $
+
+# (c) 2002 Advanced Micro Devices, Inc.
+# YOUR USE OF THIS CODE IS SUBJECT TO THE TERMS
+# AND CONDITIONS OF THE GNU LESSER GENERAL PUBLIC
+# LICENSE FOUND IN THE "README" FILE THAT IS
+# INCLUDED WITH THIS FILE
+
+#define USE_AS_BCOPY
+#define memcpy bcopy
+
+#include <sysdeps/x86_64/memcpy.S>
+

but memcpy.S doesn't seem to care about USE_AS_BCOPY (and memcpy and bcopy
have src/dest arguments swapped).

Perhaps we should check this isn't also on older distributions a problem ;(

I don't think it is, the broken patch is only in 10.3. 

either this one:

+Tue Aug 21 22:31:54 CEST 2007 - pbaudis@suse.cz
+
+- Reintroduce errorneously omitted AMD64 optimizations
+  (and fix them to compile again)
+


or this one:

+-------------------------------------------------------------------
+Fri Jul 13 18:27:49 CEST 2007 - schwab@suse.de
+
+- Update to head of glibc-2.6 branch.


or this one:

+Fri Aug  3 15:46:35 CEST 2007 - pbaudis@suse.cz
+
+- Remove the dynamic linker support for direct bindings (-Bdirect);
+  according to Michael Meeks, the performance improvement is too small
+- Split timezone package to a separate physical package
+- Update glibc to version 2.6.1


broke it

The problem is, that the partial patch which got accepted upstream
didn't include the bcopy parts, hence the wrapper can't work anymore.

As bcopy is an obsolete function I think it's easiest to not include
a fast implementation of it at all.  It's also wrong to implement bcopy
in terms of memcpy, as bcopy allows overlapping buffers, unlike memcpy.
If at all it would have to use memmove.  I'll do this.

Fixed package submitted to STABLE and 10.3 (for online update).

This update breaks 32bit-apps on x86_64. :(

After update, i couldn't start opera, seamonky (32bit) and moneyplex anymore. All apps die with "Segmentation fault".

Downgrade of glibc-* and all 32bit-apps works fine.

Please give here the output of 

% rpm -qa 'glibc*'

Once when programs crash, once when they work.

It works with "dvd-glibc":

rpm -qa glibc*
glibc-locale-2.6.1-18
glibc-i18ndata-2.6.1-18
glibc-locale-32bit-2.6.1-18
glibc-html-2.6.1-18
glibc-profile-2.6.1-18
glibc-32bit-2.6.1-18
glibc-devel-32bit-2.6.1-18
glibc-obsolete-2.6.1-18
glibc-2.6.1-18
glibc-info-2.6.1-18
glibc-devel-2.6.1-18
glibc-profile-32bit-2.6.1-18

After update:

rpm -qa "glibc*"
glibc-info-2.6.1-18.2
glibc-obsolete-2.6.1-18.2
glibc-2.6.1-18.2
glibc-devel-32bit-2.6.1-18.2
glibc-html-2.6.1-18.2
glibc-locale-32bit-2.6.1-18.2
glibc-locale-2.6.1-18.2
glibc-i18ndata-2.6.1-18.2
glibc-profile-32bit-2.6.1-18.2
glibc-profile-2.6.1-18.2
glibc-32bit-2.6.1-18.2
glibc-devel-2.6.1-18.2

Crash :(

rpm -Uvh --force glibc-* in dvddir it works again

Try to get a backtrace from a crashing program then, please.  Perhaps
something easier than opera or moneyplex crash too?  The problem is, that
we can't reproduce this here.  At least on one x86_64 machine here opera (the
32bit app) runs just fine.  With that very glibc:

% rpm -qa 'glibc*'
glibc-locale-2.6.1-18.2
glibc-info-2.6.1-18.2
glibc-32bit-2.6.1-18.2
glibc-locale-32bit-2.6.1-18.2
glibc-i18ndata-2.6.1-18.2
glibc-2.6.1-18.2
glibc-devel-2.6.1-18.2
glibc-devel-32bit-2.6.1-18.2

% file /usr/lib/opera/9.23-20070809.6/opera
/usr/lib/opera/9.23-20070809.6/opera: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.1, dynamically linked (uses shared libs), stripped
% ldd /usr/lib/opera/9.23-20070809.6/opera
 linux-gate.so.1 =>  (0xffffe000)
 libqt-mt.so.3 => /usr/lib/libqt-mt.so.3 (0xf7872000)
 libX11.so.6 => /usr/lib/libX11.so.6 (0xf7757000)
 libXext.so.6 => /usr/lib/libXext.so.6 (0xf7748000)
 libSM.so.6 => /usr/lib/libSM.so.6 (0xf773f000)
 libICE.so.6 => /usr/lib/libICE.so.6 (0xf7726000)
 libpthread.so.0 => /lib/libpthread.so.0 (0xf7710000)
 libdl.so.2 => /lib/libdl.so.2 (0xf770b000)
 libm.so.6 => /lib/libm.so.6 (0xf76e6000)
 libz.so.1 => /lib/libz.so.1 (0xf76d3000)
 libc.so.6 => /lib/libc.so.6 (0xf75a0000)
 libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xf74b4000)
 libpng12.so.0 => /usr/lib/libpng12.so.0 (0xf748f000)
 libXi.so.6 => /usr/lib/libXi.so.6 (0xf7485000)
 libXrender.so.1 => /usr/lib/libXrender.so.1 (0xf747c000)
 libXrandr.so.2 => /usr/lib/libXrandr.so.2 (0xf7475000)
 libXcursor.so.1 => /usr/lib/libXcursor.so.1 (0xf746b000)
 libXinerama.so.1 => /usr/lib/libXinerama.so.1 (0xf7467000)
 libXft.so.2 => /usr/lib/libXft.so.2 (0xf7454000)
 libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0xf73e4000)
 libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0xf73b8000)
 libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xf73aa000)
 libxcb-xlib.so.0 => /usr/lib/libxcb-xlib.so.0 (0xf73a7000)
 libxcb.so.1 => /usr/lib/libxcb.so.1 (0xf738e000)
 libXau.so.6 => /usr/lib/libXau.so.6 (0xf7389000)
 /lib/ld-linux.so.2 (0xf7f92000)
 libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0xf7383000)
 libexpat.so.1 => /lib/libexpat.so.1 (0xf7362000)

Oh, and another idea, please provide the output of 
% rpm -Va 'glibc*'

To rule out any breakage of the update as you see it.

# rpm -Va 'glibc*'
.......T    /usr/lib64/gconv/gconv-modules.cache

hmmm

Created attachment 176821 [details]
backstrace of gamix (32bit)

Created attachment 176822 [details]
strace of gamix

i've testet to little apps:

viki = works
gamix = crash, see attached files

Okay, we still can't reproduce, also not with gamix.  Please try to downgrade
only (!) glibc-32bit (maybe you need --force --nodeps).  The backtrace
is a mystery and actually just shows it breaking in the dynamic linker :-/

done:

rpm -Uvh --force --nodeps glibc-32bit-2.6.1-18.x86_64.rpm

rpm -qa "glibc*"
glibc-devel-2.6.1-18.2
glibc-32bit-2.6.1-18
glibc-2.6.1-18.2
glibc-devel-32bit-2.6.1-18.2
glibc-locale-32bit-2.6.1-18.2
glibc-i18ndata-2.6.1-18.2
glibc-info-2.6.1-18.2
glibc-locale-2.6.1-18.2

and gamix, opera etc works again...

Now, i'll try a fresh install without other repos, only suse-dvd and update on another partition.

ok, fresh install finished:

- install opera -> works
- add update repository
- YOU
- opera crash

:(

test@detlef-pc:~> rpm -qa --last | head -n 40
amarok-xine-1.4.7-37.2                        Di 09 Okt 2007 00:26:54 CEST
amarok-1.4.7-37.2                             Di 09 Okt 2007 00:26:51 CEST
fvwm2-2.5.23-1.1                              Di 09 Okt 2007 00:26:48 CEST
amarok-yauap-1.4.7-37.2                       Di 09 Okt 2007 00:26:46 CEST
nscd-2.6.1-18.2                               Di 09 Okt 2007 00:26:45 CEST
cpio-2.9-17.2                                 Di 09 Okt 2007 00:26:45 CEST
ksh-93s-48.2                                  Di 09 Okt 2007 00:26:44 CEST
wvdial-1.56-22.2                              Di 09 Okt 2007 00:26:43 CEST
glibc-locale-32bit-2.6.1-18.2                 Di 09 Okt 2007 00:26:43 CEST
gtk2-32bit-2.12.0-5.2                         Di 09 Okt 2007 00:26:42 CEST
gtk2-2.12.0-5.2                               Di 09 Okt 2007 00:26:40 CEST
amarok-libvisual-1.4.7-37.2                   Di 09 Okt 2007 00:26:39 CEST
yauap-0.2.1-21.2                              Di 09 Okt 2007 00:26:37 CEST
glibc-locale-2.6.1-18.2                       Di 09 Okt 2007 00:26:33 CEST
release-notes-10.3.17-0.1                     Di 09 Okt 2007 00:26:30 CEST
glibc-2.6.1-18.2                              Di 09 Okt 2007 00:26:29 CEST
glibc-32bit-2.6.1-18.2                        Di 09 Okt 2007 00:26:27 CEST
glibc-i18ndata-2.6.1-18.2                     Di 09 Okt 2007 00:26:24 CEST
opera-9.23-15                                 Di 09 Okt 2007 00:04:01 CEST
yast2-repair-2.15.7-59                        Mo 08 Okt 2007 23:53:31 CEST
yast2-backup-2.15.4-50                        Mo 08 Okt 2007 23:53:31 CEST
yast2-add-on-2.15.17-4                        Mo 08 Okt 2007 23:53:30 CEST
autoyast2-2.15.17-17                          Mo 08 Okt 2007 23:53:29 CEST
yast2-storage-2.15.27-4                       Mo 08 Okt 2007 23:53:28 CEST
ufraw-gimp-0.12-32                            Mo 08 Okt 2007 23:53:28 CEST
yast2-online-update-frontend-2.15.23-9        Mo 08 Okt 2007 23:53:27 CEST
gimp-help-0.9-142                             Mo 08 Okt 2007 23:53:26 CEST
autoyast2-installation-2.15.17-17             Mo 08 Okt 2007 23:53:18 CEST
yast2-update-2.15.23-21                       Mo 08 Okt 2007 23:53:17 CEST
yast2-profile-manager-2.15.0-52               Mo 08 Okt 2007 23:53:16 CEST
gimp-2.2.17-40                                Mo 08 Okt 2007 23:53:15 CEST
compiz-0.5.4-27                               Mo 08 Okt 2007 23:53:11 CEST
yast2-sudo-2.15.3-86                          Mo 08 Okt 2007 23:53:10 CEST
yast2-samba-server-2.15.7-57                  Mo 08 Okt 2007 23:53:10 CEST
yast2-registration-2.15.3-15                  Mo 08 Okt 2007 23:53:09 CEST
yast2-online-update-2.15.23-9                 Mo 08 Okt 2007 23:53:09 CEST
yast2-metapackage-handler-0.7.1-9             Mo 08 Okt 2007 23:53:08 CEST
yast2-mail-2.15.23-2                          Mo 08 Okt 2007 23:53:08 CEST
yast2-iscsi-client-2.15.2-39                  Mo 08 Okt 2007 23:53:07 CEST
yast2-inetd-2.15.1-41                         Mo 08 Okt 2007 23:53:07 CEST

*** Bug 332438 has been marked as a duplicate of this bug. ***

Ok, one idea:

I use a "very old" amd64, one of the first on 775 sockel.

What did you use, Andreas?

If we find a bug in oldstyle amd64, we get a gift! ;)

Interesting.  We noticed that the 10.3 DVD contains glibc-32bit built
from the i686 tree, whereas the glibc-32bit on the update comes from the
i386 tree (!).  That means they do contain different code, although of course
all amd64 processors should work with i586 code just fine, the same like
with i686 code.  But never say never, so it would be very interesting to not
only get the backtrace of a segfault (which wasn't that usefull after all),
but also the output of the following gdb commands when the segfault occurs:

(gdb) p $pc
(gdb) disassemble $pc $pc+48

I've also uploaded a glibc-32bit.rpm which comes from the i686 tree, from
the same sources as the update.  Please try that.  It's also build -18.2,
so you can only detect the difference with rpm -qi (the Distribution: tag
includes "i686").

So, please try the one from
  ftp://ftp.suse.de/pub/people/matz/bug328486/glibc-32bit.rpm

ok, wait for weekend, i install a fresh 10.3 on a new partition.
I'am very busy now....

Oder auf deutsch:
Ich habe im Moment zu viel auf der Liste.... :(
Werde aber am Wochenende nen paar Stunden frei machen, um das noch einmal mit einer frischen Installation durch zu spielen.

(In reply to comment #22 from Detlef Reichelt)

Well, all my three 64bit systems which now have 10.3 installed show the
same problem

 - Intel(R) Core(TM)2 Duo CPU T7300  @ 2.00GHz, Laptop
 - AMD Athlon(tm) 64 X2 Dual Core Processor 3800+, Desktop, Socket 939
 - AMD Athlon(tm) 64 X2 Dual Core Processor 4600+, Desktop, Socket AM2?

so I doubt it is processor specific. The only things common are 
that all systems are dual cores and have 2GB of memory.



(In reply to comment #23 from Michael Matz)

The i686 version (from suse.com, suse.de does not have the file) works
(tested on the T7300 and 4600+). I would guess it is a compiler code
generation problem. Both my problematic 32bit binaries are started
from extensive wrapper scripts so I am not sure how to provide the
gdb output.

(gdb) run
Starting program: /usr/bin/gamix
warning: Lowest section in system-supplied DSO at 0xffffe000 is .hash at ffffe0b4
[Thread debugging using libthread_db enabled]
[New Thread 0xf746da90 (LWP 25236)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf746da90 (LWP 25236)]
0x00000006 in ?? ()
(gdb) p $pc
$1 = (void (*)()) 0x6
(gdb) disassemble $pc $pc+48
Dump of assembler code from 0x6 to 0x36:
0x00000006:     Cannot access memory at address 0x6
(gdb)  

Your glibc-32bit.rpm works for me!

I find out, why the bug hits me (bwalle, thx for hint;) :

If i use my local mirror of the update-repo, the rpms are installed and i get this bug.

If i use an online update repo, YOU use the delta rpms, and all works fine.

(In reply to comment #27 from Detlef Reichelt)
> 
> (bwalle, thx for hint;) :

You mis-understood me. The RPM from Micha is built differently (i686 vs. i386) and that's why it works.

Re comment #26: Blaeh, right, that was already visible from the backtrace.
Someone is jumping to the wildly invalid address 0x00000006, but of course
we can't really determine where that came from :-(  Sigh.  We really need
a way for reproducing this internally I fear.

(In reply to comment #28 from Bernhard Walle)
> (In reply to comment #27 from Detlef Reichelt)
> > 
> > (bwalle, thx for hint;) :
> 
> You mis-understood me. The RPM from Micha is built differently (i686 vs. i386)
> and that's why it works.
> 

Okt 11 21:22:02 <bwalle>	also damit kein patch-rpm genommen wird

Thats what i mean, if i use the patch-rpms, it works! ;)

(In reply to comment #30 from Detlef Reichelt)
> 
> Thats what i mean, if i use the patch-rpms, it works! ;)

Hm -- really? Here, it also works when I install the full RPM.

Okay, another data point.  Please give the output of this:

% md5sum /lib/libc-2.6.1.so /lib/ld-2.6.1.so

With the broken libs of course.

From still broken system:
  75853b6c86ef1007f80e5b764b9ce50d  lib/libc-2.6.1.so
  c539d7186f4950c5841ddf96ab2d31c5  lib/ld-2.6.1.so

Okay, those are the correct ones, i.e. it's no misapplied delta-rpm or the
like.

Pfff, I don't know what to do, clearly the i586 libc somehow doesn't work
for you, but it's impossible to get nearer to the problem without reproducing
it.  Do you perhaps have other applications which also crash?  For now
we have opera, seamonkey, moneyplex and gamix.

Or would it be possible to get ssh access to a machine when the broken glibc
is installed (no root access required)?

(In reply to comment #34 from Michael Matz)
> Or would it be possible to get ssh access to a machine when the broken glibc
> is installed (no root access required)?

wait some minutes, you will get mail... ;)

mail for matz@suse.de

I have just tried reverting a system to the broken state (it 
was broken initially, than fixed by installing the i686 glibc
version from comment #23 and than tried to brake it again by
installing the broken (i586) glibc with 
"rpm -Uv --force glibc-32bit-2.6.1-18.2.x86_64.rpm" 
but it still works.

Further checking by comparing the md5sums of a broken system (which
never had the 686 version installed) and the reverted
system revealed differences in the MD5SUMs!

ld-2.6.1.so: OK
libBrokenLocale-2.6.1.so: OK
libSegFault.so: FAILED
libanl-2.6.1.so: FAILED
libc-2.6.1.so: OK
libcidn-2.6.1.so: FAILED
libcrypt-2.6.1.so: FAILED
libdl-2.6.1.so: FAILED
libkeyutils-1.2.so: OK
libm-2.6.1.so: OK
libmemusage.so: FAILED
libnsl-2.6.1.so: OK
libnss_compat-2.6.1.so: FAILED
libnss_dns-2.6.1.so: FAILED
libnss_files-2.6.1.so: OK
libnss_hesiod-2.6.1.so: FAILED
libnss_nis-2.6.1.so: FAILED
libnss_nisplus-2.6.1.so: FAILED
libpcprofile.so: FAILED
libpthread-2.6.1.so: OK
libresolv-2.6.1.so: OK
librt-2.6.1.so: OK
libthread_db-1.0.so: FAILED
libutil-2.6.1.so: FAILED
md5sum: WARNING: 14 of 24 computed checksums did NOT match


At the moment I think that somehow the automatic update process seems
to install something different than a manual install with rpm.

"rpm -qi glibc-32bit" is identical on both systems. "rpm -Vv glibc-32bit"
also shows no problems on both systems, yet the MD5SUMs don't match.

I will provide the output of these commands in a moment.

"bomba" is the reverted system, diavolo is still broken.
I just noticed that the signature date is different!
All my systems were updated from a local update repository which
I frequently rsync from gwdg.de. All my (originally 3) broken systems
were updated by either you or opensuseupdater-kde from this
local repository.


# diff -u bomba.rpm-qil.txt diavolo.rpm-qil.txt
--- bomba.rpm-qil.txt	2007-10-15 10:00:29.000000000 +0200
+++ diavolo.rpm-qil.txt	2007-10-15 10:12:21.000000000 +0200
@@ -1,10 +1,10 @@
 Name        : glibc-32bit                  Relocations: (not relocatable)
 Version     : 2.6.1                             Vendor: SUSE LINUX Products GmbH, Nuernberg, Germany
 Release     : 18.2                          Build Date: Wed 03 Oct 2007 05:32:42 PM CEST
-Install Date: Mon 15 Oct 2007 08:57:43 AM CEST      Build Host: harvey.suse.de
+Install Date: Wed 10 Oct 2007 08:39:14 AM CEST      Build Host: harvey.suse.de
 Group       : System/Libraries              Source RPM: glibc-2.6.1-18.2.src.rpm
 Size        : 2340775                          License: BSD 3-Clause; GPL v2 or later; LGPL v2 or later
-Signature   : DSA/SHA1, Wed 03 Oct 2007 05:38:46 PM CEST, Key ID a84edae89c800aca
+Signature   : DSA/SHA1, Thu 04 Oct 2007 02:20:48 PM CEST, Key ID a84edae89c800aca
 Packager    : http://bugs.opensuse.org
 URL         : http://www.gnu.org/software/libc/libc.html
 Summary     : Standard Shared Libraries (from the GNU C Library)



bomba: md5sum /lib/*.so
c539d7186f4950c5841ddf96ab2d31c5  /lib/ld-2.6.1.so
9a1cad93620046f34f541656ae84f236  /lib/libBrokenLocale-2.6.1.so
73956b49bbc14d24322acc1ecdfba570  /lib/libSegFault.so
b769a065f2b095cc447a4166512a060d  /lib/libanl-2.6.1.so
75853b6c86ef1007f80e5b764b9ce50d  /lib/libc-2.6.1.so
c2a02baceac4ec1540fc8d60aa2aab70  /lib/libcidn-2.6.1.so
9975e462a514c928df69c30fcb669cb8  /lib/libcrypt-2.6.1.so
20b0c64894a355a68df930f08c330d14  /lib/libdl-2.6.1.so
52ddb0a9af02b89de9aca86ff0ed0f82  /lib/libkeyutils-1.2.so
03407c32b78921f1fd63a866ecfbf146  /lib/libm-2.6.1.so
8a91ce6b791044a1b92c9164e0dce220  /lib/libmemusage.so
40ff2dd9dec707881a48a562aeaff4e0  /lib/libnsl-2.6.1.so
0f5889fd923aa4164a09f37e8e01656c  /lib/libnss_compat-2.6.1.so
e12305b1756285e2452f8030f56e3db1  /lib/libnss_dns-2.6.1.so
2eb73c96bd6e33ac585612f6a2ae4a6a  /lib/libnss_files-2.6.1.so
72aabab44ccdfd9dbea7a131994f308d  /lib/libnss_hesiod-2.6.1.so
89e1f938d0406470093a4ab99d887e0c  /lib/libnss_nis-2.6.1.so
e1e0321f066b0be4a138b95fcd2afba0  /lib/libnss_nisplus-2.6.1.so
4b27f5a9137587786c39bff57206d802  /lib/libpcprofile.so
70d6ae91f6e7c2389b915ac6b819a4cd  /lib/libpthread-2.6.1.so
4f1aeb5870b28812c06375fc4eff031b  /lib/libresolv-2.6.1.so
6b5190ccfb905efdab0de8154d24d119  /lib/librt-2.6.1.so
027f837b7e447478d7dd3322b2086ba6  /lib/libthread_db-1.0.so
973b8fda49ee0cf8198a97d2c6ee3ff3  /lib/libutil-2.6.1.so



diavolo: md5sum /lib/*.so
c539d7186f4950c5841ddf96ab2d31c5  /lib/ld-2.6.1.so
9a1cad93620046f34f541656ae84f236  /lib/libBrokenLocale-2.6.1.so
4397cdccc1acc15e7cc337b6812a9004  /lib/libSegFault.so
b20556e5cbd1cf36c368b3f96e7c9844  /lib/libanl-2.6.1.so
75853b6c86ef1007f80e5b764b9ce50d  /lib/libc-2.6.1.so
b3cc2e0edef8623104ac5daeb16d24e1  /lib/libcidn-2.6.1.so
d3a57cf10e098ca745365ad4ab21e3ea  /lib/libcrypt-2.6.1.so
39404cdba21509be3eac1450b832449b  /lib/libdl-2.6.1.so
52ddb0a9af02b89de9aca86ff0ed0f82  /lib/libkeyutils-1.2.so
03407c32b78921f1fd63a866ecfbf146  /lib/libm-2.6.1.so
ed7c067af689c9bde157943a5b647693  /lib/libmemusage.so
40ff2dd9dec707881a48a562aeaff4e0  /lib/libnsl-2.6.1.so
431fbf7a4cd86551b1c1301708f172ab  /lib/libnss_compat-2.6.1.so
376b5c0ee6dddf154478547f39f69b22  /lib/libnss_dns-2.6.1.so
2eb73c96bd6e33ac585612f6a2ae4a6a  /lib/libnss_files-2.6.1.so
99b43a0435777bd804804906ed8fac9c  /lib/libnss_hesiod-2.6.1.so
735b64c1edea58adbb526c6b0324d36f  /lib/libnss_nis-2.6.1.so
a6f6813ef5f0a0a576cfd757ebded5d6  /lib/libnss_nisplus-2.6.1.so
5a647a1a7ca7d139098e812852aaa187  /lib/libpcprofile.so
70d6ae91f6e7c2389b915ac6b819a4cd  /lib/libpthread-2.6.1.so
4f1aeb5870b28812c06375fc4eff031b  /lib/libresolv-2.6.1.so
6b5190ccfb905efdab0de8154d24d119  /lib/librt-2.6.1.so
949f2f4b2be8d8eed648284d4fa3c123  /lib/libthread_db-1.0.so
2b527c799f2314b8f20a689f1d80196c  /lib/libutil-2.6.1.so

Yes, the update process is broken.  With the help of the ssh account I was
able to determine that on broken systems there is a mixture of the i686
glibc and the i586 glibc installed.  The cause is the .patch.rpm on the
update server.

One after the other:
1) GA installs the i686 glibc as glibc-32bit
2) Update has the i586 glibc as glibc-32bit.rpm

Both will work fine, when installed separately.  But:

3) Update tree also has a .patch.rpm, which updates the changed files
   for the update (which are not _all_ of them), in particular it will
   not change libdl-2.6.1.so.

Installing the patch rpm (neither the delta rpm nor the full rpm will exhibit
this problem) will now replace some files on the system, e.g. ld-2.6.1.so to
the update version (from the i586 tree).  It will leave some files unupdated,
as per the patch rpm.  For instance libdl-2.6.1.so.

So, now there's a mixture of an i686 libdl-2.6.1.so and a i586 ld-2.6.1.so
installed.  This is indeed not supported by glibc.  All these files dealing
with the runtime linker have to come out of the same build (they use
internal interfaces which change over build configurations).

Therefore this segfault.

There are two bugs in here:
1) actually generating the patch.rpm went somehow wrong.
   A correct patch.rpm (from i686-GA glibc-32bit to i586-Update
   glibc-32bit _should_ have contained all files).
2) That applying the patch.rpm didn't result in exactly the same files
   as installing the full rpm obviously isn't checked.  Hence some integrity
   checking is missing in actually applying patch rpms.

Not again, that this wouldn't have happened if the update process would have
either used the delta rpm, or the full rpm.  yast2 should have chosen
the delta rpm actually.  Have you used a different installer?

CCing also mls as the patch rpm definitely is completely wrong.

And what's even more worrying is, that rpm -V doesn't show this
discrepancy in the MD5 sums of these files.  So, as far as the user is
concerned he has installed a fully valid glibc-32bit package, which just
doesn't exist at all anywhere in the world, except in the imagination of
patch-rpm :-/

(In reply to comment #39 from Michael Matz)
> Not again, that this wouldn't have happened if the update process would have
> either used the delta rpm, or the full rpm.  yast2 should have chosen
> the delta rpm actually.  Have you used a different installer?

I used either "you" or "opensuseupdater-kde" on all systems. I did some
experiments with "--exclude"-ing patch and delta rpms from my
rsync update of my local repository since they are redundant
but I am quite sure that I had a complete rsync when the problem
happened.

It seems I wasn't explicit enough, the wrong patch rpm still is on the update
server.  It needs to be removed (probably including the metadata referring
to it).  Hence assigning to Rudi.

as a quick start I'm rerunning that patch with a fixed glibc-32bit
(now the i686 one again) thus removing the broken rpm/delta/patch,
which will fix things for people updating now.

However, we will need another update to fix the systems that are affected
already.

Anja: please can I get a new SWAMPID so we can really fix this ?

sure. SWAMPID is 14207

/mounts/mirror/SuSE/ftp.opensuse.org/srv/ftp/pub/opensuse/update/10.3/rpm/x86_64/glibc-32bit-2.6.1-18.2.x86_64.patch.rpm

still is the old one.  Someone should please remove it.

*** Bug 335912 has been marked as a duplicate of this bug. ***

old patch (with rpms release-number 18.2) has been removed completely
new patch with rebuilt glibc rpms (release-number 18.3) has been checked
in and is in the queue for approval (id 528d3b710b55f4880b33cb600342e274)

released