Bug 389386

Summary: Xvnc (16bit during installation) is crashing if VNC client is running on a display with the same color depth (16bit apparently is still famous)
Product: [openSUSE] openSUSE 11.0 Reporter: Martin Mrazik <mmrazik>
Component: X.OrgAssignee: Stephan Kulow <coolo>
Status: RESOLVED FIXED QA Contact: E-mail List <xorg-maintainer-bugs>
Severity: Normal    
Priority: P2 - High CC: aj, andrew, coolo, jra, mls, sergey.error, sndirsch, sven.knauer, thomas.schallar, yxu
Version: Final   
Target Milestone: Future/Later   
Hardware: Other   
OS: openSUSE 11.0   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: y2logs
screenshot #1
screenshot #2
proposed new version of the 1.4-vnc patch
xorg-server-1.4-vnc-crash.diff
distortion in original OpenSuSE-11.0 version, see comment #51

Description Martin Mrazik 2008-05-12 15:59:14 UTC
Created attachment 214455 [details]
y2logs

This might be a duplicate of bug #358865 but I'm observing this on a i386 machine (it is a x86_64 host with i386 installation).

VNC installation crashes right after connection of the VNC client. The VNC client is asked for a password and right after the password is entered the installation crashes with "An error occurred during the installation" message.

y2logs attached. Unfortunately I don't know how to debug this better so if you need more logs please let me know. I think this one may be easy to reproduce. Attached are 2 screenshots from the text console (from KVM). This host doesn't have serial port thus I'm unable to attach the whole output.
Comment 1 Martin Mrazik 2008-05-12 15:59:37 UTC
Created attachment 214456 [details]
screenshot #1
Comment 2 Martin Mrazik 2008-05-12 15:59:44 UTC
Created attachment 214457 [details]
screenshot #2
Comment 3 Martin Mrazik 2008-05-12 16:02:59 UTC
btw. this installation is using standard setup. I just added "vnc=1 vncpassword="aaaaaaaa"" arguments to linuxrc
Comment 4 Stefan Dirsch 2008-05-13 16:50:36 UTC
Hmm. I cannot reproduce this issue when installing a x86_64 machine with Beta2-i386. I'm using vncviewer (tightvnc) as client on Factory. I'll try Beta3 as well.
Comment 5 Stefan Dirsch 2008-05-13 16:56:37 UTC
The same result. Works for me.
Comment 6 Martin Mrazik 2008-05-14 08:31:02 UTC
Mhm... I will try with beta3 (once it is ready) and I plan to reopen if I'm able to reproduce (and lower the severity as it is happening to me only).

Is there anything how can I help with debugging this?
Comment 7 Stefan Dirsch 2008-05-14 09:34:32 UTC
Ok. Possibly you can start Xvnc manually in gdb with the options you see in your screenshot #2.
Comment 8 Martin Mrazik 2008-05-22 10:48:12 UTC
I can still reproduce this with beta3plus.

I've installed beta3plus, added glibc-debuginfo and executed Xvnc in gdb. After I entered password at the client side I received SIGABRT on the server side:

----------------------cut-----------------------

(gdb) run :0 -noreset -rfbauth /root/.vnc/passwd.yast -desktop 'Installation at: amy' -geometry 800x600 -depth 16 -rfbwait 120000 -httpd /usr/share/vnc/classes -rfbport 5901 -httpport 5801 -fp /usr/share/fonts/misc,/usr/share/fonts/uni,/usr/share/fonts/truetype
Starting program: /usr/bin/Xvnc :0 -noreset -rfbauth /root/.vnc/passwd.yast -desktop 'Installation at: amy' -geometry 800x600 -depth 16 -rfbwait 120000 -httpd /usr/share/vnc/classes -rfbport 5901 -httpport 5801 -fp /usr/share/fonts/misc,/usr/share/fonts/uni,/usr/share/fonts/truetype
[Thread debugging using libthread_db enabled]
22/05/2008 18:35:33 Xvnc version X.org/xf4vnc custom version
22/05/2008 18:35:33 Copyright (C) 2001-2004 Alan Hourihane.
22/05/2008 18:35:33 Copyright (C) 2000-2004 Constantin Kaplinsky
22/05/2008 18:35:33 Copyright (C) 1999 AT&T Laboratories Cambridge
22/05/2008 18:35:33 All Rights Reserved.
22/05/2008 18:35:33 See http://www.tightvnc.com/ for information on TightVNC
22/05/2008 18:35:33 See http://xf4vnc.sf.net for xf4vnc-specific information
22/05/2008 18:35:33 Desktop name 'Installation at: amy' (amy:0)
22/05/2008 18:35:33 Protocol versions supported: 3.7, 3.3
22/05/2008 18:35:33 RGB format 5 6 5
22/05/2008 18:35:33 Listening for VNC connections on TCP port 5901
22/05/2008 18:35:33 Listening for HTTP connections on TCP port 5801
22/05/2008 18:35:33   URL http://amy:5801
Could not init font path element /usr/share/fonts/uni, removing from list!
22/05/2008 18:35:38
22/05/2008 18:35:38 Got VNC connection from client 10.20.1.145
22/05/2008 18:35:38 Using protocol version 3.7
22/05/2008 18:35:38 Enabling TightVNC protocol extensions
22/05/2008 18:35:40 Full-control authentication passed by 10.20.1.145
[New Thread 0xb7bb76c0 (LWP 4877)]
22/05/2008 18:35:40 Pixel format for client 10.20.1.145:
22/05/2008 18:35:40   16 bpp, depth 16, little endian
22/05/2008 18:35:40   true colour: max r 31 g 63 b 31, shift r 11 g 5 b 0
22/05/2008 18:35:40   no translation needed
22/05/2008 18:35:40 Using copyrect encoding for client 10.20.1.145
22/05/2008 18:35:40 Using tight encoding for client 10.20.1.145
22/05/2008 18:35:40 Using compression level 1 for client 10.20.1.145
22/05/2008 18:35:40 Using image quality level 6 for client 10.20.1.145
22/05/2008 18:35:40 Enabling X-style cursor updates for client 10.20.1.145
22/05/2008 18:35:40 Enabling cursor position updates for client 10.20.1.145
22/05/2008 18:35:40 Enabling LastRect protocol extension for client 10.20.1.145
22/05/2008 18:35:40 rfbProcessClientNormalMessage: ignoring unknown encoding -223

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xb7bb76c0 (LWP 4877)]
0xb7bfb916 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
        in ../nptl/sysdeps/unix/sysv/linux/raise.c
(gdb) bt
#0  0xb7bfb916 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0xb7bfd218 in *__GI_abort () at abort.c:88
#2  0xb7c3d3b3 in malloc_printerr (action=2, str=0xb7cf3b5b "free(): invalid pointer", ptr=0x8534da0) at malloc.c:5954
#3  0xb7c3ecdb in *__GI___libc_free (mem=0x8534da0) at malloc.c:3589
#4  0x0809a5db in ?? ()
#5  0x08534da0 in ?? ()
#6  0x00000000 in ?? ()
----------------------cut-----------------------


I know its too late for 11.0 but it would be nice to have this fixed for 11.1/sles11 (it looks like I have three affected hosts in server-room I'm using for testing). Is there anything else I can do? I can give you access to the host if you need it.
Comment 9 Stefan Dirsch 2008-05-22 12:02:13 UTC
Yes, I need access to this host.
Comment 11 Martin Mrazik 2008-05-23 09:13:51 UTC
btw its not related to x86_64/i386 HW/distribution. I just reproduced it with x86_64 on different host (but it is almost the same HW).
Comment 16 Olaf Hering 2008-05-28 11:13:44 UTC
likely related to bug #358865
Comment 17 Martin Mrazik 2008-05-30 07:49:06 UTC
(In reply to comment #16 from Olaf Hering)
> likely related to bug #358865

Very likely, yes. I'm unable to reproduce this with RC1 -> closing.
Comment 18 Sergey Akifiev 2008-06-23 14:21:18 UTC
i can confirm this with 11.0 release. Xvnc crashes the same way as reported.
only workaround i have found is to connect with java-enabled browser.

installation was made onto vmware virtual machine. everything was setted up according novell documentation. tightvnc 1.3.9 on windows platform was used as a client.
Comment 19 Vilem Marsik 2008-06-23 14:31:58 UTC
I can also confirm on following configurations:

client: tightvnc-1.3.9-15 on OpenSuSE 11 on x86_64

installed system (VNC server):
1) openSuSE 11 RC4 on x86_64 (vulture.suse.cz)
2) openSuSE 11 RC3 on x86_64 (vulture.suse.cz)
3) openSuSE 11 RC4 on i686   (patty.suse.cz)

More info on request.

Reopening the bug.
Comment 20 Stefan Dirsch 2008-06-23 15:32:34 UTC
Please tell me how I can reproduce this issue exactly.
Comment 21 Vilem Marsik 2008-06-23 15:55:02 UTC
You need to:
- start the installation with vnc=1 and vnc_password=<something>
- wait until the installer writes that you can connect via VNC
- connect from another machine via vncviewer
- supply the password
- at this moment, the vncviewer opens a window that instantly disappears; the installer on the another machine crashes at the same moment

You should perhaps have a right version of the vncviewer.

The same (incl. same machines and same vncviewer version) works fine for our newer SLES releases, the problem occurs on openSuSE only.


Comment 22 Stefan Dirsch 2008-06-23 16:01:22 UTC
Ok. I'll try to reproduce the issue. x86_64 (openSUSE 11.0) on both sides.
Comment 23 Vilem Marsik 2008-06-24 15:32:03 UTC
I just tried it with openSuSE11 on both sides (against i586), and it works OK. Yet when I disconnect the vncviewer from openSuSE11 (tightvnc-1.3.9-69.1), and reconnect from openSuSE10.3 (tightvnc-1.3.9-15), the installer crashes again.

Perhaps an issue that only appears with older versions of tightvnc?
Comment 24 Stefan Dirsch 2008-06-24 16:06:11 UTC
Cannot reproduce this issue when installing openSUSE 11.0-i386 via vnc. I connected from different machines (11.0-i386, 11.0-x86_64 and 11.0-i386) via tightvnc's vncviewer. Even at the same time. Works fine. No Xvnc crashes. I'll try to install openSUSE 11.0-x86_64 via vnc as the next step.
Comment 25 Stefan Dirsch 2008-06-24 16:43:50 UTC
Cannot reproduce this issue when installing openSUSE 11.0-x86_64 via vnc either. I connected from different machines (11.0-i386, 11.0-x86_64 and 11.0-i386) via tightvnc's vncviewer. Even at the same time. Works fine. No Xvnc crashes.

==> WORKSFORME
Comment 26 Vilem Marsik 2008-06-25 12:14:13 UTC
I know that it works when connecting from 11.0, that is what I wrote in #23. Yet it crashes when connecting from 10.3 . If you do not have any useable test machines for debugging, I can offer my ones - contact me at vmarsik at suse dot cz or vmarsik at novell dot com.
Comment 27 Andrew Joakimsen 2008-07-02 02:23:48 UTC
This is not fixed.
Comment 28 Andrew Joakimsen 2008-07-02 02:59:28 UTC

*** This bug has been marked as a duplicate of bug 390837 ***
Comment 29 Andrew Joakimsen 2008-07-02 03:03:06 UTC
Keeping unresolved bug open.
Comment 30 Stefan Dirsch 2008-07-02 05:51:54 UTC
See my comments #24/25.
Comment 31 Andrew Joakimsen 2008-07-02 05:56:44 UTC
Maybe try something besides openSUSE 11 as the client?
Comment 32 Stefan Dirsch 2008-07-02 06:07:42 UTC
Yes, I tried also 10.3-i386 (typo in my comments). I would keep this one as LATER/REMIND (since I can't reproduce it), but this resolution no longer exists, so I need to mark it as Enhancement with Milestone: Future/Later.
Comment 33 Andrew Joakimsen 2008-07-02 06:13:11 UTC
This bug is verifiable by multiple individuals.
Comment 34 Stefan Dirsch 2008-07-02 06:25:35 UTC
*** Bug 390837 has been marked as a duplicate of this bug. ***
Comment 35 Michael Schröder 2008-07-02 08:49:59 UTC
(looks like this may also be related to #351338, seems to me that the -noreset workaround doesn't help all the times)
Comment 36 Andreas Jaeger 2008-07-02 08:56:18 UTC
Ad #33: So, what is Stefan doing different from everybody where it fails?  Please read again comments #24 and #25.  Stefan tried everything he could, so help him to reproduce it, otherwise he cannot fix it.

On the other hand: If any of you that can reproduce it that easily could debug it and send more details, this would help even more!
Comment 37 Andreas Jaeger 2008-07-02 08:58:01 UTC
Andreas van dem Helge: Please stop reassing this bug around, this will not help at all and produce just frustration in my team!
Comment 38 Stefan Dirsch 2008-07-02 09:15:08 UTC
(In reply to comment #35 from Michael Schroeder)
> (looks like this may also be related to #351338, seems to me that
> the -noreset workaround doesn't help all the times)

This workaround is likely no longer required due to schwab's fix (xorg-server-1.4-vnc-memory.diff) in xorg-x11-server sources.

-------------------------------------------------------------------
Thu Mar 20 14:51:20 CET 2008 - sndirsch@suse.de
[...]
- make the memory corruption fix by schwab a seperate patch to make
  sure it won't get lost the next time I update the VNC patch

-------------------------------------------------------------------
Wed Mar 19 20:11:26 CET 2008 - schwab@suse.de

- Fix vnc server memory corruption.

diff -u -w -r xorg-server-1.4.0.90.orig//hw/vnc/init.c xorg-server-1.4.0.90/hw/vnc/init.c
--- xorg-server-1.4.0.90.orig//hw/vnc/init.c    2008-03-20 14:49:06.620576750 +0100
+++ xorg-server-1.4.0.90/hw/vnc/init.c  2008-03-20 14:49:27.769898500 +0100
@@ -827,6 +827,7 @@
        KbdDeviceOff();
        break;
     case DEVICE_CLOSE:
+       vncSetKeyboardDevice(NULL);
        if (pDev->on)
            KbdDeviceOff();
        break;
@@ -869,6 +870,7 @@
        break;
 
     case DEVICE_CLOSE:
+       vncSetPointerDevice(NULL);
        if (pDev->on)
            PtrDeviceOff();
        break;

So I don't think this is the issue here.
Comment 39 Stefan Dirsch 2008-07-02 09:26:20 UTC
See Bug #358865, comment #52. Assuming instsys is 11.0-i386/11.0-x86_64 there are about 8 combinations to be tested. I've already tested 6 of them (11.0-i386/11.0-x86_64/10.3-i386 as VNC client with both 11.0-i386/11.0-x86_64
as VNC server) and couldn't reproduce the issue.

*** This bug has been marked as a duplicate of bug 358865 ***
Comment 40 Andrew Joakimsen 2008-07-02 15:47:04 UTC
I don't want to touch Bug # 358865 (yet) but that is an issue with PPC client and server it seems not x86. So what assurance is there that the PPC bug will resolve the issue on non-PPC platforms? 

I reassigned to bnc-team-screening@forge.provo.novell.com based on the information provided here: http://en.opensuse.org/BNC_Screening_Team
Comment 41 Andrew Joakimsen 2008-07-02 15:48:26 UTC
As I said in my email to AJ I have had to spend two much time with these two bugreports to be able to dedicate any time to providing specifics regarding the VNC issue. Those details shall be provided shortly.
Comment 42 Stefan Dirsch 2008-07-02 15:57:54 UTC
(In reply to comment #40 from Andreas van dem Helge)
> I don't want to touch Bug # 358865 (yet) but that is an issue with PPC client
> and server it seems not x86. So what assurance is there that the PPC bug will
> resolve the issue on non-PPC platforms? 

Again. Please read comment #52 of Bug #358865. Unfortunately I can't look into
this issue before next week.

> So what assurance is there that the PPC bug will resolve the issue on non-PPC
> platforms? 

There is no guarantee. Never. Maybe, if I ever will be able to reproduce the issue, I might fix a different crash than occurs on your side. We'll see.

(In reply to comment #37 from Andreas Jaeger)
> Andreas van dem Helge: Please stop reassing this bug around, this will not
> help at all and produce just frustration in my team!

I fully agree. So can you please stop this reopen game, please ?!? Thanks!



*** This bug has been marked as a duplicate of bug 358865 ***
Comment 43 Andrew Joakimsen 2008-07-02 16:06:39 UTC
I also need time to gather logs and such. I had tested using UltraVNC 1.1.0.2 running on Windows Vista and also using the java viewer on the same OS with the latest Firefox 2 browser, both providing the same results.

When I provide the relevant logs in this ticket will you stop playing the re-close game? What is the issue with leaving an open bug with status of "need info" or "unverified?"


Comment 44 Stefan Dirsch 2008-07-02 16:40:58 UTC
Again. Later will vanish.
Comment 45 Andrew Joakimsen 2008-07-02 20:53:02 UTC
Verified KRDC 3.93.00 (KDE 4.0 Beta2) / TightVNC Viewer version 1.3.9; the default VNC client installed in openSUSE 10.3 is able to connect fine.

Verified UltraVNC 1.0.2 (latest stable) on Windows XP SP2 is *NOT* able to connect fine. (Previously verified issue with same viewer on Windows Vista SP1)

Previously I had tried via SSH tunneling over the internet. Running over the local LAN you can see the screen show for a brief moment the installer welcome screen.

Need to transfer logfiles and verify bug with more VNC clients.
Comment 46 Michael Schröder 2008-07-03 12:49:34 UTC
Heh, good news: I can reproduce the segfault if I attach to a VNC session from another VNC session. I suspect it crashes if you attach from a 16bit visual.

Unfortunately valgrind doesn't help much, it says:

valgrind: m_mallocfree.c:194 (mk_plain_bszB): Assertion 'bszB != 0' failed.
==9521==    at 0x3801A3BD: (within /usr/lib/valgrind/x86-linux/memcheck)
==9521==    by 0x3801A6AE: (within /usr/lib/valgrind/x86-linux/memcheck)
==9521==    by 0x38023C03: (within /usr/lib/valgrind/x86-linux/memcheck)
==9521==    by 0x38002BE0: (within /usr/lib/valgrind/x86-linux/memcheck)
==9521==    by 0x3800308D: (within /usr/lib/valgrind/x86-linux/memcheck)
==9521==    by 0x38039441: (within /usr/lib/valgrind/x86-linux/memcheck)
==9521==    by 0x3804C6B8: (within /usr/lib/valgrind/x86-linux/memcheck)

and dies :-(
Comment 47 Michael Schröder 2008-07-03 16:24:28 UTC
Created attachment 225862 [details]
proposed new version of the 1.4-vnc patch
Comment 48 Stefan Dirsch 2008-07-03 16:32:49 UTC
Thanks a lot, Michael! Would it be an option to make an additional patch on top of the old one?
Comment 49 Stefan Dirsch 2008-07-03 17:02:55 UTC
Created attachment 225869 [details]
xorg-server-1.4-vnc-crash.diff
Comment 50 Stefan Dirsch 2008-07-03 17:11:44 UTC
*** Bug 358865 has been marked as a duplicate of this bug. ***
Comment 51 Stefan Dirsch 2008-07-03 18:10:24 UTC
Package xorg-x11-Xvnc for testing will be available shortly on

  http://beta.suse.com/private/sndirsch/bug389386

On installed system start "Xvnc -depth 16 :99". On client - with 16bit color depth display - start vncviewer <servername>:99 or your Windows/Browser VNC client. With the original xorg-x11-Xvnc package from DVD this should crash Xvnc. This the updated package it shouldn't.

Please test!
Comment 52 Stefan Dirsch 2008-07-10 02:05:45 UTC
Waiting for feedback since a week now. :-( So much pressure we got to address/fix this issue. And now that packages for testing are available nobody is interested any more ?!?
Comment 53 Michael Schröder 2008-07-10 10:00:15 UTC
It's probably not easy to test, as you'd need an inst-sys with the new package...
Comment 54 Stefan Dirsch 2008-07-10 10:17:27 UTC
Michael, I provided test instructions, which do *not* depend on an updated inst-sys. See my comment #51.
Comment 55 Stefan Dirsch 2008-07-11 11:18:43 UTC
Since nobody else wanted to test it, I did this now myself by using the driver update mechanism (special driver update image) and it works for me. Indeed without this patch Xvnc crashes immediately when running the VNC client on a 16bit color depth display.
Comment 56 Stefan Dirsch 2008-07-11 12:23:47 UTC
Package and patchinfo file submitted.

/work/src/done/11.0/xorg-x11-server
/work/src/done/PATCHINFO/xorg-x11-Xvnc.patch.box

Anja, could you open a SWAMPID for this issue?
Andreas (Jaeger), (how) could we update the Driver update image (F6 at boot prompt) with xorg-x11-Xvnc:/usr/bin/Xvnc?
Comment 57 Vilem Marsik 2008-07-11 16:19:31 UTC
I made a test according comment #51. Used the original OpenSuSE-11.0 version, plus your beta version. Results:

- neither of the versions caused a crash, like that one during the installation
- the original version showed some image distortion, the beta version did not
Comment 58 Vilem Marsik 2008-07-11 16:20:49 UTC
Created attachment 227354 [details]
distortion in original OpenSuSE-11.0 version, see comment #51
Comment 61 Stefan Dirsch 2008-07-14 13:32:36 UTC
Thanks, Anja. SWAMPID is now set in

  /work/src/done/PATCHINFO/xorg-x11-Xvnc.patch.box
Comment 64 Stephan Kulow 2008-07-17 11:20:12 UTC
I submitted a new openSUSE-driverupdate to 11.0, it needs checkin
Comment 65 Stefan Dirsch 2008-07-17 12:33:52 UTC
Thanks a lot, Coolo! Finally closing as fixed.
Comment 66 Stefan Dirsch 2008-09-15 09:10:17 UTC
*** Bug 403354 has been marked as a duplicate of this bug. ***
Comment 67 Stefan Dirsch 2008-09-15 12:03:26 UTC
*** Bug 407165 has been marked as a duplicate of this bug. ***
Comment 68 Stefan Dirsch 2009-04-01 15:22:39 UTC
*** Bug 479195 has been marked as a duplicate of this bug. ***
Comment 69 Jay Ashworth 2009-04-02 16:06:51 UTC
Re 64 and 65: for those of us who don't eat, sleep and breathe the distribution build process, could anyone bother to provide instructions as to how to apply the "fix" to *an actual 11.0 installation CDROM(/image)* so that we can actually put a CD in a drive and VNC install?

Not too put too fine a point on it, gentlemen, but *the bug isn't resolved until you do that*.

So I'm reopening it.
Comment 70 Stefan Dirsch 2009-04-02 16:11:42 UTC
Coolo, please explain. Thanks.
Comment 71 Jay Ashworth 2009-04-02 17:18:04 UTC
And, looking back, I guess I should expand on what I'm after.

I'm going to do a *lot* of 11.0 installs this year; I would much prefer to spend some effort slipstreaming the fix into the install CD image, than to have to load a driver disc on every install to overwrite whatever's broken -- 66 degree machine rooms aren't my favorite hangouts.
Comment 72 Stefan Dirsch 2009-04-11 02:10:14 UTC
Apparently our project manager for openSUSE is not able or willing to provide these instructions. I suggest to accept this. The bug is fixed from our side through providing the driver-update (via network) during installation and the package update for later use of Xvnc.