Bug 385677

Summary: Xvnc segfaults when using gdm (Xvnc started by xinetd)
Product: [openSUSE] openSUSE 11.0 Reporter: Eric Ward <eward>
Component: X.OrgAssignee: Stefan Dirsch <sndirsch>
Status: RESOLVED FIXED QA Contact: E-mail List <xorg-maintainer-bugs>
Severity: Major    
Priority: P2 - High CC: federico, forgotten_U3w37yBkwc, max, vodicka.petr
Version: Factory   
Target Milestone: ---   
Hardware: i586   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Eric Ward 2008-05-01 17:25:41 UTC
Problem -
Attempting to connect to the Xvnc causes a segfault. Here is what is thrown in the /var/log/message:

May  1 04:53:06 erw3 gconfd (root-4101): starting (version 2.22.0), pid 4101 user 'root'
May  1 04:53:06 erw3 gconfd (root-4101): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
May  1 04:53:06 erw3 gconfd (root-4101): Resolved address "xml:readwrite:/root/.gconf" to a writable configuration source at position 1
May  1 04:53:06 erw3 gconfd (root-4101): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
May  1 04:53:06 erw3 gconfd (root-4101): Resolved address "xml:readonly:/etc/gconf/gconf.xml.vendor" to a read-only configuration source at position 3
May  1 04:53:06 erw3 gconfd (root-4101): Resolved address "xml:readonly:/etc/gconf/gconf.xml.schemas" to a read-only configuration source at position 4
May  1 04:53:06 erw3 klogd: Xvnc[4072]: segfault at 0 ip 08094afd sp bff59c20 error 4 in Xvnc[8048000+3b0000]
May  1 04:53:07 erw3 gdm[4075]: gkr-pam: couldn't get the user name: Conversation error
May  1 04:53:07 erw3 gconfd (root-4101): Exiting



Steps to Reproduce:
- Install an openSUSE 11.0 Beta 1 with a default gnome setup
- Once logged into the system Start YaST and configure remote administration
yast remote
- Check the "Allow Remote Administration" and "Open Port in Firewall". And save the settings
- Reboot the system.
- From a remote machine attempt to connect to the Xvnc server
vncviewer <Xvnc IP Address>:1
- The segfault should occur at this point


Expected Behavior
The user should be able to successfully connect to the Xvnc process and login to the system.
Comment 1 Stefan Dirsch 2008-05-01 17:49:58 UTC
I can't get Xvnc to start via xinetd at all, although it seems to be registered. 

# nmap -P0 -p 5900-5905 shannon
Starting Nmap 4.20 ( http://insecure.org ) at 2008-05-01 19:51 CEST
Interesting ports on shannon (10.10.0.79):
PORT     STATE    SERVICE
5900/tcp filtered vnc
5901/tcp open     vnc-1
5902/tcp filtered vnc-2
5903/tcp filtered vnc-3
5904/tcp filtered unknown
5905/tcp filtered unknown

Nmap finished: 1 IP address (1 host up) scanned in 11.346 seconds

Reinhard, does this work for you? Eric, which architecture is this?
Comment 2 Eric Ward 2008-05-01 19:49:01 UTC
I'm seeing this on both i586 and x86_64 openSUSE 11.0 beta1 systems. 
Comment 3 Stefan Dirsch 2008-05-01 19:51:36 UTC
Reinhard?
Comment 7 Reinhard Max 2008-05-05 07:33:08 UTC
Starting Xvnc via xinetd (vncviewer localhost:1) works fine for me on 11.0 Beta1.

But instead of a [xkg]dm login screen I only get the regular X server startup background with the right half distorted by horizontal stripes.

When I move the vncviewer window around for a little while, it suddenly exits saying "VNC server closed connection", but I get no segfault reported in /var/log/messages or /var/log/xinetd .
Comment 9 Stefan Dirsch 2008-05-05 08:29:28 UTC
(In reply to comment #7 from Reinhard Max)
> Starting Xvnc via xinetd (vncviewer localhost:1) works fine for me on 11.0
> Beta1.
> 
> But instead of a [xkg]dm login screen I only get the regular X server startup
> background

Same for me.

> with the right half distorted by horizontal stripes.

I cannot reproduce this.

> When I move the vncviewer window around for a little while, it suddenly exits
> saying "VNC server closed connection", but I get no segfault reported in
> /var/log/messages or /var/log/xinetd.

Same for me. Maybe this is a timeout, because no Xclient connects.



Comment 10 Stefan Dirsch 2008-05-05 08:35:57 UTC
Ok. We both didn't run a displaymanager. Therefore we got the X background. After starting gdm I see the Sig11 in /var/log/messages as well.

May  5 10:39:18 e33 klogd: Xvnc[17757]: segfault at 0 ip 45b691 sp 7fff29a4e460 error 4 in Xvnc[400000+3ea000]
Comment 11 Stefan Dirsch 2008-05-05 08:38:37 UTC
The segfault does not happen with kdm(3)/xdm.
Comment 12 Stefan Dirsch 2008-05-05 11:02:59 UTC
Program received signal SIGSEGV, Segmentation fault.
rfbSpritePictureOverlap (pPict=0xb6e9a0, x=0, y=0, w=28, h=28) at sprite.c:2020
2020        if (pPict->pDrawable->type == DRAWABLE_WINDOW)
(gdb) bt
#0  rfbSpritePictureOverlap (pPict=0xb6e9a0, x=0, y=0, w=28, h=28)
    at sprite.c:2020
#1  0x000000000045b9b4 in rfbSpriteComposite (op=1 '\001', pSrc=0xb6e9a0, 
    pMask=0xb6ea80, pDst=0xb6eb60, xSrc=<value optimized out>, 
    ySrc=<value optimized out>, xMask=0, yMask=0, xDst=0, yDst=0, 
    width=<value optimized out>, height=<value optimized out>) at sprite.c:2065
#2  0x00000000006d35fa in ProcRenderComposite (client=0xb56a00) at render.c:758
#3  0x0000000000478d22 in Dispatch () at dispatch.c:502
#4  0x000000000048c215 in main (argc=10, argv=0x7fffd1ecb118, 
    envp=<value optimized out>) at main.c:452
(gdb)

Probably related to broken RENDER support in Xvnc.
Comment 13 Stefan Dirsch 2008-05-05 14:03:12 UTC
I disabled RENDER support in Xvnc now, which fixes this issue. Fixed for Beta3.
Comment 14 Mike Fabian 2008-05-14 14:12:45 UTC
But this causes another bug, see bug #390011.
Comment 15 Federico Mena Quintero 2008-05-14 15:56:26 UTC
Hmm, but this fixes the symptom, not the cause.  What's the actual bug in Xvnc that makes it crash?  Should we reopen this so that we can keep track of the actual crasher?
Comment 16 Stefan Dirsch 2008-05-14 16:02:10 UTC
I have no clue. Do you volunteer to debug Xvnc?
Comment 17 Stefan Dirsch 2008-05-14 16:45:55 UTC
xorg-x11.changes:
-------------------------------------------------------------------
Wed May 14 18:19:15 CEST 2008 - sndirsch@suse.de

- disabled patch to disable RENDER support in Xvnc, since it broke
  24bit color depth support (bnc #390011)

==> REOPEN
Comment 18 Stefan Dirsch 2008-05-14 20:58:35 UTC
*** Bug 390468 has been marked as a duplicate of this bug. ***
Comment 19 Stefan Dirsch 2008-05-14 22:22:06 UTC
*** Bug 390478 has been marked as a duplicate of this bug. ***
Comment 20 Federico Mena Quintero 2008-05-15 18:24:23 UTC
(In reply to comment #16 from Stefan Dirsch)
> I have no clue. Do you volunteer to debug Xvnc?

I won't have time to do that any time soon, sorry :)  If the "disable RENDER" hack works and clients can deal with that, I guess it's okay to leave the bug as fixed.

You are a lot more familiar than myself with the upstream bugs.freedesktop.org for xorg --- do you know if there's a bug there about this?  We can just reference it from here and leave *our* bug as LATER or something.
Comment 21 Stefan Dirsch 2008-05-15 19:44:32 UTC
Unfortunately the "disable RENDER" hack breaks 24 color depth, so this is no longer an option. :-(
Comment 22 Stefan Dirsch 2008-05-16 10:56:12 UTC
fixed for RC1/Factory/XOrg:X11

# rpm --changelog -q xorg-x11-server
-------------------------------------------------------------------
Fri May 16 12:15:57 CEST 2008 - sndirsch@suse.de

- xorg-server-1.4-vnc-render_sig11.diff
  * fixed sig11 in RENDER code (bnc #385677)
Comment 23 Stefan Dirsch 2008-05-21 07:53:09 UTC
*** Bug 390169 has been marked as a duplicate of this bug. ***
Comment 24 JP Rosevear 2008-05-21 12:36:13 UTC
*** Bug 390169 has been marked as a duplicate of this bug. ***