Bug 406144

Summary: Xvnc: JPEG compression in tight encoding causes wrong colors/artefacts, if VNC client runs on a display with a different color depth than Xvnc.
Product: [openSUSE] openSUSE 11.0 Reporter: Andreas Schwab <schwab>
Component: X.OrgAssignee: Stefan Dirsch <sndirsch>
Status: RESOLVED UPSTREAM QA Contact: E-mail List <xorg-maintainer-bugs>
Severity: Normal    
Priority: P4 - Low CC: andrew, max
Version: Final   
Target Milestone: ---   
Hardware: Other   
OS: Other   
URL: http://sourceforge.net/tracker/index.php?func=detail&aid=2020401&group_id=48178&atid=452220
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Andreas Schwab 2008-07-03 13:11:48 UTC
The Xvnc server as well as the vnc server module don't implement the tight encoding correctly.  When connecting over vnc to an other-endian system (both big->little and little->big) while forcing tight encoding (with vncviewer -encodings tight) there are wrong colors and other artefacts.  There are no such problems when using krfb as the server, thus the client appears to be correct.  Other encodings appear to be working correctly, as well as the Xvnc server on SLES10.
Comment 1 Stefan Dirsch 2008-07-03 14:03:03 UTC
Oh well. Another VNC issue. Does this happen in all color depths? If not, which Xvnc/vncviewer color depth combinations do work? Which don't?
Comment 2 Andreas Schwab 2008-07-03 14:12:06 UTC
How can I find out?
Comment 3 Stefan Dirsch 2008-07-03 14:19:39 UTC
Use -depth <16|24> option for Xvnc. Start vncviewer on top of a 16/24 bit color depth display. I think you can fake the latter by starting vncviewer inside a Xnest server given the appropriate color depth.
Comment 4 Andreas Schwab 2008-07-03 14:41:29 UTC
Doesn't change anything.
Comment 5 Stefan Dirsch 2008-07-03 14:46:44 UTC
Ok. BTW, on SLES10 we also used xf4vnc.
Comment 6 Andreas Schwab 2008-07-03 15:04:29 UTC
Actually it's not an endian bug.  Using a same-endian server shows the same broken behaviour.
Comment 7 Stefan Dirsch 2008-07-03 15:15:05 UTC
Oops.
Comment 8 Stefan Dirsch 2008-07-03 18:37:49 UTC
This smells like another duplicate of Bug #389386. Please test the updated packages (Bug #389386, comment #51).
Comment 9 Andreas Schwab 2008-07-04 12:41:55 UTC
Doesn't change anything.
Comment 10 Stefan Dirsch 2008-07-04 13:27:40 UTC
At least tight encoding is not the default. At least not for vncviewer of our tightvnc package.

       -encodings encoding-list
              TightVNC  supports  several  different  compression  methods  to
              encode  screen  updates;  this option specifies a set of them to
              use in order of preference. Encodings  are  specified  separated
              with  spaces,  and  must thus be enclosed in quotes if more than
              one is specified. Available encodings, in default  order  for  a
              remote  connection,  are  "copyrect tight hextile zlib corre rre
              raw". For a local connection (to the same machine), the  default
              order to try is "raw copyrect tight hextile zlib corre rre". Raw
              encoding is always assumed as a last option if no other encoding
              can  be used for some reason. For more information on encodings,
              see the section ENCODINGS below.
Comment 11 Andreas Schwab 2008-07-04 14:02:04 UTC
copyrect is not a full encoding on its own, it is just used for certain operations.
Comment 12 Stefan Dirsch 2008-07-04 14:36:06 UTC
Correct.

       CopyRect
              The Copy Rectangle encoding is efficient when something is being
              moved;  the  only  data sent is the location of a rectangle from
              which data should be copied to the  current  location.  Copyrect
              could also be used to efficiently transmit a repeated pattern.

Could it be that you're seing JPEG artefacts here?

       Tight  Like Zlib encoding, Tight encoding uses zlib library to compress
              the pixel data, but it pre-processes data to  maximize  compres‐
              sion  ratios,  and  to  minimize CPU usage on compression. Also,
              JPEG compression may be used to encode color-rich  screen  areas
              (see  the  description  of  -quality and -nojpeg options above).
              Tight encoding is usually the best choice for low-bandwidth net‐
              work environments (e.g. slow modem connections).

So it might be worth a try with playing -quality / -nojpeg. But probably you tried this already, didn't you?


Comment 13 Andreas Schwab 2008-07-04 14:52:53 UTC
Completely wrong colors cannot be caused by jpeg artefacts, but jpeg compression is indeed part of the problem.
Comment 14 Stefan Dirsch 2008-07-04 14:59:17 UTC
So if you disable jpeg compression colors are correct or does it just eliminate the artefacts?
Comment 15 Andreas Schwab 2008-07-04 15:08:33 UTC
Yes, the colors are correct and there are no more distorted parts.
Comment 16 Stefan Dirsch 2008-07-14 15:05:10 UTC
Just gave it a try.

'vncviewer -quality <0-7> -encodings tight' looks broken.
'vncviewer -compresslevel <9-2> -encodings tight' looks broken.
'vncviewer -quality <9,8> -encodings tight' works for me.
'vncviewer -compresslevel <0,1> -encodings tight' works for me.

--> quality = 9 - compresslevel ?

'vncviewer -nojpeg -encodings tight' works of course (disables jpeg compression completely).

Last resort would be to disable compresslevel 2-9 / quality 0-7 in Xvnc for tightencoding.

Comment 17 Stefan Dirsch 2008-07-14 15:26:00 UTC
I'm rather clueless how and where to fix this in Xvnc sources.
Comment 19 Stefan Dirsch 2008-07-16 20:34:23 UTC
Assuming other VNC clients don't force tight encoding by default I think this is a minor issue.
Comment 20 Andreas Schwab 2008-07-16 22:26:45 UTC
The default for krdc with medium quality is tight encoding.
Comment 21 Stefan Dirsch 2008-07-17 05:15:32 UTC
Thanks. Will try to reproduce also with krdc. There are also other VNC clients on different platforms (Windows, Java running in a webbrowser, ...).
Comment 23 Stefan Dirsch 2008-07-17 09:10:55 UTC
(In reply to comment #20 from Andreas Schwab)
> The default for krdc with medium quality is tight encoding.

I can reproduce the issue with kdrc VNC client when using medium and low quality (krdc running on a 24bit(32bpp) display, "Xvnc -depth 16").

Comment 24 Stefan Dirsch 2008-09-20 00:48:21 UTC
Reported upstream. See URL.
Comment 25 Stefan Dirsch 2008-10-02 12:43:18 UTC
*** Bug 430811 has been marked as a duplicate of this bug. ***