|
Bugzilla – Full Text Bug Listing |
| Summary: | bad gnuplot_x11 performace with utf-8 | ||
|---|---|---|---|
| Product: | [openSUSE] openSUSE 11.0 | Reporter: | Harald Koenig <koenig> |
| Component: | X11 Applications | Assignee: | Dr. Werner Fink <werner> |
| Status: | RESOLVED FEATURE | QA Contact: | Stefan Dirsch <sndirsch> |
| Severity: | Normal | ||
| Priority: | P5 - None | CC: | sndirsch |
| Version: | Factory | ||
| Target Milestone: | --- | ||
| Hardware: | i686 | ||
| OS: | Other | ||
| Whiteboard: | |||
| Found By: | --- | Services Priority: | |
| Business Priority: | Blocker: | --- | |
| Marketing QA Status: | --- | IT Deployment: | --- |
Try out openSuSE 11.0 but note that UTF-8 characters used for non-ASCII letters are always multi byte characters and therefore all string operations from e.g. glibc use the wide character implemenation. Beside this for UTF-8 more fonts are required to map 65535 potential characters to their appropiate letters. This requires more than one font. (In reply to comment #1 from Werner Fink) > Try out openSuSE 11.0 but note that UTF-8 characters used for non-ASCII as mentioned above: gnuplot on 11.0-factory [beta3] does not show that problem anymore I'll pick the 11.0 sources and rebuild it on 10.3 ... > letters are always multi byte characters and therefore all string > operations from e.g. glibc use the wide character implemenation. Beside > this for UTF-8 more fonts are required to map 65535 potential characters > to their appropiate letters. This requires more than one font. I'd be happy for now if gnuplot on 10.3 works fast with just 7bit ascii ;-) (In reply to comment #1 from Dr. Werner Fink) > Try out openSuSE 11.0 ok, this is a simple benchmark with pure 11.0-factory (beta3plus?) : harald > for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done | LC_ALL=de_DE time gnuplot 0:01.77 real, 0.06 user, 0.01 sys, 4% cpu harald > for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done | LC_ALL=de_DE.utf-8 time gnuplot 0:08.99 real, 0.08 user, 0.02 sys, 1% cpu with good graphics card drivers (e.. nv/nvidia) the difference might be even worse, because my fglrx in IBM T60 for ATI X1400 shows a significant slowdown with increasing window size while "nv" has constant performace for small/large plot windows -- but that'll be another bug report... ;-) I traced the X11 protocol for this benchmark using "xmon" and found two significant differences: - with utf-8 there are _many_ ListFonts and ListFontsWithInfo requests, with de_DE there are none: harald > grep -c ListFont gp.xmon.de_DE* gp.xmon.de_DE:0 gp.xmon.de_DE.utf-8:15600 this is a historgram over all ListFonts request pattern: 1900 "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" 1900 "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" 300 "-*-medium-r-normal--16-*-ISO8859-1" 200 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-1" 200 "-*-medium-r-normal--16-*-JISX0201.1976-0" 100 "-xos4-terminus-medium-r-normal--16-160-72-72-c-80-iso8859-13" 100 "-sony-fixed-medium-r-normal--16-120-100-100-c-80-jisx0201.1976-0" 100 "-jis-fixed-medium-r-normal--16-110-100-100-c-160-jisx0208.1983-0" 100 "-isas-fangsong ti-medium-r-normal--16-160-72-72-c-160-gb2312.1980-0" 100 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-koi8-r" 100 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-9" 100 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-7" 100 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-5" 100 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-4" 100 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-3" 100 "-etl-fixed-medium-r-normal--16-160-72-72-c-80-iso8859-2" 100 "-efont-fixed-medium-r-normal--16-160-75-75-c-80-iso8859-15" 100 "-efont-biwidth-medium-r-normal--16-160-75-75-p-80-iso10646-1" 100 "-daewoo-gothic-medium-r-normal--16-120-100-100-c-160-ksc5601.1987-0" 100 "-*-medium-r-normal--16-*-KSC5601.1987-0" 100 "-*-medium-r-normal--16-*-KOI8-R" 100 "-*-medium-r-normal--16-*-JISX0208.1983-0" 100 "-*-medium-r-normal--16-*-ISO8859-9" 100 "-*-medium-r-normal--16-*-ISO8859-7" 100 "-*-medium-r-normal--16-*-ISO8859-5" 100 "-*-medium-r-normal--16-*-ISO8859-4" 100 "-*-medium-r-normal--16-*-ISO8859-3" 100 "-*-medium-r-normal--16-*-ISO8859-2" 100 "-*-medium-r-normal--16-*-ISO8859-15" 100 "-*-medium-r-normal--16-*-ISO8859-14" 100 "-*-medium-r-normal--16-*-ISO8859-13" 100 "-*-medium-r-normal--16-*-ISO10646-1" 100 "-*-medium-r-normal--16-*-GB2312.1980-0" 100 "-*-medium-r-normal--16-*-*-ISO8859-14" 100 "-*-medium-r-normal--16-*-*-*-ISO8859-14" 100 "-*-medium-r-normal--16-*-*-*-*-ISO8859-14" 100 "-*-medium-r-normal--16-*-*-*-*-*-ISO8859-14" 100 "-*-medium-r-normal--16-*-*-*-*-*-*-ISO8859-14" while the actual request sequence looks like this: "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-1" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-1" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-1" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-2" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-3" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-4" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-5" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-KOI8-R" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-7" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-9" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-13" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-14" "-*-medium-r-normal--16-*-*-ISO8859-14" "-*-medium-r-normal--16-*-*-*-ISO8859-14" "-*-medium-r-normal--16-*-*-*-*-ISO8859-14" "-*-medium-r-normal--16-*-*-*-*-*-ISO8859-14" "-*-medium-r-normal--16-*-*-*-*-*-*-ISO8859-14" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-ISO8859-15" "-*-sazanami*mincho-medium-r-normal--16-* -*-mincho-medium-r-normal--16" "-*-verdana-medium-r-normal--16-* -*-dejavu*sans-medium-r-normal--16-*-" "-*-medium-r-normal--16-*-JISX0208.1983-0" [ ...] - with utf-8 all strings are drawn one character at a time while de_DE draws a label string at once: without utf8: grep string: gp.xmon.de_DE text item 8 string: " 1" text item 8 string: "-10" text item 8 string: "-5" text item 8 string: " 0" text item 8 string: " 5" text item 8 string: " 10" text item 8 string: "1" text item 8 string: "-11.5623, -1.12474" with utf8 just for the last string (but that's still merged into a single: text item 8 string: "-" text item 8 string: "1" text item 8 string: "1" text item 8 string: "." text item 8 string: "5" text item 8 string: "6" text item 8 string: "2" text item 8 string: "3" text item 8 string: "," text item 8 string: " " text item 8 string: "-" text item 8 string: "1" text item 8 string: "." text item 8 string: "1" text item 8 string: "2" text item 8 string: "4" text item 8 string: "7" text item 8 string: "4" here is a [top of] histogram over all X11 requests: harald > grep REQUEST: gp.xmon.de_DE | sort | uniq -c | sort -nr| head 3600 ............REQUEST: PolyLine 1948 ............REQUEST: PolyText8 700 ............REQUEST: ChangeGC 502 ............REQUEST: ChangeWindowAttributes 128 ............REQUEST: CreateGC 126 ............REQUEST: FreeGC 103 ............REQUEST: OpenFont 102 ............REQUEST: CloseFont 101 ............REQUEST: MapWindow 101 ............REQUEST: ConfigureWindow harald > grep REQUEST: gp.xmon.de_DE.utf-8 | sort | uniq -c | sort -nr| head 9656 ............REQUEST: PolyText8 6200 ............REQUEST: ListFonts 3600 ............REQUEST: PolyLine 1600 ............REQUEST: ListFontsWithInfo 701 ............REQUEST: ChangeGC 502 ............REQUEST: ChangeWindowAttributes 128 ............REQUEST: CreateGC 126 ............REQUEST: FreeGC 103 ............REQUEST: OpenFont 102 ............REQUEST: GetInputFocus are you interested in the full logs ? -rw-r--r-- 1 harald users 3.3M May 23 11:14 gp.xmon.de_DE -rw-r--r-- 1 harald users 8.2M May 23 11:15 gp.xmon.de_DE.utf-8 -rw-r--r-- 1 harald users 94K May 23 11:14 gp.xmon.de_DE.bz2 -rw-r--r-- 1 harald users 184K May 23 11:15 gp.xmon.de_DE.utf-8.bz2 Please report this upstream as I'm *not* the author of gnuplot nor I'm the author of the multi byte font handling of X11. Btw: you may try the same on openSuSE 11.0 factory ... maybe the multi byte font handling with the so called font sets are faster. (In reply to comment #4 from Dr. Werner Fink) > Please report this upstream as I'm *not* the author of gnuplot > nor I'm the author of the multi byte font handling of X11. done: http://sourceforge.net/mailarchive/forum.php?thread_name=20080523125547.GA17603%40turtle.tat.physik.uni-tuebingen.de&forum_name=gnuplot-bugs (In reply to comment #5 from Dr. Werner Fink) > Btw: you may try the same on openSuSE 11.0 factory ... maybe the > multi byte font handling with the so called font sets are faster. this last test was on 11.0 factory. I'll try to set product to "11.0" but version selection only offers "Final" in bugzilla ;-( on 10.3 I'll live with fiddling LC_CTYPE before running gnuplot (at least for now;) This is not really a problem of gnuplot, it is more a problem that X11 core fonts don’t really work well when used with Fontsets in multibyte locales. I think it is better to use "set term wxt" instead of "set term x11". wxt is somewhat slower as x11 in a single byte locale, but much faster than x11 in multi byte locals. In fact the performance of wxt doesn’t significantly depend on the locale used. And it looks *much* nicer anyway. As pango is used for wxt, support for unusual character will be way better than with the X11 core fonts used for x11. Speed comparison of wxt and x11:
mfabian@magellan:~$ ( echo "set term x11" ; for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done ) | LC_ALL=de_DE time gnuplot
0.05user 0.01system 0:01.45elapsed 4%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3284minor)pagefaults 0swaps
mfabian@magellan:~$ ( echo "set term wxt" ; for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done ) | LC_ALL=de_DE time gnuplot
Failed to receive messages at scim_bridge_client_read_and_dispatch ()
An IOException occurred at handle_message ()
Command terminated by signal 6
1.38user 0.28system 0:04.19elapsed 39%CPU (0avgtext+0avgdata 0maxresident)k
16inputs+19152outputs (0major+45771minor)pagefaults 0swaps
mfabian@magellan:~$ ( echo "set term wxt" ; for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done ) | LC_ALL=de_DE.UTF-8 time gnuplot
Failed to receive messages at scim_bridge_client_read_and_dispatch ()
An IOException occurred at handle_message ()
Command terminated by signal 6
1.36user 0.29system 0:04.18elapsed 39%CPU (0avgtext+0avgdata 0maxresident)k
8inputs+19968outputs (0major+45864minor)pagefaults 0swaps
mfabian@magellan:~$ ( echo "set term x11" ; for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done ) | LC_ALL=de_DE.UTF-8 time gnuplot
0.07user 0.01system 0:15.48elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3284minor)pagefaults 0swaps
mfabian@magellan:~$
I.e. wxt is 3 times slower than x11 in single byte locales but more
than 3 times faster than x11 in multibyte locales. And the performance
of wxt is independent of the locale.
As wxt uses client side font rendering with antialiasing it is to be
expected that it is somewhat slower than x11 in single byte locales,
but it looks so much nicer that this is acceptable in my opinion.
On top of that it also gives you better support for unusual characters
and better scalability.
By the was, "set term x11 enhanced font \"dejavu sans,15\"" is also
fast independent of the locale:
mfabian@magellan:~$ ( echo "set term x11 enhanced font \"dejavu sans,15\"" ; for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done ) | LC_ALL=de_DE time gnuplot
0.05user 0.02system 0:01.51elapsed 5%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3293minor)pagefaults 0swaps
mfabian@magellan:~$ ( echo "set term x11 enhanced font \"dejavu sans,15\"" ; for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done ) | LC_ALL=de_DE.UTF-8 time gnuplot
0.03user 0.02system 0:01.52elapsed 4%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3293minor)pagefaults 0swaps
mfabian@magellan:~$
I think wxt should be the default though.
(In reply to comment #8 from Mike Fabian) > Speed comparison of wxt and x11: I neer used "wxt" so far, but my first test (64 bit?!) isn't that promising: ( echo "set term wxt" ; for i in {1..100} ; do echo "plot sin(x+$i/10.) t '$i'" ; done ) | LC_ALL=de_DE time gnuplot Command terminated by signal 11 ^^^^^^^^^ ^^^^^^^^^ I'd guess that's the 64bit-version of your > Failed to receive messages at scim_bridge_client_read_and_dispatch () > An IOException occurred at handle_message () > Command terminated by signal 6 here is my backtrace (but too helpful?!): Program terminated with signal 11, Segmentation fault. [New process 23455] #0 0x00007f44cb5a33c0 in ?? () (gdb) where #0 0x00007f44cb5a33c0 in ?? () #1 0x00007f44d068926d in exit () from /lib64/libc.so.6 #2 0x00007f44d067243d in __libc_start_main () from /lib64/libc.so.6 #3 0x0000000000414e99 in ?? () #4 0x00007fffde85d3e8 in ?? () #5 0x000000000000001c in ?? () #6 0x0000000000000001 in ?? () #7 0x00007fffde85f025 in ?? () #8 0x0000000000000000 in ?? () (gdb) > I think wxt should be the default though. IMHO at least not as long as it's throwing core files! (and maybe not as long as it's su much slower (non-utf8!)). better the gnuplot guys (or X11) will fix the real issue some time but not just cover it with new problems of "wxt"... Harald> I'd guess that's the 64bit-version of your Harald> Harald> > Failed to receive messages at scim_bridge_client_read_and_dispatch () Harald> > An IOException occurred at handle_message () Harald> > Command terminated by signal 6 I’m also using 64bit. Apart from the above message when leaving gnuplot, wxt works fine for me. Harald> (and maybe not as long as it's su much slower (non-utf8!)). Client side font rendering will never be as fast as bitmap fonts in the X11 core font system. With that argument, KDE and Gnome should have stayed with the X11 core font system with all its problems. Both desktops use client side font rendering exclusively now. This is the way to go, also for gnuplot. With X11 core fonts, i18n is an unfixable mess. And, as you see with "set term x11 enhanced font \"dejavu sans,15\"", the X11 core font system is not faster then client side font rendering here when scalable fonts are used. Our hardware has improved somewhat during the last 20 years, I think we can afford some CPU cycles for beautiful antialiased fonts and i18n support which actually works. |
gnuplot shows horrible performace with utf-8 locate! try LC_CTYPE=de_DE.utf-8 gnuplot plot sin(x) t "Foo" this will show a message gnuplot_x11: Some character sets not available and it takes about 1 second to show the simple graph on a 2+ GHz system. the cpu time is taken by the Xserver, not by gnuplot_x11 itself. gnuplot_x11 does some X11 font requests which makes the Xserver quite busy. see a small part of strace output from gnuplot_x11 below (there are ~1400 such font requests for this single plot command). you can trigger the same operations if you slowly resize the gnuplot window (with a window manager which does dynamic window resizing) -- the Xserver gets very busy and resizing/refresing is damn slow. running gnuplot with LC_CTYPE=C or LC_CTYPE=de_DE avoids that "character sets not available" message and gnuplot is fast again. gnuplot on 11.0-factory does not show that problem anymore (using x86_64 on 11.0, but this shouldn't matter?!) here some first examples of the font requests of gnuplot_x11 for _every_ redraw: writev(3, [{"1\0\7\0\1\0\22\0sazanami-ISO8859-1\0\0", 28}], 1) = 28 writev(3, [{"1\0\7\0\1\0\24\0sazanami-*-ISO8859-1", 28}], 1) = 28 writev(3, [{"1\0\10\0\1\0\26\0sazanami-*-*-ISO8859-1\0\0", 32}], 1) = 32 writev(3, [{"1\0\10\0\1\0\30\0sazanami-*-*-*-ISO8859-1", 32}], 1) = 32 writev(3, [{"1\0\t\0\1\0\32\0sazanami-*-*-*-*-ISO8859-1\0\0", 36}], 1) = 36 writev(3, [{"1\0\t\0\1\0\34\0sazanami-*-*-*-*-*-ISO8859-1", 36}], 1) = 36 writev(3, [{"1\0\n\0\1\0\36\0sazanami-*-*-*-*-*-*-ISO8859-1\0\0", 40}], 1) = 40 writev(3, [{"1\0\n\0\1\0 \0sazanami-*-*-*-*-*-*-*-ISO8859-1", 40}], 1) = 40 writev(3, [{"1\0\v\0\1\0\"\0sazanami-*-*-*-*-*-*-*-*-ISO8859-1\0\0", 44}], 1) = 44 writev(3, [{"1\0\v\0\1\0$\0sazanami-*-*-*-*-*-*-*-*-*-ISO8859-1", 44}], 1) = 44 writev(3, [{"1\0\f\0\1\0&\0sazanami-*-*-*-*-*-*-*-*-*-*-ISO8859-1\0\0", 48}], 1) = 48 writev(3, [{"1\0\f\0\1\0(\0sazanami-*-*-*-*-*-*-*-*-*-*-*-ISO8859-1", 48}], 1) = 48 writev(3, [{"1\0\r\0\1\0*\0sazanami-*-*-*-*-*-*-*-*-*-*-*-*-ISO8859-1\0\0", 52}], 1) = 52 writev(3, [{"1\0\6\0\1\0\20\00016 mincho-medium", 24}], 1) = 24 writev(3, [{"1\0\t\0\1\0\32\00016 mincho-medium-ISO8859-1\0\0", 36}], 1) = 36 writev(3, [{"1\0\t\0\1\0\34\00016 mincho-medium-*-ISO8859-1", 36}], 1) = 36 writev(3, [{"1\0\n\0\1\0\36\00016 mincho-medium-*-*-ISO8859-1\0\0", 40}], 1) = 40 writev(3, [{"1\0\n\0\1\0 \00016 mincho-medium-*-*-*-ISO8859-1", 40}], 1) = 40 writev(3, [{"1\0\v\0\1\0\"\00016 mincho-medium-*-*-*-*-ISO8859-1\0\0", 44}], 1) = 44 writev(3, [{"1\0\v\0\1\0$\00016 mincho-medium-*-*-*-*-*-ISO8859-1", 44}], 1) = 44 writev(3, [{"1\0\f\0\1\0&\00016 mincho-medium-*-*-*-*-*-*-ISO8859-1\0\0", 48}], 1) = 48 writev(3, [{"1\0\f\0\1\0(\00016 mincho-medium-*-*-*-*-*-*-*-ISO8859-1", 48}], 1) = 48 writev(3, [{"1\0\r\0\1\0*\00016 mincho-medium-*-*-*-*-*-*-*-*-ISO8859-1\0\0", 52}], 1) = 52 writev(3, [{"1\0\r\0\1\0,\00016 mincho-medium-*-*-*-*-*-*-*-*-*-ISO8859-1", 52}], 1) = 52 writev(3, [{"1\0\16\0\1\0.\00016 mincho-medium-*-*-*-*-*-*-*-*-*-*-ISO8859-1\0\0", 56}], 1) = 56 writev(3, [{"1\0\16\0\1\0000\00016 mincho-medium-*-*-*-*-*-*-*-*-*-*-*-ISO8859-1", 56}], 1) = 56 writev(3, [{"1\0\6\0\1\0\16\0verdana-medium\0\0", 24}], 1) = 24 writev(3, [{"1\0\10\0\1\0\30\0verdana-medium-ISO8859-1", 32}], 1) = 32 writev(3, [{"1\0\t\0\1\0\32\0verdana-medium-*-ISO8859-1\0\0", 36}], 1) = 36