Bug 548605

Summary: update (installation?) very slow because of gconftool-2
Product: [openSUSE] openSUSE 11.2 Reporter: Harald Koenig <koenig>
Component: GNOMEAssignee: E-mail List <gnome-bugs>
Status: RESOLVED DUPLICATE QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P3 - Medium CC: mmeeks, sbrabec, vuntz
Version: RC 1   
Target Milestone: ---   
Hardware: x86-64   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Harald Koenig 2009-10-21 00:45:06 UTC
update took *very* long for some packages.  on the text console I noticed that gconftool-2 is run *very* often and it takes muuuuch time!

here is a typical output of "ps" 

 1296 23487 23487  1296 tty2     23487 R+       0   0:00  \_ ps wwwaxjf
    1  1297  1297  1297 tty5      1297 Ss+      0   0:00 bash -l
    1  1299  1299  1299 tty6      1299 Ss+      0   0:00 bash -l
    1  1300     0     0 ?           -1 S        0   0:00 /init
 1300  1301  1301  1301 tty1      1301 Ss+      0   0:00  \_ /bin/sh /sbin/inst_setup yast
 1301  1393  1301  1301 tty1      1301 S+       0   0:00      \_ /bin/sh /sbin/yast
 1393  2307  1301  1301 tty1      1301 S+       0   0:00          \_ /bin/sh /usr/lib/YaST2/startup/YaST2.call installation initial
 2307  2783  2783  2783 tty7      2783 Ss+      0   5:23              \_ Xorg -noreset -br -deferglyphs 16 vt07
 2307  2845  1301  1301 tty1      1301 Sl+      0  10:41              \_ y2base installation ("initial") qt --noborder --auto-fonts --fullscreen
 2845  4285  1301  1301 tty1      1301 S+       0   0:00                  \_ /usr/bin/perl -w /usr/lib/YaST2/servers_non_y2/ag_zypp_repos
 2845 22907  1301  1301 tty1      1301 S+       0   0:00                  \_ rpm --root /mnt --dbpath /var/lib/rpm -U --percent --force --nodeps -- /var/cache/zypp/packages/openSUSE 11.2-0/suse/x86_64/compiz-0.7.8-42.2.x86_64.rpm
22907 22946  1301  1301 tty1      1301 S+       0   0:00                      \_ /bin/sh /var/tmp/rpm-tmp.VShO5x 1
22946 23486  1301  1301 tty1      1301 D+       0   0:03                          \_ usr/bin/gconftool-2 --makefile-uninstall-rule etc/gconf/schemas/compiz-screenshot.schemas

the time stamps in "rpm -qa --last" output show that just the installation of this one compiz.rpm took almost 6 minutes!!!
from 18:30:05 to 18:35:51 :-(

	cheese-2.28.0.1-1.3                           Tue Oct 20 18:35:51 2009
==>	compiz-0.7.8-42.2                             Tue Oct 20 18:30:05 2009
	devhelp-2.28.0.1-1.2                          Tue Oct 20 18:29:21 2009


this single rpm-tmp.VShO5x script calls gconftool-2 34 times. I've copied some of these rpm-tmp scripts if you need an example:

harald > grep -c "test -x usr/bin/gconftool-2" rpm-tmp.* | sort -t: +1n
rpm-tmp.6SNBCi:2
rpm-tmp.XbFxqJ:3
rpm-tmp.Pzb0eE:7
rpm-tmp.4HBLS5:11
rpm-tmp.7VmIVK:16
rpm-tmp.9HY48I:24
rpm-tmp.VShO5x:34



here is a postprocessed output of "rpm -qa --last" which shows the delta-time for every rpm in seconds (1st column).

those 50 "slowest" rpms sum up to 4242 seconds -- 70 minutes for 50 RPMs :-(

 346    compiz-0.7.8-42.2
 323    libgnome-2.28.0-1.2
 282    gnucash-2.2.9-2.3
 280    compiz-fusion-plugins-main-0.7.8-13.2
 255    evolution-2.28.0-1.2
 243    gnome-games-2.28.0-1.3
 164    gnome-applets-2.28.0-1.2
 148    gnome-panel-2.28.0-2.2
 128    gnome-utils-2.28.0-1.2
 114    evince-2.28.0-1.3
 103    gnome-settings-daemon-2.28.0-1.2
 102    gnome-vfs2-2.24.1-2.30
  83    kernel-xen-2.6.31.3-1.1
  82    DeviceKit-disks-007-1.2
  75    linux-kernel-headers-2.6.31-2.4
  73    epiphany-2.28.0-1.2
  66    totem-2.28.1-1.4
  66    gnome-media-2.28.0-1.3
  63    texlive-2008-12.11
  60    kernel-source-2.6.31.3-1.1
  57    gnopernicus-1.1.2-282.15
  54    nautilus-2.28.0-1.2
  48    gedit-2.28.0-1.2
  47    vino-2.28.0-1.2
  47    gnome-control-center-2.28.0-1.4
  47    OpenOffice_org-3.1.1.3-1.1
  44    devhelp-2.28.0.1-1.2
  43    planner-0.14.4-1.44
  43    gnome-main-menu-0.9.13-2.1
  42    libgnomedb-3.99.7-4.15
  42    gstreamer-0_10-plugins-good-0.10.15-2.15
  41    gnome-vfs2-devel-2.24.1-2.30
  40    seahorse-2.28.0-1.3
  39    pidgin-2.6.2-2.3
  39    gnome-power-manager-2.28.0-1.3
  37    tomboy-1.0.0-1.3
  37    gnome-packagekit-2.27.92-4.1
  36    gnome-system-monitor-2.28.0-1.2
  36    ekiga-3.2.6-1.4
  36    deskbar-applet-2.28.0-1.4
  35    eog-2.28.0-1.2
  35    brasero-2.28.1-1.1
  34    yelp-2.28.0-1.2
  34    gnome-bluetooth-2.28.1-1.2
  33    nautilus-cd-burner-2.25.3-3.24
  33    metacity-2.28.0-1.2
  33    gnome-session-2.28.0-2.1
  33    boost-devel-1.39.0-2.3
  31    kdelibs3-3.5.10-31.12
  30    yast2-control-center-gnome-2.13.4-1.2


for comparision I add all *kernel* RPMs from that same table:
  83    kernel-xen-2.6.31.3-1.1
  75    linux-kernel-headers-2.6.31-2.4
  60    kernel-source-2.6.31.3-1.1
  24    kernel-desktop-2.6.31.3-1.1
   1    patterns-openSUSE-devel_kernel-11.2-19.1
   1    kernel-firmware-20090821-3.1
   0    nfs-kernel-server-1.1.3-20.1
Comment 1 Harald Koenig 2009-11-18 16:44:59 UTC
I've done another update from 11.1 to 11.2-final: same performace problems with some gnome-related packages:

     rpm -qa --qf '%{INSTALLTIME} %{NAME} %{VERSION}\n' | sort -nr | awk '{print l-$1,$0;l=$1}' | sort -nr | less

shows the same "slow" packages:

   256  compiz-0.7.8
   240  compiz-fusion-plugins-main-0.7.8
   212  gnucash-2.2.9
   205  libgnome-2.28.0
   178  gnome-games-2.28.0
   142  evolution-2.28.0
   125  gnome-applets-2.28.0
   109  gnome-panel-2.28.0
    92  gnome-vfs2-2.24.1
    92  gnome-settings-daemon-2.28.0
    91  pullin-msttf-fonts-11.2
    83  gnome-utils-2.28.0
    81  evince-2.28.0
    80  timezone-java-2009p
    79  tvbrowser-2.7.2
    75  kernel-xen-2.6.31.5
    59  kernel-default-2.6.31.5


the compiz preuninstall script calls gconftool-2 34 times, while there
are ~250 calls alltogether (so  256 secs / 33 * 249 ~= 1931 secs just for gconftool-2 !)

	harald > rpm -q compiz  --scripts | grep -A2 ' if test -f usr/share/gconf/' | grep -c usr/bin/gconftool-2 
	33

	harald > rpm -qa  --scripts | grep -A2 ' if test -f usr/share/gconf/' | grep -c usr/bin/gconftool-2 
	249

which matches

	harald > ls /usr/share/gconf/schemas/*.schemas | wc
	    248     248   12515


gconftool-2  sucks ?!?
Comment 2 Jan Kupec 2009-11-19 19:29:31 UTC
Isn't this just the non-delayed suseconfig problem?

gconftool-2 sucks especially if it is executed for each gnome package being installed. openSUSE avoids this in CD installation (in YaST, more precisely) by setting YAST_IS_RUNNING env. variable, and packages are hacked to avoid runing suseconfig (which in turn runs gconftool). Then YaST executes suseconfig only once, at the end of installation.

We avoid doing this in zypper, because it is hack that the package management software needs to know about. Instead we encourage proper solution on the rpm level: see bug 365649, and https://features.opensuse.org/305472. (Although we could probably do this for dist-upgrade, until the feature is done).

In the meantime, use the workaround described at http://en.opensuse.org/Upgrade/Supported#Known_Issues (the one with YAST_IS_RUNNING) if needed (and vote for the feature in fate :O).

@zypp guys: anything on your mind we should do about this?

*** This bug has been marked as a duplicate of bug 365649 ***
Comment 3 Harald Koenig 2009-11-19 23:23:49 UTC
(In reply to comment #2)
> Isn't this just the non-delayed suseconfig problem?
> 
> gconftool-2 sucks especially if it is executed for each gnome package being
> installed. openSUSE avoids this in CD installation (in YaST, more precisely) by
> setting YAST_IS_RUNNING env. variable, and packages are hacked to avoid runing
> suseconfig (which in turn runs gconftool). Then YaST executes suseconfig only
> once, at the end of installation.

my update was started via boot from NET-CD and yast did the update (totally standard)!

YAST_IS_RUNNING is not evaluated at all in compiz scripts:
		
		rpm -q --scripts compiz | grep YAST_IS_RUNNING

"rpm -qa --scripts" shows YAST_IS_RUNNING being used for some SuSEconfig.fonts calls and other stuff, but I can't find any flagged calls to gconftool-2

am I missing something, or is that YAST_IS_RUNNING flag missing for gconftool-2 issues ?

> In the meantime, use the workaround described at
> http://en.opensuse.org/Upgrade/Supported#Known_Issues (the one with
> YAST_IS_RUNNING) if needed (and vote for the feature in fate :O).

thanks for this pointer, I'll keep it in mind for "zypper dup",
but in my case yast *was* running!
Comment 4 Jan Kupec 2009-11-20 10:09:11 UTC
Sorry, i was under impression that you did 'zypper dup'.

(In reply to comment #3)
> "rpm -qa --scripts" shows YAST_IS_RUNNING being used for some SuSEconfig.fonts
> calls and other stuff, but I can't find any flagged calls to gconftool-2

It could be executed via suseconfig. But then i wonder why the YAST_IS_RUNNING hack did not work.

> am I missing something, or is that YAST_IS_RUNNING flag missing for gconftool-2
> issues ?

Perhaps, we need to ask the maintainers of the pacakges.
Comment 5 Jan Kupec 2009-11-20 10:11:41 UTC
Stando, do you know something about this?
Comment 6 Stanislav Brabec 2009-11-20 12:23:47 UTC
Yes, I seen this problem as well.

I guess it is caused by the migration to merged tree in /etc/gconf/schema-install-source.

Slow installation is a cost for fast use.

I guess that measurements were done by Michael and Vincent. Do you have any ideas for improvement?

Could following work?:

- Install to multi-XML database.

- And then merge tree just once at the end.

Well, implementation of such behavior is again limited by the current RPM: lack of one-time actions: http://www.rpm.org/wiki/Problems/Scriptlets#Databaserebuild
Comment 7 Michael Meeks 2009-11-20 15:40:50 UTC
Well ... the real problem is the translations here; and merging them all one by one into the big, per-language, split out translation files.

The cost is proportional to the number of schemas, and can be substantially reduced if we merge lots of them at a time with a single gconftool-2 call.

So - some easy wins (last I looked) would be to eg. merge the umpteen, tiny, content-less schema files inside compiz into a single file ;-) prolly the same for gnome-games, and - if possible to merge several schemas at a time after the transaction (?).
Comment 8 Jan Kupec 2009-11-20 16:12:30 UTC
guys, please take this bug then :O)
Comment 10 Vincent Untz 2011-03-01 23:44:49 UTC
I'm going to mark this as dup of bug 546543, which is really the same bug.

*** This bug has been marked as a duplicate of bug 546543 ***