Bug 298716

Summary:	zypper is slow -- because it fetches unneeded patterns (for other archs)
Product:	[openSUSE] openSUSE 10.3	Reporter:	Gerald Pfeifer <gp>
Component:	libzypp	Assignee:	Duncan Mac-Vicar <dmacvicar>
Status:	RESOLVED FIXED	QA Contact:	Klaus Kämpf <kkaempf>
Severity:	Normal
Priority:	P5 - None	CC:	aj, coolo, ro
Version:	Alpha 7
Target Milestone:	---
Hardware:	All
OS:	openSUSE 10.2
Whiteboard:
Found By:	Product Management	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---
Attachments:	Proposed fix to parser/susetags/RepoParser.cc Proposed fix to repo/susetags/Downloader.cc

Description Gerald Pfeifer 2007-08-09 07:54:35 UTC

This is similar to Bugzilla #293740 which was addressed very quickly
-- thanks for that!

Using openSUSE Factory which is a combined tree, I found that libzypp
downloads all patterns in five flavors.

  rana:/var/cache/zypp/raw/factory/suse/setup/descr # ls -la xml*
  -rw-r--r-- 1 root root 3460 2007-08-07 15:05 xml-10.3-96.i586.pat
  -rw-r--r-- 1 root root 3458 2007-08-07 15:05 xml-10.3-96.ppc.pat
  -rw-r--r-- 1 root root 3464 2007-08-07 15:05 xml-10.3-96.x86_64.pat
  -rw-r--r-- 1 root root  538 2007-08-07 15:05 xml-32bit-10.3-96.x86_64.pat
  -rw-r--r-- 1 root root  532 2007-08-07 15:05 xml-64bit-10.3-96.ppc.pat

Four out of these five are probably quite useless on my old 32-bit notebook,
and while it's not a huge amount of data, it is some.

And with 470 *.pat files this probably means that more than 350 TCP/IP and
HTTP/FTP connections are opened needlessly.

Comment 1 Klaus Kämpf 2007-08-09 08:04:23 UTC

Thats probably not zyppers fault but an ommission of the 'susetags' metadata in suse/setup/descr.

Pattern file names do not imply any semantics, they're just filenames.
The architecture is _inside_ the file.
Just like RPM packages.

We need to make this information available without reading the pattern files.
Either by
- define semantics for pattern filenames
- extend the 'suse/setup/descr/patterns' file to contain more information
  than just filenames

Comment 2 Klaus Kämpf 2007-08-09 08:05:47 UTC

Fixing this is probably not too hard (are certainly worth it) but needs some effort.

Stano ?

Comment 3 Stephan Kulow 2007-08-09 08:09:32 UTC

I would like to have these problems solved after GM - there are enough real problems still :)

Comment 4 Stanislav Visnovsky 2007-08-09 08:11:13 UTC

If autobuild people can adapt their mechanisms in time, I'm fine with this. Rudi?

However, I think sticking with pattern filename matching should be enough for 10.3. I know, this is suboptimal.

Comment 5 Andreas Jaeger 2007-08-09 08:19:52 UTC

I think pattern filename matching is the easy way - let's just skip non matching archs.

Comment 6 Forgotten User zOWss6Gs9u 2007-08-11 10:59:31 UTC

*** Bug 299549 has been marked as a duplicate of this bug. ***

Comment 7 Ruediger Oertel 2007-08-13 16:04:31 UTC

the semantics for pattern filenames has been inofficially defined, please
talk to the maintainers of the patterns* packages.
(as already explained to Klaus privately).

Comment 8 Klaus Kämpf 2007-08-16 10:55:29 UTC

Talked to 10.3 prjmgr (Coolo) now and he agreed on the pattern filename semantics.
This also doesn't need any changes in autobuild.

Comment 9 Klaus Kämpf 2007-08-16 12:51:09 UTC

Created attachment 157909 [details]
Proposed fix to parser/susetags/RepoParser.cc

Comment 10 Klaus Kämpf 2007-08-16 12:52:30 UTC

The proposed fix works fine on a x86_64 system with factory.
It should be tested on an i586/i686 system ( and then commited ;-))

Comment 11 Klaus Kämpf 2007-08-16 12:54:57 UTC

Created attachment 157911 [details]
Proposed fix to repo/susetags/Downloader.cc

This is a similar fix for another susetags parser. Its not used by zypper and I do not know why its there. See https://bugzilla.novell.com/show_bug.cgi?id=300985.

Comment 13 Klaus Kämpf 2007-08-16 15:33:13 UTC

fixed and submitted to stable.

Comment 14 Klaus Kämpf 2007-08-17 07:34:54 UTC

comment #11:
The Downloader code is used if the raw metadata is not on the client yet.
See Bug 301037 (remove repo leaves 'raw' metadata intact) why I didn't see use
of this code.