Bug 1010712 - zypper outputs invalid xml for package fetchmsttfonts
Summary: zypper outputs invalid xml for package fetchmsttfonts
Status: RESOLVED FIXED
Alias: None
Product: openSUSE Distribution
Classification: openSUSE
Component: libzypp (show other bugs)
Version: Leap 42.2
Hardware: x86-64 openSUSE 42.1
: P5 - None : Normal with 5 votes (vote)
Target Milestone: ---
Assignee: E-mail List
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-17 12:57 UTC by Forgotten User -z4SIGzSs7
Modified: 2017-04-05 16:14 UTC (History)
3 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
output of zypper --quiet --non-interactive --xmlout install --auto-agree-with-licenses fetchmsttfonts (14.83 KB, text/x-log)
2016-11-21 17:14 UTC, Forgotten User -z4SIGzSs7
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Forgotten User -z4SIGzSs7 2016-11-17 12:57:59 UTC
To reproduce: On Leap 42.1 or 42.2 run

zypper --quiet --non-interactive --xmlout install --auto-agree-with-licenses fetchmsttfonts

The xml this outputs is invalid, since it contains "<<" and ">>" with some tag.

The relevant part of the output is below.

While the generated xml is fairly non-standard anyhow, dumping the output of rpm without filtering breaks xml parsers (python's xml.dom.minidom in my example, although I tried other python xml parsers)

Relevant part of the output:

Running: fetchmsttfonts-11.4-19.22-fetchmsttfonts.sh.txt  (fetchmsttfonts, /var/adm/update-scripts)

note: No proxy is used. Please set the environment variable "http_proxy"
note: to your favorite proxy, if you want to use a proxy for the download.
note:
note:   bash: export http_proxy="http://proxy.example.com:3128/"
note:   tcsh: setenv http_proxy "http://proxy.example.com:3128/"
EULA:
  Fetching   ... done

Trying to find the fastest server:
 jaist ... 1 sec
 heanet ... 0 sec
 kent ... 1 sec
 nchc ... 1 sec
 easynews ... too slow (aborted)
 waix ... 1 sec
 internode ... 2 sec
 internap ... too slow (aborted)
The winner is: >> heanet <<
Comment 1 Michael Andres 2016-11-21 15:48:54 UTC
Sorry, but I can't reproduce it:

> <message type="info">
> note: No proxy is used. Please set the environment variable &quot;http_proxy&quot;
> note: to your favorite proxy, if you want to use a proxy for the download.
> note:
> note:   bash: export http_proxy=&quot;http://proxy.example.com:3128/&quot;
> note:   tcsh: setenv http_proxy &quot;http://proxy.example.com:3128/&quot;
> EULA:
>   Fetching   ... done
> 
> Trying to find the fastest server:
>  jaist ... 1 sec
>  heanet ... 0 sec
>  kent ... 1 sec
>  nchc ... 1 sec
>  easynews ... too slow (aborted)
>  waix ... 1 sec
>  internode ... 2 sec
> >  internap ... too slow (aborted)
> The winner is: &gt;&gt; heanet &lt;&lt;
> 
> </message>

The output as message type="info" is always XML escaped. So this would be a generic error.

Could you please attach your /var/log/zypp/history file. It should contain the original output. Maybe ther's some kinfd of garbage.
Comment 2 Forgotten User -z4SIGzSs7 2016-11-21 17:14:22 UTC
Created attachment 703038 [details]
output of zypper --quiet --non-interactive --xmlout install --auto-agree-with-licenses fetchmsttfonts
Comment 3 Forgotten User -z4SIGzSs7 2016-11-21 17:15:33 UTC
Thanks for looking into this!

This also shows up in the official docker image. To reproduce:
  docker run -it opensuse:42.2 /bin/bash  # (also 42.1)
then inside the container
  zypper --quiet --non-interactive --xmlout install --auto-agree-with-licenses fetchmsttfonts

Also in /var/log/zypp/history the >> << are not escaped.

I don't get the <message type="info"> tag around the output. Neither on the terminal, nor in /var/log/zypp/history. Attached is the full stdout of the above commands in docker (identical output on my physical machine).
Comment 4 Forgotten User -z4SIGzSs7 2016-11-21 17:33:49 UTC
This also happens in the tumbleweed and 13.2 docker images (in addition to 42.1 and 42.2 I already mentioned).

I played around the the command line options and the following show the same behavior:
zypper --quiet --non-interactive --xmlout install --auto-agree-with-licenses fetchmsttfonts
zypper -xmlout install fetchmsttfonts
zypper --verbose -xmlout install fetchmsttfonts

I guess the reason is the missing <message type="info"> tag, that does not show up around that section in any of the above commands. The tag appears on other places of the output though.

Please let me know if you need more details. I hope the docker images help with reproducibility.
Comment 5 Michael Andres 2016-11-22 12:56:03 UTC
(In reply to Robin Roth from comment #3)
> Also in /var/log/zypp/history the >> << are not escaped.
That's ok; zypp/history contains the 'raw' script output.

I'm now able to reproduce it. Will be fixes asap.
Comment 6 Michael Andres 2016-11-22 17:26:01 UTC
Fixed in
  zypper 1.13.14  SLE-12-SP2  Code-42_2  Factory
  zypper 1.12.46  SLE-12-SP1  Code-42_1
  zypper 1.11.58  SLE-12
Comment 7 Forgotten User -z4SIGzSs7 2016-11-23 07:29:34 UTC
Thanks for the quick fix!
Comment 8 Forgotten User -z4SIGzSs7 2016-11-28 09:28:05 UTC
Will this come as an update in 42.2? Is so, what is the ETA?
Comment 9 Andreas Stieger 2016-11-28 09:51:51 UTC
https://github.com/openSUSE/zypper/commit/58e97f2233891b4553fe7cd3921f5a50a770218c
https://github.com/openSUSE/zypper/commit/0e9c4cf1b020d553e79f3519eec0eec8610077a0
https://github.com/openSUSE/zypper/commit/fe8b42772d9b27024087b71a4a2f0d72464f2831

(In reply to Robin Roth from comment #8)
> Will this come as an update in 42.2?

Yes.

> Is so, what is the ETA?

There is no particular schedule for this. It will be included with the next important update for the package management stack.
Comment 10 Michael Andres 2016-11-28 10:01:31 UTC
JFYI: The package is already available in our zypp devel project (https://build.opensuse.org/project/show/zypp:SLE-12-SP2-Branch). 

I just made it build against openSUSE_Leap_42.2, so those (unofficial) packages will be available at http://download.opensuse.org/repositories/zypp:/SLE-12-SP2-Branch/openSUSE_Leap_42.2 within the next 24 hrs. If it's urgent for you and they fit your system, you can give them a try.
Comment 11 Swamp Workflow Management 2016-12-16 19:08:21 UTC
SUSE-RU-2016:3187-1: An update that has 12 recommended fixes can now be installed.

Category: recommended (low)
Bug References: 1003748,1004096,1010712,731333,964932,980263,980901,982379,983141,984494,986694,992302
CVE References: 
Sources used:
SUSE Linux Enterprise Server 12-LTSS (src):    libsolv-0.6.22-2.22.6.1, libzypp-14.44.1-2.53.5, zypper-1.11.59-2.49.1
Comment 12 Swamp Workflow Management 2017-01-04 15:08:56 UTC
SUSE-RU-2017:0019-1: An update that has 9 recommended fixes can now be installed.

Category: recommended (low)
Bug References: 1003748,1004096,1007273,1010712,1010952,1014265,731333,975777,975794
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP2 (src):    libsolv-0.6.24-2.25.3, libzypp-16.3.2-25.1
SUSE Linux Enterprise Server for Raspberry Pi 12-SP2 (src):    libsolv-0.6.24-2.25.3, libzypp-16.3.2-25.1, zypper-1.13.14-16.9
SUSE Linux Enterprise Server 12-SP2 (src):    libsolv-0.6.24-2.25.3, libzypp-16.3.2-25.1, zypper-1.13.14-16.9
SUSE Linux Enterprise Desktop 12-SP2 (src):    libsolv-0.6.24-2.25.3, libzypp-16.3.2-25.1, zypper-1.13.14-16.9
Comment 13 Swamp Workflow Management 2017-01-08 00:08:53 UTC
openSUSE-RU-2017:0060-1: An update that has 9 recommended fixes can now be installed.

Category: recommended (low)
Bug References: 1003748,1004096,1007273,1010712,1010952,1014265,731333,975777,975794
CVE References: 
Sources used:
openSUSE Leap 42.2 (src):    libsolv-0.6.24-3.1, libzypp-16.3.2-3.1, zypper-1.13.14-3.1
Comment 14 Swamp Workflow Management 2017-03-29 19:12:23 UTC
SUSE-RU-2017:0861-1: An update that has 11 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1007273,1010712,1014265,1024909,1025440,1028492,1030136,1030827,1030919,731333,926844
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP1 (src):    libsolv-0.6.26-2.39.1, libzypp-15.24.2-42.1
SUSE Linux Enterprise Server 12-SP1 (src):    libsolv-0.6.26-2.39.1, libzypp-15.24.2-42.1, zypper-1.12.50-40.3
SUSE Linux Enterprise Desktop 12-SP1 (src):    libsolv-0.6.26-2.39.1, libzypp-15.24.2-42.1, zypper-1.12.50-40.3
Comment 15 Swamp Workflow Management 2017-04-05 16:14:33 UTC
openSUSE-RU-2017:0938-1: An update that has 11 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1007273,1010712,1014265,1024909,1025440,1028492,1030136,1030827,1030919,731333,926844
CVE References: 
Sources used:
openSUSE Leap 42.1 (src):    libsolv-0.6.26-19.1, libzypp-15.24.2-19.2, zypper-1.12.50-19.1