Bug 327839

Oh, and btw, should creating an image without a template (though not done here) work? As in, will it use its own template?

I'm also wondering if this is related to the fact that the second stage of outputting all depends, i.e. 

  yast2-users-2.15.36-22@i586
    Requires:
      yast2-2.15.58-6@i586 (installed)
      perl-Digest-SHA1-2.11-59@i586 (installed)
      libgcc42-4.2.1_20070724-13@i586 (installed)
      yast2-security-2.15.1-17@noarch (installed)
      yast2-core-2.15.12-2@i586 (installed)
      yast2-country-2.15.19-8@i586 (installed)
      libstdc++42-4.2.1_20070724-13@i586 (installed)
      yast2-pam-2.14.0-122@noarch (installed)
      yast2-perl-bindings-2.15.3-21@i586 (installed)
      yast2-ldap-client-2.15.12-29@noarch (installed)
      perl-X500-DN-0.29-13@i586 (installed)
    Required By:
      yast2-samba-server-2.15.7-49@noarch (installed)
      yast2-inetd-2.15.1-33@noarch (installed)
      yast2-mail-2.15.22-2@noarch (installed)
      yast2-sudo-2.15.3-78@noarch (installed)
      yast2-profile-manager-2.15.0-44@i586 (installed)

etc etc.. happens incredibly slowly. Like 2 lines a second. In the terminal it outputs _all_ the lines straight away. Is there something in YaST that slows down the output of the text? Because that stage alone seems to take 10 minutes or so, in comparison to 1 second in the shell. And, in the shell I _do_ have --logfile terminal on.

- Please attach the YaST logs from the time you started creating the imagefrom yast2 kiwi. 

- I didn't try that config.xml, but I can (tomorrow)

- Strange things from the screenshot: I do not know, but the output you see there is generated by kiwi itself, YaST doesn't do anything more than calling kiwi (for the exact command we must look into y2logs)

- Slow output: I haven't seen seen it being slow, I'll try to reproduce.

- Missing template: I'm not exactly sure what do you mean, actually config.xml is the main (and may be only) part of template YaST uses; I assume you imported it before creating the image.

Created attachment 174444 [details]
y2logs

> - I didn't try that config.xml, but I can (tomorrow)

That would be great.

> - Slow output: I haven't seen seen it being slow, I'll try to reproduce.

10 minutes was being a little too kind, in fact. It's more like 25 minutes :o

> - Missing template: I'm not exactly sure what do you mean, actually config.xml
is the main (and may be only) part of template YaST uses; I assume you imported
it before creating the image.

Of course, but I was just wondering if you are supposed to be able to create an image without specifying any template file at the top. I presume you are meant to be able to (since it doesn't suggest otherwise), because that seems broken too.. I could file a bug report for that too, if you like.

> Of course, but I was just wondering if you are supposed to be able to create an
> image without specifying any template file at the top. I presume you are meant
> to be able to (since it doesn't suggest otherwise), because that seems broken
> too.. I could file a bug report for that too, if you like.

Yes, it is supposed to work, the templates can be found at /usr/share/YaST2/data/product-creator/kiwi_templates. I juste tested it today, and it worked... so I'm looking forward to your (separate) report.

In your logs (at the very end of /var/log/YaST2/y2log file) you can find this lines:

2007-09-24 20:30:08 <1> opensuse(4582) [YCP] Kiwi.ycp:511 calling 'kiwi --root /tmp/YaST2-04582-mWbGsl/myphysical --prepare /tmp/YaST2-04582-mWbGsl/openSUSE-10.3-RC1 --logfile terminal'
2007-09-24 20:51:52 <1> opensuse(4582) [YCP] Kiwi.ycp:549 calling 'kiwi --create /tmp/YaST2-04582-mWbGsl/myphysical -d /home/francis/kiwi --logfile terminal'
2007-09-24 21:01:30 <3> opensuse(4582) [liby2] genericfrontend.cc(signal_handler):59 got signal 2 at YCP file Kiwi.ycp:499



This shows you where is the temporary configuration that is actually used for creating the image (/tmp/YaST2-04582-mWbGsl/openSUSE-10.3-RC1). It's in temporary directory, which is deleted after you close YaST module. So please, before you close YaST, copy the content of the temporary directory (which will be different in next run) out and try calling kiwi on this config from the command line to see if the speed will be the same. Attach also the config.xml from that directory (here /tmp/YaST2-04582-mWbGsl/openSUSE-10.3-RC1/config.xml) so we can look what's the difference to the original one (maybe I skipped some important keys while importing it).

Created attachment 174573 [details]
kiwi output

- I can confirm the "strange output" - see attachment for more info (the kiwi output) - I don't know what's up there but I hope nothing bad. Marcus, could you explain?

- I can confirm the slowness - yes, it's bad, I try to work on it. I expect to release a fix with the online update or on build service.

That's strange it seems the mkisofs call has failed (exit code != 0) but
the messages displayed doesn't point to an error. I'm sorry can't tell you
more. I have created an ISO today and it has worked

Hmm 

Created attachment 174579 [details]
config.xml

This is config.xml which my YaST instance uses to build the image. Looks like the only changes yast2 module modified in the imported file (see comment 0) were: limit type to primary one (no problem as we are building ISO now), omit "locale" setting in preferences section and "replaceable" status of repositories. Could it hurt?

Marcus, could you test with the config file above in comment 10?

Initializing image system on: /tmp/testroot...

stay tuned ;)

It created successfully. I was using the same file mentioned in comment #10
The image is rather big without compression:

   kiwi.pl -p /suse/ms/xxx/ --root /tmp/testroot
   kiwi.pl --create /tmp/testroot -d /tmp/

   ll /tmp/openSUSE-10.3-RC1.i686-1.3.0.iso -rw-r--r-- 1 root root 1.7G 2007-09-25 19:11 /tmp/openSUSE-10.3-RC1.i686-1.3.0.iso

but it works

The mkisofs call was:

   mkisofs -p KIWI-Team - http://kiwi.berlios.de -publisher SUSE LINUX Products GmbH, suse@suse.de -R -J -pad -joliet-long -sort /var/tmp/m_cd-I16381 -no-emul-boot -boot-load-size 4 -boot-info-table -b boot/i386/loader/isolinux.bin -c boot/i386/boot.catalog -hide boot/i386/boot.catalog -hide-joliet boot/i386/boot.catalog -o /tmp//openSUSE-10.3-RC1.i686-1.3.0.iso /var/tmp/m_cd-p16382 /tmp/kiwi-cdboot.X12040/kiwi-cdboot-11612/CD

==> couldn't reproduce any of the issues mentioned:

The issue doesn't happen for me when using KIWI directly too... it only happens with YaST. Since the bug seems very real to me, and I'm presuming you only tried it directly with kiwi, not with YaST Image Creator, I'll reopen it for now..

Jiri - should I open up another bug report to track the slowness issue, or is it ok just trackign it here?

Reopening...

Francis: yes, for the speed problem it would be better to open separate report, thanks.

Back to this one: the mkisofs issue _does_ happen when using kiwi directly for me with the file attached in comment 10. So the problem might be in my configuration (but it is just installed RC1, nothing special) or with the file itself which is modified by YaST and slightly different from the initial one from comment 0 (but this is also strange when it works for Marcus with exactly that file).

Francis, could you also try calling kiwi on the file modified by YaST, not on the original one? You should be able to find it in /var/lib/YaST2/product-creator/images/openSUSE-10.3-RC1

Marcus, as your test (comment 13) works and my not (with the same file, comment 10), maybe the only differnce in our setup are the repository paths again. Couldn't it be another problem of ftp source? (Sounds strange, as unrelated to mkisofs, but that's the only difference i can see...)

Yup, the problem still occurs if I use the config.xml as modified by YaST.

Created attachment 174723 [details]
kiwi logfile

...so it must be a kiwi bug somehow/somewhere I guess. Attaching my log file too.

mkisofs fails when I remove flags="unified" from the type section. If I leave it there, it works for me.

for me it has worked without flags="unified" too. Maybe this is a problem
of big files or too many files for mkisofs. The problem is I don't see
an error of mkisofs just the reordering information which is ok

So I can't reproduce it here, can someone try to check that again and
inspect the output data by mkisofs. The following patch could be applied
to /usr/share/kiwi/modules/KIWIImage.pm

Index: KIWIImage.pm
===================================================================
--- KIWIImage.pm        (revision 649)
+++ KIWIImage.pm        (working copy)
@@ -977,6 +977,12 @@
                $kiwi -> failed ();
                $kiwi -> error  ("Failed to create ISO image: $data");
                $kiwi -> failed ();
+
+               my $stop;
+               print "BREAK\n";
+               print "CALL: $CD/isolinux.sh $main::RootTree/CD $name\n";
+               chomp ($stop = <>);
+
                if (! -d $main::RootTree.$baseSystem) {
                        qx (rm -rf $main::RootTree);
                        qx (rm -rf $tmpdir);

it will stop the process in case of an error and print the command which
was used to call mkisofs. Simply recall the command and check if you can find
an error. If you type any key at the "BREAK" shell kiwi will proceed and
will cleanup the environment.

Thanks 

   

Created attachment 174787 [details]
stdout of isolinux.sh  call

BREAK
CALL: /usr/share/kiwi/image/isoboot/suse-10.3/cdboot/isolinux.sh /tmp/kiwi-cdboot.qm3881/kiwi-cdboot-3777/CD /tmp/openSUSE-10.3-RC1.i686-1.3.0.iso

Created attachment 174788 [details]
stderr output of isolinux.sh call

There's some error message at the end, but I don't understand what is wrong. Anyway - the whole file is stderr of "usr/share/kiwi/image/isoboot/suse-10.3/cdboot/isolinux.sh /tmp/kiwi-cdboot.qm3881/kiwi-cdboot-3777/CD /tmp/openSUSE-10.3-RC1.i686-1.3.0.iso"

The mkisofs command called from isolinux.sh was:

mkisofs -p KIWI-Team - http://kiwi.berlios.de -publisher SUSE LINUX Products GmbH, suse@suse.de -R -J -pad -joliet-long -sort /var/tmp/m_cd-em8177 -no-emul-boot                                -boot-load-size 4 -boot-info-table             -b boot/i386/loader/isolinux.bin                                 -c boot/i386/boot.catalog                               -hide boot/i386/boot.catalog           -hide-joliet boot/i386/boot.catalog -o /tmp/openSUSE-10.3-RC1.i686-1.3.0.iso /var/tmp/m_cd-cu8178 /tmp/kiwi-cdboot.qm3881/kiwi-cdboot-3777/CD

This is the problem. There are files with wrong encodings.
This is not a kiwi bug. It's a distro bug

Using IPW21001.FW;1 for  /tmp/kiwi-cdboot.qm3881/kiwi-cdboot-3777/CD/read-only-system/lib/firmware/ipw2100-1.3-i.fw (ipw2100-1.3-p.fw)
Incorrectly encoded string (n�on) encountered.
Possibly creating an invalid Joliet extension. Aborting.


This happens only with the "old" live system setup. Because in this case
there are files stored as files in the iso filesystem. With the new system
or if you use compression kiwi generates images which are loop mounted later.
With the old system we want to avoid yet another loop mount indirection and
therefore files are placed directly on the iso

I'm sorry I can't fix this

wrong close flag

Because the LATER and REMIND resolutions have been removed, the resolution of this bug has changed from REMIND to WONTFIX. If this bug needs to be reconsidered, reopen it and set a future "Target Milestone for Fix."

(In reply to comment #24)
> This is the problem. There are files with wrong encodings.
> This is not a kiwi bug. It's a distro bug
> 
> Using IPW21001.FW;1 for 
> /tmp/kiwi-cdboot.qm3881/kiwi-cdboot-3777/CD/read-only-system/lib/firmware/ipw2100-1.3-i.fw
> (ipw2100-1.3-p.fw)
> Incorrectly encoded string (n�on) encountered.
> Possibly creating an invalid Joliet extension. Aborting.

This is a message that definitely did never exist in mkisofs.

It is obvious that your problem is caused by the fact that
you don't use mkisofs but a defective fork....

Note that the defective fork "genisoimage" has a well known
bug that is related to UTF-8 file name encoding.

There is a simple way to avoid the problem: upgrade to original
software from:

ftp://ftp.berlios.de/pub/cdrecord/alpha/ 
 
http://cdrecord.berlios.de/ 
 
Note thet Suse even has related binary packages.