Bug 549158

Summary: installation fails with "Copying the live image to hard disk failed"
Product: [openSUSE] openSUSE 11.2 Reporter: a b <alex.gorbachenko>
Component: YaST2Assignee: Jiri Srain <jsrain>
Status: RESOLVED FIXED QA Contact: Jiri Srain <jsrain>
Severity: Major    
Priority: P2 - High CC: aschnell, coolo, forgotten_cyMTVkUc77, mao.huanxue
Version: Final   
Target Milestone: ---   
Hardware: All   
OS: openSUSE 11.2   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: yast log
screenshot
yast installation logs

Description a b 2009-10-22 12:39:07 UTC
Created attachment 323696 [details]
yast log

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.1.3) Gecko/20090909 SUSE/3.5.3-3.2 Firefox/3.0.1 YB/3.5.1

bug in 11.2-rc1 (x86_64 and i386 version).

Reproducible: Always

Steps to Reproduce:
1. boot from livecd or usb stick (i use openSUSE-KDE4-LiveCD-Build0336-i686.iso on usb stick and openSUSE-KDE4-LiveCD-Build0336-x86_64.iso on cd);

2. in "suggested partitioning" chose "Partition Based" and press "Edit Partition Setup";

3. split disk into:
 - /boot - 128 mb;
 - swap;
 - / - 8 Gb;
 - /usr - 12 Gb;
 - /var - 4Gb or more;
 - /opt - 12 Gb;
 - /home - any size;

or any other sizes. prerequisite: creation of /usr and /var partition.

Actual Results:  
lots or free space on any partition, but installation always fail with "copying the image failed". 

results from my vbox machine:

linux@linux:~> df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/loop0            3.0G  2.3G  646M  79% /
udev                  504M  244K  504M   1% /dev
/dev/sda6              13G  330M   12G   3% /mnt
/dev/sda1             130M  7.4M  116M   7% /mnt/boot
/dev/sda9             2.0G   68M  1.9G   4% /mnt/opt
/dev/sda7             9.9G  2.2G  7.2G  23% /mnt/usr
/dev/sda8             5.0G  185M  4.5G   4% /mnt/var


very, very strange behaviour. if hdd split into:

/boot
swap
/
/home

all works fine. some problems with /usr or /var or both of them.
Comment 1 a b 2009-10-22 12:41:23 UTC
Created attachment 323697 [details]
screenshot
Comment 2 Arvin Schnell 2009-10-23 10:01:00 UTC
I cannot see a problem related to partitioning.
Comment 3 a b 2009-10-29 11:41:35 UTC
Created attachment 324663 [details]
yast installation logs
Comment 4 a b 2009-10-29 11:43:26 UTC
same problem with 11-rc2 (openSUSE-KDE4-LiveCD-Build0339-i686.iso).
Comment 5 mao huanxue 2009-11-03 14:02:52 UTC
x86_64 RC1 & RC2 LiveCD
I got this problem when trying to install the 11.2 RC1 with the LiveCD.
and I thought it was about the the disk problem, and I got the same error 
in the 11.2 RC2 too.
Copying the live image to hard disk failed.
I tested it on the VMware Workstation 7.0 (Trial Version) just now, the same 
error.
all the md5sum of the livecd image is correct.
should it be the problem of LiveCD's filesystem ?
I did not see the error in the reloaded livecd of 11.1 and 11.2 M2.
hope this can be solved before 11.2 releases.
Comment 6 Jiri Srain 2009-11-05 13:16:56 UTC
According to the log, the underlying command fails, however, I don't see its error message.

if you can reproduce that bug, once it occurrs and the system is still mounted, can you run following command?

mkfifo /tmp/YaST2-03169-la8Vgm/system_clone_fifo_1 ;
         mkfifo /tmp/YaST2-03169-la8Vgm/system_clone_fifo_2 ;
         tar -C / --numeric-owner -cSf /tmp/YaST2-03169-la8Vgm/system_clone_fifo_1 --one-file-system . &
         dd bs=1048576 if=/tmp/YaST2-03169-la8Vgm/system_clone_fifo_1 of=/tmp/YaST2-03169-la8Vgm/system_clone_fifo_2 >&2 &
         jobs -l >&2;
         tar -C /mnt --numeric-owner -xSf /tmp/YaST2-03169-la8Vgm/system_clone_fifo_2

and find out which process (and why) fails?
Comment 7 a b 2009-11-06 13:00:49 UTC
hmhm.

parallel execution of:

linux:~ # tar -C / --numeric-owner -cSf /tmp/YaST2-03200-Ik5EuY/system_clone_fifo_1 --one-file-system .
[skip]
tar: /tmp/YaST2-03200-Ik5EuY/system_clone_fifo_1: Wrote only 4096 of 10240 bytes
tar: Error is not recoverable: exiting now

and:

linux:~ # df -h && date
Filesystem            Size  Used Avail Use% Mounted on
/dev/loop0            3.0G  2.8G  195M  94% /
udev                  504M  248K  504M   1% /dev
/dev/sda6             2.0G  206M  1.7G  11% /mnt
/dev/sda1              69M  7.2M   58M  12% /mnt/boot
/dev/sda10             14G  167M   13G   2% /mnt/home
/dev/sda9             4.0G   72M  3.7G   2% /mnt/opt
/dev/sda7             6.0G  2.2G  3.5G  39% /mnt/usr
/dev/sda8             2.0G   82M  1.8G   5% /mnt/var
Fri Nov  6 15:36:30 MSK 2009
linux:~ # df -h && date
Filesystem            Size  Used Avail Use% Mounted on
/dev/loop0            3.0G  2.9G  134M  96% /
udev                  504M  248K  504M   1% /dev
/dev/sda6             2.0G  206M  1.7G  11% /mnt
/dev/sda1              69M  7.2M   58M  12% /mnt/boot
/dev/sda10             14G  167M   13G   2% /mnt/home
/dev/sda9             4.0G   72M  3.7G   2% /mnt/opt
/dev/sda7             6.0G  2.2G  3.5G  39% /mnt/usr
/dev/sda8             2.0G   82M  1.8G   5% /mnt/var
Fri Nov  6 15:36:49 MSK 2009
linux:~ # df -h && date
Filesystem            Size  Used Avail Use% Mounted on
/dev/loop0            3.0G  3.0G   16K 100% /
udev                  504M  248K  504M   1% /dev
/dev/sda6             2.0G  206M  1.7G  11% /mnt
/dev/sda1              69M  7.2M   58M  12% /mnt/boot
/dev/sda10             14G  167M   13G   2% /mnt/home
/dev/sda9             4.0G   72M  3.7G   2% /mnt/opt
/dev/sda7             6.0G  2.2G  3.5G  39% /mnt/usr
/dev/sda8             2.0G   82M  1.8G   5% /mnt/var
Fri Nov  6 15:37:58 MSK 2009

in order that fix this bug, size of /dev/loop0 must be greather than 3 Gb, right ?
Comment 8 Jiri Srain 2009-11-06 13:18:29 UTC
Did you run all the commands in parallel? That's strange, since fifo should be emptied countinuously by the other processes...
Comment 9 a b 2009-11-06 14:43:42 UTC
no :)

now, i run one more test:

linux:~ # mkfifo /tmp/YaST2-03136-f8bFS4/system_clone_fifo_1 ; mkfifo /tmp/YaST2-03136-f8bFS4/system_clone_fifo_2 ;

linux:~ # tar -C / --numeric-owner -cSf /tmp/YaST2-03136-f8bFS4/system_clone_fifo_1 --one-file-system . & dd bs=1048576 if=/tmp/YaST2-03136-f8bFS4/system_clone_fifo_1 of=/tmp/YaST2-03136-f8bFS4/system_clone_fifo_2 >&2 & jobs -l >&2; tar -C /mnt --numeric-owner -xSf /tmp/YaST2-03136-f8bFS4/system_clone_fifo_2                                                                       
[1] 8821
[2] 8822
[1]-  8821 Running                 tar -C / --numeric-owner -cSf /tmp/YaST2-03136-f8bFS4/system_clone_fifo_1 --one-file-system . &
[2]+  8822 Running                 dd bs=1048576 if=/tmp/YaST2-03136-f8bFS4/system_clone_fifo_1 of=/tmp/YaST2-03136-f8bFS4/system_clone_fifo_2 1>&2 &                                         

[skip]

tar: ./usr/share/zoneinfo/posix/W-SU: Cannot hard link to `./etc/localtime': Invalid cross-device link
tar: ./usr/share/zoneinfo/posix/Europe/Moscow: Cannot hard link to `./etc/localtime': Invalid cross-device link
tar: ./usr/share/zoneinfo/W-SU: Cannot hard link to `./etc/localtime': Invalid cross-device link
tar: ./usr/share/zoneinfo/Europe/Moscow: Cannot hard link to `./etc/localtime': Invalid cross-device link
2+71272 records in
2+71272 records out
2215915520 bytes (2.2 GB) copied, 442.381 s, 5.0 MB/s
tar: ./home/linux/.pulse: time stamp 2009-11-06 18:55:33 is 6472.239163428 s in the future
tar: ./home/linux/.kde4: time stamp 2009-11-06 18:55:28 is 6467.23890968 s in the future
tar: ./etc/sysconfig: time stamp 2009-11-06 18:54:20 is 6399.238626445 s in the future
tar: ./var/run: time stamp 2009-11-06 19:42:00 is 9259.238583706 s in the future
tar: .: time stamp 2009-11-06 18:53:57 is 6376.238490741 s in the future
tar: Exiting with failure status due to previous errors
[1]-  Done                    tar -C / --numeric-owner -cSf /tmp/YaST2-03136-f8bFS4/system_clone_fifo_1 --one-file-system .
[2]+  Done                    dd bs=1048576 if=/tmp/YaST2-03136-f8bFS4/system_clone_fifo_1 of=/tmp/YaST2-03136-f8bFS4/system_clone_fifo_2 1>&2


looks like a bug in timezone-2009m-2.2.i586 package:

linux@linux:~> rpm -q --scripts $(rpm -qf /etc/localtime)
postinstall scriptlet (using /bin/sh):
if [ -f /etc/sysconfig/clock ];
then
    . /etc/sysconfig/clock
    if [ -n "$TIMEZONE" -a -f /etc/localtime -a -f /usr/share/zoneinfo/$TIMEZONE ]; then
        new=$(mktemp /etc/localtime.XXXXXXXX) || exit 1
        cp -l /usr/share/zoneinfo/$TIMEZONE $new 2>/dev/null || cp -fp /usr/share/zoneinfo/$TIMEZONE $new
        mv -f $new /etc/localtime
    else
        [ ! -f /etc/localtime ] || echo "WARNING: Not updating /etc/localtime with new zone file" >&2
    fi
fi


"cp -l" make hardlinks instead of copies.
Comment 10 a b 2009-11-09 08:05:55 UTC
probably same bug in the "smolt" or "libzypp" pakage:

tar: ./etc/smolt/uuid: Cannot hard link to `./var/lib/zypp/AnonymousUniqueId'

hardlinks in live system is very, very bad idea. installation will always fails, because it is impossible to create cross-device hardlinks.
Comment 11 Jiri Srain 2009-11-09 12:00:05 UTC
Ah, now I see the problem, many thanks for this analysis.

I don't know whether there is any option for tar to solve that when un-tarring (I at least didn't find any), what can be done is to use --hard-dereference option for the tar command creating the archive (according to the documentation). This should ensure that hardlinks will be handled properly.

In general, hard links in live system make sense - they save the disk space, which is in case of CD very limited, however, causes a problem here.
Comment 12 a b 2009-11-13 08:42:02 UTC
crutches for 11.2 release installation:

1. boot into live system (kde or gnome);
2. run terminal emulator;
3. type "su -";
4. cd /usr/share/YaST2/modules;
5. open ImageInstallation.ycp in editor (vim ImageInstallation.ycp);
6. go to line 910;
7. add "--hard-derefernce" into tar options;
8. ycpc -c ImageInstallation.ycp;
9. run installation;
10. ...
11. PROFIT

patch:

--- ImageInstallation.ycp.orig  2009-11-13 08:21:03.150679080 +0000
+++ ImageInstallation.ycp       2009-11-13 08:21:34.172020411 +0000
@@ -907,7 +907,7 @@ global boolean FileSystemCopy (string fr
     cmd = sformat (
        "mkfifo %3 ;
         mkfifo %4 ;
-        tar -C %1 --numeric-owner -cSf %3 --one-file-system . &
+        tar -C %1 --hard-dereference --numeric-owner -cSf %3 --one-file-system . &
         dd bs=1048576 if=%3 of=%4 >&2 &
         jobs -l >&2;
         tar -C %2 --numeric-owner -xSf %4",
Comment 13 Jiri Srain 2009-11-19 10:02:02 UTC
Does that mean that with this change live installation works for you?
Comment 14 a b 2009-11-19 10:30:07 UTC
Yes
Comment 15 Jiri Srain 2009-12-09 09:29:55 UTC
Thanks for confirmation.
Coolo, do you plan to release updated live media for 11.2?
Comment 16 Stephan Kulow 2009-12-09 09:46:41 UTC
not now, but e.g. the kde team plans to release a spin off with updates.
Comment 17 Jiri Srain 2009-12-09 10:03:52 UTC
OK,  then I will fix it in SVN also in the 11.2 branch (apply patch from comment#12) and in case new release is planned, it can be grabbed.
Comment 18 Jiri Srain 2009-12-09 11:53:21 UTC
Patch applied in SVN for 11.3, 11.2 and SDK11-SP1.
Comment 19 Adam Spiers 2010-01-16 15:54:43 UTC
This one burnt me and I couldn't find anything preventing it from burning other users in the near future, so I added it to 

http://en.opensuse.org/Bugs:Most_Annoying_Bugs_11.2#Installation

and also added a link to that page from 

http://en.opensuse.org/OpenSUSE_11.2

I think we could be a bit better about communicating these kinds of issues.

(In reply to comment #16)
> not now, but e.g. the kde team plans to release a spin off with updates.

Does this mean the GNOME live image won't be fixed?  I saw the problem with both GNOME and KDE images.
Comment 20 Forgotten User cyMTVkUc77 2015-11-03 08:43:02 UTC
i affected currently by this bug in my installation of OpenSuse 13.2 edulife version.