Bug 711829

Summary: lseek broken with redirected output?
Product: [openSUSE] openSUSE 12.1 Reporter: Marcus Rückert <mrueckert>
Component: BasesystemAssignee: E-mail List <bnc-team-screening>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: aj, kukuk, matz, rguenther, ro
Version: Factory   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: the broken generated file
strace -s 1024 -o callback.func.strace.pipe.txt $(RUBY) $(srcdir)/mkcallback.rb | tee $@
strace -s 1024 -o callback.func.strace.redirect.txt $(RUBY) $(srcdir)/mkcallback.rb > $@
strace -s 1024 -o callback.func.strace.redirect.sp1.txt $(RUBY) $(srcdir)/mkcallback.rb > $@
backtraces of all lseeks

Description Marcus Rückert 2011-08-11 17:41:16 UTC
during the build of ruby some files get generated with ruby itself.
This also only happens with ruby 1.8.7p352. (see home:darix/ruby). we want to push out this version as update for sle11.

the snippet from the makefile looks like this:
[[[
cbtable.func: $(srcdir)/mkcbtable.rb ./dlconfig.rb
        @echo "Generating cbtable.func"
        @$(RUBY) $(srcdir)/mkcbtable.rb > $@
]]]

up to SLE 11 SP1 and even 11.3 to factory this worked nicely. on SP2 it creates garbage output. (see attached callback.func) the first line and parts of the 2nd line get written to the end of the file instead of the beginning. but all the rest of the file is then written correctly.

funnily ... when you change the makefile snippet to
[[[
cbtable.func: $(srcdir)/mkcbtable.rb ./dlconfig.rb
        @echo "Generating cbtable.func"
        @$(RUBY) $(srcdir)/mkcbtable.rb | tee $@
]]]

it works as expected.

Rudi thinks this might have something to do with the
lseek(1, 0, SEEK_CUR)

i will attach the strace for both variants.
Comment 1 Marcus Rückert 2011-08-11 17:42:28 UTC
Created attachment 445433 [details]
the broken generated file
Comment 2 Marcus Rückert 2011-08-11 17:43:12 UTC
Created attachment 445434 [details]
strace -s 1024 -o callback.func.strace.pipe.txt $(RUBY) $(srcdir)/mkcallback.rb | tee $@
Comment 3 Marcus Rückert 2011-08-11 17:43:37 UTC
Created attachment 445435 [details]
strace -s 1024 -o callback.func.strace.redirect.txt $(RUBY) $(srcdir)/mkcallback.rb > $@
Comment 4 Marcus Rückert 2011-08-11 18:16:25 UTC
Created attachment 445440 [details]
 strace -s 1024 -o callback.func.strace.redirect.sp1.txt $(RUBY) $(srcdir)/mkcallback.rb > $@
Comment 5 Richard Biener 2011-08-12 11:17:21 UTC
For reference we just saw

dup(1)                                  = 4
...
write(1, "static v...
...
lseek(4, 0, SEEK_SET)                   = 0
close(4)                                = 0
...
write(1, "rg11, long arg12, lon...

which looks like a bug in ruby.
Comment 6 Ruediger Oertel 2011-08-12 12:03:56 UTC
caused by a change in glibc: fclose does an lseek now.

http://sourceware.org/bugzilla/show_bug.cgi?id=12724

2011-05-13  Ulrich Drepper  <drepper@gmail.com>

        [BZ #12724]
        * libio/fileops.c (_IO_new_file_close_it): Always flush when
        currently writing and seek to current position when not.
        * libio/Makefile (tests): Add bug-fclose1.
        * libio/bug-fclose1.c: New file.


according to kukuk:
-> Michael Matz, please remove this change from the SP2 glibc.
Comment 7 Marcus Rückert 2011-08-12 12:12:10 UTC
Created attachment 445594 [details]
backtraces of all lseeks

according to the strace the last lseek on the dup-ed FD is causing the problem. the stack trace for the last lseek is:

[[[
#0  0x00002b619964b9f0 in lseek64 () from /lib64/libc.so.6
#1  0x00002b61995e6598 in _IO_new_file_close_it () from /lib64/libc.so.6
#2  0x00002b61995d91b0 in fclose@@GLIBC_2.2.5 () from /lib64/libc.so.6
#3  0x000000000043b2af in fptr_finalize (noraise=<optimized out>, fptr=<optimized out>) at io.c:2316
#4  rb_io_fptr_cleanup (fptr=0x775f60, noraise=2) at io.c:2339
#5  0x000000000043e0a8 in rb_io_fptr_finalize (fptr=0x775f60) at io.c:2352
]]]
Comment 8 Marcus Rückert 2011-08-12 13:13:00 UTC
Adding AJ for factory.
Comment 9 Michael Matz 2011-08-12 14:39:19 UTC
POSIX describes a very surprising behaviour here.  It seems when they
introduced this wording they didn't think at all about duped file descriptors.

Nevertheless also the current glibc behaviour doesn't work as specified by
POSIX.  And to make matters even worse they have proposed to change the
wording again to:

      the file offset of the underlying open file description shall be
      set to the file position of the stream if the stream is the active
      handle to the underlying file description.

See http://austingroupbugs.net/view.php?id=87#c838 and
http://austingroupbugs.net/view.php?id=87#c895 .

glibc doesn't currently implement this behaviour at all.
For SP2 I'm reverting this whole thing (SR 14154).  For factory I'd propose to
wait until POSIX is settled.
Comment 10 Michael Matz 2011-08-12 14:41:08 UTC
Therefore fixed for beta4.
Comment 11 Andreas Jaeger 2011-08-12 15:03:25 UTC
Michael, could you report this upstream, please?
Comment 12 Michael Matz 2011-08-12 15:25:34 UTC
Resetting product so it's accessible externally.
Comment 13 Andreas Jaeger 2011-08-14 11:00:00 UTC
glibc 2.13 does not contain the patch - and thus nothing to do for factory. I queued it for glibc 2.15 to check the situation then.