Bug 600948

Summary: EXT4-fs error (device sda6): ext4_lookup: deleted inode referenced:
Product: [openSUSE] openSUSE 11.2 Reporter: Forgotten User --EoyBps8f <forgotten_--EoyBps8f>
Component: KernelAssignee: E-mail List <kernel-maintainers>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: erico.mendonca, jeffm, rcoe
Version: Final   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard: maint:released:11.3:35790
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Forgotten User --EoyBps8f 2010-04-29 14:35:29 UTC
User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.1.9) Gecko/20100317 SUSE/3.5.9-0.1.1 Firefox/3.5.9

Quite regularly a partition is not mounted (sda3 ext3) and the boot log shows:

Apr 25 18:04:50 linux-qija kernel: imklog 4.4.1, log source = /proc/kmsg 
started.
Apr 25 18:04:50 linux-qija kernel: [    9.598225] EXT4-fs error (device sda6): 
ext4_lookup: deleted inode referenced: 486
Apr 25 18:04:50 linux-qija kernel: [    9.601760] EXT4-fs error (device sda6): 
ext4_lookup: deleted inode referenced: 486

Jan 15 12:37:32 linux-qija kernel: [    9.018451] EXT4-fs error (device sda6): 
ext4_lookup: deleted inode referenced: 530
Jan 15 12:37:32 linux-qija kernel: [    9.023844] EXT4-fs error (device sda6): 
ext4_lookup: deleted inode referenced: 530
sda6 (ext4) is the partition that has / on it.

After one runs fsck.ext4 on it works again for a couple of times. I ran fsck.ext4 including badblocks read/write test on the partition and no errors where found.

This happens after normal shutdowns, i.e. no hard resets causing this.

I found quite some text about this issue via google, yet I am no expert so I cannot tell whether it really is the same issue. there seems to be a patch though.

https://bugzilla.kernel.org/show_bug.cgi?id=14354
http://marc.info/?l=linux-ext4&m=125663763907323&w=2
https://bugzilla.kernel.org/show_bug.cgi?id=13201
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/528981


Reproducible: Sometimes
Comment 1 Jeff Mahoney 2010-05-04 21:48:19 UTC
Are you running the latest update kernel? The patch in http://marc.info/?l=linux-ext4&m=125663763907323&w=2 was included in 2.6.31.8.
Comment 2 Forgotten User --EoyBps8f 2010-05-05 17:18:36 UTC
Linux linux-qija 2.6.31.12-0.2-desktop #1 SMP PREEMPT 2010-03-16 21:25:39 
+0100 x86_64 x86_64 x86_64 GNU/Linux

Is the kernel on that system.
Comment 3 Erico Mendonca 2010-07-01 13:47:53 UTC
Could it be related to this? 
https://bugzilla.kernel.org/show_bug.cgi?id=16165

It's definitely affecting people using ext4 on OpenSUSE, including me:

<pre>
erico@docmobile:~/Download> ./aio_corrupt  testdata
Success!
erico@docmobile:~/Download> rm testdata
erico@docmobile:~/Download> ./aio_corrupt  testdata
Corrupted buffer!

</pre>

I'm using the exact same kernel version as Sven noted above:

<pre>
Linux docmobile 2.6.31.12-0.2-desktop #1 SMP PREEMPT 2010-03-16 21:25:39 +0100 x86_64 x86_64 x86_64 GNU/Linux
</pre>
Comment 4 Forgotten User --EoyBps8f 2010-07-01 14:44:38 UTC
Could the patches from the kernel bug report mentioned above please be included into some test kernel package for opensuse?
Comment 5 Erico Mendonca 2010-07-05 14:05:55 UTC
I started adapting the patches against (our) kernel version to test it. Could take a while, the original patches don't apply cleanly on 2.6.31.12-0.2.
Comment 6 Erico Mendonca 2010-07-07 20:45:54 UTC
It seems to depend on a few specific 2.6.34 structures, will be easier to apply it to a newer kernel.

BTW, I just tried kernel 2.6.34-12-desktop from 11.3-RC2, and the problem is still there...
Comment 8 Jeff Mahoney 2010-09-07 16:12:03 UTC
I've just applied the patches related to bko#16165 to the openSUSE 11.3 branch. 
The next kernel pushed out to the repo will have the fixes.

aio-dio-hole-filling-race from xfstests will trigger this issue reliably on the unpatched kernel and I am unable to reproduce with the patched kernel.

-------------------------------------------------------------------
Tue Sep  7 14:53:16 CEST 2010 - jeffm@suse.com

- patches.fixes/direct-io-move-aio_complete-into-end_io:
  direct-io: move aio_complete into ->end_io (bnc#600948).
- patches.fixes/ext4-move-aio-completion-after-unwritten-extent-conversion:
  ext4: move aio completion after unwritten extent conversion
  (bnc#600948).
- patches.fixes/xfs-move-aio-completion-after-unwritten-extent-conversion:
  xfs: move aio completion after unwritten extent conversion
  (bnc#600948).
- patches.kabi/direct-io-move-aio_complete-into-end_io-kabi:
  KABI: Workaround dio_iodone_t prototype change (bnc#600948).
Comment 9 Forgotten User --EoyBps8f 2010-09-07 16:38:49 UTC
So 11.2 will not get these fixes because it is in security maintenance mode and those are non-security fixes?
Comment 10 Jeff Mahoney 2010-09-07 17:10:09 UTC
Yes. There was also a decent amount of work involved in ensuring that the kABI wouldn't be broken by the changes. Otherwise it'd be easy to just tack them on.
Comment 11 Jan Kara 2010-09-08 13:04:39 UTC
*** Bug 618804 has been marked as a duplicate of this bug. ***
Comment 12 Swamp Workflow Management 2010-09-17 14:30:03 UTC
Update released for: kernel-debug, kernel-debug-base, kernel-debug-base-debuginfo, kernel-debug-debuginfo, kernel-debug-debugsource, kernel-debug-devel, kernel-debug-devel-debuginfo, kernel-default, kernel-default-base, kernel-default-base-debuginfo, kernel-default-debuginfo, kernel-default-debugsource, kernel-default-devel, kernel-default-devel-debuginfo, kernel-desktop, kernel-desktop-base, kernel-desktop-base-debuginfo, kernel-desktop-debuginfo, kernel-desktop-debugsource, kernel-desktop-devel, kernel-desktop-devel-debuginfo, kernel-devel, kernel-pae, kernel-pae-base, kernel-pae-base-debuginfo, kernel-pae-debuginfo, kernel-pae-debugsource, kernel-pae-devel, kernel-pae-devel-debuginfo, kernel-source, kernel-source-vanilla, kernel-syms, kernel-trace, kernel-trace-base, kernel-trace-base-debuginfo, kernel-trace-debuginfo, kernel-trace-debugsource, kernel-trace-devel, kernel-trace-devel-debuginfo, kernel-vanilla, kernel-vanilla-base, kernel-vanilla-base-debuginfo, kernel-vanilla-debuginfo, kernel-vanilla-debugsource, kernel-vanilla-devel, kernel-vanilla-devel-debuginfo, kernel-xen, kernel-xen-base, kernel-xen-base-debuginfo, kernel-xen-debuginfo, kernel-xen-debugsource, kernel-xen-devel, kernel-xen-devel-debuginfo, preload-kmp-default, preload-kmp-desktop
Products:
openSUSE 11.3 (debug, i586, x86_64)