From: Marcel Holtmann <holtmann@redhat.com>
User-Agent: Thunderbird 1.5.0.7 (X11/20060913)
To: vendor-sec@lst.de
Subject: [vendor-sec] shmat hangs by simultaneous shmctl (embargoed)
Errors-To: vendor-sec-admin@lst.de
Date: Wed, 27 Sep 2006 17:42:51 +0200

[-- Anhang #1 --]
[-- Typ: text/plain, Kodierung: 7bit, Größe: 2,1K --]

Hi,

this got reported by a customer and we were first treating that like a
normal bug, but it actually has a security implication. It can cause
your complete system to deadlock. We assigned CVE-2006-4342 to it.

We believe that this is a RHEL3 only issue caused by one of our own
backports during the 2.5 days. However it might worth giving the
reproducer a try. This issue is embargoed until we release a hotfix
kernel with a fix for it in it. This is expected to be public really
soon and I can't do anything about it anymore. Sorry. However as said,
we don't expect anyone else to be vulnerable.

This is the description of the actual problem from on of our developers:


Deadlock happens on smp machines when a shmat() function runs at the
same time that shmctl() is used to remove the same shm. While the shmat
process waits at the spinlock in ipc_lock function, the shmctl function
deletes the path to the kern_ipc_perm structure which holds the spinlock
variable and frees the memory location. Because of this, the
ipc_unlock() run for shmctl() does not unlock the spinlock and the shmat
waits indefinitely at the spinlock.

This is serious issue for RHEL3 users especially applications using IPC
shared-memory. This deadlock brings other deadlock in IPC features (ex.
IPC semaphore) since deadlocked process holds readlock of
BR_SEMAPHORE_LOCK which are shared by all IPC features.

The code in question was backported from the 2.5 kernel with the patch
linux-2.4.21-semaphore-scale.patch by Ingo Molnar. This code does not
appear upstream in the 2.4 kernels. The Red Hat implementation on the
2.4 kernel is done using 'Big Reader' read-write locks. The patch was
written by modifying the code from the patch provided in bz # 91146 to
use brlocks. The attached reproducer recreates the issue.


I am also attaching the reproducer. Please keep in mind that this one
should be embargoed forever. It is not meant for the public.
- Compile shmat.c and shmctl.c
- Run both simultaneously on MP system
- Repeat running 

Once it happen, continuous error message from shmat will stop and the
two CPUs will be deadlocked.

Regards

Marcel