Subject: [vendor-sec] X.Org server exposes kernel vulnerability
Date: Sun, 20 Jun 2010 22:29:30 +0200
From: Matthieu Herrb <matthieu.herrb@laas.fr>
To: vendor-sec@lst.de

Hi,

xorg-security got the attached vulnerability report from  Rafal Wojtczuk
last week.

A short analysis shows that the problems looks like a generic kernel
vulnerability that didn't get much attention 5 years ago when the
initial paper was published at CanSecWest 2005. But Rafal found out that
the X server has a number of properties that make it a good vector to
exploit that.

Rafal suggests that setting RLIMIT_AS in the X server to 1.5GB on
x86_32 and to something like 10GB on 64 arches should be enough to fix
the issue.

He agrees to keep that under embargo  until august 1st.
Please keep him CC'ed.

there's no CVE id allocated for that afaik.

--- Cut ---
Xorg server is prone to large memory management vulnerabilities

1) Summary

        A malicious authenticated client can force Xorg server to fill all
        of its address space. This may result in the process stack being in an
unexpected region and execution of arbitrary code with server privileges
(root). Both x86_32 and x86_64 platforms are affected, at least when
running on Linux.

        Note that depending on the system configuration, by default local
unprivileged users may be able to start an instance of Xorg server that
requires no authentication and exploit it. Also if an attacker exploits
a vulnerability in a GUI application (e.g. web browser), he will have
ability to attack X server.

        In case of a local attacker that can use MIT-SHM extension, the
exploit is very reliable.

2) Affected and patched software versions
...

3) Attack scenario

        The class of vulnerabilities mentioned in the title is described by
Gael Delalleu at [1]. Applying this paper to X server case, an attacker
can instruct Xorg server to allocate many large pixmaps. This may result
in the stack and mmapped memory regions becoming close, and allows all
the tricks described in the Gael's paper.

        In fact, X server case is special, because of MIT-SHM extension.
        Local attacker can almost completely exhaust X server's address apace, then
create a shared memory segment S and force X server to attach it at the
only available region left, which will be close above the stack. Then
attacker instructs X server to call a recursive function, which results
in the stack being extended and the stack pointer being moved to S for a
brief period of time (during recursion). Attacker can then write to S;
this will overwrite the stack locations and allow arbitrary code
execution. So, unlike in Gael's paper, we will not need to trigger data
structures corruption by expanding the stack; we will be able to write
to the expanded stack directly, which makes this attack 100% reliable
(and somehow unique).

        Without MIT-SHM, it is possible for an attacker to allocate a pixmap
and have X server place the stack top there for a moment, but when
attacker instructs X server to write to this pixmap, the stack is
already in a safe location (because the recursing function has already
completed).

Attack scenario (in case of Linux):
Step 1:
In case of x86_64 platform, instruct X server to allocate as many
32000x32000 pixmaps (largest allowed by X) as possible. Note that on
x86_64 platform, not all 64bit address space is available - legal
addresses must be canonical (top 16bits must be all 0 or all 1), so we
have only 49bits address space. Linux and X have their additional
restrictions on the mmap return value; as a result, ca 36000 pixmaps
exhaust all address space. Note that X server does not initialize pixmap
contents (just reserves a VMA for it);
my tests showed that only about 800MB of RAM is needed for this step.
Again, this step is not necessary in x86_32 case.
Step 2:

shm_seg_size=1<<24
while shm_seg_size >= 4096
        shm_seg=shmget(..., shm_seg_size,...)
        have X attach shm_seg
        if XShmAttach fails, then shm_seg_size/=2
done
Similarly, the action of creating and attaching a shared memory segment
require resident memory only for the control structures, not for all
segment content; thus little RAM is required.
On Linux, the maximum number of shm segments is limited by shmmni kernel
variable (exported in /proc/sys/kernel/shmmni). By default it is 4096,
and that is why in case of a 64bits platform we need to shrink the
available address space by allocating pixmaps first.
Step 3:
Allocate windows arranged so that when X processes them, some function F
is called recursively. Trigger F recursion.
Step 4:
Find a shm segment S that has nonzero content. Nonzero content means X
stack has been resident in this segment during F recursion.
Step 5:
Spawn a process W that continuously overwrites the bottom page of S with
custom payload
Step 6:
Trigger F recursion. When one of Fs returns, it will pick the return
address from our payload. It is a race (W must write to stack after F
has placed its return address there, but before F returns), but reliably
winnable.

4) The fix
[probably prefault stack pages and place a guard PROT_NONE page above
it. Not perfect, maybe something else will pop up]

5) Other notes
        The attack has been reproduced on Fedora 13 default install, both 32
and 64bits. Local users (say, logged in via ssh) can run "Xorg :1" and
attack this process. SELinux in the enforcing mode neither prevents
exploitation nor limits the executed code capabilities - it is
impossible to sandbox a process that requires iopl privileges. OpenBSD
privilege-separated X server may resist root compromise; it was not
verified.
        On Fedora, Xorg executable base is not randomized, so we may happily
return into locations in Xorg executable (they are at constant
addresses): into execl@got in case of x86_32, or into middle of
os/utils.c!System in case of x86_64. If Xorg executable base was
randomized (PIE executable), its base would leak in the stack content,
so it would not help, either.

        If an attacker is not local (or cannot use MIT-SHM), the attack is
still possible - expanded stack may overwrite other data structures.
This has not been researched, and probably would be much less reliable.

        In case of Qubes [2] architecture, X server maps pages from
        untrusted VMs, so its address space can be attacked, too. The difference
with a vanilla Linux+Xorg installation is that untrusted parties (VMs) do
not talk to X server directly: the access is proxied by a qubes_guid
process, and very limited set of actions are actually executed on a real
X server (BTW, it prevents all attacks against X server known so far).
The attack described here is unique, as it potentially can be conducted
by only creating windows, which obviously must be allowed. However, one
of the proactive safety measures
implemented in qubes_guid since the very beginning was to detect the
suspiciously high number of allocations and ask the user (in dom0) for
permission to continue. It prevents the attack.

6) Credits
Rafal Wojtczuk <rafal@invisiblethingslab.com>

7) References
[1] Gael Delalleu, "Large memory management vulnerabilities",
http://cansecwest.com/core05/memory_vulns_delalleau.pdf
[2] Qubes OS project, http://www.qubes-os.org