Bug 962558

Summary: invalid instruction in pthread_rwlock_unlock()
Product: [openSUSE] openSUSE Tumbleweed Reporter: Cyril Hrubis <chrubis>
Component: BasesystemAssignee: E-mail List <bnc-team-screening>
Status: RESOLVED INVALID QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: schwab
Version: Current   
Target Milestone: ---   
Hardware: x86-64   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Cyril Hrubis 2016-01-19 11:29:13 UTC
Found by LTP test pthread_rwlock_unlock_4-2.

The what the testcase does is:

* Main thread initializes rwlock and locks it for reading

* New thread is created which tries to unlock the lock
  which is expected to yield EPERM

Now this is technically undefined to but even POSIX recommends to return with EPERM and it has worked fine in Linux beforehand.

Now the test ends with illegal instruction:

Dump of assembler code for function pthread_rwlock_unlock:
   0x00007ffff7bcab10 <+0>:     sub    $0x8,%rsp
   0x00007ffff7bcab14 <+4>:     mov    0x1c(%rdi),%esi
   0x00007ffff7bcab17 <+7>:     mov    0x18(%rdi),%r9d
   0x00007ffff7bcab1b <+11>:    mov    %rdi,%rdx
   0x00007ffff7bcab1e <+14>:    cmp    $0x1,%esi
   0x00007ffff7bcab21 <+17>:    sbb    %r8d,%r8d
   0x00007ffff7bcab24 <+20>:    not    %r8d
   0x00007ffff7bcab27 <+23>:    and    $0x80,%r8d
   0x00007ffff7bcab2e <+30>:    test   %r9d,%r9d
   0x00007ffff7bcab31 <+33>:    jne    0x7ffff7bcab48 <pthread_rwlock_unlock+56>
   0x00007ffff7bcab33 <+35>:    mov    0x4(%rdi),%edi
   0x00007ffff7bcab36 <+38>:    test   %edi,%edi
   0x00007ffff7bcab38 <+40>:    jne    0x7ffff7bcab48 <pthread_rwlock_unlock+56>
=> 0x00007ffff7bcab3a <+42>:    xend   
   0x00007ffff7bcab3d <+45>:    xor    %eax,%eax
   0x00007ffff7bcab3f <+47>:    add    $0x8,%rsp
   0x00007ffff7bcab43 <+51>:    retq

The test source can be seen at:

https://github.com/linux-test-project/ltp/blob/master/testcases/open_posix_testsuite/conformance/interfaces/pthread_rwlock_unlock/4-2.c

Glibc version (from rpm -qi glibc):

Version     : 2.22
Release     : 6.1
Architecture: x86_64
Comment 1 Cyril Hrubis 2016-01-19 11:36:39 UTC
And it looks like the pthread_rwlock_unlock() returned 0 previously (glibc 2.19) and fails with illegal instruction in glibc 2.21 as well. So it looks like it was undefined behavior previously as well.
Comment 2 Andreas Schwab 2016-01-19 11:52:30 UTC
As you already discovered, this is undefined.
Comment 3 Cyril Hrubis 2016-01-19 12:37:53 UTC
FYI: these testcases were disabled on Linux in:

https://github.com/linux-test-project/ltp/commit/8965f0dec3e8dfa7fe72823f65629c92ff7e1e48