Bug 702363

Summary: Tumbleweed - /var/lock/subsys is missing
Product: [openSUSE] openSUSE 11.4 Reporter: Marco Röben <mroeben>
Component: BasesystemAssignee: Greg Kroah-Hartman <gregkh>
Status: RESOLVED WONTFIX QA Contact: E-mail List <qa-bugs>
Severity: Critical    
Priority: P5 - None CC: mike
Version: Final   
Target Milestone: ---   
Hardware: x86-64   
OS: openSUSE 11.4   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Marco Röben 2011-06-27 12:52:54 UTC
User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1

Installing the Tumbleweed packages is modifying the startup process somehow, that some services (cifs, lm_sensors, nfc-rpc.idmap, syslogd) are needing the directory /var/lock/subsys. But the directory is not created during the start of the system. Manually creating the directory after startup and re-starting the services works, but it is deleted on every reboot.

Maybe its related to the script in /etc/init.d/boot.localfs where a /run directory and several bind mounts are defined. Change to systemd???

Reproducible: Always

Steps to Reproduce:
1. install tumbleweed
2. reboot
3.
Actual Results:  
processes are failing during boot

Expected Results:  
processes should succeed to start

No additional information, but if it is really related to a change to systemd, please wait until the change has been successfully be done in OS12.1. There should only go stable stuff into tumbleweed, otherwise there's no need to have tumbleweed in addition to factory.
Comment 1 Marcus Meissner 2011-06-27 14:20:00 UTC
deinstall systemd I guess.
Comment 2 Marco Röben 2011-06-27 16:05:22 UTC
I didn't installed it or at least on purpose. I have to check if it's really installed. Then the question remains why it is installed without notifying the user? My point is: Please, only do these kind of changes if it is known to be working. Otherwise tumbleweed is as dangerous as factory and there's no need for tumbleweed.
Comment 3 Greg Kroah-Hartman 2011-06-27 16:37:28 UTC
You must have installed it sometime on your own, or something else installed
it as it was not part of the default 11.4 install image.

Just uninstall it and you should be fine.

But note, I can not reproduce this problem at all, with systemd installed
on my Tumbleweed test systems, or my main development machines, so I do
not understand what is causing the problem to happen.

As for "dangerous", using Tumbleweed can cause minor problems like this at times,
that's par for the course here.  A simple uninstall of a package solves this one,
or, if you want to help out in debugging the root cause here, that would be great, reopen the bug with whatever you can find as again, I can't duplicate this
here to try to resolve it myself.
Comment 4 Marco Röben 2011-06-28 07:41:50 UTC
As I guessed, systemd is not installed.

zypper se sysemd
Daten des Repositorys laden ...
Installierte Pakete lesen...
                                         
S | Name             | Zusammenfassung                 | Typ
--+------------------+---------------------------------+-----------               
  | systemd          | A System and Session Manager    | Paket
  | systemd          | A System and Session Manager    | Quellpaket
  | systemd-gtk      | Graphical front-end for systemd | Paket
  | systemd-sysvinit | System V init tools             | Paket


And here are some additional information from the german opensuse mailinglist, where someone also stumbled about this. I am trying to translate.

The package aaa_base is creating a tmpfs-filesystem in /run and tries to bind-mount /var/run and /var/lock. Somehow the directories inside these two are ignored. There should be a workaround to delete these bind-mounts in /etc/init.d/boot.localfs.

This I haven't tried since I am not sure where these are defined exactly and I cannot risk to end up with a completely broken system (at least not today. :-))


For the german speaking people, here is the original text:

"ich bin da auch gerade drüber gestolpert. Das in Tumbleweed enthaltene Paket 
aaa_base legt ein tmpfs-Dateisystem unter /run an und richtet dann 
entsprechende bind-Mounts unter /var/run und /var/lock ein. Die per Paket-
Installation eingerichteten Verzeichnisse unterhalb /var/run und /var/lock 
werden dabei vollständig ignoriert und das führt dann dazu, dass diverse 
Startskripte aufgrund der fehlenden Verzeichnisse auf die Nase fallen. Um das 
System wieder vernünftig zum Laufen zu bringen, muss man die bind-Mounts in 
/etc/init.d/boot.localfs auskommentieren."
Comment 5 Michael Buchau 2011-06-28 18:43:55 UTC
This is not an issue with systemd per se. The culprit is the package aaa_base-11.5-91.1. Marcus probably assumed that it got released to tumbleweed because of some dependency with systemd.

If there is no dependency, just remove aaa_base from tumbleweed and all is well.

If not, the package should be modified to not do bind mounts of the tmpfs /run on /var/run and /var/lock by default, since that breaks a lot of packages depending on the directory structure below /var/run and /var/lock. Just make it configurable through some switch in /etc/sysconfig defaulting to off - at least until all packages depending on the subdirectories below /var/run and /var/lock have been modified accordingly and have also been added to tumbleweed.
Comment 6 Greg Kroah-Hartman 2011-06-28 19:23:19 UTC
Yes, we updated aaa_base because of systemd, but it should still work.

What package is causing a problem with the changes in aaa_base?  Let me know and I'll be sure to update it in tumbleweed.

Again, I can't duplicate this here at all, so either I am not running the same services as Marcus, and one of them is the problem, or something else odd is happening...

Marcus, what services are you running at startup time?  How about your filesystem setup, is it relying on anything "odd"?
Comment 7 Michael Buchau 2011-06-28 21:58:11 UTC
On my system, the packages failing to start on boot are

nfs-client /var/lock/subsys/nfs-rpc.idmapd
nfs-server /var/lock/subsys/nfsserver-rpc.idmapd
pcsc-lite /var/lock/subsys/pcscd
smpppd  /var/run/smpppd/
samba-client /var/lock/subsys/cifs

Contrary to Marco's findings, rsyslogd starts without problems.
Comment 8 Greg Kroah-Hartman 2011-06-28 22:39:45 UTC
Ok, I've updated nfs, pcsc-lite, and smpppd, but samba is wierd, that should
be fixed, I'll work on tracking that down.

The build server should take about 24 more hours due to KDE updates, so after the next update, if these packages, or any others, still cause problems, please can
someone let me know.

I'll go look at fixing aaa_base now as well...
Comment 9 Marco Röben 2011-06-29 08:14:14 UTC
Thank you guys for picking this up and providing help, even though you cannot reproduce it. Special thank to Michael for helping out.
Comment 10 Greg Kroah-Hartman 2011-08-12 23:16:03 UTC
Should now be resolved, right?
Comment 11 Michael Buchau 2011-08-13 14:45:14 UTC
Well, it's not completely resolved yet. There are still some packages fiddling around in /var/lock/subsys, but they aren't prepared for creating the directory if it's missing due to /run/lock being bind-mounted on /var/lock.

Here's what a quick check on an 11.4+Tumbleweed system for references of /var/lock/subsys in the init scripts and missing an mkdir brings up:

dhcpv6-1.0.22-13.1              /etc/init.d/dhcp6{r,s}
drbd-8.3.8.1-4.3                /etc/init.d/drbd
OpenIPMI-2.0.16-7.1             /etc/init.d/ipmi
libvirt-client-0.8.8-0.12.1     /etc/init.d/libvirt-guests
sensors-3.2.0-9.12.1            /etc/init.d/lm_sensors
nagios-3.2.3-3.4.1              /etc/init.d/nagios
openais-1.1.4-3.1               /etc/init.d/openais
pcsc-lite-1.7.4-2.2             /etc/init.d/pcscd
scsirastools-1.5.8-15.1         /etc/init.d/sg{disk,raid}
smartmontools-5.40-10.1         /etc/init.d/smartd
smolt-1.4.3-1.6.1               /etc/init.d/smolt
klogd-1.4.1-730.1               /etc/init.d/syslog
samba-client-3.6.0-4.1          /etc/init.d/cifs
tomcat6-6.0.32-6.5              /etc/init.d/tomcat6

IMHO the quickest fix would be to create /run/lock/subsys in /etc/init.d/boot (package aaa_base) and be done with it. I haven't checked if systemd handles this.
Comment 12 Greg Kroah-Hartman 2011-09-20 00:09:34 UTC
systemd has been removed from Tumbleweed for this, and other reasons, so this will no longer be an issue.