Bugzilla – Bug 968042
X (xdm.service) does not start after systemd 210 upgrade
Last modified: 2018-01-13 20:06:24 UTC
Created attachment 666712 [details] journalctl -b output (kernel booted with systemd.log_level=debug) on systemd-210 having the issue As requested by Franck Bui in bug 965475, a new report, with added detail: The following happens on a Lenovo T410 laptop * KDE install (KDE:/Current repo active) * with an NFS3 mount in /etc/fstab * and NetworkManager active (without any fancy extra config, just LAN) after the systemd-210 upgrade (package systemd-210-40.1) on openSUSE 13.1 X does not start. ntp does not start. The NFS volume _is_ mounted. The attachment "xdm-does-not-start.journal" shows the situation.
Created attachment 666713 [details] same systemd-210 install, but NFS mount removed from fstab After disabling my NFS mount in fstab, and rebooting, X starts again, NTP starts again, the dependency cycle is gone. Attachment xdm-does-not-start.nonfsmount shows the log from that boot.
Created attachment 666714 [details] logfile after downgrading to systemd-208 Downgrading the system to systemd-208 using the following commands, and reenabling my NFS mount in fstab, the problem GOES AWAY, too. (additional note: system needs hard reset / power cycle after such downgrade, once) zypper in -y --oldpackage systemd=208 systemd-sysvinit=208 systemd-32bit=208 udev=208 libudev1=208 libgudev-1_0-0=208
Created attachment 666721 [details] journalctl -b output (kernel booted with systemd.log_level=debug) on systemd-210 having the issue
Created attachment 666725 [details] same systemd-210 install, but NFS mount removed from fstab sorry, had to replace the attachments, did not properly boot with systemd.log_level=debug before.
Created attachment 666728 [details] logfile after downgrading to systemd-208 In the journal output after downgrading to 208, you can see that that NFS mount unit does fail there, too, presumably because NetworkManager has not already done its thing. The mount does appear automatically later. However, on the 208 system, that does not hinder NTP or X to start. It _does_ break postfix starting (for which I see I had a cronjob workaround for a long time already...) Hope this helps to get the issues fixed.
(In reply to Patrick Schaaf from comment #0) > Created attachment 666712 [details] > journalctl -b output (kernel booted with systemd.log_level=debug) on > systemd-210 having the issue > you forgot to remove the "quiet" option from the command line as I suggested in the other bug.
(In reply to Franck Bui from comment #6) > > you forgot to remove the "quiet" option from the command line as I suggested > in the other bug. Yes, I noticed that (in fact I forgot to update grub config after changing defaults) I uploaded the attachments anew (replaced) with another round and correct boot options
On 13.1, xdm package provides /etc/init.d/xdm which include the following LSB header: ### BEGIN INIT INFO # Provides: xdm display-manager # Required-Start: $remote_fs dbus ... systemd v210 turns the "Required-Start" into a "Requires=" native dependency. It's a specific Suse behaviour AFAICS and it has been introduced in v210, I guess. However on 13.2, xdm package does not ship the sysvinit script anymore, so I suppose that's the reason why no one hits this issue previously. The fix might be in the xdm package. @Werner, any idea ? I'm asking because you seem to be involved in the xdm package as well...
(In reply to Franck Bui from comment #8) Indeed the new xdm package has now a native service unit /usr/lib/systemd/system/display-manager.service which then executes the script /usr/lib/X11/display-manager From the journal log I see systemd[1]: samba2.mount mount process exited, code=exited status=32 systemd[1]: Failed to mount /samba2. systemd[1]: Dependency failed for Remote File Systems. systemd[1]: Dependency failed for Postfix Mail Transport Agent. systemd[1]: Dependency failed for LSB: X Display Manager. systemd[1]: Dependency failed for LSB: Network time protocol daemon (ntpd). The question rises *which* of the dependencies does cause this. My guess is the tag `$remote_fs' and I also guess that the postfix service as well as the LSB SysVinit scripts for ntpd also require `$remote_fs' aka remote-fs.target. Then the problem seems to be the first line which indicates that `Remote File Systems' aka remote-fs.target does fail. I'd like to see the output of systemctl status remote-fs.target as well as systemctl list-dependencies --before remote-fs.target which should include xdm/display-manager and also the output of systemctl list-dependencies --after remote-fs.target to see what is missed ... the only problem is that cut&paste does not show the colors of the dots (green for loaded /red for not loaded)
(In reply to Dr. Werner Fink from comment #9) > (In reply to Franck Bui from comment #8) > > Indeed the new xdm package has now a native service unit > > /usr/lib/systemd/system/display-manager.service > > which then executes the script > > /usr/lib/X11/display-manager > > From the journal log I see > systemd[1]: samba2.mount mount process exited, code=exited status=32 > systemd[1]: Failed to mount /samba2. > systemd[1]: Dependency failed for Remote File Systems. > systemd[1]: Dependency failed for Postfix Mail Transport Agent. > systemd[1]: Dependency failed for LSB: X Display Manager. > systemd[1]: Dependency failed for LSB: Network time protocol daemon > (ntpd). > > The question rises *which* of the dependencies does cause this. Well that's probably due to: graphical.target -> xdm.service -> $remote_fs (remote-fs.target) assuming that graphical target is the default one.
> I'd like to see the output of > systemctl status remote-fs.target > as well as > systemctl list-dependencies --before remote-fs.target > which should include xdm/display-manager and also the output of > systemctl list-dependencies --after remote-fs.target > to see what is missed ... the only problem is that cut&paste does not > show the colors of the dots (green for loaded /red for not loaded) Here it is. Immediately (5 seconds or so) after boot: lat17:~ # systemctl status remote-fs.target remote-fs.target - Remote File Systems Loaded: loaded (/usr/lib/systemd/system/remote-fs.target; enabled) Drop-In: /run/systemd/generator/remote-fs.target.d └─50-insserv.conf.conf Active: inactive (dead) Docs: man:systemd.special(7) Feb 25 14:25:06 lat17 systemd[1]: Dependency failed for Remote File Systems. lat17:~ # systemctl list-dependencies --before remote-fs.target remote-fs.target green ├─autofs.service RED ├─gpm.service RED ├─ntp.service RED ├─postfix.service green ├─systemd-user-sessions.service RED └─xdm.service lat17:~ # systemctl list-dependencies --after remote-fs.target remote-fs.target green ├─fix.service green ├─samba2.mount green └─remote-fs-pre.target green └─nfs.service lat17:~ # uptime 14:25pm up 0:00, 1 user, load average: 0.62, 0.21, 0.07
> Immediately (5 seconds or so) after boot: > > lat17:~ # systemctl status remote-fs.target > remote-fs.target - Remote File Systems > Loaded: loaded (/usr/lib/systemd/system/remote-fs.target; enabled) > Drop-In: /run/systemd/generator/remote-fs.target.d > └─50-insserv.conf.conf > Active: inactive (dead) > Docs: man:systemd.special(7) An additional observation: after I start postfix by hand (systemctl start postfix), that remote-fs.target service turns green "all by itself": lat17:~ # systemctl status remote-fs.target remote-fs.target - Remote File Systems Loaded: loaded (/usr/lib/systemd/system/remote-fs.target; enabled) Drop-In: /run/systemd/generator/remote-fs.target.d └─50-insserv.conf.conf Active: active since Thu 2016-02-25 14:27:49 CET; 6s ago Docs: man:systemd.special(7)
Created attachment 666881 [details] systemd debug messages, for real now... OMG, I'm stupid. Forgot that we have journald configured to not log debug messages to the journal, but only to syslog..... This attachment, corresponding to the situation described in the previous comments (ending after I started postfix by hand), finally shows the full debug messages (taken from syslog)...
(In reply to Patrick Schaaf from comment #12) ??? How does postfix service looks like? IMHO it should look like [Unit] Description=Postfix Mail Transport Agent Requires=var-run.mount After=var-run.mount nss-lookup.target network.target time-sync.target After=amavis.service mysql.service cyrus.service ldap.service openslp.service ypbind.service Conflicts=sendmail.service exim.service [...]
(In reply to Dr. Werner Fink from comment #14) > (In reply to Patrick Schaaf from comment #12) > > ??? > > How does postfix service looks like? IMHO it should look like > > [Unit] > Description=Postfix Mail Transport Agent > Requires=var-run.mount > After=var-run.mount nss-lookup.target network.target time-sync.target > After=amavis.service mysql.service cyrus.service ldap.service > openslp.service ypbind.service > Conflicts=sendmail.service exim.service > > [...] It reads like this (last change of file 18. October 2013) : [Unit] Description=Postfix Mail Transport Agent Requires=var-run.mount nss-lookup.target network.target remote-fs.target time-sync.target After=var-run.mount nss-lookup.target network.target remote-fs.target time-sync.target After=amavis.service mysql.service cyrus.service ldap.service openslp.service ypbind.service Conflicts=sendmail.service exim.service
Then the service unit of postfix has to be fixed.
(In reply to Dr. Werner Fink from comment #16) > Then the service unit of postfix has to be fixed. Sure enough, after removing remote-fs.target from postfix.service Requires=, and also removing $remote_fs from /etc/init.d/xdm and /etc/init.d/ntp, all three start fine on my test system. But anyway, isn't there (before and after the 208 / 210) switch something generally fishy with the interaction of NetworkManager and NFS mounting / remote-fs.target? The delayed network upping of NetworkManager obviously does redo the previously failed NFS mount (my /samba2 thing), but that does not help any other services which require the already failed remote-fs.target. That smells wrong. No idea what to do about it, though.
Created attachment 666978 [details] debug messages after downgrade to 208 syslog extract of my test after downgrading to 208, with systemd debug messages embedded, for comparison with attachment 666881 [details]
(In reply to Dr. Werner Fink from comment #16) > Then the service unit of postfix has to be fixed. Beside this , I see 2 issues here: - Required-Start: $remote_fs is translated to a "Requires=$remote_fs" native dependency. This is due to a Suse specific patch "1018-Make-LSB-Skripts-know-about-Required-and-Should.patch" (you wrote it seems). This makes the sysv-generator not compatible to the old behaviour of sysvinit anymore AFAICS. And that might be the main reason why this fails now. - it's not clear why xdm needs $remote_fs to be started.
As a workaround, I have built myself locally a workaround package with three %triggerin scripts. With that workaround in place, my laptop again boots cleanly. I have it installed on other clients that did not show the issue, with no adverse effects, either. Time for somebody to fix these xdm, ntp, and postfix packages! %triggerin -- xdm [ $1 -gt 0 ] || exit 0 [ -r /etc/init.d/xdm ] || exit 0 sed -i -e 's/$remote_fs\s*//' /etc/init.d/xdm exit 0 %triggerin -- ntp [ $1 -gt 0 ] || exit 0 [ -r /etc/init.d/ntp ] || exit 0 sed -i -e 's/$remote_fs\s*//' /etc/init.d/ntp exit 0 %triggerin -- postfix [ $1 -gt 0 ] || exit 0 [ -r /usr/lib/systemd/system/postfix.service ] || exit 0 sed -i -e 's/remote-fs.target\s*//' /usr/lib/systemd/system/postfix.service exit 0
(In reply to Franck Bui from comment #19) > > - it's not clear why xdm needs $remote_fs to be started. Possibly to wait for /home/<user> ?
See also bug#971331. As long as I only have one NFS mount in /etc/fstab, everything works fine. I cna't see why it is different from what is described here. With two mounts, we get a race condition due to two mounts both trying to start rpc.statd.
openSUS 13.1 is out of support.