Bugzilla – Bug 827654
open-iscsi should be updated to integrate with systemd
Last modified: 2013-10-29 10:00:10 UTC
+++ This bug was initially created as a clone of Bug #821695 +++ This bug was cloned from bnc#821695 so that it could track the update of open-iscsi on openSUSE to support systemd.
See bnc#821695 for dependencies. The plan is to update openSUSE Factory/13.1 for systemd -- not sure if such a change will be appropriate for openSUSE 12.3, but will decide at that time.
Raising to P3 ... this should be done in time for openSUSE 13.1 ...
Hannes mentioned that another aspect of this is that open-iscsi currently uses a udev daemon to mount iSCSI devices, so that will have to be updated to use systemd as well.
Bumping this to P2, as it needs to get done soon for openSUSE 12.3. I have found 3 git commits in public open-iscsi that address systemd integration, so I will start with them.
Additional complication discovered: in addition to supporting /etc/init.d/iscsid with a iscsid.service and iscsid.socket file, there is the question of the boot-time file /etc/init.d/boot.iscsid-early. I will have to research this file further to see what happens to it in the systemd world
Oh, that's not that hard. What I did for multipath was to have systemd intercept the netlink socket, used for communication between the admin program and the daemon. If the same trick would be applied to iscsi it would mean that each call to iscsiadm would automatically start iscsid. At the same time iscsid can and should be started within the initrd, so the only thing left to be implemented is a re-exec. The main problem we have currently is that we can only do a full restart of the iscsid. So whenever the iscsid has to be restarted it needs to be terminated properly, and the new instance need to re-initialize itself before it can operate properly. This is a rather long window, during which we cannot process any connection failures. Which hits us particularly hard during init, as the daemon has to be shut down from the initrd, _then_ the real init takes over, all scripts in boot.d are executed before iscsid is finally started during rc phase. That is a rather long time with heavy I/O going on, so it is quite likely to hit an I/O interruption then. But iscsid isn't running. So we're stuck and booting hangs. With system we should be able to do a proper re-exec; if systemd would control _all_ filedescriptors iscsid uses (even internally) then no events on these fds would be lost during restart. So for re-exec iscsid would just have to flush the internal queue, and do an 'execve' on the pathname of the original program. iscsid would then startup, pull the fds from systemd, and would be ready to go. With that the boot.iscsid-early can be dropped and iscsid can be started during normal system boot. -
(additionaly ... i wouild suggest to change this for Factory only for now and shake out bugs before doing it for 12.3 ... to avoid regressions ;)
I found this rather interesting documentation on systemd support for Root Storage via a daemon (such as iscsid): http://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons/ Quote: > ... we hereby clarify that we do not support storage > technology setups where the storage daemons are being > run from the storage it maintains itself. In other words: > a storage daemon backing the root file system cannot > be stored on the root file system itself. This means that the iscsid that starts at initd time must continue to run during the transition from initrd to full running mode, and that iscsid can only be stopped when the OS stops and drops back to the initrd state, where the tangle that is the root OS mount can be unwound. In the case of iscsi, it is not quite as bad as all this, because the iscsi daemon (iscsid) is only needed to handle errors, not for storage access under normal conditions. So a restart of iscsid in our case means there is no error handling/recovery, but it does not mean to access to the root disc under normal conditions. Until our systemd and initrd support the initrd images running until OS shutdown time, we may have to live with this temporary loss of error handling during the initrd to full OS transition.
Hey, What do you guys do for multipath/multipathd? We might have similar issues.
(In reply to comment #9) > Hey, > > What do you guys do for multipath/multipathd? We might have similar issues. If you mean what does openSUSE do for multipathd-systemd interaction, I believe the answer is nothing yet. I believe that until the initrd environment is updated on openSUSE to better match the "systemd plan", we will not be able to have a fully-supported (i.e. iscsid running all the time) iscsi-boot environment.
Hehe. Not quite. I've been updating multipath to interact with systemd (socket-based activation and all); patches should go upstream soon. What I'm still fighting with is the automatic multipathd startup on boot. There we either - start multipath always and have the service stopped if no multipath devices are found (kinda like the current behaviour) or - start multipath only for the devices known to be multipathed. This requires some configuration file, which needs to be parsed by systemd / udev. Or udev calling 'multipath' in 'check' mode, which will then cause multipathd to be started. Not quite decided yet. Nor coded.
I grabbed 3 commits from upstream open-iscsi that support systemd's socket-based activation, and I compiled openSUSE 12.3 open-iscsi with these changes, installing just iscsiadm and iscsid. In addition, I took the systemd file iscsid.service and iscsid.socket which where part of that commit, and I updated them to (1) remove mention/use of iscsiuio, since this is not yet present in openSUSE, and (2) move iscsiadm's path from /usr/bin to /sbin, which is where we keep it in openSUSE. After installing these two updated systemd files in /usr/lib/systemd/system, and running "systemctl --system daemon-reload", I then enable them both with "systemctl enable iscsid.service iscsid.socket", then turn off the iscsid service with "chkconfig iscsid off". Oh, and I also had to comment out the iscsid.startup line in /etc/iscsi/iscsid.conf, so that iscsiadm didn't try to start iscsid itself. After rebooting, iscsid is not running, but if I try to access it, e.g. "iscsiadm -m node -l" to login to pre-discovered nodes, iscsid gets started up automatically by systemd. Pretty cool. Still need to figure out how to get rid of the boot.iscsid-early service, which is only used when there are pre-existing iSCSI sessions at boot time (i.e. when booting from an iSCSI-enabled network card), and even then the only work it does is to ensure that those pre-connected nodes have "startup = onboot" set. Not sure how to get systemd to run a service script at boot time, but somebody has to make sure that boot-required volumes have "onboot" set, right?
(In reply to comment #12) > Not sure how to get systemd to run a service script at boot time, but somebody > has to make sure that boot-required volumes have "onboot" set, right? http://en.opensuse.org/openSUSE:Systemd_packaging_guidelines you'll probably need to request a change in systemd-presets-branding-openSUSE
(In reply to comment #3) > Hannes mentioned that another aspect of this is that open-iscsi currently uses > a udev daemon to mount iSCSI devices, so that will have to be updated to use > systemd as well. I talked to Kay Sievers, and he said that the mount optino 'nofail' is implemented already in systemd. And that systemd would mount any devices when listed in /etc/fstab. So that should work as expected, and we can remove the udevmountd. Which is okay.
(In reply to comment #14) > (In reply to comment #3) > > Hannes mentioned that another aspect of this is that open-iscsi currently uses > > a udev daemon to mount iSCSI devices, so that will have to be updated to use > > systemd as well. > > I talked to Kay Sievers, and he said that the mount optino 'nofail' is > implemented already in systemd. And that systemd would mount any devices when > listed in /etc/fstab. So that should work as expected, and we can remove the > udevmountd. Which is okay. Hannes: Once I had the service files set up correctly, I found this to be true. In openSUSE the udevmountd is not used by udev, anyway. I set the fstab mount options to "nofail" for a remote device, and when the iSCSI device becomes available systemd does indeed mount it.
I have a set of changes that seems to work now. The only problems seems to be: 1. iSCSI-root volumes (or volumes needed at boot time using iSCSI, in general) were not tested, and 2. Arrangements will still need to be made to ensure that the iscsid.socket interface is enabled by default For openSUSE Factory, should I remove the systemV-init files (like /etc/init.d/iscsid and /etc/init.d/boot.iscsid-early)? The patch set I currently has removes these files (as well as "/sbin/iscsid"). When systemd is running, these commands get rerouted to systemd anyway. But what I'm not clear about is whether there will be an option to use SysV-init on openSUSE Factory? If so, perhaps we should keep these files around (even though they would be rarely used). Help from a systemd-vs-sysv-init expert?
Response from systemd expert is that sysv-init files are no longer needed for openSUSE Factory / 13.1.
Indeed
Changes submitted to OBS for Factory open-iscsi, request id 201518.
First set of changes rejected: the open-iscsi.changes file was from SP3. Resubmitted as request ID 201939.
This is an autogenerated message for OBS integration: This bug (827654) was mentioned in https://build.opensuse.org/request/show/202146 Factory / open-iscsi
This is an autogenerated message for OBS integration: This bug (827654) was mentioned in https://build.opensuse.org/request/show/202543 Factory / open-iscsi
Changes to systemd-presets-branding-openSUSE, which contains a list of services that are automatically enabled when installed, have been submitted as OBS request id 203672. I will try to get these changes into openSUSE 13.1, but it is already at RC1.
i dont like starting the service by default
(In reply to comment #24) > i dont like starting the service by default Marcus: I believe I understand your reluctance, but since iscsid will become a socket-activated service, the daemon will never be started unless the caller runs a command that requires the daemon, or unless they have iSCSI volumes. So even though enabled, the iscsid daemon will not be running by default. I have submitted changes to enable iscsid.socket and iscsi.service, by default, to the build service.
Setting NEEDSINFO for answer to question in previous comment.
open-iscsi Factory changes submitted to openSUSE 13.1 Update, for possible inclusion in openSUSE 13.1 RC2, as Request: #204239.
I will clone this bug, creating a new bug to track enabling of the iscsi service when using systemd. Clearing NEEDSINFO for this one since it had no effect.
This is an autogenerated message for OBS integration: This bug (827654) was mentioned in https://build.opensuse.org/request/show/204239 13.1 / open-iscsi
This will be in 13.1 and is in Factory, so closing the bug as fixed.
This is an autogenerated message for OBS integration: This bug (827654) was mentioned in https://build.opensuse.org/request/show/205076 Factory / systemd-presets-branding-openSUSE