Bug 747783

Summary: systemd: 'service openvpn start' triggers spamming of all terminals
Product: [openSUSE] openSUSE 12.1 Reporter: Gerald Pfeifer <gp>
Component: BasesystemAssignee: Frederic Crozat <fcrozat>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: fcrozat, forgotten_62H0P2WxbG, haenig, hpj, lmuelle, msvec, mt
Version: Final   
Target Milestone: ---   
Hardware: x86-64   
OS: openSUSE 12.1   
URL: https://dareclama.ro
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Gerald Pfeifer 2012-02-18 19:18:04 UTC
User-Agent:       Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0

1. become root
2. # service openvpn start
3. Watch as every terminal will be spammed with something like

Broadcast message from root@trevally.site (Fri, 17 Feb 2012 17:28:14 +0100):

Password entry required for 'Enter Auth Username:' (PID 17770).
Please enter password with the systemd-tty-ask-password-agent tool!


This does not make any sense since I am already in the terminal with
the prompt; why bother all other terminals?


Reproducible: Always
Comment 1 Frederic Crozat 2012-02-20 09:37:24 UTC
But do you get the correct prompt in the active terminal and are able to type the passphrase ?
Comment 2 Gerald Pfeifer 2012-02-21 19:12:09 UTC
Yes, the active terminal works just as expected.  It's the spamming
of all the others that is the issue.
Comment 3 Frederic Crozat 2012-03-30 12:04:13 UTC
*** Bug 754952 has been marked as a duplicate of this bug. ***
Comment 4 Frederic Crozat 2012-07-25 07:38:17 UTC
*** Bug 772860 has been marked as a duplicate of this bug. ***
Comment 5 Frederic Crozat 2012-07-25 07:43:08 UTC
I've discussed this bug with upstream and it is very tricky to get it fixed because the password prompt is asynchronous to the request and you could also get several query at the same time.

Excerpt from IRC discussion:
juin 25 13:45:05 <fcrozat>	mezcalero: while I have a grab on you, a quick question. systemd-tty-ask-password-agent "wall" on all tty, even when you are on the terminal where the request is originating (and you have the prompt). Would it be ok if I (try to) cook a patch to make sure it "walls" only when active tty isn't the one where the prompt is running ?
juin 25 15:47:50 <mezcalero>	fcrozat: hmm, actually we have a logic in there that avoids the active tty
juin 25 15:47:57 <mezcalero>	fcrozat: must be broken for you i guess?
juin 25 15:48:06 <mezcalero>	fcrozat: oh
juin 25 15:48:07 <mezcalero>	hmm
juin 25 15:48:17 <mezcalero>	nah, different thing
juin 25 15:48:22 <fcrozat>	mezcalero: ah, ok ;)
juin 25 15:48:28 <mezcalero>	or hmmm
juin 25 15:48:45 *	mezcalero needs to look up what i hacked there
juin 25 15:49:09 <fcrozat>	:)
juin 25 15:51:18 <mezcalero>	fcrozat: hmm, so wall_tty_block() in tty-ask-password-agent.c should already do what you are asking for?
juin 25 15:51:26 <mezcalero>	fcrozat: it uses fifos as locks
juin 25 15:53:03 <fcrozat>	mezcalero: well, it doesn't (or something isn't). See https://bugzilla.novell.com/show_bug.cgi?id=747783 for bug report
juin 25 15:53:19 <fcrozat>	(I plugged openvpn to use tty-ask-password)
juin 25 15:53:24 <mezcalero>	fcrozat: hmm, so the code flow might be surprising
juin 25 15:53:38 <mezcalero>	fcrozat: but basically in systemctl we spawn into the background on instance of thy agent
juin 25 15:53:48 <mezcalero>	another instance of the agent is run for the wall stuff
juin 25 15:54:54 <fcrozat>	but could we prevent the wall instance to start if we are on the active terminal ? Does this behaviour would be acceptable for you ?
juin 25 15:55:22 <mezcalero>	right now the tty instance blocks out the other
juin 25 15:55:37 <mezcalero>	by creating creating a lock for that tty in question
juin 25 15:55:43 <mezcalero>	the wall instance will honour that lock
juin 25 15:55:49 <mezcalero>	and not display the message on that screen
juin 25 15:56:05 <mezcalero>	but what i do wonder about now is whether there might be a race here
juin 25 15:56:23 <mezcalero>	i.e. that the wall instance runs quicker and doesn't see the tty lock
juin 25 15:57:00 <fcrozat>	well, it does work
juin 25 15:57:05 <mezcalero>	it does?
juin 25 15:57:05 <fcrozat>	it doesn't wall on the active terminal
juin 25 15:57:11 <fcrozat>	it wall on all others :)
juin 25 15:57:20 <mezcalero>	hmm, so, what's the question?
juin 25 15:57:43 <fcrozat>	could we prevent to wall on all others, if we are in the active terminal when the fifo was started ?
juin 25 15:58:10 <mezcalero>	uh
juin 25 15:58:16 <mezcalero>	well, there is a bit of a problem there
juin 25 15:58:42 <mezcalero>	there is no way to figure out whether a question is being asked due to the local tty
juin 25 15:59:15 <mezcalero>	i.e. if somebody plugs in a usb key that is encrypted at the same time as you type "systemctl start somethingelse.service" and both need a passowrd
juin 25 15:59:23 <mezcalero>	then i assume you'd one wall to show, and the other now
juin 25 15:59:24 <mezcalero>	not
juin 25 16:00:00 <mezcalero>	but the thing is that we really have no information about which UI action actually triggered a password being needed
juin 25 16:00:07 <mezcalero>	resp. whether there was any UI action at all
juin 25 16:00:21 <mezcalero>	for example, it might just be your iscsi server being powered on
juin 25 16:00:25 <mezcalero>	and things like this
juin 25 16:00:36 <fcrozat>	specially since I'm using this stuff for openvpn credentials too :)
juin 25 16:00:42 <mezcalero>	so, there is really no clear chain of events, that we could rely on and use to hide some walls, and show the others
juin 25 16:00:48 <fcrozat>	(I got two bug reports from the same guy, one for crypsetup, one for openvpn ;)
juin 25 16:01:08 <mezcalero>	i mean, at least i know no nice way to do this...
juin 25 16:01:33 <mezcalero>	maybe the best suggestion is to ask him to use "systemctl mask" on the wall agent?
juin 25 16:01:58 <fcrozat>	well, seems a ugly workaround for an "ugly" behaviour (from a usability perspective) ;)
juin 25 16:02:15 <mezcalero>	well, i mean, i am open to ideas on this
juin 25 16:02:24 <mezcalero>	if you figure out a nice way, i am all ears
juin 25 16:02:29 <mezcalero>	but i see no nice way
juin 25 16:03:37 <fcrozat>	well, if we can differentiate stuff coming from a event from a "started from a service by user", we would have a way
Comment 6 Michal Svec 2012-07-25 08:24:30 UTC
Well the thing is that it works for the first time after reboot and is "only"
broken for all the other cases. One would assume that stopping the service
should lead to the same state as if the service wasn't started, but apparently
that's not the case, which is really weird. If we can figure out what's the
difference in behavior (strace,ltrace,...) between those two cases that could
help with the resolution.

But moreover, using wall(1) for telling anything to the user nowadays is just
braindead. With all the modern desktop environments and their elaborate (and
pretty huge, nothing lightweight) notification and communication infrastructure
this should not be needed at all.
Comment 7 Thomas Hänig 2012-11-13 12:14:53 UTC
SCNR, but such things happen if one breaks 'one program for one task' with each and everything becoming service/server/systemwide.

What speaks against e.g. openvpn running on ist own?

My "solution", because I'm the one and only user (OpenSuSE 12.2, openvpn 2.2.2):
# echo > /usr/bin/wall
Comment 8 Michal Svec 2013-02-11 17:19:13 UTC
Any news on this?
Comment 9 Frederic Crozat 2013-02-11 17:34:29 UTC
(In reply to comment #8)
> Any news on this?

no and it is really not very high in my todo list (moreover, since upstream doesn't see a solution for it)
Comment 10 Michal Svec 2013-02-11 17:55:42 UTC
Hmm. It's actually very annoying (having a lot of terminals open and using
this like almost every day).

Is there at least a workaround? 'echo >/usr/bin/wall' mentioned in #c7 does
not work, systemd seems to be doing the wall some other way.

BTW on 12.2 there's no difference between the system after reboot and later
(like what I mentioned in comment 6).
Comment 11 Frederic Crozat 2013-02-12 10:23:44 UTC
try "systemctl mask systemd-ask-password-wall.service" as root

it will prevent the wall service for asking password to start..
Comment 12 Lars Müller 2013-05-07 09:15:48 UTC
We still see the same issue with openSUSE 12.3.  Masking the systemd-ask-password-wall.service as suggested in comment #11 doesn't change the habit.

Also masking the following services alone doesn't help:

systemctl mask systemd-ask-password-console.path
systemctl mask systemd-ask-password-plymouth.path
systemctl mask systemd-ask-password-wall.path
systemctl mask systemd-ask-password-console.service
systemctl mask systemd-ask-password-plymouth.service
systemctl mask systemd-ask-password-wall.service

in addition we need to stop these services too:

systemctl stop systemd-ask-password-console.path
systemctl stop systemd-ask-password-plymouth.path
systemctl stop systemd-ask-password-wall.path
systemctl stop systemd-ask-password-console.service
systemctl stop systemd-ask-password-plymouth.service
systemctl stop systemd-ask-password-wall.service

Then while starting the openvpn service doesn't lead to the described symptoms.
Comment 13 Marius Tomaschewski 2013-05-08 09:26:37 UTC
When it is started at boot time, systemd tools are used and they
broadcast/wall. You can provide the username / password using
systemd-tty-ask-password-agent.

To avoid the wall messages, you can set OPENVPN_AUTOSTART=" " in
/etc/sysconfig/openvpn to avoid automatic starts and start the
configs later manually using

   "rcopenvpn [re]start xyz"

to start /etc/openvpn/xyz.conf. Then openvpn starts directly (no
systemd redirection to systemctl), asks on the console only (and
joins openvpn.service cgroup, so "status" shows it later].

This is IMO the only workaround we can provide here.

BTW: On factory all is done using systemd mechanisms, the init script
and variable are gone, use:

   systemctl enable openvpn@xyz.service

and then e.g.:

   systemctl [re]start openvpn@xyz.service

or also:

   systemctl [re]start openvpn.target

to start all configs. When called manually, prompt is on console.
Otherwise you can provide it via systemd-tty-ask-password-agent.
Comment 14 Frederic Crozat 2013-05-15 14:03:33 UTC
after careful thinking, even if it isn't very pretty (because we will loose notification for some events), we should probably disable by default the ask-password wall service.

I'll look into it for Factory (I don't want to change behavior on a shipped distribution).
Comment 15 Frederic Crozat 2013-10-01 16:55:38 UTC
ok, I was able to reproduce the issue in a VM.

YaST2-Second-Stage.service (and FirstBoot.service) should have:
cryptsetup.target added to their After= line.

This will ensure all YaST* services will be started after cryptsetup is being handled (I thought the dependency on local-fs.target was enough, but apparently, it isn't).

I would also suggest to add this dependency to Factory too.
Comment 16 Frederic Crozat 2013-10-01 16:57:12 UTC
discard comment 15, I used the wrong bug page :)
Comment 17 Frederic Crozat 2013-11-21 13:52:50 UTC
closing as fixed for 13.1
Comment 18 Forgotten User 62H0P2WxbG 2017-04-11 07:50:48 UTC
You should add this to OpenVPN client .conf file:

auth-retry nointeract

Have a good day!