Bug 1191347

Summary:	Yast2 Disk masks all systemd x.mount units
Product:	[openSUSE] openSUSE Distribution	Reporter:	itteam itteam <itteam>
Component:	YaST2	Assignee:	YaST Team <yast-internal>
Status:	RESOLVED FIXED	QA Contact:	Jiri Srain <jsrain>
Severity:	Normal
Priority:	P5 - None	CC:	ancor, aschnell, dgonzalez
Version:	Leap 15.3
Target Milestone:	---
Hardware:	All
OS:	openSUSE Leap 15.3
URL:	https://trello.com/c/Cs2LigI1
Whiteboard:
Found By:	---	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---
Attachments:	journal y2log

Description itteam itteam 2021-10-05 16:54:08 UTC

Created attachment 852942 [details]
journal

We've come across an issue whereby opening yast2 disk management causes all systemd x.mount units to be masked until reboot or manual intervention. On some systems this is causing issues restarting services which depend on these (eg nrpe with After=var-run.mount), which present an error:

Failed to restart nrpe.service: Unit var.mount is masked.

It's simple enough to trigger the masking with yast2 disk as follows.

1. Boot system, confirm var.mount for example is not masked:

# systemctl status var.mount
● var.mount - /var
     Loaded: loaded (/etc/fstab; generated)
     Active: active (mounted) since Tue 2021-10-05 17:17:26 BST; 1min 31s ago
      Where: /var
       What: /dev/mapper/vg_system-lv_var
       Docs: man:fstab(5)
             man:systemd-fstab-generator(8)
      Tasks: 0 (limit: 7030)
     CGroup: /system.slice/var.mount

Oct 05 17:17:25 someserver1 systemd[1]: Mounting /var...
Oct 05 17:17:26 someserver1 systemd[1]: Mounted /var.


2. Run yast2 disk

> sudo yast2 disk

3. Accept the warning prompt and proceed to Abort or Finish without making any changes.

4. Check the status again of var.mount and find that it is masked (note the log of changed status, this was achieved setting journalctl to debug mode with systemctl log-level debug):

# systemctl status var.mount
● var.mount - /var
     Loaded: masked (Reason: Unit var.mount is masked.)
     Active: active (mounted) since Tue 2021-10-05 17:17:26 BST; 2min 50s ago
      Where: /var
       What: /dev/mapper/vg_system-lv_var
      Tasks: 0 (limit: 7030)
     CGroup: /system.slice/var.mount

Oct 05 17:17:25 someserver1 systemd[1]: Mounting /var...
Oct 05 17:17:26 someserver1 systemd[1]: Mounted /var.
Oct 05 17:19:26 someserver1 systemd[1]: var.mount: Changed dead -> mounted
Oct 05 17:19:37 someserver1 systemd[1]: var.mount: Changed dead -> mounted

5. It's actually masking every mount unit it seems:

# ll /run/systemd/system
total 0
lrwxrwxrwx 1 root root 9 Oct  5 17:19 \x2esnapshots.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 backup.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 boot-grub2-i386\x2dpc.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 boot-grub2-x86_64\x2defi.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 data.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 dev-disk-by\x2did-dm\x2dname\x2dvg_system\x2dlv_swap.swap -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 dev-disk-by\x2did-dm\x2duuid\x2dLVM\x2dNn2wUiPvm30gKUVSZzdj7ZcrG3W8F0qgJkggHvpUF2wzkNeQpQmB1Vb4sk6o43zY.swap -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 dev-disk-by\x2duuid-61a850a3\x2de6cd\x2d4156\x2dbdfe\x2d59c040afdf8d.swap -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 dev-dm\x2d0.swap -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 dev-mapper-vg_system\x2dlv_swap.swap -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 dev-vg_system-lv_swap.swap -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 home.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 opt.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 root.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 srv.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 tmp.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 usr-local.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 var-lock.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 var-run.mount -> /dev/null
lrwxrwxrwx 1 root root 9 Oct  5 17:19 var.mount -> /dev/null

6. At this point the service cannot be restarted:
# systemctl restart nrpe
Failed to restart nrpe.service: Unit var.mount is masked.

Either rebooting, or removing the symlink and doing a daemon reload is sufficient to resolve individual units, in this case nrpe also needs tmp.mount (possibly due to the PrivateTmp=true directive):

# rm /run/systemd/system/var.mount
# systemctl daemon-reload
# systemctl restart nrpe
Failed to restart nrpe.service: Unit tmp.mount is masked.
# rm /run/systemd/system/tmp.mount
# systemctl daemon-reload
# systemctl restart nrpe
(starts fine)

This latter behaviour though is where there is a divergence, the service will only fail to restart on *some* systems, despite the unit file (/usr/lib/systemd/system/nrpe.service) being identical and all systems seeing the same masking behaviour triggered by yast.

So a few questions are raised, namely why is yast2 disk masking the x.mount units, is this expected, and is it supposed to leave them in this state? But then also why is it only having a tangible effect on some systems?

Attached is the systemd journal (debug level).

Comment 1 Ancor Gonzalez Sosa 2021-10-08 09:16:16 UTC

(In reply to itteam itteam from comment #0)
> 
> So a few questions are raised, namely why is yast2 disk masking the x.mount
> units, is this expected, and is it supposed to leave them in this state?

No, is not supposed to leave them in that state. It should have unmasked them on quitting. Please attach the YaST logs to debug why that didn't happen.

Comment 2 itteam itteam 2021-10-08 09:27:03 UTC

Created attachment 853015 [details]
y2log

YaST2 log as requested

Comment 3 Ancor Gonzalez Sosa 2021-10-08 11:00:48 UTC

(In reply to itteam itteam from comment #2)
> Created attachment 853015 [details]
> y2log
> 
> YaST2 log as requested

Relevant parts.

When the Partitioner starts:

> y2storage/inhibitors.rb(inhibit):32 inhibit
> set udev ANACONDA property
> Executing "/usr/bin/udevadm control --property\=ANACONDA\=yes".
> Status: 0
>
> inhibitors/udisks.rb(inhibit):30 inhibit udisks
> inhibit udisks failed The name org.freedesktop.UDisks was not provided by any .service files; caused by 3 sender=org.freedesktop.DBus -> dest=:1.18 serial=3 reply_serial=3 path=; interface=; member= error_name=org.freedesktop.DBus.Error.ServiceUnknown
>
> inhibitors/systemd_units.rb(inhibit):28 mask systemd units
> Executing "/usr/lib/YaST2/bin/mask-systemd-units --mask".
> Error output: /usr/bin/systemctl --runtime mask -- '\x2esnapshots.mount' 'boot-grub2-i386\x2dpc.mount' 'boot-grub2-x86_64\x2defi.mount' ��� home.mount opt.mount root.mount srv.mount ��� tmp.mount usr-local.mount ��� ��� ��� var-lock.mount var-run.mount var.mount 'dev-disk-by\x2did-dm\x2dname\x2dsystem\x2dswap.swap' 'dev-disk-by\x2did-dm\x2duuid\x2dLVM\x2dPnp3nScjnUuDzX0DYgfil6F0lPEn549s3JgyVwLOp4GUeNfLLmxBU6neU7IpkVOc.swap' 'dev-disk-by\x2duuid-09c0242a\x2d58b9\x2d4867\x2dac0b\x2df9b2e20092bc.swap' 'dev-dm\x2d1.swap' 'dev-mapper-system\x2dswap.swap' dev-system-swap.swap
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Unit \xe2\x97\x8f.service does not exist, proceeding anyway.
> Error output: Unit \xe2\x97\x8f.service does not exist, proceeding anyway.
> Error output: Unit \xe2\x97\x8f.service does not exist, proceeding anyway.
> Error output: Unit \xe2\x97\x8f.service does not exist, proceeding anyway.
> Error output: Unit \xe2\x97\x8f.service does not exist, proceeding anyway.
> Error output: Created symlink /run/systemd/system/\x2esnapshots.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/boot-grub2-i386\x2dpc.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/boot-grub2-x86_64\x2defi.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/\xe2\x97\x8f.service ��� /dev/null.
> Error output: Created symlink /run/systemd/system/home.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/opt.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/root.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/srv.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/tmp.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/usr-local.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/var-lock.mount ��� /dev/null. 
> Error output: Created symlink /run/systemd/system/var-run.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/var.mount ��� /dev/null.
> Error output: Created symlink /run/systemd/system/dev-disk-by\x2did-dm\x2dname\x2dsystem\x2dswap.swap ��� /dev/null.
> Error output: Created symlink /run/systemd/system/dev-disk-by\x2did-dm\x2duuid\x2dLVM\x2dPnp3nScjnUuDzX0DYgfil6F0lPEn549s3JgyVwLOp4GUeNfLLmxBU6neU7IpkVOc.swap ��� /dev/null.
> Error output: Created symlink /run/systemd/system/dev-disk-by\x2duuid-09c0242a\x2d58b9\x2d4867\x2dac0b\x2df9b2e20092bc.swap ��� /dev/null.
> Error output: Created symlink /run/systemd/system/dev-dm\x2d1.swap ��� /dev/null.
> Error output: Created symlink /run/systemd/system/dev-mapper-system\x2dswap.swap ��� /dev/null.
> Error output: Created symlink /run/systemd/system/dev-system-swap.swap ��� /dev/null.
> Status: 0


When the Partitioner quits:

> y2storage/inhibitors.rb(uninhibit):40 uninhibit
> unmask systemd units
> Executing "/usr/lib/YaST2/bin/mask-systemd-units --unmask".
> Error output: /usr/bin/systemctl --runtime unmask -- ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ���
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?). > Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Invalid unit name "���" escaped as "\xe2\x97\x8f" (maybe you should use systemd-escape?).
> Error output: Removed /run/systemd/system/\xe2\x97\x8f.service.
> Status: 0
>
> inhibitors/mdadm_auto_assembly.rb(uninhibit):41 unset udev ANACONDA property
> Executing "/usr/bin/udevadm control --property\=ANACONDA\=".
> Status: 0

Comment 4 Ancor Gonzalez Sosa 2021-10-08 11:03:40 UTC

(In reply to Ancor Gonzalez Sosa from comment #3)
> 
> Error output: /usr/bin/systemctl --runtime unmask -- ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ���

That's a clearly malformed command!

Comment 5 David Diaz 2021-10-14 14:31:44 UTC

Hi there!

It looks like a problem with the systemctl list-units output, which (in some cases) includes some glyphs to represent the status of the unit. Thus, when a unit is `masked` or `not-found` a ● (dot) is shown (at least when using systemd >= 246).

> 
> $ systemctl list-units --full --all --type mount --no-legend 
>    -.mount                        loaded    active   mounted Root Mount
>  ● boot-efi.mount                 masked    active   mounted /boot/efi
> 

Thus, `systemctl --runtime (un)mask` call at [1] is receiving the glyph instead the unit name, which cause the issue.

I found a patch in systemd repository [2] which should fix the issue by hiding the glyph when --no-legend flag is provided, but I'm not sure if it has arrived to SLE yet. According to my tests, I'd say no.

Fortunately, adding the --plain flag to the command makes the output looks like before. I have tested it in SLE-15-SP3 and Tumbleweed and it works. Patch [3] is on the way, under review.


[1] https://github.com/yast/yast-storage-ng/blob/5a02f4a9cf8750d1a8a2859498042a06968e2454/src/bin/mask-systemd-units#L41
[2] https://github.com/systemd/systemd/pull/15081
[3] https://github.com/yast/yast-storage-ng/pull/1234

Comment 8 Swamp Workflow Management 2021-10-27 13:22:57 UTC

SUSE-RU-2021:3536-1: An update that has three recommended fixes can now be installed.

Category: recommended (low)
Bug References: 1187270,1191109,1191347
CVE References: 
JIRA References: 
Sources used:
SUSE Linux Enterprise Module for Basesystem 15-SP3 (src):    yast2-storage-ng-4.3.56-3.12.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.

Comment 9 Swamp Workflow Management 2021-10-27 13:50:37 UTC

openSUSE-RU-2021:3536-1: An update that has three recommended fixes can now be installed.

Category: recommended (low)
Bug References: 1187270,1191109,1191347
CVE References: 
JIRA References: 
Sources used:
openSUSE Leap 15.3 (src):    yast2-storage-ng-4.3.56-3.12.1

Comment 18 Stefan Hundhammer 2022-09-21 15:56:16 UTC

This is already fixed, just the bug was never closed.