Bug 980521

Summary: gvfs fuse mounts do not appear breaking access for legacy apps
Product: [openSUSE] openSUSE Tumbleweed Reporter: Martin Szulecki <novell>
Component: X.OrgAssignee: Dr. Werner Fink <werner>
Status: RESOLVED FIXED QA Contact: E-mail List <xorg-maintainer-bugs>
Severity: Normal    
Priority: P5 - None CC: eich, mstaudt, novell, zaitor
Version: Current   
Target Milestone: ---   
Hardware: x86-64   
OS: SUSE Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: openSUSE Tumbleweed 20160516 GVFS fuse mounts not appearing
Log of gdm starting up on vt8 breaking gvfsd-fuse mounting
/etc/X11/xdm/sys.xsession

Description Martin Szulecki 2016-05-18 10:47:02 UTC
Created attachment 677410 [details]
openSUSE Tumbleweed 20160516 GVFS fuse mounts not appearing

I noticed that on openSUSE Tumbleweed latest snapshot, including the 20160516 live CD, GVFS fuse mounts for accessing remote locations using "/run/user/1000/gvfs" are not working (see attachment).

Manually restarting "gvfsd-fuse" makes them appear again if mounts are added/removed. Therefore this breaks all legacy applications which access a mount's files using the filesystem and not with GIO directly.

This can be tested by logging in, then running "gvfs-mount localtest:///".
The mountpoint does not appear in "/run/user/$(id)/gvfs".

After some investigation with upstream (See https://bugzilla.gnome.org/show_bug.cgi?id=766433) it kind of points towards a problem around the DBUS session bus.

Since you know the gdm/gnome-session/dbus-launch procedure of openSUSE better you might be able to help.
Comment 1 Bjørn Lie 2016-05-18 11:12:10 UTC
What makes this even more fun is that I can't reproduce :-/

bjolie@haldis:~> ls /run/user/1000/gvfs
bjolie@haldis:~> gvfs-mount localtest:///
bjolie@haldis:~> ls /run/user/1000/gvfs
localtest:
bjolie@haldis:~> ls /run/user/1000/gvfs/localtest\:/
bin   dev  home  lib64       mnt  proc  run   selinux  sys  usr
boot  etc  lib   lost+found  opt  root  sbin  srv      tmp  var
bjolie@haldis:~>
Comment 2 Martin Szulecki 2016-05-18 11:57:27 UTC
Nasty. I also have systems where I can't reproduce it (Macbooks).

However it is visible when booting the live CD in a VM (the path is /run/user/999/gvfs in that case and use rootfstype=ramfs if you get a kernel panic) and some desktop systems.

Could you please check if "ps ax | grep gvfsd" only gives you a single gvfsd instance?
Comment 3 Bjørn Lie 2016-05-18 12:23:29 UTC
bjolie@drude:~> ps ax | grep gvfsd
 1084 ?        Sl     0:00 /usr/lib/gvfs/gvfsd
 1089 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-fuse /run/user/1000/gvfs -f -o big_writes
 1192 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-metadata
 1497 pts/0    S+     0:00 grep --color=auto gvfsd
bjolie@drude:~> su -
Passord: 
drude:~ # ps ax | grep gvfsd
 1084 ?        Sl     0:00 /usr/lib/gvfs/gvfsd
 1089 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-fuse /run/user/1000/gvfs -f -o big_writes
 1192 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-metadata
 1558 pts/0    S+     0:00 grep --color=auto gvfsd
drude:~ # exit
logout
bjolie@drude:~> ls /run/user/1000/
dconf  gdm  gnome-shell  gvfs  keyring  krb5cc  pulse  systemd
bjolie@drude:~> ls /run/user/1000/gvfs
bjolie@drude:~> gvfs-mount localtest:///
bjolie@drude:~> 
bjolie@drude:~> ls /run/user/1000/gvfs
localtest:
bjolie@drude:~> ps ax | grep gvfsd
 1084 ?        Sl     0:00 /usr/lib/gvfs/gvfsd
 1089 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-fuse /run/user/1000/gvfs -f -o big_writes
 1192 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-metadata
 1601 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-localtest --spawner :1.1 /org/gtk/gvfs/exec_spaw/0
 1625 pts/0    S+     0:00 grep --color=auto gvfsd
bjolie@drude:~> su -
Passord: 
drude:~ # ps ax | grep gvfsd
 1084 ?        Sl     0:00 /usr/lib/gvfs/gvfsd
 1089 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-fuse /run/user/1000/gvfs -f -o big_writes
 1192 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-metadata
 1601 ?        Sl     0:00 /usr/lib/gvfs/gvfsd-localtest --spawner :1.1 /org/gtk/gvfs/exec_spaw/0
 1684 pts/0    R+     0:00 grep --color=auto gvfsd
drude:~ # 


----
So nope

Since I assume you are running a clean TW system, mind doing
 osc getbinaries openSUSE:Factory gvfs standard x86_64

and installing those?
I don't really expect the versionbump to make a diff, but lets test anyway.
Comment 4 Martin Szulecki 2016-05-18 16:15:47 UTC
Ok, thanks. It kind of proves my point.

On systems with only a single "gvfsd" running after login everything works normally as the DBUS session bus is the same.

On systems with two "gvfsd" running after login the issue comes up as "gvfsd-fuse" doesn't get the mount messages most likely because it's using a different DBUS session bus.

I can reproduce the issue on a clean openSUSE Tumbleweed Live CD system in a VM (see screenshot). The other systems can not be considered absolutely clean installs.

It isn't fixed after installing obs binary packages directly and I am nevertheless using a branched GNOME:Factory "gvfs" with added debugging on affected systems.

Also tested running the same system with nouveau instead of the NVIDIA blob and got a single "gvfsd" with working mounts...

It smells like a nasty thing somewhere in the gdm/login/xsession process.
Comment 5 Martin Szulecki 2016-05-20 12:10:38 UTC
Created attachment 677749 [details]
Log of gdm starting up on vt8 breaking gvfsd-fuse mounting

Upstream thinks it's an openSUSE issue.
I updated the upstream bug with more information and reassigned to gdm for a final clarification and also attached the same file here fore completeness.

Do you know how to determine what started the second gvfsd instance?
Does the pstree in the usptream bug look correct as, if I understood correctly, upstream says "gdm-x-session" should also be run as gdm user?
Comment 6 Martin Szulecki 2016-05-20 18:29:56 UTC
Ok, well. Looks like it is indeed openSUSE related.

I found out what resets the dbus session bus address.
It is the DBUS check within "/etc/X11/xdm/sys.xsession".

If I comment out the whole "Check if a dbus is required for e.g. plain xdm sessions" block, the dbus user session is correctly propagated to gnome-session and all afterwards starts up correctly. GVFS works and only has a single process running.

This also applies to the openSUSE TW Live CD... gvfs works in that case again, too.

Still need to check why the logic fails.
Comment 7 Martin Szulecki 2016-05-20 23:17:18 UTC
So, when "gdm-x-session" runs "/etc/gdm/Xsession" is called.
Due to SuSE specific patches that also runs "/etc/X11/xdm/Xsession" which runs "/etc/X11/xdm/sys.xsession".

However, when "gdm-x-session" is run, it spawns a new user session bus already if one was not yet started for the user.

Thus DBUS_SESSION_BUS_ADDRESS will be populated with a valid session.

Once within the "sys.xsession" script, it will destroy the valid session with the line "test -n "$dpid" || unset DBUS_SESSION_BUS_ADDRESS", which causes a new dbus user session to be created by the script and causes havok later on for the dbus user session pingpong of GVFS and other GNOME related components.

As far as I understand this really messy startup procedure... I think it should not unset a valid DBUS_SESSION_BUS_ADDRESS and therefore not start a new dbus-daemon in the "gdm" case to fix proper GNOME session startup for "gdm".
Comment 8 Egbert Eich 2016-05-21 05:28:18 UTC
Werner, this dbus code in sys.xsession is your code.
Please have a look.
Comment 9 Dr. Werner Fink 2016-05-24 13:37:54 UTC
(In reply to Martin Szulecki from comment #7)

Please could you add two lines

 printenv
 set -x

after the comment

 #
 # Check if a dbus is required for e.g. plain xdm sessions
 #

in /etc/X11/xdm/sys.xsession and attach the users X session log with curruent environment as well as with the resulting trace of the following shell commands.
Comment 10 Egbert Eich 2016-05-24 14:13:23 UTC
Werner, I've done so already. Point is, that this loop does not work:

        for suid in "${HOME}/.dbus/session-bus/"${mid}* ; do
            test -e "$suid" || break
            grep -q $guid "$suid" || continue
            dpid=$(grep -E '^DBUS_SESSION_BUS_PID=[[:digit:]]+' "$suid")
            test /proc/${dpid#*=}/exe -ef $dbusdaemon && continue
            unset DBUS_SESSION_BUS_ADDRESS
            break
        done

There is no file in ${HOME}/.dbus/session-bus/ containing $guid. In fact, there isn't even a file in this directory that was created at or after the start of the gdm session.
DBUS_SESSION_BUS_ADDRESS is set and gets unset due to the failure to verify it.

I don't know how it can be verified and frankly I think we have to give up the idea that we need to and should instead trust DBUS_SESSION_BUS_ADDRESS.
Comment 11 Dr. Werner Fink 2016-05-24 14:27:51 UTC
(In reply to Egbert Eich from comment #10)

Ahh ... yes, this was one of my assumptions, that is a change in the behaviour of how dbus-daemon has works or is used.  Currently I've trouble to test this as my X session is not supported anymore by gdm
Comment 12 Dr. Werner Fink 2016-05-24 14:44:32 UTC
Maybe the dbus-send command is the better way to test if DBUS_SESSION_BUS_ADDRESS is valid
Comment 13 Dr. Werner Fink 2016-05-24 15:43:31 UTC
Created attachment 678158 [details]
/etc/X11/xdm/sys.xsession

Yet an other /etc/X11/xdm/sys.xsession. With this now the dbus-send command is used to check for a valid dbus-daemon session. For testing please replace the current /etc/X11/xdm/sys.xsession with the attachment.  Note that you might run

  chmod 755 /etc/X11/xdm/sys.xsession

afterwards.  Now does this new version fix the problem?
Comment 14 Martin Szulecki 2016-05-24 23:38:49 UTC
The provided (In reply to Dr. Werner Fink from comment #13)
> Created attachment 678158 [details]
> /etc/X11/xdm/sys.xsession
> 
> Yet an other /etc/X11/xdm/sys.xsession. With this now the dbus-send command
> is used to check for a valid dbus-daemon session. For testing please replace
> the current /etc/X11/xdm/sys.xsession with the attachment.  Note that you
> might run
> 
>   chmod 755 /etc/X11/xdm/sys.xsession
> 
> afterwards.  Now does this new version fix the problem?

The "sys.xsession" has a syntax error in line 122.
After fixing it, I can confirm it works.

There is also an example in the man page of dbus-launch regarding testing for the dbus-daemon but I am not sure if this is the "correct" way:
https://dbus.freedesktop.org/doc/dbus-launch.1.html
Comment 15 Dr. Werner Fink 2016-05-25 08:14:04 UTC
(In reply to Martin Szulecki from comment #14)

Oops yes that misses a `then', thanks a lot for debugging!
Comment 16 Dr. Werner Fink 2016-05-25 08:35:37 UTC
(In reply to Martin Szulecki from comment #14)

> There is also an example in the man page of dbus-launch regarding testing for 
> the dbus-daemon but I am not sure if this is the "correct" way:
> https://dbus.freedesktop.org/doc/dbus-launch.1.html

Yep, that is correct ... in sys.xsession this is extend to use dbus-launch
with the option --exit-with-session to let dbus-daemon become one of the parent processes of the final X session leader (session of window manager).  This is done the `set' which prefix the final command line with the dbus-launch command and its options.
Comment 17 Dr. Werner Fink 2016-05-25 10:53:21 UTC
Have a look at home:WernerFink:branches:X11:XOrg/xdm done for this bug here but not submitted due bug boo#972787.
Comment 18 Egbert Eich 2016-05-25 12:26:49 UTC
Werner, could it be that you forgot to tar up the files?
I cannot find any change in there but the changelog itself.

Moreover, let's treat this independent from boo#972787. We need to get this fixed now. We can later merge these fixes back to boo#972787.
I know, this is far from ideal, we really should have a git repo for the files in xdm.tar.bz2.
Max is currently looking into another issue with another file in this tar ball, I'd like to sync these up so he won't have to do a 2nd submission.
Comment 19 Dr. Werner Fink 2016-05-25 12:31:34 UTC
(In reply to Egbert Eich from comment #18)

Thanks for spotting ... now the xdm.tar.bz2 should be uptodate
Comment 20 Martin Szulecki 2016-05-26 10:19:32 UTC
I can confirm that the branched "xdm" packages work fine on my real box and the openSUSE live CD to fix the issue due to the updated "sys.xsession" script.

+1 for adding a git repo of the "xdm tar"...
Comment 21 Martin Szulecki 2016-06-02 18:28:51 UTC
It would be good to get this in asap as it does not depend on the gpg changes.

The problem already exists in the distro since April and any new user that hits it might be one less to become a longtime user of openSUSE... ;)
Comment 22 Bjørn Lie 2016-06-19 18:06:38 UTC
https://build.opensuse.org/request/show/400372

As far as I can tell this is fixed by the above sub to TW, but moving bug to xorg for verification and closing.
Comment 23 Max Staudt 2016-06-20 08:21:34 UTC
That SR looks good to me, too. Thanks!
Comment 24 Egbert Eich 2016-06-20 08:57:48 UTC
Meanwhile the fix has been submitted independently of bsc#972787. Closing.