Bug 1175487

Summary: Emacs 27.1: package downloads & compilation yields "end of file during parsing" error
Product: [openSUSE] openSUSE Tumbleweed Reporter: Philipp Reichmuth <philipp.reichmuth>
Component: X11 ApplicationsAssignee: Dr. Werner Fink <werner>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: dmacvicar, philipp.reichmuth, psychonaut, roger.whittaker, sean, toganm, werner
Version: CurrentFlags: werner: SHIP_STOPPER?
Target Milestone: ---   
Hardware: x86-64   
OS: openSUSE Tumbleweed   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: differences of fnal configurations

Description Philipp Reichmuth 2020-08-19 12:38:31 UTC
DESCRIPTION
Emacs 27.1 has a bug related to HTTP(S) downloads and/or package compilation that causes downloads and compilation of ELPA packages using Quelpa (https://github.com/quelpa/quelpa) to fail. Compilation of downloaded files gives an error: "error in process sentinel: End of file during parsing". 

Being a bit of a novice, it is hard for me to pinpoint the error, but it's reproducible when installing the Spacemacs Emacs distribution (https://github.com/syl20bnr/spacemacs/). This is because Spacemacs, upon installation, downloads and installs lots of packages. This will yield dozens of "end of file during parsing" messages.

This error is usually indicative of missed parentheses, but it seems to be related to the download operation, because (a) it is not reproducible (it concerns different files every time), and (b) the files in question are downloaded from their master repositories and are unlikely to contain missing parentheses.

HOW TO REPRODUCE

Installation of Spacemacs according to the recommended procedure:
- Delete ./emacs.d
- Delete .spacemacs
- Download spacemacs using `git clone --branch develop https://github.com/syl20bnr/spacemacs ~/.emacs.d` 
- Start Emacs 27.1.

EXPECTED BEHAVIOUR

Clean installation of Spacemacs.

OBSERVED BEHAVIOUR

Spacemacs begins downloading and compiling packages from ELPA. A significant, but not reproducible number of these fail to compile and give the following error message:

error in process sentinel: async-handle-result: End of file during parsing
error in process sentinel: End of file during parsing

This happens regardless of whether I use the "develop" (0.300) or the stable "master" (0.200) branch of Spacemacs, and regardless of whether I use HTTPS or HTTP (with `emacs --insecure`).

Different packages are concerned every time.

Two examples of messages from this installation are here: 
https://pastebin.com/AzPJ0FZr and https://pastebin.com/VQzD6zgY
Comment 1 Dr. Werner Fink 2020-08-19 13:41:14 UTC
Wrong maintainer ... seems more a problem of spacemacs ...

here I can open URL ... e.g. https://www.gnu.org/gnu/thegnuproject.html

What I see from the pastbin ... thereare a lot of gone lisp functions which are not part of GNU Emacs 27.1 anymore ... or you have to require/load them like emacs-lisp/timer.el
Comment 2 Philipp Reichmuth 2020-08-19 15:32:22 UTC
For the record, this seems to be related to MELPA currenly being down, and to how Emacs behaves with respect to HTTP connection issues. 

Emacs 27.1 from the TW repository seems to give up quite quickly when it can't connect to melpa.org. After the connection fails, it seems to to receive some junk (probably an error message either from the server or locally, couldn't check which), which then gets interpreted as Lisp code and throws an error. 

Spacemacs is just particularly visible case of this, because upon installation there's lots of downloading and compiling going on. 

For comparison, I tried Emacs 27.1 from Andreas Schwab's repository instead, which tries to connect to melpa.org for much longer time until it gives up gracefully. In the Spacemacs case, this means that the first install with all the package downloads takes much longer, but in the end you get a working install.

In the meantime, while MELPA is down, the easiest workaround is to use a different mirror, e.g. by putting something like this in your init.el:

(require 'package)
(setq package-archives
   '(("melpa" . "https://raw.githubusercontent.com/d12frosted/elpa-mirror/master/melpa/")
     ("org"   . "https://raw.githubusercontent.com/d12frosted/elpa-mirror/master/org/")
     ("gnu"   . "https://raw.githubusercontent.com/d12frosted/elpa-mirror/master/gnu/")))
;; (setq package-check-signature nil) ;; probably not necessary
(package-initialize)

If I put this into the Spacemacs configuration, with Emacs 27.1 from the TW repository I still get spurious errors (such as org-mode failing to load), with the version from Andreas Schwab's repository it workes fine. So in the meantime I'll use that as a workaround.
Comment 3 Sean O'Connell 2020-08-20 02:04:50 UTC
This problem is not isolated to Spacemacs. I am running Emacs with a bespoke vanilla config, and encountered the same issue today upon upgrading from 26.3-3.5 to 27.1-1.1 (TW). Further, the problem is not related solely to MELPA (although I saw the same issues with downloading packages). The error also presents itself in Elfeed when downloading an enclosure (e.g. a podcast episode). The download fails with "error in process sentinel: End of file during parsing". The problem may be related to TLS. The download link that failed was https. My RSS feeds are a mix of http and https, and Elfeed was able to pull new articles - however I did not verify if the feeds it successfully pulled from were http or https.

Rolling back (via snapper), removing ~/.emacs.d/elpa/* and re-running Emacs 26.3-3.5 with the same configuration as prior re-downloaded all packages (via 'use-package' statements) without error. Elfeed downloads worked again, as well.
Comment 4 Dr. Werner Fink 2020-08-20 05:31:13 UTC
(In reply to Philipp Reichmuth from comment #2)

> If I put this into the Spacemacs configuration, with Emacs 27.1 from the TW
> repository I still get spurious errors (such as org-mode failing to load),
> with the version from Andreas Schwab's repository it workes fine. So in the
> meantime I'll use that as a workaround.

And what exactly is different to that version?
Comment 5 Philipp Reichmuth 2020-08-20 07:02:59 UTC
(In reply to Dr. Werner Fink from comment #4)
> (In reply to Philipp Reichmuth from comment #2)
> 
> > If I put this into the Spacemacs configuration, with Emacs 27.1 from the TW
> > repository I still get spurious errors (such as org-mode failing to load),
> > with the version from Andreas Schwab's repository it workes fine. So in the
> > meantime I'll use that as a workaround.
> 
> And what exactly is different to that version?

That version seems to have a more conservative patchset and slightly different compile-time features. 

Disclaimer: I don't know enough about the Emacs build system to be able to pinpoint the relevant differences from a .spec file; also they're quite big, 270k and 30k respectively.

Patchsets:
==========
https://build.opensuse.org/package/view_file/openSUSE:Factory/emacs/emacs.spec?expand=1 :
Source:         ftp://ftp.gnu.org/gnu/emacs/emacs-%{version}.tar.xz
Source1:        app-defaults.Emacs
Source2:        site-lisp.tar.bz2
Source3:        dot.gnu-emacs
Source4:        emacs-rpmlintrc
Source5:        emacs.sh
Source6:        ftp://ftp.gnu.org/gnu/emacs/emacs-%{version}.tar.xz.sig
# https://ftp.gnu.org/gnu/gnu-keyring.gpg
Source7:        %{name}.keyring
Source8:        emacs-%{version}-pdf.tar.xz
Patch:          emacs-27.1.dif
# PATCH-FIX-UPSTREAM Adjust to GnuPG 2.1 key listing change
Patch2:         emacs-24.4-glibc.patch
Patch4:         emacs-24.3-asian-print.patch
Patch5:         emacs-24.4-ps-bdf.patch
Patch7:         emacs-24.1-ps-mule.patch
Patch8:         emacs-24.4-nonvoid.patch
Patch11:        emacs-24.4-xim.patch
Patch12:        emacs-24.3-x11r7.patch
Patch15:        emacs-24.3-iconic.patch
Patch16:        emacs-24.4-flyspell.patch
Patch23:        emacs-25.1-custom-fonts.patch
# this patch works with both ImageMagick-6 and ImageMagick-7 for us,
# but that is because we ship /usr/include/ImageMagick-7/wand compat
# symlink
Patch24:        emacs-25.2-ImageMagick7.patch
Patch25:        emacs-26.1-xft4x11.patch
Patch26:        emacs-27.1-pdftex.patch

https://build.opensuse.org/package/view_file/home:AndreasSchwab:emacs:27/emacs/emacs.spec?expand=1 :
Source:         http://ftp.gnu.org/gnu/emacs/emacs-%{version}.tar.xz
Source2:        site-lisp.tar.bz2
Source3:        dot.gnu-emacs
Source4:        emacs-rpmlintrc
Source5:        emacs.sh
Patch:          emacs-25.3.dif
Patch5:         emacs-24.4-ps-bdf.patch
Patch7:         emacs-24.1-ps-mule.patch
Patch11:        emacs-24.4-xim.patch
Patch12:        emacs-24.3-x11r7.patch
# PATCH-FIX-SUSE bsc#1096354 -- emacs-x11 can't use X core fonts for the menu bar
Patch28:        emacs-25.3-xft4x11.patch
Patch99:        pdump.patch


Compile-time features:
======================
https://build.opensuse.org/package/view_file/openSUSE:Factory/emacs/emacs.spec?expand=1
DESKTOP="--with-x \
	 --with-xim \
	 --with-sound \
	 --with-xpm \
	 --with-jpeg \
	 --with-tiff \
	 --with-gif \
	 --with-png \
	 --with-rsvg \
	 --with-dbus \
	 --with-xft \
	 --without-gpm \
"
    GTK="${DESKTOP} \
	 --with-x-toolkit=gtk3 \
	 --with-toolkit-scroll-bars \
	 --x-includes=%{_x11inc} \
	 --x-libraries=%{_x11lib} \
	 --with-libotf \
	 --with-m17n-flt \
%if %{with cairo}
	 --with-cairo \
%endif
	 --with-xwidgets \
"
[...]
   COMP="--disable-build-details \
%if %{with mailutils}
	 --without-pop
	 --with-mailutils
%else
	 --with-pop \
%endif
	 --without-hesiod \
	 --with-gameuser=:games \
	 --with-kerberos \
	 --with-kerberos5 \
	 --with-file-notification=inotify \
	 --with-modules \
	 --enable-autodepend \
"
if (($(getconf LONG_BIT) < 62))
then
    COMP="${COMP} --with-wide-int"
fi

https://build.opensuse.org/package/view_file/home:AndreasSchwab:emacs:27/emacs/emacs.spec?expand=1 :
DESKTOP="--with-x \
	 --with-sound \
	 --with-xpm \
	 --with-jpeg \
	 --with-tiff \
	 --with-gif \
	 --with-png \
	 --with-rsvg \
	 --with-dbus \
	 --with-xft \
"
    GTK="${DESKTOP} \
%if 0%{?suse_version} > 1130
	 --with-x-toolkit=gtk3 \
%ifnarch ppc
%if 0%{?suse_version} > 1310
	 --with-xwidgets \
%endif
%endif
%else
	 --with-x-toolkit=gtk2 \
%endif
	 --with-toolkit-scroll-bars \
	 --x-includes=%{_x11inc} \
	 --x-libraries=%{_x11lib} \
	 --with-libotf \
	 --with-m17n-flt \
%if %{with cairo} && 0%{?suse_version} > 1130
	 --with-cairo \
%endif
"
   COMP="--with-pop \
	 --without-hesiod \
	 --with-kerberos \
	 --with-kerberos5 \
	 --with-xim \
	 --with-wide-int \
	 --with-gpm \
	 --with-file-notification=inotify \
	 --with-modules \
	 --enable-autodepend \
	 --with-file-notification=gfile \
"
Comment 6 Dr. Werner Fink 2020-08-20 08:32:35 UTC
(In reply to Philipp Reichmuth from comment #5)
> (In reply to Dr. Werner Fink from comment #4)
> > (In reply to Philipp Reichmuth from comment #2)
> > 
> > > If I put this into the Spacemacs configuration, with Emacs 27.1 from the TW
> > > repository I still get spurious errors (such as org-mode failing to load),
> > > with the version from Andreas Schwab's repository it workes fine. So in the
> > > meantime I'll use that as a workaround.
> > 
> > And what exactly is different to that version?
> 
> That version seems to have a more conservative patchset and slightly
> different compile-time features. 

No of the patches do change timings at download. And the version of
Factory uses more BuildRequires to enable the autodetection
of the configure script.

Teh only difference I see is that in Andreas version the resulting
configuration ignores --with-file-notification=inotify

> 
> Disclaimer: I don't know enough about the Emacs build system to be able to
> pinpoint the relevant differences from a .spec file; also they're quite big,
> 270k and 30k respectively.

Ahm ... which version of which repostitory do you have installed?

> 
> Patchsets:
> ==========
> https://build.opensuse.org/package/view_file/openSUSE:Factory/emacs/emacs.
> spec?expand=1 :
> Source:         ftp://ftp.gnu.org/gnu/emacs/emacs-%{version}.tar.xz
> Source1:        app-defaults.Emacs
> Source2:        site-lisp.tar.bz2
> Source3:        dot.gnu-emacs
> Source4:        emacs-rpmlintrc
> Source5:        emacs.sh
> Source6:        ftp://ftp.gnu.org/gnu/emacs/emacs-%{version}.tar.xz.sig
> # https://ftp.gnu.org/gnu/gnu-keyring.gpg
> Source7:        %{name}.keyring
> Source8:        emacs-%{version}-pdf.tar.xz
> Patch:          emacs-27.1.dif
> # PATCH-FIX-UPSTREAM Adjust to GnuPG 2.1 key listing change
> Patch2:         emacs-24.4-glibc.patch
> Patch4:         emacs-24.3-asian-print.patch
> Patch5:         emacs-24.4-ps-bdf.patch
> Patch7:         emacs-24.1-ps-mule.patch
> Patch8:         emacs-24.4-nonvoid.patch
> Patch11:        emacs-24.4-xim.patch
> Patch12:        emacs-24.3-x11r7.patch
> Patch15:        emacs-24.3-iconic.patch
> Patch16:        emacs-24.4-flyspell.patch
> Patch23:        emacs-25.1-custom-fonts.patch
> # this patch works with both ImageMagick-6 and ImageMagick-7 for us,
> # but that is because we ship /usr/include/ImageMagick-7/wand compat
> # symlink
> Patch24:        emacs-25.2-ImageMagick7.patch
> Patch25:        emacs-26.1-xft4x11.patch
> Patch26:        emacs-27.1-pdftex.patch
> 
> https://build.opensuse.org/package/view_file/home:AndreasSchwab:emacs:27/
> emacs/emacs.spec?expand=1 :
> Source:         http://ftp.gnu.org/gnu/emacs/emacs-%{version}.tar.xz
> Source2:        site-lisp.tar.bz2
> Source3:        dot.gnu-emacs
> Source4:        emacs-rpmlintrc
> Source5:        emacs.sh
> Patch:          emacs-25.3.dif
> Patch5:         emacs-24.4-ps-bdf.patch
> Patch7:         emacs-24.1-ps-mule.patch
> Patch11:        emacs-24.4-xim.patch
> Patch12:        emacs-24.3-x11r7.patch
> # PATCH-FIX-SUSE bsc#1096354 -- emacs-x11 can't use X core fonts for the
> menu bar
> Patch28:        emacs-25.3-xft4x11.patch
> Patch99:        pdump.patch
> 
> 
> Compile-time features:
> ======================
> https://build.opensuse.org/package/view_file/openSUSE:Factory/emacs/emacs.
> spec?expand=1
> DESKTOP="--with-x \
> 	 --with-xim \
> 	 --with-sound \
> 	 --with-xpm \
> 	 --with-jpeg \
> 	 --with-tiff \
> 	 --with-gif \
> 	 --with-png \
> 	 --with-rsvg \
> 	 --with-dbus \
> 	 --with-xft \
> 	 --without-gpm \
> "
>     GTK="${DESKTOP} \
> 	 --with-x-toolkit=gtk3 \
> 	 --with-toolkit-scroll-bars \
> 	 --x-includes=%{_x11inc} \
> 	 --x-libraries=%{_x11lib} \
> 	 --with-libotf \
> 	 --with-m17n-flt \
> %if %{with cairo}
> 	 --with-cairo \
> %endif
> 	 --with-xwidgets \
> "
> [...]
>    COMP="--disable-build-details \
> %if %{with mailutils}
> 	 --without-pop
> 	 --with-mailutils
> %else
> 	 --with-pop \
> %endif
> 	 --without-hesiod \
> 	 --with-gameuser=:games \
> 	 --with-kerberos \
> 	 --with-kerberos5 \
> 	 --with-file-notification=inotify \
> 	 --with-modules \
> 	 --enable-autodepend \
> "
> if (($(getconf LONG_BIT) < 62))
> then
>     COMP="${COMP} --with-wide-int"
> fi
> 
> https://build.opensuse.org/package/view_file/home:AndreasSchwab:emacs:27/
> emacs/emacs.spec?expand=1 :
> DESKTOP="--with-x \
> 	 --with-sound \
> 	 --with-xpm \
> 	 --with-jpeg \
> 	 --with-tiff \
> 	 --with-gif \
> 	 --with-png \
> 	 --with-rsvg \
> 	 --with-dbus \
> 	 --with-xft \
> "
>     GTK="${DESKTOP} \
> %if 0%{?suse_version} > 1130
> 	 --with-x-toolkit=gtk3 \
> %ifnarch ppc
> %if 0%{?suse_version} > 1310
> 	 --with-xwidgets \
> %endif
> %endif
> %else
> 	 --with-x-toolkit=gtk2 \
> %endif
> 	 --with-toolkit-scroll-bars \
> 	 --x-includes=%{_x11inc} \
> 	 --x-libraries=%{_x11lib} \
> 	 --with-libotf \
> 	 --with-m17n-flt \
> %if %{with cairo} && 0%{?suse_version} > 1130
> 	 --with-cairo \
> %endif
> "
>    COMP="--with-pop \
> 	 --without-hesiod \
> 	 --with-kerberos \
> 	 --with-kerberos5 \
> 	 --with-xim \
> 	 --with-wide-int \
> 	 --with-gpm \
> 	 --with-file-notification=inotify \
> 	 --with-modules \
> 	 --enable-autodepend \
> 	 --with-file-notification=gfile \
> "
Comment 7 Dr. Werner Fink 2020-08-20 08:34:09 UTC
Created attachment 840840 [details]
differences of fnal configurations
Comment 8 Dr. Werner Fink 2020-08-20 08:38:27 UTC
(In reply to Dr. Werner Fink from comment #7)
> Created attachment 840840 [details]
> differences of fnal configurations

OK ... also

-  Should Emacs use mmap(2) for buffer allocation?         no
+  Should Emacs use mmap(2) for buffer allocation?         yes

in Andreas version standard buffer allocation is used and

-  Does Emacs use -llcms2?                                 no
+  Does Emacs use -llcms2?                                 yes

that is the libcms2 (Little CMS Engine - A color managment library and tools.
) is in Andreas version not found

... why gpm should be used for X based GNU Emacs ... don't know
Comment 9 Dr. Werner Fink 2020-08-20 08:55:34 UTC
From NEWS25

 ** 'configure' now prefers inotify to gfile for file notification,
 unless gfile is explicitly requested via --with-file-notification='gfile'.

as in NEWS for 26 and 27 is nothing written else about I prefer inotify
Comment 10 Philipp Reichmuth 2020-08-20 09:20:13 UTC
(In reply to Dr. Werner Fink from comment #6)
> (In reply to Philipp Reichmuth from comment #5)
> > (In reply to Dr. Werner Fink from comment #4)
> > > (In reply to Philipp Reichmuth from comment #2)
> > > 
> > > > If I put this into the Spacemacs configuration, with Emacs 27.1 from the TW
> > > > repository I still get spurious errors (such as org-mode failing to load),
> > > > with the version from Andreas Schwab's repository it workes fine. So in the
> > > > meantime I'll use that as a workaround.
> > > 
> > > And what exactly is different to that version?
> > 
> > That version seems to have a more conservative patchset and slightly
> > different compile-time features. 
> 
> No of the patches do change timings at download. And the version of
> Factory uses more BuildRequires to enable the autodetection
> of the configure script.
> 
> Teh only difference I see is that in Andreas version the resulting
> configuration ignores --with-file-notification=inotify

I'm sorry that I can't pinpoint it better. I'm not a developer and I've been using OpenSUSE for about two months. 

Maybe there are some differences in the packaged elisp files. 

All I can say is that if I remove the mainline version and install the version from Andreas Schwab's repository, and make no further changes, it works, and if I go back to the mainline version, it stops working.

> > Disclaimer: I don't know enough about the Emacs build system to be able to
> > pinpoint the relevant differences from a .spec file; also they're quite big,
> > 270k and 30k respectively.
> 
> Ahm ... which version of which repostitory do you have installed?

Sorry, that should have been 252k and 30.7k.

zypper search -s emacs gives:

   | Name   | Typ        | Version   | Arch   | Repository
---+--------+------------+-----------+--------+----------------------------
[...]
i+ | emacs | Paket      | 27.1-12.1  | x86_64 | home:AndreasSchwab:emacs:27
v  | emacs | Paket      | 27.1-12.1  | i586   | home:AndreasSchwab:emacs:27
v  | emacs | Paket      | 27.1-1.1   | x86_64 | openSUSE:Tumbleweed
v  | emacs | Paket      | 27.1-1.1   | x86_64 | repo-oss
v  | emacs | Paket      | 27.1-1.1   | i586   | openSUSE:Tumbleweed
v  | emacs | Paket      | 27.1-1.1   | i586   | repo-oss
[...]

Here are the respective repositories (from zypper lr -e):

[https-download.opensuse.org-b9d8ea21]
name=home:AndreasSchwab:emacs:27
enabled=1
autorefresh=1
baseurl=https://download.opensuse.org/repositories/home:/AndreasSchwab:/emacs:/27/f/
type=rpm-md
priority=90
keeppackages=0

[https-download.opensuse.org-57bd66ed]
name=openSUSE:Tumbleweed
enabled=1
autorefresh=1
baseurl=https://download.opensuse.org/repositories/openSUSE:/Tumbleweed/standard/
type=rpm-md
keeppackages=0

[repo-oss]
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/tumbleweed/repo/oss
type=rpm-md
keeppackages=0
Comment 11 Dr. Werner Fink 2020-08-20 12:36:56 UTC
In the repository editors there is now an emacs 27.1 without mmap for buffer allocation
Comment 12 Sean O'Connell 2020-08-20 17:37:05 UTC
(In reply to Dr. Werner Fink from comment #11)
> In the repository editors there is now an emacs 27.1 without mmap for buffer
> allocation

I've setup a test machine running TW 20200818 and installed emacs-27.1-337 from the editors repo. Problem persists as before:

    error in process sentinel: async-handle-result: End of file during parsing
    error in process sentinel: End of file during parsing

when installing packages from MELPA/GNU ELPA and when downloading enclosures in Elfeed. Elfeed downloads fail entirely. It seems like package installations do complete, but report the above error.

I can leave this test environment in place if you'd like me to test further package builds or provide specific logs.
Comment 13 Togan Muftuoglu 2020-08-20 19:34:55 UTC
I am running the latest Tumbleweed # VERSION="20200818" with just emacs from the editors repo .

When calling /usr/bin/emacs-gtk or simply emacs-gtk here is an extra message. 

Warning: game dir ’/var/games/emacs’: Permission denied

When just using /usr/bin/emacs the permissioin message is not there but melpa package async (async-20200809.501/async.el) is broken with sentinel error message but with /usr/bin/emacs-gtk the error is gone and the package works as it should

But must say whatever worked like a charm in 26.3 turns out to be an issue as 27.1 is very picky how things are loaded and from what I can see how I order them in the init file makes a difference.

I am not going to make another bug report but the man page needs fixing

man emacs says

================================================
FILES
       /usr/local/share/info — files for the Info documentation browser.  The complete text of the Emacs reference manual is included in a convenient tree structured form.  Also includes  the  Emacs  Lisp
       Reference Manual, useful to anyone wishing to write programs in the Emacs Lisp extension language, and the Introduction to Programming in Emacs Lisp.

       /usr/local/share/emacs/$VERSION/lisp — Lisp source files and compiled files that define most editing commands.  Some are preloaded; others are autoloaded from this directory when used.

       /usr/local/libexec/emacs/$VERSION/$ARCH — various programs that are used with GNU Emacs.

       /usr/local/share/emacs/$VERSION/etc — various files of information.

       /usr/local/share/emacs/$VERSION/etc/DOC.*  —  contains the documentation strings for the Lisp primitives and preloaded Lisp functions of GNU Emacs.  They are stored here to reduce the size of Emacs  proper.

===================================================
Comment 14 Sean O'Connell 2020-08-20 20:15:56 UTC
I have done further testing on this issue and have isolated the problem to interaction with the 'async' package (https://github.com/jwiegley/emacs-async).

There are two async package versions currently available:

async 1.9.3 (gnu)
async 20200809.501 (melpa)

I have tested the Emacs packages from the TW repo, from the Editors repo, and from Andreas Schwab's Emacs repo.

Andreas' build has no issue with either async package (gnu or melpa).

The builds from TW and Editors both present the "error in process sentinel: async-handle-result.." error once either async package (gnu or melpa) is loaded and a network file download operation is performed (e.g. installing a package or downloading an enclosure in Elfeed).
Comment 15 Dr. Werner Fink 2020-08-21 07:28:55 UTC
(In reply to Sean O'Connell from comment #14)
> I have done further testing on this issue and have isolated the problem to
> interaction with the 'async' package
> (https://github.com/jwiegley/emacs-async).
> 
> There are two async package versions currently available:
> 
> async 1.9.3 (gnu)
> async 20200809.501 (melpa)
> 
> I have tested the Emacs packages from the TW repo, from the Editors repo,
> and from Andreas Schwab's Emacs repo.
> 
> Andreas' build has no issue with either async package (gnu or melpa).
> 
> The builds from TW and Editors both present the "error in process sentinel:
> async-handle-result.." error once either async package (gnu or melpa) is
> loaded and a network file download operation is performed (e.g. installing a
> package or downloading an enclosure in Elfeed).

Hmmm ... there is one difference between Andreas approach and my ... in my spec file the whole lisp tree is byte compiled with the fresh (t)emacs (boot strapped).  Whereas Andreas takes the byte copiled lisp file as they are.

In principle this should not make a difference ... otherwise there is a real bug or the soource tar ball is not complete
Comment 16 Dr. Werner Fink 2020-08-21 07:36:45 UTC
(In reply to Togan Muftuoglu from comment #13)
> I am running the latest Tumbleweed # VERSION="20200818" with just emacs from
> the editors repo .
> 
> When calling /usr/bin/emacs-gtk or simply emacs-gtk here is an extra
> message. 
> 
> Warning: game dir ’/var/games/emacs’: Permission denied
> 
> When just using /usr/bin/emacs the permissioin message is not there but
> melpa package async (async-20200809.501/async.el) is broken with sentinel
> error message but with /usr/bin/emacs-gtk the error is gone and the package
> works as it should

The warning is about you are not member of the group games,

Oug? You've said that with /usr/bin/emacs-gtk the error is gone?  If so you might check is there is a background process running for a emacs server

  ps ux | grep emacs

this because there exists a systemd user service unit which might start such a daemon server:

  werner/emacs> ps aux | grep emacs
  werner   21109 18.9  0.7 255536 63320 ?        Ssl  09:32   0:01 /usr/bin/emacs --fg-daemon
  werner   21129  0.0  0.0   3400   608 pts/0    S+   09:32   0:00 grep emacs

If you reinstall emacs you might restart this user service unit

  systemctl --user restart emacs

if this is not the case then one of the environment variables in the script /usr/bin/emacs cause the broken behaviour.

> I am not going to make another bug report but the man page needs fixing
> 
> man emacs says

Ah ... OK that isn't deliverd from configure script to the manual pages :(

> 
> ================================================
> FILES
>        /usr/local/share/info — files for the Info documentation browser. 
> The complete text of the Emacs reference manual is included in a convenient
> tree structured form.  Also includes  the  Emacs  Lisp
>        Reference Manual, useful to anyone wishing to write programs in the
> Emacs Lisp extension language, and the Introduction to Programming in Emacs
> Lisp.
> 
>        /usr/local/share/emacs/$VERSION/lisp — Lisp source files and compiled
> files that define most editing commands.  Some are preloaded; others are
> autoloaded from this directory when used.
> 
>        /usr/local/libexec/emacs/$VERSION/$ARCH — various programs that are
> used with GNU Emacs.
> 
>        /usr/local/share/emacs/$VERSION/etc — various files of information.
> 
>        /usr/local/share/emacs/$VERSION/etc/DOC.*  —  contains the
> documentation strings for the Lisp primitives and preloaded Lisp functions
> of GNU Emacs.  They are stored here to reduce the size of Emacs  proper.
> 
> ===================================================
Comment 17 Togan Muftuoglu 2020-08-21 07:49:36 UTC
(In reply to Dr. Werner Fink from comment #16)
> (In reply to Togan Muftuoglu from comment #13)
> > I am running the latest Tumbleweed # VERSION="20200818" with just emacs from
> > the editors repo .
> > 
> > When calling /usr/bin/emacs-gtk or simply emacs-gtk here is an extra
> > message. 
> > 
> > Warning: game dir ’/var/games/emacs’: Permission denied
> > 
> > When just using /usr/bin/emacs the permissioin message is not there but
> > melpa package async (async-20200809.501/async.el) is broken with sentinel
> > error message but with /usr/bin/emacs-gtk the error is gone and the package
> > works as it should
> 
> The warning is about you are not member of the group games,



The user is not a member of it, yet it doesn't explain the fact why when I enter emacs or /usr/bin/emacs as the command this warning doesn't come up. So I am not so sure being a member of the user group or not is the culprit here.

The settings are default installation, hence if games should be one of the default groups the user be part of it then install logic is not fully integrated in to the process. 




> 
> Oug? You've said that with /usr/bin/emacs-gtk the error is gone?  If so you
> might check is there is a background process running for a emacs server
> 
>   ps ux | grep emacs
> 
> this because there exists a systemd user service unit which might start such
> a daemon server:
> 
>   werner/emacs> ps aux | grep emacs
>   werner   21109 18.9  0.7 255536 63320 ?        Ssl  09:32   0:01
> /usr/bin/emacs --fg-daemon
>   werner   21129  0.0  0.0   3400   608 pts/0    S+   09:32   0:00 grep emacs


I am not running it via systemd or via daemon option in the startup but rather through the init.el by using the server package


 (add-hook 'after-init-hook
           (lambda ()
             (require 'server)
             (unless (server-running-p)
               (server-start))))
Comment 18 Dr. Werner Fink 2020-08-21 08:07:52 UTC
(In reply to Togan Muftuoglu from comment #17)
> (In reply to Dr. Werner Fink from comment #16)
> > (In reply to Togan Muftuoglu from comment #13)
> > > I am running the latest Tumbleweed # VERSION="20200818" with just emacs from
> > > the editors repo .
> > > 
> > > When calling /usr/bin/emacs-gtk or simply emacs-gtk here is an extra
> > > message. 
> > > 
> > > Warning: game dir ’/var/games/emacs’: Permission denied
> > > 
> > > When just using /usr/bin/emacs the permissioin message is not there but
> > > melpa package async (async-20200809.501/async.el) is broken with sentinel
> > > error message but with /usr/bin/emacs-gtk the error is gone and the package
> > > works as it should
> > 
> > The warning is about you are not member of the group games,
> 
> The user is not a member of it, yet it doesn't explain the fact why when I
> enter emacs or /usr/bin/emacs as the command this warning doesn't come up.
> So I am not so sure being a member of the user group or not is the culprit
> here.
> 
> The settings are default installation, hence if games should be one of the
> default groups the user be part of it then install logic is not fully
> integrated in to the process. 

The emacs server does not show this messages, yes!  But starting it by hand
as server it should:

  werner/emacs> /usr/bin/emacs-gtk -Q --fg-daemon

  Warning: due to a long standing Gtk+ bug
  https://gitlab.gnome.org/GNOME/gtk/issues/221
  Emacs might crash when run in daemon mode and the X11 connection is unexpectedly lost.
  Using an Emacs configured with --with-x-toolkit=lucid does not have this problem.
  Warning: game dir '/var/games/emacs': Permission denied
  Starting Emacs daemon.

Hmmm ... I should remove the warning about the daemon mode, that seems to be fixed

To check the membership a simple call of the program id(1) should show all groups which the user is part of.

> > Oug? You've said that with /usr/bin/emacs-gtk the error is gone?  If so you
> > might check is there is a background process running for a emacs server
> > 
> >   ps ux | grep emacs
> > 
> > this because there exists a systemd user service unit which might start such
> > a daemon server:
> > 
> >   werner/emacs> ps aux | grep emacs
> >   werner   21109 18.9  0.7 255536 63320 ?        Ssl  09:32   0:01
> > /usr/bin/emacs --fg-daemon
> >   werner   21129  0.0  0.0   3400   608 pts/0    S+   09:32   0:00 grep emacs
> 
> I am not running it via systemd or via daemon option in the startup but
> rather through the init.el by using the server package
>
>  (add-hook 'after-init-hook
>            (lambda ()
>              (require 'server)
>              (unless (server-running-p)
>                (server-start))))

It seems that this has the same result ;)

The question rises: if you've restarted the emacs server process, does the failure with the fresh emacs server process happens again?
Comment 19 Dr. Werner Fink 2020-08-21 08:12:42 UTC
(In reply to Dr. Werner Fink from comment #18)

> 
> Hmmm ... I should remove the warning about the daemon mode, that seems to be
> fixed

That is part of the source, nmot of my script

 werner/emacs> grep -A3 -rs 'Warning: due to a long standing Gtk+ bug' emacs-27.1
 emacs-27.1/src/emacs.c:      fputs ("\nWarning: due to a long standing Gtk+ bug\nhttps://gitlab.gnome.org/GNOME/gtk/issues/221\n\
 emacs-27.1/src/emacs.c-Emacs might crash when run in daemon mode and the X11 connection is unexpectedly lost.\n\
 emacs-27.1/src/emacs.c-Using an Emacs configured with --with-x-toolkit=lucid does not have this problem.\n",
 emacs-27.1/src/emacs.c-      stderr);
Comment 20 Togan Muftuoglu 2020-08-21 09:22:10 UTC
(In reply to Dr. Werner Fink from comment #18)
> To check the membership a simple call of the program id(1) should show all
> groups which the user is part of.


uid=1000(toganm) gid=100(users) groups=100(users),494(wheel),497(mail)

plus getent groups says 

users:x:100:games

Unless I understand differently all users are member of the games group so it is not necessary for an user to be specifically member of the games group as it should be inherited.



> > 
> > I am not running it via systemd or via daemon option in the startup but
> > rather through the init.el by using the server package
> >
> >  (add-hook 'after-init-hook
> >            (lambda ()
> >              (require 'server)
> >              (unless (server-running-p)
> >                (server-start))))
> 
> It seems that this has the same result ;)
> 
> The question rises: if you've restarted the emacs server process, does the
> failure with the fresh emacs server process happens again?

Mind you emacs is called via emacs-gtk
So here is what I did started with my current configuration which per above elisp starts the server process. 

(server-mode) which restarts the server there is no warning 

To test further I started emacs without the server enabled this time calling with "/usr/bin/emacs" and plain simple "emacs" No error regarding permissions on '/var/games' 

started server I tried (server-mode) (server-start) options to see if there is any difference none whatsoever there is no messager regarding permissions.

So I tried installing a package via the package menu 

Package refresh done
Packages to install: 1 (company-fuzzy-20200712.49).  Proceed? (y or n) y
Operation [ Install 1 ] started
Saving file /home/toganm/.emacs.d/personal/emacs-sane-custom-settings.el...
Wrote /home/toganm/.emacs.d/personal/emacs-sane-custom-settings.el
Contacting host: melpa.org:443 [2 times]
  INFO     Scraping files for company-fuzzy-autoloads.el... 
Generating autoloads for company-fuzzy.el...done
  INFO     Scraping files for company-fuzzy-autoloads.el...done
Wrote /home/toganm/.emacs.d/elpa-27.1/company-fuzzy-20200712.49/company-fuzzy-autoloads.el [2 times]
Loading async...done
Operation [ Install 1 ] finished
error in process sentinel: async-handle-result: End of file during parsing
error in process sentinel: End of file during parsing

So with calling emacs with or without server-mode or restarting the server process does not produce permission error but, use of async package is broken

Just to be sure started this time with emacs-gtk yes permission error is there howevere using async package works flawlessly.

Enabling (server-mode) does not produce permission errors.

So hope this helps you to figure out what is going on
Comment 21 Togan Muftuoglu 2020-08-21 09:37:56 UTC
(In reply to Togan Muftuoglu from comment #20)
> (In reply to Dr. Werner Fink from comment #18)
> > To check the membership a simple call of the program id(1) should show all
> > groups which the user is part of.
> 
> 
> uid=1000(toganm) gid=100(users) groups=100(users),494(wheel),497(mail)
> 
> plus getent groups says 
> 
> users:x:100:games
> 
> Unless I understand differently all users are member of the games group so
> it is not necessary for an user to be specifically member of the games group
> as it should be inherited.


Maybe my understanding is not correct just for the sake of curiosity I added myself to the games group and the error message is gone
Comment 22 Dr. Werner Fink 2020-08-21 10:13:13 UTC
Just tried a few packages from elpa .... installing works flawless but delete shows

 Delete package ?xpm-1.0.4?? y
 Saving file /suse/werner/.gnu-emacs-custom...
 Wrote /suse/werner/.gnu-emacs-custom
 Package ?xpm-1.0.4? deleted.
 package--with-response-buffer-1: Wrong type argument: stringp, nil

 Install package ?async-1.9.3?? y
 Saving file /suse/werner/.gnu-emacs-custom...
 Wrote /suse/werner/.gnu-emacs-custom
 Contacting host: elpa.gnu.org:443 [2 times]
 Parsing tar file...done
 Extracting...done
   INFO     Scraping files for async-autoloads.el... 
 Generating autoloads for async-bytecomp.el...done
 Generating autoloads for async-pkg.el...done
 Generating autoloads for async-test.el...done
 Generating autoloads for async.el...done
 Generating autoloads for dired-async.el...done
 Generating autoloads for smtpmail-async.el...done
   INFO     Scraping files for async-autoloads.el...done
 Wrote /suse/werner/.emacs.d/elpa/async-1.9.3/async-autoloads.el [2 times]
 Checking /suse/werner/.emacs.d/elpa/async-1.9.3... [2 times]
 Wrote /suse/werner/.emacs.d/elpa/async-1.9.3/async-bytecomp.elc
 Checking /suse/werner/.emacs.d/elpa/async-1.9.3... [2 times]
 Wrote /suse/werner/.emacs.d/elpa/async-1.9.3/async-test.elc
 Checking /suse/werner/.emacs.d/elpa/async-1.9.3...
 Wrote /suse/werner/.emacs.d/elpa/async-1.9.3/async.elc
 Checking /suse/werner/.emacs.d/elpa/async-1.9.3...
 Wrote /suse/werner/.emacs.d/elpa/async-1.9.3/dired-async.elc
 Checking /suse/werner/.emacs.d/elpa/async-1.9.3...
 Wrote /suse/werner/.emacs.d/elpa/async-1.9.3/smtpmail-async.elc
 Checking /suse/werner/.emacs.d/elpa/async-1.9.3...
 Done (Total of 5 files compiled, 2 skipped)

The error has been thrown by package--with-response-buffer-1
which is the common lisp function defined in lisp/emacs-lisp/package.el
and used by the macro package--with-response-buffer
Comment 23 Togan Muftuoglu 2020-08-21 10:46:18 UTC
(In reply to Dr. Werner Fink from comment #22)
> Just tried a few packages from elpa .... installing works flawless but
> delete shows
> 
>  Delete package ?xpm-1.0.4?? y
>  Saving file /suse/werner/.gnu-emacs-custom...
>  Wrote /suse/werner/.gnu-emacs-custom
>  Package ?xpm-1.0.4? deleted.
>  package--with-response-buffer-1: Wrong type argument: stringp, nil

> The error has been thrown by package--with-response-buffer-1
> which is the common lisp function defined in lisp/emacs-lisp/package.el
> and used by the macro package--with-response-buffer


Weird as when I am running emacs-gtk I have no problems with it installing or deleting wise.
Comment 24 Dr. Werner Fink 2020-08-21 11:16:49 UTC
Found this https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34094
Comment 25 Togan Muftuoglu 2020-08-21 12:19:16 UTC
(In reply to Dr. Werner Fink from comment #24)
> Found this https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34094

With emacs-27.1-341.1 running normal /usr/bin/emacs (as defined by the Emacs.desktop) no problem in using async both installing or deleting.

touch would
Comment 26 OBSbugzilla Bot 2020-08-21 12:50:06 UTC
This is an autogenerated message for OBS integration:
This bug (1175487) was mentioned in
https://build.opensuse.org/request/show/828462 Factory / emacs
Comment 27 Dr. Werner Fink 2020-08-21 12:53:07 UTC
(In reply to Togan Muftuoglu from comment #25)
> (In reply to Dr. Werner Fink from comment #24)
> > Found this https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34094
> 
> With emacs-27.1-341.1 running normal /usr/bin/emacs (as defined by the
> Emacs.desktop) no problem in using async both installing or deleting.
> 
> touch would

Thanks a lot for your feeedback!

This new pdumper feature is very fragile, if it is missed the emacs dumps and loads which then leads to strange errors.
Comment 28 Sean O'Connell 2020-08-21 13:20:32 UTC
Also confirming that emacs-27.1-341.1 from the editors repo seems to have resolved my observed issues. Downloads complete with no errors thrown now after loading 'async'. Thanks!
Comment 29 Togan Muftuoglu 2020-08-21 14:09:30 UTC
Just pure cosmetics but these do need some TLC as well from the man page of emacs 

/usr/share/emacs/$VERSION/lisp 
/usr/lib/emacs/$VERSION/$ARCH 
/usr/share/emacs/$VERSION/etc
/usr/share/emacs/$VERSION/etc/DOC.*
Comment 30 Dr. Werner Fink 2020-08-24 13:34:59 UTC
(In reply to Togan Muftuoglu from comment #29)
> Just pure cosmetics but these do need some TLC as well from the man page of
> emacs 
> 
> /usr/share/emacs/$VERSION/lisp 
> /usr/lib/emacs/$VERSION/$ARCH 
> /usr/share/emacs/$VERSION/etc
> /usr/share/emacs/$VERSION/etc/DOC.*

This should be gone now, as well as the missing pdumper file for the executed emacs binary from the wrapper script.
Comment 31 OBSbugzilla Bot 2020-08-24 13:50:06 UTC
This is an autogenerated message for OBS integration:
This bug (1175487) was mentioned in
https://build.opensuse.org/request/show/828946 Factory / emacs
Comment 32 Togan Muftuoglu 2020-08-24 14:24:17 UTC
What is your reasoning to disable harfbuzz which is default on normally
Comment 33 Togan Muftuoglu 2020-08-24 14:32:09 UTC
(In reply to Togan Muftuoglu from comment #32)
> What is your reasoning to disable harfbuzz which is default on normally

Nevermind need either a cup of coffee or a new pair of glasses. Of course it doesn't make sense with noX version.
Comment 34 OBSbugzilla Bot 2020-08-28 14:20:07 UTC
This is an autogenerated message for OBS integration:
This bug (1175487) was mentioned in
https://build.opensuse.org/request/show/830281 Factory / emacs
Comment 35 Dr. Werner Fink 2020-09-01 07:50:16 UTC
*** Bug 1175915 has been marked as a duplicate of this bug. ***
Comment 36 OBSbugzilla Bot 2021-01-26 15:30:13 UTC
This is an autogenerated message for OBS integration:
This bug (1175487) was mentioned in
https://build.opensuse.org/request/show/866880 Factory / emacs