Bug 927250

Summary: ncurses has strange characters
Product: [openSUSE] openSUSE Tumbleweed Reporter: Freek de Kruijf <freek>
Component: BasesystemAssignee: Thomas Blume <thomas.blume>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: arvidjaar, bwiedemann, chcao, cristian_anita, crrodriguez, fabian, fcrozat, forgotten_CR1ueQjYM4, forgotten_DHIkF8sU1p, freek, fvogt, gs, ismail, jengelh, meissner, mpluskal, systemd-maintainers, thomas.blume
Version: 201503*   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: dracut-add-trivial-vconsole-fontmap.patch
Attached is the requested xz compressed journal log.
run-systemd-vconsole-setup-after-plymouth.patch

Description Freek de Kruijf 2015-04-15 09:54:44 UTC
When booting my Tumbleweed system I don't get the normal green splash screen, but three strange looking pairs of characters, which change during the boot process.
After booting and starting YaST2 in ncurses mode, the line drawing character do not appear, but some strange characters instead.

# hwinfo --gfxcard
34: PCI 500.0: 0300 VGA compatible controller (VGA)             
  [Created at pci.328]
  Unique ID: Ddhb.zvQlrOmSRQ8
  Parent ID: _Znp.ZJmKoWxd6BF
  SysFS ID: /devices/pci0000:00/0000:00:02.0/0000:05:00.0
  SysFS BusID: 0000:05:00.0
  Hardware Class: graphics card
  Model: "ATI RV770 [Radeon HD 4850]"
  Vendor: pci 0x1002 "ATI Technologies Inc"
  Device: pci 0x9442 "RV770 [Radeon HD 4850]"
  SubVendor: pci 0x1787 "Hightech Information System Ltd."
  SubDevice: pci 0x2266 
  Driver: "radeon"
  Driver Modules: "drm"
  Memory Range: 0xd0000000-0xdfffffff (ro,non-prefetchable)
  Memory Range: 0xfe9f0000-0xfe9fffff (rw,non-prefetchable)
  I/O Ports: 0xe000-0xefff (rw)
  Memory Range: 0xfe9c0000-0xfe9dffff (ro,non-prefetchable,disabled)
  IRQ: 34 (83245 events)
  I/O Ports: 0x3c0-0x3df (rw)
  Module Alias: "pci:v00001002d00009442sv00001787sd00002266bc03sc00i00"
  Driver Info #0:
    Driver Status: radeon is active
    Driver Activation Cmd: "modprobe radeon"
  Config Status: cfg=no, avail=yes, need=no, active=unknown
  Attached to: #9 (PCI bridge)

Primary display adapter: #34
Comment 1 Bernhard Wiedemann 2015-04-21 14:08:24 UTC
please check
systemctl status systemd-vconsole-setup.service
Comment 3 Dr. Werner Fink 2015-04-21 14:22:40 UTC
splash -> plymouth
Comment 4 Freek de Kruijf 2015-04-21 15:25:53 UTC
(In reply to Bernhard Wiedemann from comment #1)
> please check
> systemctl status systemd-vconsole-setup.service

# systemctl status systemd-vconsole-setup.service
● systemd-vconsole-setup.service - Setup Virtual Console
   Loaded: loaded (/usr/lib/systemd/system/systemd-vconsole-setup.service; static; vendor preset: disabled)
   Active: active (exited) since di 2015-04-21 16:57:29 CEST; 4min 57s ago
     Docs: man:systemd-vconsole-setup.service(8)
           man:vconsole.conf(5)
 Main PID: 134 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/systemd-vconsole-setup.service

apr 21 16:57:29 eiktum systemd-vconsole-setup[134]: mapscrn: cannot open map file _trivial_
apr 21 17:02:22 eiktum.beelaertsict.nl systemd[1]: Started Setup Virtual Console.
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
#

It seems a file _trivial_ is missing in my Tumbleweed. I have another Tumbleweed system which does not have this problem.
I do have a file /usr/share/kbd/consoletrans/trivial on my system.

The last line about log output incomplete is nonsense. I just booted the system.
Comment 5 Dr. Werner Fink 2015-04-21 16:27:23 UTC
(In reply to Freek de Kruijf from comment #4)

How often I had metioned this ... don't know:  Plymouth locks the virtual consoles with the ioctl TIOCSLCKTRMIOS (1).  That is if plymouthd is running then the systemd-vconsole-setup should never run!  Do this before plymouthd has been started or after plymouthd has released the virtual concoles.

(1) man:tty_ioctl(4)

  Locking the termios structure
     The termios structure of a terminal can be locked.  The lock is itself
     a termios structure, with non-zero bits or fields indicating a locked
     value.

     TIOCGLCKTRMIOS struct termios *argp
            Gets the locking status of the termios structure of the terminal.

     TIOCSLCKTRMIOS const struct termios *argp
            Sets the locking status of the termios structure of the terminal.
            Only root (more precisely: a process with the CAP_SYS_ADMIN
            capability) can do this.
Comment 6 Thomas Blume 2015-04-22 05:58:38 UTC
(In reply to Freek de Kruijf from comment #4)
> 
> apr 21 16:57:29 eiktum systemd-vconsole-setup[134]: mapscrn: cannot open map
> file _trivial_
> apr 21 17:02:22 eiktum.beelaertsict.nl systemd[1]: Started Setup Virtual
> Console.
> Warning: Journal has been rotated since unit was started. Log output is
> incomplete or unavailable.
> #
> 
> It seems a file _trivial_ is missing in my Tumbleweed. I have another
> Tumbleweed system which does not have this problem.
> I do have a file /usr/share/kbd/consoletrans/trivial on my system.
> 

That was already reported in bug 926742 and is a dracut issue.
A simple patch is provided and in evaluation by the dracut maintainers.
However, this message is most probably unrelated to the observer behaviour. 
Since bug 926742 is internal, I'm adding the patch as reference.
Comment 7 Thomas Blume 2015-04-22 05:59:21 UTC
Created attachment 631891 [details]
dracut-add-trivial-vconsole-fontmap.patch

Patch from bug 926742
Comment 8 Chenzi Cao 2015-04-22 07:08:27 UTC
Hi Thomas, if you have made this issue fixed, would you mind to help close this bug? Thank you!
Comment 9 Thomas Blume 2015-04-22 08:18:06 UTC
(In reply to Chenzi Cao from comment #8)
> Hi Thomas, if you have made this issue fixed, would you mind to help close
> this bug? Thank you!

The patch doesn't fix the issue reported.
It is only addressing the error message, e.g.:

apr 21 16:57:29 eiktum systemd-vconsole-setup[134]: mapscrn: cannot open map file _trivial_

But this is unrelated to the problem.
So, I guess we cannot close the bug yet.
Comment 10 Freek de Kruijf 2015-04-22 08:35:13 UTC
(In reply to Thomas Blume from comment #9)
> (In reply to Chenzi Cao from comment #8)
> > Hi Thomas, if you have made this issue fixed, would you mind to help close
> > this bug? Thank you!
> 
> The patch doesn't fix the issue reported.
> It is only addressing the error message, e.g.:
> 
> apr 21 16:57:29 eiktum systemd-vconsole-setup[134]: mapscrn: cannot open map
> file _trivial_
> 
> But this is unrelated to the problem.
> So, I guess we cannot close the bug yet.

Indeed. The problem is still there.

I added the line(s) mentioned in the patch, which were not present, but I still get the error message about the missing file, trivial.
Comment 11 Thomas Blume 2015-04-22 10:11:34 UTC
(In reply to Freek de Kruijf from comment #10)
> 
> Indeed. The problem is still there.
> 
> I added the line(s) mentioned in the patch, which were not present, but I
> still get the error message about the missing file, trivial.

Just tested on a recent tumbleweed install.
Before applying the patch, I see this:

-->--
# journalctl -axb | grep 'systemd-vconsole-setup\['
Apr 22 11:53:30 linux-lwge systemd-vconsole-setup[94]: /usr/bin/setfont failed with error code 1.
Apr 22 11:53:30 linux-lwge systemd-vconsole-setup[94]: mapscrn: cannot open map file _trivial_
Apr 22 11:53:31 linux-lwge systemd-vconsole-setup[222]: /usr/bin/setfont failed with error code 1.
--<--

Afterwards, I only see this:

-->--
# journalctl -axb | grep 'systemd-vconsole-setup\['
Apr 22 12:00:52 linux-lwge systemd-vconsole-setup[446]: /usr/bin/setfont failed with error code 71.
--<--

but that's a different problem.
Did you recreate the initrd after making the change?
Comment 12 Freek de Kruijf 2015-05-03 10:50:06 UTC
After the upgrade to kernel-desktop 4.0.0 booting the system gives the normal green splash screen with the curved lines. So this part is solved. However using yast with ncurses on tty1 gives still gives strange characters instead of the characters that draw lines on the screen. So I assume the wrong character set is configured for that console.
Also messages in Dutch which may have a character like ë is not displayed as one character but as two, a bit strange, characters.
Comment 13 Freek de Kruijf 2015-07-26 21:33:35 UTC
(In reply to Freek de Kruijf from comment #12)
> After the upgrade to kernel-desktop 4.0.0 booting the system gives the
> normal green splash screen with the curved lines. So this part is solved.
> However using yast with ncurses on tty1 gives still gives strange characters
> instead of the characters that draw lines on the screen. So I assume the
> wrong character set is configured for that console.
> Also messages in Dutch which may have a character like ë is not displayed as
> one character but as two, a bit strange, characters.

When the command "systemctl restart systemd-vconsole-setup.service"is given before
 invoking yast the line drawing characters, as they should appear, are shown. This command is only necessary on tty1. So during start up somehow this command is needed.
Comment 14 Thomas Blume 2015-07-27 12:30:01 UTC
(In reply to Freek de Kruijf from comment #13)
> 
> When the command "systemctl restart systemd-vconsole-setup.service"is given
> before
>  invoking yast the line drawing characters, as they should appear, are
> shown. This command is only necessary on tty1. So during start up somehow
> this command is needed.

Thanks for the details.
Can you please remove the boot option 'quiet' and add the boot option: 

udev.log-priority=debug

and afterwards provide the boot log (journalctl -axb) from this machine?
Comment 15 Freek de Kruijf 2015-07-27 16:47:07 UTC
Created attachment 642070 [details]
Attached is the requested xz compressed journal log.
Comment 18 Robert Milasan 2015-07-30 09:00:04 UTC
Thomas, what about this?

--- 90-vconsole.rules.orig	2015-07-30 10:58:12.820740385 +0200
+++ 90-vconsole.rules	2015-07-30 10:58:37.804090176 +0200
@@ -8,4 +8,4 @@
 # Kernel resets vconsole state when changing console drivers so run
 # systemd-vconsole-setup when fbcon loads
 
-ACTION=="add", SUBSYSTEM=="graphics", KERNEL=="fbcon", RUN+="/usr/lib/systemd/systemd-vconsole-setup"
+ACTION=="add", SUBSYSTEM=="graphics", ENV{DEVPATH}=="*/fbcon", RUN+="/usr/lib/systemd/systemd-vconsole-setup"

I've ran some tests and seems to work. Please check it and let me know.
Comment 19 Robert Milasan 2015-07-30 09:04:06 UTC
OK, maybe I misunderstood the issue, but how exactly do we reproduce this?

NOTE: I have a VM running Factory/Leap.
Comment 20 Andrei Borzenkov 2015-07-30 09:11:42 UTC
(In reply to Robert Milasan from comment #18)
> Thomas, what about this?
> 
> --- 90-vconsole.rules.orig	2015-07-30 10:58:12.820740385 +0200
> +++ 90-vconsole.rules	2015-07-30 10:58:37.804090176 +0200
> @@ -8,4 +8,4 @@
>  # Kernel resets vconsole state when changing console drivers so run
>  # systemd-vconsole-setup when fbcon loads
>  
> -ACTION=="add", SUBSYSTEM=="graphics", KERNEL=="fbcon",
> RUN+="/usr/lib/systemd/systemd-vconsole-setup"
> +ACTION=="add", SUBSYSTEM=="graphics", ENV{DEVPATH}=="*/fbcon",
> RUN+="/usr/lib/systemd/systemd-vconsole-setup"
> 

How these two lines differ? I.e. how is it possible to have $DEVPATH not ending with $KERNEL?
Comment 21 Robert Milasan 2015-07-30 09:15:41 UTC
Thats what I just said, I misunderstood, would like to know how to reproduce, why the crap comment?
Comment 22 Thomas Blume 2015-07-30 10:13:48 UTC
(In reply to Robert Milasan from comment #21)
> Thats what I just said, I misunderstood, would like to know how to
> reproduce, why the crap comment?

In order to reproduce, just install a recent factory and make multi-user.target the default.
When you log in to a physical tty of the machine and start yast, you will see that the line characters are all thrashed.
Comment 23 Thomas Blume 2015-07-30 10:15:07 UTC
Created attachment 642468 [details]
run-systemd-vconsole-setup-after-plymouth.patch

I was talking to Werner. We need to make sure that systemd-vconsole-setup runs once after plymouth has quit.
The simplest seems to be adding it to plymouth-quit-wait.service.

Please see the patch attached.
Would this be acceptable?
Comment 24 Fabian Vogt 2015-11-02 13:33:05 UTC
As far as I can see, this is the same issue as bug 857527, which I reported ages ago. I'll make it a duplicate of this.

I noticed that systemd-vconsole-setup is failing in the initrd, as LANG=POSIX, somehow. I'm not sure where LANG is set to the wrong value, but it means that vconsole-setup tries to call loadkeys without -u, resulting in strange error messages, which I noticed in the journal.
The workaround is quite simple, add
    EnvironmentFile=-/etc/locale.conf
to /usr/lib/systemd/system/systemd-vconsole-setup.service and run mkinitrd.
All charset, ncurses and similiar issues are now gone!
The real issue is that systemd-vconsole-setup does not read the locale config files itself.
Comment 25 Fabian Vogt 2015-11-02 13:33:52 UTC
*** Bug 857527 has been marked as a duplicate of this bug. ***
Comment 26 Fabian Vogt 2015-11-02 13:39:09 UTC
Also: This affects tty1 only as it's the only tty that is activated during the initrd phase.
Comment 27 Cristian ANITA 2015-11-09 16:57:25 UTC
*** Bug 954118 has been marked as a duplicate of this bug. ***
Comment 28 Dr. Werner Fink 2015-11-11 08:32:09 UTC
There are three different scenarios, first of all with plymouth which causes that systems-vconsole-setup is called very early in dracut and with LANG set to the value in /etc/locale.conf.  During plymouth(d) is running all calls of systems-vconsole-setup will come (partly) to nothing.

Then we have the case without plymouth (rpm -e plymouth plymouth-dracut && mkinitrd).  In this case the systems-vconsole-setup is called later than in first case and now includes LANG and LC_CTYPE from /etc/locale.conf and /etc/sysconfig/language (see ROOT_USES_LANG case "ctype").  My guess is that now dracut has included /etc/sysconfig/language and therefore LC_CTYPE is set as well.

The last case is the udev rule /usr/lib/udev/rules.d/90-vconsole.rules with the line

  ACTION=="add", SUBSYSTEM=="graphics", KERNEL=="fbcon", RUN+="/usr/lib/systemd/systemd-vconsole-setup"

which changes also the virtual consoles and is only in action if no plymouth is active as otherwise systems-vconsole-setup will come (partly) to nothing.

Without plymouth the last case does set UTF8 even with LANG=POSIX in /etc/locale.conf and ROOT_USES_LANG not set in /etc/sysconfig/language.

That is that  all three calls uses different an environment respectively the locale due to the function is_locale_utf8() from systemd-210/src/shared/util.c or systemd-git/src/basic/locale-util.c
Comment 29 Fabian Vogt 2015-11-11 15:25:45 UTC
I looked a bit deeper into this. There seem to be two different, yet related issues:

systemd-vconsole-setup.service calls systemd-vconsole-setup with LANG=POSIX during the initrd. So tty1 has utf8 disabled, always. The cause is patch handle-root_uses_lang-value-in-etc-sysconfig-language.patch in systemd, setting LANG unconditionally. I wrote Frederic an E-Mail about that.

/usr/lib/udev/rules.d/90-vconsole.rules calls systemd-vconsole-setup without environment variables. This means that utf8 is always enabled. This happens after tty1 has been set up, so only tty2-ttyInf are affected. I'm not sure what could be done about that, maybe a call to "systemctl restart systemd-vconsole-setup.service" in the udev rule is more appropriate here than directly calling "systemd-vconsole-setup", which may also lead to race conditions.
Comment 30 Fabian Vogt 2015-11-12 16:16:15 UTC
The first part of the bug is now fixed in dracut and systemd and will reach factory shortly. One of the fixes is enough to get a working tty, so it should be fixed in the next snapshot.

The second part will probably not get fixed, it seems like that keyboard in XLATE/ascii mode is not properly implemented upstream. So the udev rule won't interfere with the UTF-8 consoles.
Having UTF-8 disabled is not recommended and not the default, so that should not be an issue.
Comment 31 Thomas Blume 2015-11-13 12:33:31 UTC
*** Bug 744108 has been marked as a duplicate of this bug. ***
Comment 32 Bernhard Wiedemann 2015-12-02 10:00:11 UTC
This is an autogenerated message for OBS integration:
This bug (927250) was mentioned in
https://build.opensuse.org/request/show/347140 13.2 / systemd
Comment 33 Bernhard Wiedemann 2015-12-03 21:00:09 UTC
This is an autogenerated message for OBS integration:
This bug (927250) was mentioned in
https://build.opensuse.org/request/show/347379 13.1 / systemd
Comment 36 Bernhard Wiedemann 2016-01-13 12:00:09 UTC
This is an autogenerated message for OBS integration:
This bug (927250) was mentioned in
https://build.opensuse.org/request/show/353405 13.1 / systemd
Comment 37 Swamp Workflow Management 2016-02-03 14:47:51 UTC
openSUSE-RU-2016:0320-1: An update that has 146 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 737690,742774,750845,818044,838475,841544,849870,852015,852021,852232,853293,854884,856389,856392,856858,857204,858864,859072,859365,860574,860937,861316,861489,863217,864745,864904,865834,866732,866933,867128,867663,867664,867840,868019,868230,868439,868931,869142,869603,872929,873432,873444,874665,875502,876587,876694,877021,877674,878525,880438,880732,881125,881559,881942,882393,882714,883565,884271,884403,885232,885288,886211,886599,886852,888178,888215,888612,889297,889357,890977,892096,892162,892300,893797,895087,896664,897799,897801,897803,898233,898240,898432,900558,901481,902240,902901,903009,903963,904214,904517,904828,905550,906709,906900,907318,907393,908476,909358,910643,911347,912030,912334,913517,916420,918118,919095,920195,921831,921898,921920,926169,927250,927457,928265,931388,932284,933365,933512,933521,933533,934077,934901,937512,937900,938908,939571,940264,941576,944132,944799,945282,947212,948458,948555,948705,949574,949683,949739,950510,951265,951663,953241,954336,954781,955635,961576
CVE References: 
Sources used:
openSUSE 13.1 (src):    systemd-210-40.1, systemd-mini-210-40.1
Comment 38 Swamp Workflow Management 2016-02-25 20:26:08 UTC
SUSE-RU-2016:0586-1: An update that has 17 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 927250,942946,948458,948555,949574,954336,954781,955469,955770,958295,958935,958937,961226,961576,962080,964355,965475
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP1 (src):    systemd-210-95.1
SUSE Linux Enterprise Server 12-SP1 (src):    systemd-210-95.1
SUSE Linux Enterprise Desktop 12-SP1 (src):    systemd-210-95.1
Comment 39 Swamp Workflow Management 2016-02-25 20:29:26 UTC
SUSE-RU-2016:0587-1: An update that has 15 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 927250,942946,948458,948555,949574,955469,955770,958295,958935,958937,961226,961576,962080,964355,965475
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12 (src):    systemd-210-70.39.1
SUSE Linux Enterprise Server 12 (src):    systemd-210-70.39.1
SUSE Linux Enterprise Desktop 12 (src):    systemd-210-70.39.1
Comment 40 Swamp Workflow Management 2016-03-18 13:15:03 UTC
openSUSE-RU-2016:0815-1: An update that has 17 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 927250,942946,948458,948555,949574,954336,954781,955469,955770,958295,958935,958937,961226,961576,962080,964355,965475
CVE References: 
Sources used:
openSUSE Leap 42.1 (src):    systemd-210-89.1, systemd-mini-210-89.1