Bug 1165366

Summary: dracut has trouble with /etc/services and /etc/protocols
Product: [openSUSE] openSUSE Tumbleweed Reporter: Per Jessen <per>
Component: OtherAssignee: dracut maintainers <dracut-maintainers>
Status: RESOLVED DUPLICATE QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: daniel, mt, per, thomas.blume, wicked-maintainers
Version: Current   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Per Jessen 2020-03-01 13:39:38 UTC
I'm setting up some hardware for burn-in and wanted to install
Tumbleweed on an NFS root.  When trying to rebuild the initrd:

https://files.jessen.ch/mm29-dracut-output.txt

dracut-install: ERROR: installing '/etc/services'
dracut-install: ERROR: installing '/etc/protocols'
dracut:
FAILED: /usr/lib/dracut/dracut-install -D /var/tmp/dracut.Lm1PZh/initramfs -a /etc/services /etc/nsswitch.conf /etc/rpc /etc/protocols /etc/idmapd.conf
Command line is not complete. Try option "help"
sed: can't read /etc/sysconfig/network/ifcfg-: No such file or directory

I thought someone might want to know.
Comment 1 Per Jessen 2020-03-01 15:37:27 UTC
*** Bug 1165367 has been marked as a duplicate of this bug. ***
Comment 2 Per Jessen 2020-03-01 15:38:10 UTC
*** Bug 1165368 has been marked as a duplicate of this bug. ***
Comment 3 Per Jessen 2020-03-01 15:39:06 UTC
Unfortunately, no more logs available, we've installed leap 15.2.  should be easy to reproduce though.
Comment 4 Stefan Hundhammer 2020-03-04 18:08:48 UTC
There was a change recently with moving files from /etc to /usr/etc .
/etc/services and /etc/protocols were some of those files, and the transition was not exactly smooth. You might have fallen into the same trap.

https://en.opensuse.org/openSUSE:Packaging_UsrEtc
https://lists.opensuse.org/opensuse-factory/2020-02/msg00027.html

(there are probably some more threads about this on the opensuse-factory mailing list)
Comment 5 Daniel Molkentin 2020-03-27 14:13:58 UTC
15.2 should suffer from the exact same Problems.

Anyway, the proposed upstream fix is here:

https://github.com/dracutdevs/dracut/pull/778

The error: "sed: can't read /etc/sysconfig/network/ifcfg-: No such file or directory"

is unrelated and caused by the lines:

    bootproto=$(sed -n "/BOOTPROTO/s/BOOTPROTO='\([[:alpha:]]*6\?\)4\?'/\1/p" /etc/sysconfig/network/ifcfg-$ifname)
    if [ $bootproto ]; then
        printf 'ip=%s:%s ' ${ifname} ${bootproto}
    else
        printf 'ip=%s:static ' ${ifname}
    fi

Which tool are you using to manage the network?
Comment 6 Per Jessen 2020-03-27 14:30:57 UTC
(In reply to Daniel Molkentin from comment #5)

> Which tool are you using to manage the network?

Wicked.
Comment 7 Daniel Molkentin 2020-03-27 14:42:36 UTC
Wicked maintainers: Does wicked create the ifcfg- files, or just read from them as legacy source of information? If so, any guidance on how the code should work to determine a DHCP vs static IP boot?
Comment 8 Marius Tomaschewski 2020-03-30 12:53:07 UTC
(In reply to Daniel Molkentin from comment #7)
> Wicked maintainers: Does wicked create the ifcfg- files, or just read from
> them as legacy source of information? If so, any guidance on how the code
> should work to determine a DHCP vs static IP boot?

Wicked never creates any ifcfg files or another kind of interface configuration.
This is done by yast2-network or the user.

Wicked reads the following configurations by default:
   - ibft firmware config from sysfs
   - ifcfg-* files from /etc/sysconfig/network/
   - *.xml files from /etc/wicked/ifconfig/

the last source is kind of experimental / not (yet) really supported because
99.9% of the system does not have any clue about.

The BOOTPROTO can be set to "dhcp", "dhcp4", "dhcp6" + combinations with e.g.
"+autoip" behind, ... So I'd use:

  case $BOOTPROTO in
  static)
     printf 'ip=%s:static ' ${ifname}
     ;;
  *dhcp*)
     printf 'ip=%s:%s ' ${ifname} ${bootproto}
     ;;
  esac

There is a BOOTPROTO=6to4 artifact which is basically also BOOTPROTO=static.
Comment 9 Daniel Molkentin 2020-03-30 13:38:08 UTC
(In reply to Marius Tomaschewski from comment #8)
> (In reply to Daniel Molkentin from comment #7)
> > Wicked maintainers: Does wicked create the ifcfg- files, or just read from
> > them as legacy source of information? If so, any guidance on how the code
> > should work to determine a DHCP vs static IP boot?
> 
> Wicked never creates any ifcfg files or another kind of interface
> configuration.
> This is done by yast2-network or the user.

Ok, as expected

> case $BOOTPROTO in
>   static)
>      printf 'ip=%s:static ' ${ifname}
>      ;;
>   *dhcp*)
>      printf 'ip=%s:%s ' ${ifname} ${bootproto}
>      ;;
>   esac
> 
> There is a BOOTPROTO=6to4 artifact which is basically also BOOTPROTO=static.

I just realized that this is actually caused by a patch from Thomas (9cc1b76087c3ff17609a28f2142c2c62cd6cc97b) which is now also part of 050. I didn't realize it at the time but we created a circular dependency between the nfs and the network-legacy module this way.

This will break for the network-manager module and for the upcoming network-wicked one. Honestly: I think the network storage modules has no business setting up the network (i.e. it was wrong before, but not bad, because there was only one network backend). 

However, the reverse might be true, e.g. for iBPF, the network module needs to know a bit about the storage part and set things up. The only thing I think we _should_ do is ensure that the network module (and hence one of its backends) is loaded. That is already happening in depends().
Comment 10 Thomas Blume 2020-03-30 16:05:40 UTC
(In reply to Daniel Molkentin from comment #9)
> (In reply to Marius Tomaschewski from comment #8)
> > (In reply to Daniel Molkentin from comment #7)
> > > Wicked maintainers: Does wicked create the ifcfg- files, or just read from
> > > them as legacy source of information? If so, any guidance on how the code
> > > should work to determine a DHCP vs static IP boot?
> > 
> > Wicked never creates any ifcfg files or another kind of interface
> > configuration.
> > This is done by yast2-network or the user.
> 
> Ok, as expected
> 
> > case $BOOTPROTO in
> >   static)
> >      printf 'ip=%s:static ' ${ifname}
> >      ;;
> >   *dhcp*)
> >      printf 'ip=%s:%s ' ${ifname} ${bootproto}
> >      ;;
> >   esac
> > 
> > There is a BOOTPROTO=6to4 artifact which is basically also BOOTPROTO=static.
> 
> I just realized that this is actually caused by a patch from Thomas
> (9cc1b76087c3ff17609a28f2142c2c62cd6cc97b) which is now also part of 050. I
> didn't realize it at the time but we created a circular dependency between
> the nfs and the network-legacy module this way.

Hm, the error:

sed: can't read /etc/sysconfig/network/ifcfg-: No such file or directory

indicates that $ifname is empty.
That should normally get filled via:

ifname=$(ip -o route get to $nfs_address | sed -n 's/.*dev \([^ ]*\).*/\1/p')

Per could you paste the output of the command:

ip -o route get to $YOUR_NSFSSERVERIP | sed -n 's/.*dev \([^ ]*\).*/\1/p'

where $YOUR_NSFSSERVERIP is the ip of your nfsserver?

on your nfsroot machine?
Comment 11 Thomas Blume 2020-03-30 16:33:51 UTC
btw. this is a static ip setup, but the ip= parameter doesn't contain the static ip:

dracut: root=nfs4:rootserver:/srv/nfsroot/mm29:rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.6.232,local_lock=none,addr=192.168.2.128
ip=:static

that looks quite broken.
Comment 12 Per Jessen 2020-03-30 17:22:46 UTC
(In reply to Thomas Blume from comment #11)
> btw. this is a static ip setup, but the ip= parameter doesn't contain the
> static ip:
> 
> dracut:
> root=nfs4:rootserver:/srv/nfsroot/mm29:rw,relatime,vers=4.1,rsize=131072,
> wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,
> clientaddr=192.168.6.232,local_lock=none,addr=192.168.2.128
> ip=:static
> 
> that looks quite broken.

Hi Thomas

yes, I think that is due to an issue with constructing a working resolv.conf - that way 'rootserver' does not get resolved.  May be related to bug#1165547.  During installation, the setup is dynamic, but later I switch to a static ditto. 
When I reported this, we were still mostly pre-corona, I don't remember where I left it.  I think I might just start from scratch.
Comment 13 Per Jessen 2020-03-31 16:42:32 UTC
Well, starting from scratch - see Bug#1168215
Comment 14 Per Jessen 2020-04-02 12:33:22 UTC
(In reply to Thomas Blume from comment #11)
> btw. this is a static ip setup, but the ip= parameter doesn't contain the
> static ip:
> 
> dracut:
> root=nfs4:rootserver:/srv/nfsroot/mm29:rw,relatime,vers=4.1,rsize=131072,
> wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,
> clientaddr=192.168.6.232,local_lock=none,addr=192.168.2.128
> ip=:static
> 
> that looks quite broken.

The above is caused by building the initrd without a proper /etc/resolv.conf.  I think that is what the installer does. Every time I install on NFS I always have to rebuild the initrd to make it work.  

With a working /etc/resolv.conf:  (on a new 15.2 install)

dracut: root=nfs4:rootserver:/srv/nfsroot/mole28:rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp6,timeo=600,retrans=2,sec=sys,clientaddr=2a03:7520:4c68:1:ff99:ffff:0:49fc,local_lock=none,addr=2a03:7520:4c68:1::128
ifname=eth0:00:30:48:d3:75:74 ip=eth0:dhcp
Comment 15 Thomas Blume 2020-04-03 06:24:47 UTC
(In reply to Per Jessen from comment #14)
> > dracut:
> > root=nfs4:rootserver:/srv/nfsroot/mm29:rw,relatime,vers=4.1,rsize=131072,
> > wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,
> > clientaddr=192.168.6.232,local_lock=none,addr=192.168.2.128
> > ip=:static
> > 
> > that looks quite broken.
> 
> The above is caused by building the initrd without a proper
> /etc/resolv.conf.  I think that is what the installer does. Every time I
> install on NFS I always have to rebuild the initrd to make it work.  
> 
> With a working /etc/resolv.conf:  (on a new 15.2 install)
> 
> dracut:
> root=nfs4:rootserver:/srv/nfsroot/mole28:rw,relatime,vers=4.1,rsize=131072,
> wsize=131072,namlen=255,hard,proto=tcp6,timeo=600,retrans=2,sec=sys,
> clientaddr=2a03:7520:4c68:1:ff99:ffff:0:49fc,local_lock=none,addr=2a03:7520:
> 4c68:1::128
> ifname=eth0:00:30:48:d3:75:74 ip=eth0:dhcp

Thanks for the details, that looks like on my testmachine.

So, there seems to be an issue creating resolv.conf.
Is there any mention of resolv.conf in the installer logs and does it show any problem?
Comment 16 Per Jessen 2020-04-03 08:15:51 UTC
Sorry, I should have referred you to bug#1165547
Comment 17 Thomas Blume 2020-04-03 10:14:52 UTC
(In reply to Per Jessen from comment #16)
> Sorry, I should have referred you to bug#1165547

I see, thanks for the referrer.
So this one actually looks like caused by bug#1165547.
Are you ok if I close it as duplicate?
Comment 18 Per Jessen 2020-04-03 10:41:01 UTC
Yep, let us close this one - the original problem with /etc/services seems to have gone away anyway.

*** This bug has been marked as a duplicate of bug 1165547 ***