Bug 1024740

Hi Ludwig,

(In reply to Ludwig Nussel from comment #0)
> systemd-machine-id-setup shouldn't be called in %post. /etc/machine-id will
> be generated at fist boot if needed.

I'm not sure what is supposed to generate it at first boot: when systemd is started /etc is usually mounted read only.

> This is especially important for
> appliance builds to avoid an identical file in all images.

Shouldn't the provisioning process take care of this ?

(In reply to Franck Bui from comment #1)
> Hi Ludwig,
> 
> (In reply to Ludwig Nussel from comment #0)
> > systemd-machine-id-setup shouldn't be called in %post. /etc/machine-id will
> > be generated at fist boot if needed.
> 
> I'm not sure what is supposed to generate it at first boot: when systemd is
> started /etc is usually mounted read only.

https://github.com/systemd/systemd/blob/master/src/firstboot/firstboot.c#L422

> > This is especially important for
> > appliance builds to avoid an identical file in all images.
> 
> Shouldn't the provisioning process take care of this ?

It does. The provisioning process is systemd-firstboot. One has to
delete /etc/machine-id when building the appliance so
systemd-firstboot can re-create it :-)

(In reply to Ludwig Nussel from comment #2)
> (In reply to Franck Bui from comment #1)
> > Hi Ludwig,
> > 
> > (In reply to Ludwig Nussel from comment #0)
> > > systemd-machine-id-setup shouldn't be called in %post. /etc/machine-id will
> > > be generated at fist boot if needed.
> > 
> > I'm not sure what is supposed to generate it at first boot: when systemd is
> > started /etc is usually mounted read only.
> 
> https://github.com/systemd/systemd/blob/master/src/firstboot/firstboot.c#L422
> 

systemd-firstboot is not supposed to run after a regular installation.

> > > This is especially important for
> > > appliance builds to avoid an identical file in all images.
> > 
> > Shouldn't the provisioning process take care of this ?
> 
> It does. The provisioning process is systemd-firstboot. One has to
> delete /etc/machine-id when building the appliance so
> systemd-firstboot can re-create it :-)

Then make the provisioning super process does that ;)

(In reply to Franck Bui from comment #3)
> (In reply to Ludwig Nussel from comment #2)
> > (In reply to Franck Bui from comment #1)
> > > Hi Ludwig,
> > > 
> > > (In reply to Ludwig Nussel from comment #0)
> > > > systemd-machine-id-setup shouldn't be called in %post. /etc/machine-id will
> > > > be generated at fist boot if needed.
> > > 
> > > I'm not sure what is supposed to generate it at first boot: when systemd is
> > > started /etc is usually mounted read only.
> > 
> > https://github.com/systemd/systemd/blob/master/src/firstboot/firstboot.c#L422
> 
> systemd-firstboot is not supposed to run after a regular installation.

Since it's installed and enabled by default (not even via presets)
it looks like the contrary is the case :-) For all actions
systemd-firstboot offers, it first checks if they are actually
necessary. So it really seems to be made to fill in the gaps left
open by the installer.

> > > > This is especially important for
> > > > appliance builds to avoid an identical file in all images.
> > > 
> > > Shouldn't the provisioning process take care of this ?
> > 
> > It does. The provisioning process is systemd-firstboot. One has to
> > delete /etc/machine-id when building the appliance so
> > systemd-firstboot can re-create it :-)
> 
> Then make the provisioning super process does that ;)

How would you do that if the deliverable is a bootable disk image?
Stuff like /etc/machine-id has to be created per instance, ie at
first boot and that's what systemd-firstboot is for. Doesn't make
sense to reimplement it. Right now every single kiwi file has to
delete a number of files that are actually specific to the
deployment/machine. That doesn't scale.

See also https://fate.suse.com/320479

(In reply to Ludwig Nussel from comment #4)
> 
> Since it's installed and enabled by default (not even via presets)
> it looks like the contrary is the case :-) For all actions
> systemd-firstboot offers, it first checks if they are actually
> necessary. So it really seems to be made to fill in the gaps left
> open by the installer.
> 

I don't think systemd-firstboot service is supposed to create /etc/machine-id.

See systemd-firstboot(1) man page and the description of "--setup-machine-id":

   --setup-machine-id
           Initialize the system's machine ID to a random ID. This only
           works in combination with --root=.

> How would you do that if the deliverable is a bootable disk image?

The tool used to create the disk image might want to remove the /etc/machine-id and create an empty one. In this case systemd would create a transient machine-id during the first boot, which would be committed later by systemd-machine-id-commit.service.

Or maybe we should create an empty machine-id in %post....

(In reply to Franck Bui from comment #5)
> [...]
> The tool used to create the disk image might want to remove the
> /etc/machine-id and create an empty one. In this case systemd would create a
> transient machine-id during the first boot, which would be committed later
> by systemd-machine-id-commit.service.
> 
> Or maybe we should create an empty machine-id in %post....

Ok, sounds good too.

after thinking more about it, why does the systemd-firstboot not use the same mechanism as systemd-machine-id-setup? Ie don't enforce a random uuid but take the external one too if specified.

(In reply to Ludwig Nussel from comment #7)
> after thinking more about it, why does the systemd-firstboot not use the
> same mechanism as systemd-machine-id-setup? Ie don't enforce a random uuid
> but take the external one too if specified.

I'm not sure to understand.

systemd-firstboot (the binary) with --root option can take an external uuid via --machine-id= option. However if /etc/machine-id file already exists it will be a nop.

yes, one can specify it but that is not possible in a generic way :) systemd-machine-id-setup has some built in algorithm to select or generate the machine-id.

well I don't really know. I'm not sure why systemd-firstboot gained the ability to setup /etc/machine-id in the first place.

systemd-machine-id-setup seems to be the tool for this purpose.

I'm still not sure that creating an empty file would be a good idea: some packages might rely on it during the installation process.

Also there are probably others files that need to be cleaned up during the creation of the disk images (ssh host keys, etc...) and if so machine-id could be simply part of the files to be reinit.

(In reply to Franck Bui from comment #11)
> I'm still not sure that creating an empty file would be a good idea: some
> packages might rely on it during the installation process.

What for? To take it and copy it elsewhere? That would be equally bad. 

> Also there are probably others files that need to be cleaned up during the
> creation of the disk images (ssh host keys, etc...) and if so machine-id
> could be simply part of the files to be reinit.

ssh keys are created upon first start of the service. "part of the files to reinit" is exactly the problem. An externally maintained list does not scale.

(In reply to Ludwig Nussel from comment #12)
> (In reply to Franck Bui from comment #11)
> > I'm still not sure that creating an empty file would be a good idea: some
> > packages might rely on it during the installation process.
> 
> What for? To take it and copy it elsewhere? That would be equally bad. 
> 

I don't know, I just want to make sure that it won't fail or cause any regressions later.

How could we test it ?

Maybe we can do the change in Factory first and see how it goes ?

Since you opened the bug against 42.3, we should at least make the change to SLE12-SP2 (not sure if it's also needed by SP1/42.1).

(In reply to Franck Bui from comment #13)
> (In reply to Ludwig Nussel from comment #12)
> > (In reply to Franck Bui from comment #11)
> > > I'm still not sure that creating an empty file would be a good idea: some
> > > packages might rely on it during the installation process.
> > 
> > What for? To take it and copy it elsewhere? That would be equally bad. 
> > 
> 
> I don't know, I just want to make sure that it won't fail or cause any
> regressions later.
> 
> How could we test it ?

By changing it and waiting for any impact :-)

> Maybe we can do the change in Factory first and see how it goes ?

Sure!

> Since you opened the bug against 42.3, we should at least make the change to
> SLE12-SP2 (not sure if it's also needed by SP1/42.1).

Feel free to reassign to Factory.

@Dominique, any objections we test this change in Factory first ?

(In reply to Franck Bui from comment #15)
> @Dominique, any objections we test this change in Factory first ?

No objection - but be prepared for reverts if weird stuff appears

I plan to use "%config /etc/machine-id" in the spec file as this file shouldn't be changed via a package update. 

Does that sound good or can you see something more appropriate ?

Thanks.

(In reply to Franck Bui from comment #17)
> I plan to use "%config /etc/machine-id" in the spec file as this file
> shouldn't be changed via a package update. 
> 
> Does that sound good or can you see something more appropriate ?
> 
> Thanks.

%config(noreplace) ?

See
http://people.ds.cam.ac.uk/jw35/docs/rpm_config.html
To find what behavior you really want

%config for one will put the packaged file in place - putting the users version in a .rpmsave backup

%config or %config(noreplace) shouldn't make any difference as machine-id file shouldn't be changed through a RPM update, ie it should always be kept as an empty file.

But I think you're right and %config(noreplace) should be safer even if it's theoretically not needed.

Changes submitted to Base:System
https://build.opensuse.org/request/show/479267

The relevant change is not part of Factory/TW.

I'm closing this bug now, if you think it also should be fixed in SLE12/Leap please reopen.

This change seem to cause an issue running tests during the build-process:

https://build.opensuse.org/package/show/home:weberho:UNSTABLE:python-systemd/python-systemd

[   59s]         if machineid is None:
[   59s] >           machineid = _id128.get_machine().hex
[   59s] E           IOError: [Errno 123] No medium found
[   59s]
[   59s] systemd/journal.py:387: IOError

The change not to create machine-id in %post is good. But something should create a non-empty machine-id in the build chroot for packages. It's not just systemd, but also other software which can assume that it's present. In Fedora, mock does that: https://github.com/rpm-software-management/mock/blob/devel/py/mockbuild/buildroot.py#L224.

@Zbigniew, welcome and thanks for contributing to openSUSE ! :)

> It's not just systemd, but also other software which can assume that it's present.

Well the issue here is not really the software itself but the scripts involved to install software packages. But I can see the point.

> In Fedora, mock does that: https://github.com/rpm-software-management/mock/blob/devel/py/mockbuild/buildroot.py#L224.

We could probably do that as well, Ludwig what do you think ?

BTW it seems that even if mock takes care of populating /etc/machine-id if it doesn't exist, Fedora still calls systemd-machine-id-setup in %post.

> Fedora still calls systemd-machine-id-setup in %post.

Hm, indeed. I think we should change this.

https://github.com/openSUSE/obs-build/pull/351