Bug 1057640

Summary: Custom variables not recognized
Product: [openSUSE] openSUSE Distribution Reporter: Tony Mechelynck <antoine.mechelynck>
Component: libzyppAssignee: E-mail List <zypp-maintainers>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Major    
Priority: P5 - None CC: aalzayed
Version: Leap 42.3   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: Community User Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Tony Mechelynck 2017-09-07 16:57:59 UTC
I don't know if this is a program bug or a documentation bug (i.e. a lack of clarity in the documentation).

I tried to create a custom variable for use by zypper, as explained in the output of "man zypper" under "Custom Variables", as follows:

1. Create a file named /etc/zypp/vars.d/server
2. Make the first line of that file equal to
       ftp.halifax.rwth-aachen.de/opensuse
(with no blanks at left).
3. In /etc/repos.d/all.repo (the only *.repo I have in that directory) replace every instance of the above string on the baseurl= lines (one per repo except Packman and the DVD) by ${server}

the idea being to replace that one line by
    download.opensuse.org
when the present outage will be _completely_ a thing of the past, so that all URLs will be reassigned in one fell swoop. (Then run "zypper ref" again to make sure it's OK.)

Then try "zypper ref"
I expected zypper to resolve my custom variable, refresh all repositories, and tell me that they were now up to date. Instead, I got an error message in red telling me that ${server} was an unknown variable.

If the above behaviour is expected, then the manpage is unclear; conversely, if what I did was right, then there is a bug in zypper.
Comment 1 Michael Andres 2017-09-07 17:27:06 UTC
No, that's how it is intended to be used  

BUT it looks like the URL parser rejects this before the variables are substituted. It works in the URLs path but not in the host part. I'll have to look into this, but it's probably not that easy to fix....
Comment 2 Michael Andres 2017-09-08 08:04:28 UTC
Grrrr. This will be a shomewhat bigger fix. 

WORKAROUND:
As custom variables in the host part of the URL are not working, you can just mimic this by using a (local) RIS service:

- Create directory and file ./repo/repoindex.xml somwhere on your disk (or accessible via http,nfs, etc), E.g. my service for the Leap-42.3 repos:
> <repoindex      disturl="http://download.opensuse.org"
>                 distsub="leap/"
>                 distver="42.3"
>             debugenable="false"
>            sourceenable="false">
> 
>   <repo url="%{disturl}/distribution/%{distsub}%{distver}/repo/oss"
>                 alias="repo-oss"
>                  name="%{alias} (%{distver})"
>               enabled="true"
>           autorefresh="true"/>
> 
>   <repo url="%{disturl}/distribution/%{distsub}%{distver}/repo/non-oss"
>                 alias="repo-non-oss"
>                  name="%{alias} (%{distver})"
>               enabled="true"
>           autorefresh="true"/>
> 
> 
>   <repo url="%{disturl}/update/%{distsub}%{distver}/oss"
>                 alias="update-oss"
>                  name="%{alias} (%{distver})"
>               enabled="true"
>           autorefresh="true"/>
> 
>   <repo url="%{disturl}/update/%{distsub}%{distver}/non-oss"
>                 alias="upadte-non-oss"
>                  name="%{alias} (%{distver})"
>               enabled="true"
>           autorefresh="true"/>
> 
> 
>   <repo url="%{disturl}/debug/distribution/%{distsub}%{distver}/repo/oss"
>                 alias="debug-oss"
>                  name="%{alias} (%{distver})"
>               enabled="%{debugenable}"
>           autorefresh="true"/>
> 
>   <repo url="%{disturl}/debug/distribution/%{distsub}%{distver}/repo/oss"
>                 alias="debug-non-oss"
>                  name="%{alias} (%{distver})"
>               enabled="%{debugenable}"
>           autorefresh="true"/>
> 
> 
>   <repo url="%{disturl}/source/distribution/%{distsub}%{distver}/repo/oss"
>                 alias="source-oss"
>                  name="%{alias} (%{distver})"
>               enabled="%{sourceenable}"
>           autorefresh="true"/>
> 
>   <repo url="%{disturl}/source/distribution/%{distsub}%{distver}/repo/non-oss"
>                 alias="source-non-oss"
>                  name="%{alias} (%{distver})"
>               enabled="%{sourceenable}"
>           autorefresh="true"/>
> </repoindex>
%{VAR} always refers to the value defined in the <repoindex > tag (reserved names are 'ttl' and 'alias'). %{alias} which is available within a <repo > tag after the alias was defined.

- Given the file is located at '/somewhere'/repo/repoindex.xml then add the service by
> zypper sa /somewhere openSUSE

- manually refresh the service (incl. the repos) with
> zypper refs -r
Refreshing the service will evaluate the repoindex.xml and adjust the Services repos accordingly. The repos alias will be prefixed by the service name:
> #  | Alias                   | Name                  | Enabled | GPG Check | Refresh
> ---+-------------------------+-----------------------+---------+-----------+--------
>  9 | openSUSE:debug-non-oss  | debug-non-oss (42.3)  | No      | ----      | ----   
> 10 | openSUSE:debug-oss      | debug-oss (42.3)      | No      | ----      | ----   
> 11 | openSUSE:repo-non-oss   | repo-non-oss (42.3)   | Yes     | (r ) Yes  | Yes    
> 12 | openSUSE:repo-oss       | repo-oss (42.3)       | Yes     | (rp) Yes  | Yes    
> 13 | openSUSE:source-non-oss | source-non-oss (42.3) | No      | ----      | ----   
> 14 | openSUSE:source-oss     | source-oss (42.3)     | No      | ----      | ----   
> 15 | openSUSE:upadte-non-oss | upadte-non-oss (42.3) | Yes     | (rp) Yes  | Yes    
> 16 | openSUSE:update-oss     | update-oss (42.3)     | Yes     | (rp) Yes  | Yes    

- After editing the repoindex.xml (e.g. disturl=) you must manually refresh the service to takeover the changes, or you turn on autorefresh for the service (depends on how often you chnage the content)
> zypper ms -r openSUSE
Comment 3 Tony Mechelynck 2017-09-08 08:53:28 UTC
In reply to coment #2:

The workaround I used is different: Besides the /etc/zypp/repos.d/all.repo (which is the repositories list zypper uses) I created two backup versions: /etc/zypp/repos.d/all.repo.aachen.bak (with the URLs of the Halifax backup at RWTH Aachen) and /etc/zypp/repos.d/all.repo.bak (with the "master" URLs at download.opensuse.org). When the time comes, I can copy one or the other of the .bak files onto the all.repo to switch all repositories from one server to another.

Adding the server name to the "name" of the repo is a nice idea, I think I'll implement it. At the moment (and after resolution of ${releasever} == 42.3 in all "name" and "baseurl" values) I have the following list of repos:

linux-2iyu:~ # zypper lr -u
Repository priorities in effect:                                                                                                          (See 'zypper lr -P' for details)
      99 (default priority) :  8 repositories
     140 (lowered priority) :  1 repository
     150 (lowered priority) :  4 repositories

#  | Alias   | Name                          | Enabled | GPG Check | Refresh | URI
---+---------+-------------------------------+---------+-----------+---------+---------------------------------------------------------------------------------------
 1 | DVD     | Leap-42.3-DVD                 | Yes     | (r ) Yes  | No      | iso:///?iso=openSUSE-Leap-42.3-DVD-x86_64.iso&url=dir%3A%2Fboot%2Fiso
 2 | NOSS    | Leap-42.3-NonOSS              | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/distribution/leap/42.3/repo/non-oss/
 3 | OSS     | Leap-42.3-OSS                 | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/distribution/leap/42.3/repo/oss/
 4 | dbNO    | Leap-42.3-Debug-NonOSS        | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/debug/distribution/leap/42.3/repo/non-oss/
 5 | dbO     | Leap-42.3-Debug-OSS           | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/debug/distribution/leap/42.3/repo/oss/
 6 | dbuNO   | Leap-42.3-Debug-Update-NonOSS | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/debug/update/leap/42.3/non-oss/
 7 | dbuO    | Leap-42.3-Debug-Update-OSS    | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/debug/update/leap/42.3/oss/
 8 | packman | Leap-42.3-Packman             | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/packman/suse/openSUSE_Leap_42.3/
 9 | sNOSS   | Leap-42.3-Source-NonOSS       | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/source/distribution/leap/42.3/repo/non-oss/
10 | sOSS    | Leap-42.3-Source-OSS          | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/source/distribution/leap/42.3/repo/oss/
11 | uNOSS   | Leap-42.3-Update-NonOSS       | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/update/leap/42.3/non-oss/
12 | uOSS    | Leap-42.3-Update-OSS          | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/update/leap/42.3/oss/
13 | uT      | Leap-42.3-Update-Test         | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/opensuse/update/leap/42.3-test/
linux-2iyu:~ #
Comment 4 Tony Mechelynck 2017-09-10 16:19:01 UTC
:-/ RWTH Aachen is synchronizing by rsync from stage, which is down: this explains why no changes were visible. I went back to download.opensuse.org and plop! suddenly a lot of updates appeared.
Comment 5 Michael Andres 2017-11-10 10:58:30 UTC
*** Bug 1067605 has been marked as a duplicate of this bug. ***
Comment 6 Michael Andres 2017-11-22 09:56:01 UTC
Fixed in 
  libzypp-17.0.2 (TW/CODE15)
  libzypp-16.17.5 (Leap 42.2/3, SLE12-SP2/3)
Comment 7 Michael Andres 2017-11-22 10:15:55 UTC
JFYI: Variable substitution within an URIs authority is limited to host and port. Bash style definition of default and alternate values is not supported. No variables can be used in an URIs scheme, user and password.
Comment 8 Tony Mechelynck 2017-11-22 10:17:49 UTC
(In reply to Michael Andres from comment #6)
> Fixed in 
>   libzypp-17.0.2 (TW/CODE15)
>   libzypp-16.17.5 (Leap 42.2/3, SLE12-SP2/3)

The fix seems noy yet to have been compiled and uploaded:

linux-2iyu:~ # zypper lr -u
Repository priorities in effect:                                                                                                          (See 'zypper lr -P' for details)
      99 (default priority) :  8 repositories
     140 (lowered priority) :  1 repository
     150 (lowered priority) :  4 repositories

#  | Alias   | Name                               | Enabled | GPG Check | Refresh | URI
---+---------+------------------------------------+---------+-----------+---------+-------------------------------------------------------------------------
 1 | DVD     | boot:Leap-42.3-DVD                 | Yes     | (r ) Yes  | No      | iso:///?iso=openSUSE-Leap-42.3-DVD-x86_64.iso&url=dir%3A%2Fboot%2Fiso
 2 | NOSS    | SUSE:Leap-42.3-NonOSS              | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/distribution/leap/42.3/repo/non-oss/
 3 | OSS     | SUSE:Leap-42.3-OSS                 | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/distribution/leap/42.3/repo/oss/
 4 | dbNO    | SUSE:Leap-42.3-Debug-NonOSS        | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/debug/distribution/leap/42.3/repo/non-oss/
 5 | dbO     | SUSE:Leap-42.3-Debug-OSS           | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/debug/distribution/leap/42.3/repo/oss/
 6 | dbuNO   | SUSE:Leap-42.3-Debug-Update-NonOSS | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/debug/update/leap/42.3/non-oss/
 7 | dbuO    | SUSE:Leap-42.3-Debug-Update-OSS    | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/debug/update/leap/42.3/oss/
 8 | packman | RWTH:Leap-42.3-Packman             | Yes     | (r ) Yes  | Yes     | http://ftp.halifax.rwth-aachen.de/packman/suse/openSUSE_Leap_42.3/
 9 | sNOSS   | SUSE:Leap-42.3-Source-NonOSS       | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/source/distribution/leap/42.3/repo/non-oss/
10 | sOSS    | SUSE:Leap-42.3-Source-OSS          | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/source/distribution/leap/42.3/repo/oss/
11 | uNOSS   | SUSE:Leap-42.3-Update-NonOSS       | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/update/leap/42.3/non-oss/
12 | uOSS    | SUSE:Leap-42.3-Update-OSS          | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/update/leap/42.3/oss/
13 | uT      | SUSE:Leap-42.3-Update-Test         | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/update/leap/42.3-test/
linux-2iyu:~ # zypper ref
Repository 'boot:Leap-42.3-DVD' is up to date.
Repository 'SUSE:Leap-42.3-NonOSS' is up to date.
Repository 'SUSE:Leap-42.3-OSS' is up to date.
Repository 'SUSE:Leap-42.3-Debug-NonOSS' is up to date.
Repository 'SUSE:Leap-42.3-Debug-OSS' is up to date.
Repository 'SUSE:Leap-42.3-Debug-Update-NonOSS' is up to date.
Repository 'SUSE:Leap-42.3-Debug-Update-OSS' is up to date.
Repository 'RWTH:Leap-42.3-Packman' is up to date.
Repository 'SUSE:Leap-42.3-Source-NonOSS' is up to date.
Repository 'SUSE:Leap-42.3-Source-OSS' is up to date.
Repository 'SUSE:Leap-42.3-Update-NonOSS' is up to date.
Repository 'SUSE:Leap-42.3-Update-OSS' is up to date.
Repository 'SUSE:Leap-42.3-Update-Test' is up to date.
All repositories have been refreshed.
linux-2iyu:~ # zypper info libzypp
Loading repository data...
Reading installed packages...


Information for package libzypp:
--------------------------------
Repository     : SUSE:Leap-42.3-Update-OSS
Name           : libzypp
Version        : 16.15.6-12.1
Arch           : x86_64
Vendor         : openSUSE
Installed Size : 7.4 MiB
Installed      : Yes
Status         : up-to-date
Source package : libzypp-16.15.6-12.1.src
Summary        : Package, Patch, Pattern, and Product Management
Description    :
    Package, Patch, Pattern, and Product Management
Comment 10 Swamp Workflow Management 2018-01-02 20:09:42 UTC
SUSE-RU-2018:0004-1: An update that has 5 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1057640,1067605,1068708,1071466,969569
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP3 (src):    libzypp-16.17.7-2.20.2
SUSE Linux Enterprise Server 12-SP3 (src):    libzypp-16.17.7-2.20.2, zypper-1.13.39-21.13.7
SUSE Linux Enterprise Desktop 12-SP3 (src):    libzypp-16.17.7-2.20.2, zypper-1.13.39-21.13.7
SUSE Container as a Service Platform ALL (src):    libzypp-16.17.7-2.20.2, zypper-1.13.39-21.13.7
Comment 11 Tony Mechelynck 2018-01-03 16:34:40 UTC
Patch 7628 has appeared on Leap 42.3 Update-Test repo, upgrading libzypp to version 16.17.7-18.1 and zypper, zypper-log and zypper-aptitude to 1.13.39-14.1.
Comment 12 Swamp Workflow Management 2018-01-04 20:09:52 UTC
SUSE-RU-2018:0021-1: An update that has 16 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1035946,1036002,1047233,1053671,1057188,1057634,1057640,1058695,1058783,1062561,1064999,1067605,1068708,1071466,661410,969569
CVE References: 
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP2 (src):    libsolv-0.6.30-2.27.10.11, libzypp-16.17.7-27.39.1
SUSE Linux Enterprise Server for Raspberry Pi 12-SP2 (src):    libsolv-0.6.30-2.27.10.11, libzypp-16.17.7-27.39.1, zypper-1.13.39-18.25.6
SUSE Linux Enterprise Server 12-SP2 (src):    libsolv-0.6.30-2.27.10.11, libzypp-16.17.7-27.39.1, zypper-1.13.39-18.25.6
SUSE Linux Enterprise Desktop 12-SP2 (src):    libsolv-0.6.30-2.27.10.11, libzypp-16.17.7-27.39.1, zypper-1.13.39-18.25.6
OpenStack Cloud Magnum Orchestration 7 (src):    libsolv-0.6.30-2.27.10.11, libzypp-16.17.7-27.39.1, zypper-1.13.39-18.25.6
Comment 13 Swamp Workflow Management 2018-01-05 23:07:48 UTC
openSUSE-RU-2018:0032-1: An update that has 5 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1057640,1067605,1068708,1071466,969569
CVE References: 
Sources used:
openSUSE Leap 42.3 (src):    libzypp-16.17.7-18.1, zypper-1.13.39-14.1
Comment 14 Swamp Workflow Management 2018-01-05 23:09:25 UTC
openSUSE-RU-2018:0033-1: An update that has 16 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1035946,1036002,1047233,1053671,1057188,1057634,1057640,1058695,1058783,1062561,1064999,1067605,1068708,1071466,661410,969569
CVE References: 
Sources used:
openSUSE Leap 42.2 (src):    libsolv-0.6.30-5.6.1, libzypp-16.17.7-5.15.1, zypper-1.13.39-5.15.1