Bugzilla – Bug 1170848
VUL-0: CVE-2023-32189: Handling SSH key in SUSE Manager when bootstrapping new clients
Last modified: 2024-05-06 12:31:17 UTC
I'd like the security team to look at a recent feature that we added in SUSE Manager (SUMA). Timeframe: The feature should go to SUSE Manager 4.1, branch freeze for RC1 is on July the 9th. Intro: - SUSE Manager manages (among other things) salt minions - adding a new client (salt minion) to SUMA can be done with a procedure called "bootstrap" - for invoking the bootstrap procedure, the user passes information about the target client to SUMA Server (hostname, SSH port, SSH password) - then, SUMA Server uses Salt SSH to prepare the client (installing required packages, setting up configuration files, ...) - after bootstrapping, the client registration is run (Salt SSH not involved anymore). A registered the system can be managed by SUMA. - (Salt SSH uses SSH under the hood) - for bootstrapping, we supported the SSH password authentication only until now - we added support for authentication with SSH key recently and I'd like to ask you to take a look at it. The new feature: - upon bootstrapping the user passes their SSH key file (and optionally a passphrase) to SUMA - SUMA executes the bootstrap procedure, during which it stores the SSH key in a file (in the /srv/susemanager/salt/salt_ssh/temp_bootstrap_keys directory) - (the file is stored with user:group salt:salt and permissions 600) - SUMA deletes the SSH key file both when the bootstrap succeeded or failed - user can invoke the bootstrap procedure from the SUMA Web UI or XMLRPC - in case of the Web UI, the user uses a HTML <input type="file" /> to select the local file. The file is then read in the browser using FileReader and its contents is sent to SUMA Server when initiating the bootstrap procedure. The user is also informed about temporary storing of the key on SUMA server in the bootstrap web page. - in case of the XMLRPC, user passes the SSH key file contents as a string - in both XMLRPC and Web UI, the communication goes over HTTPS The code: I'm not sure how to present you the code, so for now I included the links to github with full pull requests and then "shortcuts" to interesting parts. Shall I also attach a tarball with the patches, so we won't rely on github? The full pull requests: - backend + XMLRPC: - https://github.com/uyuni-project/uyuni/pull/2069 - + this bugfix: https://github.com/uyuni-project/uyuni/pull/2165/commits/21aeff4ff69fd4ce9cd5c81b04e8edad1678d5b5 - Web UI: https://github.com/uyuni-project/uyuni/pull/2095 "Interesting parts": - passing the SSH key in XMLRPC: https://github.com/uyuni-project/uyuni/pull/2069/files#diff-708ec02b367f9305b2f066e165b6b5a5 - passing the SSH key in Web UI: https://github.com/uyuni-project/uyuni/pull/2095/files#diff-b8754718aa908385849b4429c1e0a837R218 - the SSH key creation & deletion - original: https://github.com/uyuni-project/uyuni/pull/2069/files#diff-9cd8c01fc88e5cc5d7159ba37599533aR549 - bugfix: https://github.com/uyuni-project/uyuni/pull/2165/commits/21aeff4ff69fd4ce9cd5c81b04e8edad1678d5b5 Let me know if you need any further assistance/info.
Hi, thanks for reaching out! We will have a look at this next week.
Sorry that nothing happened with this request. We had various changes in staffing in the security team. Both Malte and Hans are no longer with SUSE. Has this feature already been released by now? Does it still make sense to do the review?
> Has this feature already been released by now? Yes. The feature has been in SUSE Manager 4.1 since April 2020. > Does it still make sense to do the review? From my POV yes: if there is a security problem in the feature, then we should fix it.
(In reply to fkobzik@suse.com from comment #3) > > Does it still make sense to do the review? > > From my POV yes: if there is a security problem in the feature, then we should fix it. Okay. I have heard in my team that either Hans or Malte might have had some communication with your team about this review outside of this bug (via email?). Do you know anything about that? It seems it was Hans that investigated this, he wrote in a work report in June: ``` * SUSE Manager security: ssh key handling - Proposal of a solution that avoids writing private keys to disk - A solution without uploading private keys needs some discussion with SUMA team (has drawbacks; most likely not feasible in short term, maybe not at all) - No further news fom SUMA team regarding a demo/intro for us ``` Do you know anything about this proposal he made?
> Okay. I have heard in my team that either Hans or Malte might have had some communication with your team about this review outside of this bug (via email?). Do you know anything about that? No, I don't remember anything about further communication. There might have been some rocket.chat communication with Malte, but it's gone now. I was also grepping through mails and found nothing. > Do you know anything about this proposal he made? Sadly not.
Removing block, that was an accident.
tomcat is reading and writing in that directory. So this PR will not work. We would need to do bigger changes. Maybe we can change the group and 750 would work. But we need to have a deeper look.
Maybe Michael this ticket was out of your radar during the time. Can you help me in figuring out the chosen direction?
PR is ready for review: https://github.com/SUSE/spacewalk/pull/23206 I'd personally do more testing but so far looks good. I've created some instructions for upgrade, depending if we want to do key migration or not: Method 1 - With key rotation 1 - Offboard all ssh minions 2 - Check if proper clean-up has been done in those minions (if offboards has been correct, proper cleanup is done): - Check ssh keypair `mgr_own_id` is not present in $HOME/.ssh - Check that in $HOME/.ssh/authorized_keys file, keys `salt@server` and `susemanager-own-ssh-push` are not present 3 - Delete keypair `mgr_ssh_id` at susemanager server `rm -rf /srv/susemanager/salt/salt_ssh/*` 4 - Upgrade susemanager 5 - Onboard again salt ssh minions Method 2 - Keeping the same old key 1 - Create the new folder with the right permissions `mkdir -p /var/lib/salt/.ssh && chown salt:salt -R /var/lib/salt && chmod 0700 /var/lib/salt/.ssh` 2 - Move the keys to its new folder `mv /srv/susemanager/salt/salt_ssh/* /var/lib/salt/.ssh` 3 - Upgrade susemanager Thanks, /Miguel
Hi Paolo, Could you please assign us a CVE number for this security incident? Thanks!
Hi Miguel, it's Johannes from our team that assigns CVE from our pool. Now he is on Winter FTO, so I guess we have to wait 2024 to have it.
Hi Johannes, can you please assign a CVE for this?
Please use CVE-2023-32189
(In reply to Johannes Segitz from comment #25) > Please use CVE-2023-32189 Thanks
Lifiting embargo. Fixed in SUMA 4.3.11
SUSE-SU-2024:0513-1: An update that solves three vulnerabilities, contains one feature and has 39 security fixes can now be installed. Category: security (important) Bug References: 1170848, 1210911, 1211254, 1211560, 1211912, 1213079, 1213507, 1213738, 1213981, 1214077, 1214791, 1215166, 1215514, 1215769, 1215810, 1215813, 1215982, 1216114, 1216394, 1216437, 1216550, 1216657, 1216753, 1216781, 1216988, 1217069, 1217209, 1217588, 1217784, 1217869, 1218019, 1218074, 1218075, 1218089, 1218094, 1218146, 1218490, 1218615, 1218669, 1218849, 1219577, 1219850 CVE References: CVE-2023-32189, CVE-2024-22231, CVE-2024-22232 Jira References: MSQA-719 Sources used: openSUSE Leap 15.4 (src): release-notes-susemanager-proxy-4.3.11-150400.3.79.1, release-notes-susemanager-4.3.11-150400.3.100.1 SUSE Manager Proxy 4.3 (src): release-notes-susemanager-proxy-4.3.11-150400.3.79.1 SUSE Manager Retail Branch Server 4.3 (src): release-notes-susemanager-proxy-4.3.11-150400.3.79.1 SUSE Manager Server 4.3 (src): release-notes-susemanager-4.3.11-150400.3.100.1 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2024:0485-1: An update that solves two vulnerabilities, contains one feature and has 44 security fixes can now be installed. Category: security (important) Bug References: 1170848, 1210911, 1211254, 1211560, 1211912, 1213079, 1213507, 1213738, 1213981, 1214077, 1214791, 1215166, 1215514, 1215769, 1215810, 1215813, 1215982, 1216114, 1216394, 1216437, 1216550, 1216609, 1216657, 1216753, 1216781, 1216988, 1217069, 1217209, 1217588, 1217784, 1217869, 1218019, 1218074, 1218075, 1218089, 1218094, 1218146, 1218490, 1218615, 1218669, 1218837, 1218849, 1219151, 1219449, 1219577, 1219850 CVE References: CVE-2023-31582, CVE-2023-32189 Jira References: MSQA-719 Sources used: SUSE Manager Proxy 4.3 Module 4.3 (src): mgr-daemon-4.3.8-150400.3.12.5, susemanager-build-keys-15.4.10-150400.3.23.5, spacewalk-client-tools-4.3.18-150400.3.24.7, spacecmd-4.3.26-150400.3.33.5, spacewalk-backend-4.3.27-150400.3.38.2, spacewalk-web-4.3.37-150400.3.39.7, patterns-suse-manager-4.3-150400.5.9.5, spacewalk-certs-tools-4.3.22-150400.3.25.1 SUSE Manager Server 4.3 Module 4.3 (src): supportutils-plugin-susemanager-4.3.10-150400.3.18.5, susemanager-sls-4.3.40-150400.3.44.1, susemanager-build-keys-15.4.10-150400.3.23.5, prometheus-postgres_exporter-0.10.1-150400.3.9.5, subscription-matcher-0.35-150400.3.19.5, spacewalk-web-4.3.37-150400.3.39.7, spacewalk-backend-4.3.27-150400.3.38.2, jose4j-0.5.1-150400.3.6.2, spacewalk-utils-4.3.19-150400.3.21.5, susemanager-sync-data-4.3.16-150400.3.22.2, liberate-formula-0.1.0-150400.10.3.3, cobbler-3.3.3-150400.5.39.5, spacewalk-setup-4.3.19-150400.3.30.5, spacewalk-client-tools-4.3.18-150400.3.24.7, inter-server-sync-0.3.2-150400.3.27.5, susemanager-4.3.34-150400.3.45.5, uyuni-reportdb-schema-4.3.9-150400.3.12.7, spacecmd-4.3.26-150400.3.33.5, prometheus-formula-0.8.0-150400.3.6.5, susemanager-docs_en-4.3-150400.9.53.5, saltboot-formula-0.1.1701196218.b6b8ca1-150400.3.15.3, susemanager-schema-4.3.24-150400.3.36.7, spacewalk-java-4.3.71-150400.3.74.2, patterns-suse-manager-4.3-150400.5.9.5, spacewalk-certs-tools-4.3.22-150400.3.25.1, grafana-formula-0.10.0-150400.3.15.5 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Fixed in SUMA 4.3.11
what package was actually affected here?
Key rotation documented and backported to manager-4.3.
SUSE-SU-2024:1532-1: An update that solves one vulnerability, contains one feature and has 33 security fixes can now be installed. Category: security (important) Bug References: 1170848, 1208572, 1214340, 1214387, 1216085, 1217204, 1217874, 1218764, 1218805, 1218931, 1218957, 1219061, 1219233, 1219634, 1219875, 1220001, 1220101, 1220169, 1220194, 1220221, 1220376, 1220705, 1220726, 1220903, 1220980, 1221111, 1221182, 1221279, 1221465, 1221571, 1221784, 1221922, 1222110, 1222347 CVE References: CVE-2023-51775 Jira References: MSQA-760 Maintenance Incident: [SUSE:Maintenance:33591](https://smelt.suse.de/incident/33591/) Sources used: SUSE Manager Proxy 4.3 (src): release-notes-susemanager-proxy-4.3.12-150400.3.82.3 SUSE Manager Retail Branch Server 4.3 (src): release-notes-susemanager-proxy-4.3.12-150400.3.82.3 SUSE Manager Server 4.3 (src): release-notes-susemanager-4.3.12-150400.3.108.2 openSUSE Leap 15.4 (src): release-notes-susemanager-proxy-4.3.12-150400.3.82.3, release-notes-susemanager-4.3.12-150400.3.108.2 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
SUSE-SU-2024:1507-1: An update that solves one vulnerability, contains one feature and has 32 security fixes can now be installed. Category: security (moderate) Bug References: 1170848, 1208572, 1214340, 1214387, 1216085, 1217204, 1217874, 1218764, 1218805, 1218931, 1218957, 1219061, 1219233, 1219634, 1219875, 1220101, 1220169, 1220194, 1220221, 1220376, 1220705, 1220726, 1220903, 1220980, 1221111, 1221182, 1221279, 1221465, 1221571, 1221784, 1221922, 1222110, 1222347 CVE References: CVE-2023-51775 Jira References: MSQA-760 Maintenance Incident: [SUSE:Maintenance:33406](https://smelt.suse.de/incident/33406/) Sources used: SUSE Manager Proxy 4.3 Module 4.3 (src): spacewalk-certs-tools-4.3.23-150400.3.28.5, spacewalk-client-tools-4.3.19-150400.3.27.5, uyuni-common-libs-4.3.10-150400.3.18.4, mgr-daemon-4.3.9-150400.3.15.5, spacewalk-web-4.3.38-150400.3.42.6, spacewalk-backend-4.3.28-150400.3.41.7, spacecmd-4.3.27-150400.3.36.5 SUSE Manager Server 4.3 Module 4.3 (src): smdba-1.7.13-0.150400.4.12.4, susemanager-docs_en-4.3-150400.9.56.4, uyuni-reportdb-schema-4.3.10-150400.3.15.6, susemanager-4.3.35-150400.3.48.6, spacewalk-backend-4.3.28-150400.3.41.7, spacewalk-java-4.3.73-150400.3.79.1, susemanager-sync-data-4.3.17-150400.3.25.4, cobbler-3.3.3-150400.5.42.5, jose4j-0.5.1-150400.3.9.4, susemanager-sls-4.3.41-150400.3.47.6, spacecmd-4.3.27-150400.3.36.5, uyuni-common-libs-4.3.10-150400.3.18.4, spacewalk-certs-tools-4.3.23-150400.3.28.5, spacewalk-web-4.3.38-150400.3.42.6, inter-server-sync-0.3.3-150400.3.30.4, susemanager-schema-4.3.25-150400.3.39.5, supportutils-plugin-susemanager-4.3.11-150400.3.21.4, spacewalk-client-tools-4.3.19-150400.3.27.5, image-sync-formula-0.1.1711646883.4a44375-150400.3.18.4, spacewalk-config-4.3.13-150400.3.15.5, subscription-matcher-0.37-150400.3.22.4 NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.