Bug 1052681

Summary: nvidia-gfxG03-kmp-default-340.102...x86_64.rpm "Digest verification failed"
Product: [openSUSE] openSUSE Distribution Reporter: Forgotten User NCG4lRmxtN <forgotten_NCG4lRmxtN>
Component: X11 3rd Party DriverAssignee: E-mail List <xorg-maintainer-bugs>
Status: RESOLVED FIXED QA Contact: Stefan Dirsch <sndirsch>
Severity: Normal    
Priority: P5 - None CC: ddadap, forgotten_NCG4lRmxtN
Version: Leap 42.2   
Target Milestone: ---   
Hardware: x86-64   
OS: openSUSE 42.2   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Forgotten User NCG4lRmxtN 2017-08-08 06:01:19 UTC
I encountered the error described, below, while using zypper to install updates this evening.

The download URI is http://download.nvidia.com/opensuse/leap/42.2/

The README file at that location states "The driver RPMs hosted in this location are entirely built, maintained and supported by Novell/SUSE. NVIDIA hosts them as a courtesy to Novell, however all problems and support requests related to these RPMs should be reported to Novell via their bug tracking system: http://bugzilla.novell.com".

I cleared the cache, tried again, and the error persisted. I even downloaded the file directly and the checksum still fails.

Error:

"Retrieving: nvidia-gfxG03-kmp-default-340.102_k4.4.27_2-18.1.x86_64.rpm ........................................[done (758.2 KiB/s)]

Warning: Digest verification failed for file 'nvidia-gfxG03-kmp-default-340.102_k4.4.27_2-18.1.x86_64.rpm'
[/var/cache/zypp/packages/nVidia_Graphics_Drivers/x86_64/nvidia-gfxG03-kmp-default-340.102_k4.4.27_2-18.1.x86_64.rpm]

  expected d9844df9a3ffa3c110d16c447293a633ca4e71013c52e3c14011145742f1bfa0
  but got  cef17e27677f0e66bd0b00d48f66d94479699fd4aebe1929d62d2425457715b9

Accepting packages with wrong checksums can lead to a corrupted system and in extreme cases even to a system compromise."

Thank you!
Comment 1 Stefan Dirsch 2017-08-08 09:31:26 UTC
Cannot reproduce that issue. Tried installing all packages provided for Leap 42.2. Did you run

  zypper ref

before?

I guess it is/was an interim failure, NVIDIA unfortunately often has when updating their CDN. Repositories have been updated about 10 hours ago.

Can you please try it again?
Comment 2 Stefan Dirsch 2017-08-08 09:54:45 UTC
Sorry, my testing wasn't correct. Indeed I can reproduce the issue with G03 packages in 42.2 repo. G02 and G04 in this repo are working though.

For some reason G03 packages in 42.3 are also broken, but G02 and G04 again fine.
Comment 3 Stefan Dirsch 2017-08-08 09:55:25 UTC
Daniel, could you have a look, please?
Comment 4 Forgotten User NCG4lRmxtN 2017-08-08 12:33:45 UTC
(In reply to Stefan Dirsch from comment #2)
> Sorry, my testing wasn't correct. Indeed I can reproduce the issue with G03
> packages in 42.2 repo. G02 and G04 in this repo are working though.
> 
> For some reason G03 packages in 42.3 are also broken, but G02 and G04 again
> fine.

Thank you, Stefan! My plan was to wait it out, but I checked and reported it when I found that no one else had.
Comment 5 Daniel Dadap 2017-08-08 15:01:30 UTC
Apologies, this appears to be due to a stale cache at NVIDIA's CDN. There was a recent repository update which updated the packages for the G02 and G04 drivers, without updating the G03 drivers. I guess the process for generating the packages doesn't necessarily produce bit-identical outputs when generating the same package more than once.

I'll update the repository maintenance scripts to explicitly purge the CDN cache for the entire repo when updating to hopefully avoid this sort of problem in the future. In the meantime, I've performed a manual purge of the cache for the Leap 42.2 and 42.3 repos, and the file mentioned in comment #1 appears to be checksumming correctly now:

$ curl https://download.nvidia.com/opensuse/leap/42.2/x86_64/nvidia-gfxG03-kmp-default-340.102_k4.4.27_2-18.1.x86_64.rpm | sha256sum -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 3641k  100 3641k    0     0  3799k      0 --:--:-- --:--:-- --:--:-- 3833k
d9844df9a3ffa3c110d16c447293a633ca4e71013c52e3c14011145742f1bfa0  -
$ curl https://download.nvidia.com/opensuse/leap/42.2/repodata/repomd.xml | grep filelists.xml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1510  100  1510    0     0  32942      0 --:--:-- --:--:-- --:--:-- 43142
  <location href="repodata/bdd58bdda5fe8263d19c23c3e51996271dc446a4d44f601bd1d7eb530643722d-filelists.xml.gz"/>
$ curl https://download.nvidia.com/opensuse/leap/42.2/repodata/bdd58bdda5fe8263d19c23c3e51996271dc446a4d44f601bd1d7eb530643722d-filelists.xml.gz | zgrep nvidia-gfxG03-kmp-default
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  6045  100  6045    0     0   129k      0 --:--:-- --:--:-- --:--:--  168k
<package pkgid="d9844df9a3ffa3c110d16c447293a633ca4e71013c52e3c14011145742f1bfa0" name="nvidia-gfxG03-kmp-default" arch="x86_64">
Comment 6 Daniel Dadap 2017-08-08 15:03:13 UTC
And of course by comment #1 I actually meant the bug description.
Comment 7 Stefan Dirsch 2017-08-08 16:14:19 UTC
Thanks, Daniel. I can confirm, that G03 packages now have the correct checksum. Just verified. Can be installed now without problems.

Your assumption is correct. The process for generating the packages doesn't necessarily produce bit-identical outputs when generating the same package more than once. :-( It has been a goal since ever, but we never got there unfortunately. So you can't rely on having no changes, when filenames don't change. Thanks for adjusting your process to keep this in consideration.

To be honest this CDN cache appears to be pretty stupid to me assuming, that file contents don't change, when filenames don't change. I guess you agree here. But
I see, that you likely can't change this system.

(In reply to Daniel Dadap from comment #6)
> And of course by comment #1 I actually meant the bug description.
Sure, understood this.

Thanks again!
Comment 8 Daniel Dadap 2017-08-08 17:01:04 UTC
(In reply to Stefan Dirsch from comment #7)

> To be honest this CDN cache appears to be pretty stupid to me assuming, that
> file contents don't change, when filenames don't change. I guess you agree
> here. But I see, that you likely can't change this system.


To be totally fair to the CDN, it doesn't make any assumptions about whether file contents have changed: it relies on the customer to dirty the cache when making updates. Unfortunately, the process by which I publish new content to a staging server which in turn makes the content available to the CDN (and which, as you guessed, is beyond my control) does not automatically invalidate the cache when a file's contents have changed, which is why I have to include that step in the part of the process that I do have control over.

In the case where new content appears under a new filename, explicitly dirtying the cache isn't necessary because there's no preexisting cached data, so requests for that resource generates a cache miss, causing the resource to be loaded into the cache. In any case, these repos aren't huge, and there are very few truly static files within them, so clearing the cache for the entire repo whenever the repos are updated is cheaper than inspecting each individual file to see whether it changed.
Comment 9 Forgotten User NCG4lRmxtN 2017-08-08 23:08:51 UTC
Just a quick note to say "Thank you!" You guys rock!