|
Bugzilla – Full Text Bug Listing |
| Summary: | libzypp (zypper/YaST) can't add certain repos anymore (mirrors out of sync) | ||
|---|---|---|---|
| Product: | [openSUSE] openSUSE 10.2 | Reporter: | Wolfgang Rosenauer <wolfgang> |
| Component: | libzypp | Assignee: | Peter Poeml <poeml> |
| Status: | RESOLVED FIXED | QA Contact: | Klaus Kämpf <kkaempf> |
| Severity: | Major | ||
| Priority: | P5 - None | CC: | aj, casualprogrammer, crrodriguez, dmacvicar, forgotten_t-K-SYtr-p, novell, trango |
| Version: | Final | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | Other | ||
| Whiteboard: | |||
| Found By: | Other | Services Priority: | |
| Business Priority: | Blocker: | --- | |
| Marketing QA Status: | --- | IT Deployment: | --- |
| Bug Depends on: | |||
| Bug Blocks: | 239483 | ||
| Attachments: | zypper.log | ||
|
Description
Wolfgang Rosenauer
2007-01-24 10:01:28 UTC
Created attachment 114590 [details]
zypper.log
It's really interesting since the above happens on two different machines with 10.2 and the same repos added. So something not local to the machines triggered an error in libzypp and caused some corruption. Hmm, someone has any suggestion how to it to work again? Maybe zypp only gets confused very seldom and so this issue is not of high priority but for me it is. So any idea how to fix the current zypp state at least as a workaround? As it just came up on IRC: I never ever had ZMD/rug installed on these systems. 2007-01-24 11:00:34 <5> Hygiea(8180) [base] Exception.cc(log):94 SourceImpl.cc(getPossiblyCachedMetadataFile):400 THROW: SourceImpl.cc(getPossiblyCachedMetadataFile):400: http://software.opensuse.org/download/openSUSE:/Tools/openSUSE_10.2./repodata/filelists.xml.gz fails checksum verification. But the checksum seems fine nevertheless. Puzzling. Christoph, could it be related to the redirector of openSUSE? Getting files from various hosts which are not 100% in sync? Aha, it has indeed succeeded on the third try for me. I've now used ftp://ftp4.gwdg.de/pub/opensuse/repositories/openSUSE:/Tools/openSUSE_10.2 and it worked. So yes, it seems to be a redirector problem. Interesting is that it happened only for the above two repositories for me. (I have tried others and they worked the first time). Strange behaviour at least. It is not directly related to the redirector. It is related to the fact that we don't set sensible cache controlling headers. (Assuming that the mirrors have exact copies.) We now set cache-control: no-cache, must-revalidate headers on repo-md files, on repos.opensuse.org. However, mirrors that don't do that could still serve files which are cached even though they are outdated. Thus, we need to find a way to cooperate with our mirrors so they also set those header (or don't mirror those files so they are always served by us...). Harald, the redirector implements some kind of "session handling" to make sure to redirector the user to only one mirror for those kind of metadata files. However, as Peter already elaborated, we seem to be hitting some nasty caching-related problems right now. Meanwhile I'm pretty sure that most issues referenced here are not caused by inconsistent mirrors, or the fact that we redirect things, but rather to caching of repo metadata by web caches in between. Meanwhile, we set an appropriate Cache-Control header on those repo metadata which are frequently re-created by the buildservice. Today, I added Expires header also, just in case that cache doesn't implement Cache-Control, (or some client software like zypper, yum, smart) makes their own caching decision and doesn't have a complete implementation. The fact that we redirect comes _only_ into play because if metadata is pulled from a mirror, because on mirrors we have no influence on the HTTP headers, and can't fix them as described above. Thus, since 2007-01-29 18:20:53 +0100 we are no longer redirecting metadata requests to mirrors at all. For packages it should not matter that much, whether they get stale or mirrors are inconsistent, because the filename is altered at each rebuild. A squid with default configuration will deliver HTTP content from cache without revalidation up to 72 hours, so please allow some time for our latest changes to become effective. By the way, all this just displays how well the integrity checking of repositories is done :) I'm regarding the bug fixed; please reopen if the problem persists. *** Bug 239483 has been marked as a duplicate of this bug. *** *** Bug 251275 has been marked as a duplicate of this bug. *** (In reply to comment #11) > The fact that we redirect comes _only_ into play because if metadata is > pulled from a mirror, because on mirrors we have no influence on the > HTTP headers, and can't fix them as described above. Thus, since > 2007-01-29 18:20:53 +0100 we are no longer redirecting metadata requests > to mirrors at all. According to bug 251275, this seems not to be true. Problems persist. Peter, reassigning the bug to you, please reassign as/if needed. *** Bug 239625 has been marked as a duplicate of this bug. *** Why do we do these redirects, anyway? To pick up the best mirror for each file accessed? What about redirect just once per session (to just one mirror)? The idea as such seems brilliant, you only need to register one source, then depending on load or whatever, you distribute the requests to all available servers subsequently. Problem is, how is synchronicity checked and used as selection criterion. Thanks for reopening the bug. Testing with the media file referenced in bug 251275, I found that I did a mistake. In my last attempt to fix this, I added a ZrkadloExcludeFileMask for the Factory tree, matching on files which do not contain digits. However, the expression was erroneously written to match on the entire request URI, instead of the last path component. - ZrkadloExcludeFileMask "^[^0-9]*$" + ZrkadloExcludeFileMask "/[^0-9]*$" Sorry, my bad. It didn't become apparent in my testing, because I didn't test with URIs which t contain digits, like it is the case for the the media.1 directory... |