Bug 641257

Paraphrased from Peter Poeml from "Re: [opensuse] One repo in the pool killing updates - probably utexas.edu - can it be fixed?"

Using aria2c directly, we can see what it does with that:

 % aria2c --log=- --log-level=debug '--user-agent=ZYpp 7.7.5 (aria2 version 1.9.1) openSUSE-11.3-x86_64' --summary-interval=1 --follow-metalink=mem --check-integrity=true --file-allocation=none --server-stat-of=/home/poeml/.aria2/aria2.stats --server-stat-if=/home/poeml/.aria2/aria2.stats --uri-selector=adaptive --use-head=false --max-tries=5 --max-concurrent-downloads=2 --header=Pragma: --connect-timeout=180 'http://download.opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm.metalink?country=us' | grep Host
Host: download.opensuse.org
Host: widehat.opensuse.org
Host: ftp.halifax.rwth-aachen.de
Host: ftp5.gwdg.de
Host: ftp.roedu.net
Host: 130.57.19.201

It contacts 5 servers, which is the built-in default for concurrency. The job finishes after 1 or 2 seconds for me, which is about the same time to download the package from a single source in this case. 

<snip>

Sigh. When testing the same as above from the US, aria2c does not contact 5 mirrors; it contacts only one, and if that is utexas.edu, the download takes a minute instead of 2-4 seconds. The log shows that only one server is contacted, and it is reproducible:

 % aria2c --log=- --log-level=debug '--user-agent=ZYpp 7.7.5 (aria2 version 1.9.1) openSUSE-11.3-x86_64' --summary-interval=1 --follow-metalink=mem --check-integrity=true --file-allocation=none --server-stat-of=/home/poeml/.aria2/aria2.stats --server-stat-if=/home/poeml/.aria2/aria2.stats --uri-selector=adaptive --use-head=false --max-tries=5 --max-concurrent-downloads=2 --header=Pragma: --connect-timeout=180 'http://download.opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm.metalink?country=us' | grep Host
Host: download.opensuse.org
Host: 130.57.19.201

The worst is, I don't know why this differs. Apart from the aria2c version everything should be exactly the same. In the latter case, I used the newest version, 1.10.3.

<snip - next reply from Peter>

I found http://sourceforge.net/tracker/?func=detail&aid=3043968&group_id=159897&atid=813673 which mentions a changed behaviour in aria2c 1.10.x, compared to 1.9.x. Indeed, when I try the mentioned options "--max-connection-per-server=4 --min-split-size=1M" aria2c behaves again as I would expect it:


poeml@download:~/Images> time aria2c --log=- --log-level=debug --summary-interval=1 --follow-metalink=mem --check-integrity=true --file-allocation=none --use-head=false --max-tries=5 --connect-timeout=180 --max-connection-per-server=1 --min-split-size=1M 'http://download.opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm.metalink?country=us' | grep Host
Host: download.opensuse.org
2010-09-22 11:30:19.744906 DEBUG - UsedHost=1, ftp.utexas.edu
2010-09-22 11:30:19.745048 DEBUG - UsedHost=1, 130.57.19.201
2010-09-22 11:30:19.745057 DEBUG - UsedHost=1, ftp.utexas.edu
2010-09-22 11:30:19.745179 DEBUG - UsedHost=1, 130.57.19.201
2010-09-22 11:30:19.745189 DEBUG - UsedHost=1, ftp.utexas.edu
2010-09-22 11:30:19.745198 DEBUG - UsedHost=1, ftp5.gwdg.de
2010-09-22 11:30:19.745318 DEBUG - UsedHost=1, 130.57.19.201
2010-09-22 11:30:19.745328 DEBUG - UsedHost=1, ftp.utexas.edu
2010-09-22 11:30:19.745337 DEBUG - UsedHost=1, ftp5.gwdg.de
2010-09-22 11:30:19.745345 DEBUG - UsedHost=1, widehat.opensuse.org
Host: 130.57.19.201
Host: ftp.halifax.rwth-aachen.de
Host: widehat.opensuse.org
Host: ftp5.gwdg.de
Host: ftp.utexas.edu
Host: 130.57.19.201
aria2c --log=- --log-level=debug --summary-interval=1 --follow-metalink=mem    0.00s user 0.24s system 5% cpu 4.089 total
grep Host  0.00s user 0.05s system 1% cpu 4.053 total


As the log excerpt shows, aria2c contacts not only one server, and it finishes the download in about 4 seconds (instead of one minute) even though the slow utexas.edu mirror was used (among others).

That's how it should be.

So, if you see the same happening, and if you are indeed using a new aria2c (1.10.x), then one should figure out how zypper/yast can use the right options (or if this is a bug).

<snip - next reply from Carlos E.R.>

Then I think that aria2c should be upgraded (via update repo, recomended update) on all openSUSE distros. That would reduce the problem of missing mirrors in the US.

<me>

And backport to all supported releases :p