nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael <>
Subject Re[2]: what contibute to fetch slowing down
Date Mon, 03 Oct 2005 00:36:45 GMT
3mbit, 100 threads = 15 pages/sec
cpu is low during fetch, so its bandwidth limit.

AC> More than 1 million pages were fetched, but it took several days at current
AC> speed - just too slow. I'm planning to get more bandwidth. Could someone
AC> share their experience on what stable rate (pages/sec) can be achieved using
AC> 3 mbps or 10 mbps inbound connection?


View raw message