nutch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Tomblin <ptomb...@xcski.com>
Subject What to do about sites with Disallow: * and a sitemap?
Date Thu, 17 Sep 2009 15:26:35 GMT
Is there a way to make Nutch look at the pages of a site based on its
Sitemap?

-- 
http://www.linkedin.com/in/paultomblin

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message