nutch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Susam Pal" <susam....@gmail.com>
Subject Re: site alias
Date Fri, 06 Jul 2007 07:32:26 GMT
I have faced this issue. I block the duplicate domain using the URL
filters. So only one domain is crawled by the bot and the other domain
is ignored.

Regards,
Susam Pal
http://susam.in/

On 7/6/07, Nuther <nuther@proservice.ge> wrote:
> Hi,
> I was wondering if nutch has alias option
> Let's say we have two domains www.site1.com and www.site2.com that point on
> one site. How can I tell nutch that they pooint on that site? This is problem
> because there are a lot of duplicates in search results.
> Thanks.
>
> --
> Regards,
>  Nuther                          mailto:nuther@proservice.ge

Mime
View raw message