Mailing-List: contact nutch-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: nutch-dev@lucene.apache.org
Received-SPF: pass (herse.apache.org: local policy)
Message-ID: <45C33C72.8000901@getopt.org>
Date: Fri, 02 Feb 2007 14:28:18 +0100
From: Andrzej Bialecki <ab@getopt.org>
User-Agent: Thunderbird 1.5.0.9 (Windows/20061207)
MIME-Version: 1.0
To: nutch-dev@lucene.apache.org
Subject: Re: Generator.java bug?
References: <001901c746c1$16b8a2e0$6403a8c0@GALTOP>
In-Reply-To: <001901c746c1$16b8a2e0$6403a8c0@GALTOP>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Gal Nitzan wrote:
> Hi,
>
>  
>
> After many failures of generate "Generator: 0 records selected for fetching,
> exiting ..." I made a post about it a few days back.
>
>  
>
> I narrowed down to the following function:
>
>  
>
> public Path generate(Path dbDir, Path segments, int numLists, long topN,
> long curTime, boolean filter, boolean force)
>
>  
>
> in the following if:  if (readers == null || readers.length == 0 ||
> !readers[0].next(new FloatWritable()))
>
>  
>
>  
>
> It turns out that the: "!readers[0].next(new FloatWritable())" is the
> culprit.
>   

Well, this condition simply checks if the result is not empty. When we 
open Reader[] on a SequenceFile, each reader corresponds to a 
part-xxxxx. There must be at least one part, so we use the one at index 
0. If we cannot retrieve at least one entry from it, then it logically 
follows that the file is empty, and we bail out.

-- 
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com