nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MilleBii (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (NUTCH-770) Timebomb for Fetcher
Date Sat, 05 Dec 2009 16:51:21 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786443#action_12786443
] 

MilleBii edited comment on NUTCH-770 at 12/5/09 4:50 PM:
---------------------------------------------------------

Tried it succesfully on a windows platform.

It does not work on a Ubuntu, pseudo-distributed hadoop configuration with two mappers running
in parallel ????



      was (Author: millebii):
    Tried it succesfully on a windows platform.

It does not work on a Ubuntu, pseudo-distributed hadoop configuration with mappers running
in parallel ????


  
> Timebomb for Fetcher
> --------------------
>
>                 Key: NUTCH-770
>                 URL: https://issues.apache.org/jira/browse/NUTCH-770
>             Project: Nutch
>          Issue Type: Improvement
>            Reporter: Julien Nioche
>            Assignee: Andrzej Bialecki 
>             Fix For: 1.1
>
>         Attachments: log-770, NUTCH-770-v2.patch, NUTCH-770-v3.patch, NUTCH-770.patch
>
>
> This patch provides the Fetcher with a timebomb mechanism. By default the timebomb is
not activated; it can be set using the parameter fetcher.timebomb.mins. The number of minutes
is relative to the start of the Fetch job. When the number of minutes is reached, the QueueFeeder
skips all remaining entries then all active queues are purged. This allows to keep the Fetch
step under comtrol and works well in combination with NUTCH-769

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message