nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alban Mouton <alba...@gmail.com>
Subject State of nutchbase
Date Sat, 05 Dec 2009 14:56:49 GMT
Hello,

I have looked a little into nutch code and mailing lists. I think the
nutchbase branch (http://issues.apache.org/jira/browse/NUTCH-650) is very
interesting, with a good potential to improve code clarity and flexibility
(I find data structure quite obscure in current version). The issue is
untouched since last august, so my question is : can nutchbase really be
part of nutch 1.1 ? Is there still much work to do or is it almost ready ?
Is it a worthy issue for an interested developer with a (still !) limited
knowledge of the project ?

So far I have only tried to run nutchbase in eclipse by applying the
tutorial (http://wiki.apache.org/nutch/RunNutchInEclipse1.0) but I run in
errors when building, mostly from Parser and tests. I may start by cleaning
this up.

Eclipse build errors:

Description    Resource    Path    Location    Type
FetcherOutputFormat cannot be resolved to a type
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 362    Java Problem
Generator.GENERATE_MAX_PER_HOST_BY_IP cannot be resolved
TestGenerator.java    /nutchbase/src/test/org/apache/nutch/crawl    line
202    Java Problem
ParseImpl cannot be resolved to a type    ArcSegmentCreator.java
/nutchbase/src/java/org/apache/nutch/tools/arc    line 229    Java Problem
ParseImpl cannot be resolved to a type    BasicFields.java
/nutchbase/src/java/org/apache/nutch/indexer/field    line 335    Java
Problem
ParseImpl cannot be resolved to a type    ExtParser.java
/nutchbase/src/plugin/parse-ext/src/java/org/apache/nutch/parse/ext    line
138    Java Problem
ParseImpl cannot be resolved to a type    MSBaseParser.java
/nutchbase/src/plugin/lib-parsems/src/java/org/apache/nutch/parse/ms    line
108    Java Problem
ParseImpl cannot be resolved to a type    OOParser.java
/nutchbase/src/plugin/parse-oo/src/java/org/apache/nutch/parse/oo    line
103    Java Problem
ParseImpl cannot be resolved to a type    PdfParser.java
/nutchbase/src/plugin/parse-pdf/src/java/org/apache/nutch/parse/pdf    line
155    Java Problem
ParseImpl cannot be resolved to a type    RSSParser.java
/nutchbase/src/plugin/parse-rss/src/java/org/apache/nutch/parse/rss    line
187    Java Problem
ParseImpl cannot be resolved to a type    SWFParser.java
/nutchbase/src/plugin/parse-swf/src/java/org/apache/nutch/parse/swf    line
113    Java Problem
ParseImpl cannot be resolved to a type    TestIndexingFilters.java
/nutchbase/src/test/org/apache/nutch/indexer    line 45    Java Problem
ParseImpl cannot be resolved to a type    TestMoreIndexingFilter.java
/nutchbase/src/plugin/index-more/src/test/org/apache/nutch/indexer/more
line 61    Java Problem
ParseImpl cannot be resolved to a type    TextParser.java
/nutchbase/src/plugin/parse-text/src/java/org/apache/nutch/parse/text
line 55    Java Problem
ParseImpl cannot be resolved to a type    ZipParser.java
/nutchbase/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip    line
105    Java Problem
ParseResult cannot be resolved    ExtParser.java
/nutchbase/src/plugin/parse-ext/src/java/org/apache/nutch/parse/ext    line
137    Java Problem
ParseResult cannot be resolved    MSBaseParser.java
/nutchbase/src/plugin/lib-parsems/src/java/org/apache/nutch/parse/ms    line
107    Java Problem
ParseResult cannot be resolved    OOParser.java
/nutchbase/src/plugin/parse-oo/src/java/org/apache/nutch/parse/oo    line
103    Java Problem
ParseResult cannot be resolved    PdfParser.java
/nutchbase/src/plugin/parse-pdf/src/java/org/apache/nutch/parse/pdf    line
155    Java Problem
ParseResult cannot be resolved    RSSParser.java
/nutchbase/src/plugin/parse-rss/src/java/org/apache/nutch/parse/rss    line
187    Java Problem
ParseResult cannot be resolved    SWFParser.java
/nutchbase/src/plugin/parse-swf/src/java/org/apache/nutch/parse/swf    line
113    Java Problem
ParseResult cannot be resolved    TextParser.java
/nutchbase/src/plugin/parse-text/src/java/org/apache/nutch/parse/text
line 55    Java Problem
ParseResult cannot be resolved    ZipParser.java
/nutchbase/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip    line
105    Java Problem
ParseResult cannot be resolved to a type    ArcSegmentCreator.java
/nutchbase/src/java/org/apache/nutch/tools/arc    line 159    Java Problem
ParseResult cannot be resolved to a type    CCParseFilter.java
/nutchbase/src/plugin/creativecommons/src/java/org/creativecommons/nutch
line 267    Java Problem
ParseResult cannot be resolved to a type    CCParseFilter.java
/nutchbase/src/plugin/creativecommons/src/java/org/creativecommons/nutch
line 267    Java Problem
ParseResult cannot be resolved to a type    ExtParser.java
/nutchbase/src/plugin/parse-ext/src/java/org/apache/nutch/parse/ext    line
69    Java Problem
ParseResult cannot be resolved to a type    FeedParser.java
/nutchbase/src/plugin/feed/src/java/org/apache/nutch/parse/feed    line
106    Java Problem
ParseResult cannot be resolved to a type    FeedParser.java
/nutchbase/src/plugin/feed/src/java/org/apache/nutch/parse/feed    line
108    Java Problem
ParseResult cannot be resolved to a type    FeedParser.java
/nutchbase/src/plugin/feed/src/java/org/apache/nutch/parse/feed    line
108    Java Problem
ParseResult cannot be resolved to a type    FeedParser.java
/nutchbase/src/plugin/feed/src/java/org/apache/nutch/parse/feed    line
211    Java Problem
ParseResult cannot be resolved to a type    FeedParser.java
/nutchbase/src/plugin/feed/src/java/org/apache/nutch/parse/feed    line
221    Java Problem
ParseResult cannot be resolved to a type    HTMLLanguageParser.java
/nutchbase/src/plugin/languageidentifier/src/java/org/apache/nutch/analysis/lang
line 90    Java Problem
ParseResult cannot be resolved to a type    HTMLLanguageParser.java
/nutchbase/src/plugin/languageidentifier/src/java/org/apache/nutch/analysis/lang
line 90    Java Problem
ParseResult cannot be resolved to a type    MSBaseParser.java
/nutchbase/src/plugin/lib-parsems/src/java/org/apache/nutch/parse/ms    line
64    Java Problem
ParseResult cannot be resolved to a type    MSExcelParser.java
/nutchbase/src/plugin/parse-msexcel/src/java/org/apache/nutch/parse/msexcel
line 40    Java Problem
ParseResult cannot be resolved to a type    MSPowerPointParser.java
/nutchbase/src/plugin/parse-mspowerpoint/src/java/org/apache/nutch/parse/mspowerpoint
line 44    Java Problem
ParseResult cannot be resolved to a type    MSWordParser.java
/nutchbase/src/plugin/parse-msword/src/java/org/apache/nutch/parse/msword
line 43    Java Problem
ParseResult cannot be resolved to a type    OOParser.java
/nutchbase/src/plugin/parse-oo/src/java/org/apache/nutch/parse/oo    line
63    Java Problem
ParseResult cannot be resolved to a type    PdfParser.java
/nutchbase/src/plugin/parse-pdf/src/java/org/apache/nutch/parse/pdf    line
69    Java Problem
ParseResult cannot be resolved to a type    RSSParser.java
/nutchbase/src/plugin/parse-rss/src/java/org/apache/nutch/parse/rss    line
80    Java Problem
ParseResult cannot be resolved to a type    RelTagParser.java
/nutchbase/src/plugin/microformats-reltag/src/java/org/apache/nutch/microformats/reltag
line 68    Java Problem
ParseResult cannot be resolved to a type    RelTagParser.java
/nutchbase/src/plugin/microformats-reltag/src/java/org/apache/nutch/microformats/reltag
line 68    Java Problem
ParseResult cannot be resolved to a type    SWFParser.java
/nutchbase/src/plugin/parse-swf/src/java/org/apache/nutch/parse/swf    line
64    Java Problem
ParseResult cannot be resolved to a type    SWFParser.java
/nutchbase/src/plugin/parse-swf/src/java/org/apache/nutch/parse/swf    line
125    Java Problem
ParseResult cannot be resolved to a type    TestFeedParser.java
/nutchbase/src/plugin/feed/src/test/org/apache/nutch/parse/feed    line
94    Java Problem
ParseResult cannot be resolved to a type    TextParser.java
/nutchbase/src/plugin/parse-text/src/java/org/apache/nutch/parse/text
line 41    Java Problem
ParseResult cannot be resolved to a type    ZipParser.java
/nutchbase/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip    line
55    Java Problem
The constructor Fetcher(Configuration) is undefined    TestFetcher.java
/nutchbase/src/test/org/apache/nutch/fetcher    line 100    Java Problem
The constructor Fetcher(Configuration) is undefined    TestFetcher.java
/nutchbase/src/test/org/apache/nutch/fetcher    line 177    Java Problem
The constructor Generator(Configuration) is undefined    TestFetcher.java
/nutchbase/src/test/org/apache/nutch/fetcher    line 94    Java Problem
The constructor Generator(Configuration) is undefined
TestGenerator.java    /nutchbase/src/test/org/apache/nutch/crawl    line
312    Java Problem
The constructor Injector(Configuration) is undefined    TestFetcher.java
/nutchbase/src/test/org/apache/nutch/fetcher    line 90    Java Problem
The constructor Injector(Configuration) is undefined    TestInjector.java
/nutchbase/src/test/org/apache/nutch/crawl    line 70    Java Problem
The constructor NutchWritable(ParseImpl) is undefined
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 229    Java Problem
The import org.apache.nutch.fetcher.FetcherOutputFormat cannot be
resolved    ArcSegmentCreator.java
/nutchbase/src/java/org/apache/nutch/tools/arc    line 44    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 50    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
BasicFields.java    /nutchbase/src/java/org/apache/nutch/indexer/field
line 61    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
ExtParser.java
/nutchbase/src/plugin/parse-ext/src/java/org/apache/nutch/parse/ext    line
26    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
MSBaseParser.java
/nutchbase/src/plugin/lib-parsems/src/java/org/apache/nutch/parse/ms    line
39    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
PdfParser.java
/nutchbase/src/plugin/parse-pdf/src/java/org/apache/nutch/parse/pdf    line
41    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
RSSParser.java
/nutchbase/src/plugin/parse-rss/src/java/org/apache/nutch/parse/rss    line
41    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
TestExtParser.java
/nutchbase/src/plugin/parse-ext/src/test/org/apache/nutch/parse/ext    line
26    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
TestIndexingFilters.java    /nutchbase/src/test/org/apache/nutch/indexer
line 26    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
TestMSWordParser.java
/nutchbase/src/plugin/parse-msword/src/test/org/apache/nutch/parse/msword
line 26    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
TestMoreIndexingFilter.java
/nutchbase/src/plugin/index-more/src/test/org/apache/nutch/indexer/more
line 29    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
TestZipParser.java
/nutchbase/src/plugin/parse-zip/src/test/org/apache/nutch/parse/zip    line
26    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
ZipParser.java
/nutchbase/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip    line
33    Java Problem
The import org.apache.nutch.parse.ParseImpl cannot be resolved
ZipTextExtractor.java
/nutchbase/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip    line
41    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 51    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
ExtParser.java
/nutchbase/src/plugin/parse-ext/src/java/org/apache/nutch/parse/ext    line
21    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
FeedParser.java
/nutchbase/src/plugin/feed/src/java/org/apache/nutch/parse/feed    line
43    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
HTMLLanguageParser.java
/nutchbase/src/plugin/languageidentifier/src/java/org/apache/nutch/analysis/lang
line 33    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
MSBaseParser.java
/nutchbase/src/plugin/lib-parsems/src/java/org/apache/nutch/parse/ms    line
40    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
MSExcelParser.java
/nutchbase/src/plugin/parse-msexcel/src/java/org/apache/nutch/parse/msexcel
line 20    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
MSPowerPointParser.java
/nutchbase/src/plugin/parse-mspowerpoint/src/java/org/apache/nutch/parse/mspowerpoint
line 20    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
MSWordParser.java
/nutchbase/src/plugin/parse-msword/src/java/org/apache/nutch/parse/msword
line 21    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
PdfParser.java
/nutchbase/src/plugin/parse-pdf/src/java/org/apache/nutch/parse/pdf    line
37    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
RSSParser.java
/nutchbase/src/plugin/parse-rss/src/java/org/apache/nutch/parse/rss    line
36    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
RelTagParser.java
/nutchbase/src/plugin/microformats-reltag/src/java/org/apache/nutch/microformats/reltag
line 38    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
TestFeedParser.java
/nutchbase/src/plugin/feed/src/test/org/apache/nutch/parse/feed    line
32    Java Problem
The import org.apache.nutch.parse.ParseResult cannot be resolved
ZipParser.java
/nutchbase/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip    line
34    Java Problem
The method calculate(WebTableRow, Parse) in the type Signature is not
applicable for the arguments (Content, Parse)    ArcSegmentCreator.java
/nutchbase/src/java/org/apache/nutch/tools/arc    line 187    Java Problem
The method calculate(WebTableRow, Parse) in the type Signature is not
applicable for the arguments (Content, Parse)    ArcSegmentCreator.java
/nutchbase/src/java/org/apache/nutch/tools/arc    line 208    Java Problem
The method fetch(String, int, boolean) from the type Fetcher is not
visible    TestFetcher.java
/nutchbase/src/test/org/apache/nutch/fetcher    line 178    Java Problem
The method fetch(String, int, boolean) in the type Fetcher is not applicable
for the arguments (Path, int, boolean)    TestFetcher.java
/nutchbase/src/test/org/apache/nutch/fetcher    line 101    Java Problem
The method generate(String, long, long, boolean) in the type Generator is
not applicable for the arguments (Path, Path, int, int, long, boolean,
boolean)    TestGenerator.java
/nutchbase/src/test/org/apache/nutch/crawl    line 313    Java Problem
The method generate(String, long, long, boolean) in the type Generator is
not applicable for the arguments (Path, Path, int, long, long, boolean,
boolean)    TestFetcher.java
/nutchbase/src/test/org/apache/nutch/fetcher    line 95    Java Problem
The method getData() is undefined for the type Parse
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 200    Java Problem
The method getData() is undefined for the type Parse
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 211    Java Problem
The method getData() is undefined for the type Parse
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 213    Java Problem
The method getData() is undefined for the type Parse
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 216    Java Problem
The method getData() is undefined for the type Parse
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 230    Java Problem
The method getData() is undefined for the type Parse
ArcSegmentCreator.java    /nutchbase/src/java/org/apache/nutch/tools/arc
line 244    Java Problem
The method getData() is undefined for the type Parse    BasicFields.java
/nutchbase/src/java/org/apache/nutch/indexer/field    line 386    Java
Problem
The method getData() is undefined for the type Parse    BasicFields.java
/nutchbase/src/java/org/apache/nutch/indexer/field    line 395    Java
Problem
The method getData() is undefined for the type Parse
CCIndexingFilter.java
/nutchbase/src/plugin/creativecommons/src/java/org/creativecommons/nutch
line 55    Java Problem
The method getData() is undefined for the type Parse
CCParseFilter.java
/nutchbase/src/plugin/creativecommons/src/java/org/creativecommons/nutch
line 280    Java Problem
The method getData() is undefined for the type Parse
CCParseFilter.java
/nutchbase/src/plugin/creativecommons/src/java/org/creativecommons/nutch
line 286    Java Problem
The method getData() is undefined for the type Parse
CCParseFilter.java
/nutchbase/src/plugin/creativecommons/src/java/org/creativecommons/nutch
line 291    Java Problem
The method getData() is undefined for the type Parse
FeedIndexingFilter.java
/nutchbase/src/plugin/feed/src/java/org/apache/nutch/indexer/feed    line
76    Java Problem

Mime
View raw message