flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: Flume Compile 1.5.0 for HBase 0.98.3
Date Sun, 27 Jul 2014 21:52:23 GMT
Something built against HBase 0.98.2-hadoop2 will definitely work with any later release that
has also been built against Hadoop 2. We just released 0.98.4 last week. 

IIRC, setting the umask like that was needed for Hadoop 1 miniclusters. I haven't personally
needed to set it when working with Hadoop 2 miniclusters. Sometimes I switch back to Hadoop
1 and always forget. 


> On Jul 27, 2014, at 9:16 AM, Hari Shreedharan <hshreedharan@cloudera.com> wrote:
> 
> If you remove the -Dhbase.version and use the default one (0.98.2-hadoop2), then it should
build fine (at least it builds fine for me). Flume built against hbase-98 should be able to
write to hbase 0.98.3.
> 
>  Also, I remember a hadoop minicluster requirement that requires umask be set to 022
(not sure if that is required now). 
> 
> 
>> On Sun, Jul 27, 2014 at 2:18 AM, ac@hsk.hk <ac@hsk.hk> wrote:
>> Hi Hari,
>> 
>> I have finally managed to compile it successfully,  many thanks for your help!!
>> 
>> 
>> I edited pom.xml and deleted the following lines  (around line 247), and stop all
Hadoop and HBase 
>> ### DELETED
>>           <dependency>
>>             <groupId>org.apache.hadoop</groupId>
>>             <artifactId>hadoop-minicluster</artifactId>
>>             <version>${hadoop.version}</version>
>>             <scope>test</scope>
>>           </dependency>
>> ### DELETED
>> 
>> By the way, I found some warning at the beginning of the compilation
>> 
>> [WARNING] Some problems were encountered while building the effective model for org.apache.flume.flume-ng-sinks:flume-ng-elasticsearch-sink:jar:1.5.0.1
>> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be
unique: org.apache.httpcomponents:httpclient:jar -> duplicate declaration of version (?)
@ line 86, column 17
>> 
>> Is there a way to fix this warning too?  How to find the full path and file name
about "line 86 column 17"?
>> 
>> Regards
>> Arthur
>> 
>> 
>> 
>> 
>> 
>> 
>> [INFO] Scanning for projects...
>> [WARNING] 
>> [WARNING] Some problems were encountered while building the effective model for org.apache.flume.flume-ng-sinks:flume-ng-elasticsearch-sink:jar:1.5.0.1
>> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be
unique: org.apache.httpcomponents:httpclient:jar -> duplicate declaration of version (?)
@ line 86, column 17
>> [WARNING] 
>> [WARNING] It is highly recommended to fix these problems because they threaten the
stability of your build.
>> [WARNING] 
>> [WARNING] For this reason, future Maven versions might no longer support building
such malformed projects.
>> [WARNING] 
>> …
>> INFO] Apache Flume ...................................... SUCCESS [1.297s]
>> [INFO] Flume NG SDK ...................................... SUCCESS [2.772s]
>> [INFO] Flume NG Configuration ............................ SUCCESS [0.874s]
>> [INFO] Flume NG Core ..................................... SUCCESS [4.033s]
>> [INFO] Flume NG Sinks .................................... SUCCESS [0.106s]
>> [INFO] Flume NG HDFS Sink ................................ SUCCESS [1.946s]
>> [INFO] Flume NG IRC Sink ................................. SUCCESS [0.595s]
>> [INFO] Flume NG Channels ................................. SUCCESS [0.053s]
>> [INFO] Flume NG JDBC channel ............................. SUCCESS [0.892s]
>> [INFO] Flume NG file-based channel ....................... SUCCESS [2.439s]
>> [INFO] Flume NG Spillable Memory channel ................. SUCCESS [0.721s]
>> [INFO] Flume NG Node ..................................... SUCCESS [1.545s]
>> [INFO] Flume NG Embedded Agent ........................... SUCCESS [0.722s]
>> [INFO] Flume NG HBase Sink ............................... SUCCESS [1.893s]
>> [INFO] Flume NG ElasticSearch Sink ....................... SUCCESS [1.447s]
>> [INFO] Flume NG Morphline Solr Sink ...................... SUCCESS [1:13.893s]
>> [INFO] Flume NG Kite Dataset Sink ........................ SUCCESS [8.526s]
>> [INFO] Flume Sources ..................................... SUCCESS [0.020s]
>> [INFO] Flume Scribe Source ............................... SUCCESS [0.485s]
>> [INFO] Flume JMS Source .................................. SUCCESS [0.994s]
>> [INFO] Flume Twitter Source .............................. SUCCESS [0.591s]
>> [INFO] Flume legacy Sources .............................. SUCCESS [0.017s]
>> [INFO] Flume legacy Avro source .......................... SUCCESS [0.619s]
>> [INFO] Flume legacy Thrift Source ........................ SUCCESS [0.956s]
>> [INFO] Flume NG Clients .................................. SUCCESS [0.020s]
>> [INFO] Flume NG Log4j Appender ........................... SUCCESS [1:10.438s]
>> [INFO] Flume NG Tools .................................... SUCCESS [1.188s]
>> [INFO] Flume NG distribution ............................. SUCCESS [9.275s]
>> [INFO] Flume NG Integration Tests ........................ SUCCESS [15.610s]
>> [INFO] ------------------------------------------------------------------------
>> [INFO] BUILD SUCCESS
>> [INFO] ------------------------------------------------------------------------
>> [INFO] Total time: 3:24.410s
>> [INFO] Finished at: Sun Jul 27 15:30:11 HKT 2014
>> [INFO] Final Memory: 61M/6296M
>> [INFO] ------------------------------------------------------------------------
> 

Mime
View raw message