flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Ravindran <rahu...@yahoo.com>
Subject Re: Hadoop jars
Date Tue, 20 Nov 2012 00:04:14 GMT
Thanks for the responses.

Good to know that the only external dependencies are Hadoop and Hbase. We will deploy those
components only on boxes which are going to have those sinks set up.

 From: Hari Shreedharan <hshreedharan@cloudera.com>
To: user@flume.apache.org 
Sent: Monday, November 19, 2012 3:29 PM
Subject: Re: Hadoop jars

Flume installs all required binaries, except for Hadoop (and the dependencies it would pull
in) and HBase. This is because Flume, like most other Hadoop ecosystem components is meant
to work against binary incompatible versions of Hadoop (Hadoop-1/Hadoop2). So instead of packaging
hadoop jars with Flume, we expect Hadoop to be available on the machines you are running Flume
on. Once you install Hadoop you should not have any dependency issues. Same is true for HBase.


Hari Shreedharan

On Monday, November 19, 2012 at 2:33 PM, Mohit Anchlia wrote:
Easiest way is to install cdh binary and point your flume's classpath to it.
>On Mon, Nov 19, 2012 at 2:25 PM, Roshan Naik <roshan@hortonworks.com> wrote:
>Currently, unfortunately, i dont think there is any such documentation. 
>>A  very general answer would be..Normally this list would depend on the source/sink/channel
you are using.
>>I think it would be nice if the user manual did list these external dependencies for
each component.
>>I am not the expert on HDFS sink.. but i dont see why it would depend on anything
more than HDFS itself. 
>>On Mon, Nov 19, 2012 at 2:18 PM, Rahul Ravindran <rahulrv@yahoo.com> wrote:
>>Are there other such libraries which will need to be downloaded? Is there a well-defined
location for the hadoop jar and any other jars that flume may depend on?
View raw message