flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Shreedharan" <hshreedha...@cloudera.com>
Subject RE: Java API for Flume Agent
Date Tue, 20 Jan 2015 20:22:37 GMT
Can you send the code you used? 




Thanks, Hari

On Tue, Jan 20, 2015 at 8:48 AM, Carlotta Hicks <Carlotta.Hicks@sas.com>
wrote:

> I added in an HDFS sink.  But, I am getting this FlumeException:  Expected one source
and got 0 .
> The only source for the embedded agent is 'embedded', correct?  Not sure how to proceed
from here.  
> Need help.
> -----Original Message-----
> From: Ashish [mailto:paliwalashish@gmail.com] 
> Sent: Saturday, January 17, 2015 7:58 AM
> To: user@flume.apache.org
> Subject: Re: Java API for Flume Agent
> Try a bit of hacking in
> org.apache.flume.agent.embedded.EmbeddedAgentConfiguration class, and add HBase Sink
to allowed sinks in the section
> private static final String[] ALLOWED_SINKS = {
>   SINK_TYPE_AVRO,
>   // Add Hbase sink here <<<
> };
> See if it works and let us know.
> HTH!
> On Sat, Jan 17, 2015 at 10:19 AM, Carlotta Hicks <Carlotta.Hicks@sas.com> wrote:
>> Thanks Joey!  I would love to use the embedded agent.  It is exactly what I am looking
for.  However, I can't use the embedded agent because with the embedded agent, the sink must
be avro.  I need the sink to be hbase.
>> I am assuming that requirements has not changed.  Please correct me if I am wrong.
>> ________________________________________
>> From: Joey Echeverria <joey@cloudera.com>
>> Sent: Friday, January 16, 2015 3:47 PM
>> To: user@flume.apache.org
>> Subject: Re: Java API for Flume Agent
>>
>> You could use an embedded flume agent[1]. This will do the same thing 
>> that the Flume flow does but run flume inside of another Java 
>> application.
>>
>> Let me know if that meets your needs.
>>
>> -Joey
>>
>> [1] http://flume.apache.org/FlumeDeveloperGuide.html#embedded-agent
>>
>>
>> On Fri, Jan 16, 2015 at 10:30 AM, Carlotta Hicks <Carlotta.Hicks@sas.com> wrote:
>>> I need to create an agent using java API to load an HBase table using 
>>> a CSV file.  I am able to accomplish this using flume-ng and an agent 
>>> with a spooldir directory source, file channel and HBase sink.
>>>
>>>
>>>
>>> I need information on how to accomplish this via a java API.
>>>
>>>
>>>
>>> Any help would be appreciated.
>>>
>>>
>>>
>>> -CM
>>>
>>>
>>
>>
>>
>> --
>> Joey Echeverria
> --
> thanks
> ashish
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Mime
View raw message