flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <j.guilm...@accenture.com>
Subject RE: Flume HTTP sink for binary data
Date Mon, 27 Oct 2014 18:47:07 GMT
Hello Ari, Otis,

Yes I could contribute on this: I’m currently developing it. I’ve followed the pattern
described below, reusing existing Sink as a model.

I’m working on generic options that appear to me as mandatory for a wider usage that mine
only:

-          Load balancing

-          Authentication

-          Caching of Http connexion instead of creating one per event.

-          Adding HTTP headers from config.

I will follow up on this.

Regards
________________________________________________
Jean-François Guilmard


From: Hari Shreedharan [mailto:hshreedharan@cloudera.com]
Sent: lundi 27 octobre 2014 19:29
To: user@flume.apache.org
Cc: user@flume.apache.org
Subject: Re: Flume HTTP sink for binary data

You could build one using Apache HttpClient - just look at any other sink (like hbase sink
or rolling file sink) and follow the same pattern.

Thanks,
Hari


On Mon, Oct 27, 2014 at 9:29 AM, Otis Gospodnetic <otis.gospodnetic@gmail.com<mailto:otis.gospodnetic@gmail.com>>
wrote:
Hi,

Shouldn't something like that exist in Flume?

Jean-François, is this something you could contribute?  Maybe we can use yours or work on
it if you can put up a patch?  If not, I think we can take one of the existing implementations
you mentioned and try to improve that and contribute that...... but if your stuff is better,
somewhat ready, and contributable.....

Thanks,
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


On Mon, Oct 27, 2014 at 12:09 PM, <j.guilmard@accenture.com<mailto:j.guilmard@accenture.com>>
wrote:
Hi Pavel,

I have found 2 initiatives on this.

https://github.com/hammer/flume/blob/f140709502abf286d9e25a174caa24629a776448/plugins/http/src/java/com/cloudera/flume/handlers/http/HttpPostSink.java
https://github.com/josealvarezmuguerza/flume-http-sink/blob/master/src/main/java/org/apache/flume/sink/HttpSink.java

None of them appeared to me with the code quality/flexibility I expected, so I guess it might
be worth going further?

Good luck and let me know if you find other alternatives. (I’m currently developing my own
code, based on Apache HttpClient )

Best regards.
________________________________________________
Jean-François Guilmard


From: Pavel Zalunin [mailto:wr4bbit@gmail.com<mailto:wr4bbit@gmail.com>]
Sent: lundi 27 octobre 2014 16:44
To: user@flume.apache.org<mailto:user@flume.apache.org>
Subject: Flume HTTP sink for binary data

Hi,

We need to send binary data to our http endpoint, I looked at built-in sinks ( https://flume.apache.org/FlumeUserGuide.html#flume-sinks)
and can't find http sink. Are there opensourced implementations of such thing?

Pavel.

________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our systems for the purposes
of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________

www.accenture.com<http://www.accenture.com>


Mime
View raw message