flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: How to use HTTPS with HTTPSource?
Date Thu, 27 Jun 2013 01:50:21 GMT
Sure, I am working on the test cases and Httpclient is giving me tough time
with SSL. If it doesn't work, shall write simple SSL client to test it.


On Wed, Jun 26, 2013 at 8:13 PM, shushuai zhu <sszhu@yahoo.com> wrote:

> Ashish, thx. Will try your solution. Please also kindly send a
> notice after you submit the patch.
>
> Shushuai
>    *From:* Ashish <paliwalashish@gmail.com>
> *To:* user@flume.apache.org; shushuai zhu <sszhu@yahoo.com>
> *Sent:* Wednesday, June 26, 2013 5:28 AM
>
> *Subject:* Re: How to use HTTPS with HTTPSource?
>
> The example I shared works for Jetty 9, Flume uses 6.1.26.
>
> Here is a basic version. Add these lines in HTTPSource in start() method.
> This is a basic version. You would like to read these properties from flume
> config in configure(). This is small stuff.
>
> SslSocketConnector sslSocketConnector = new SslSocketConnector();
> sslSocketConnector.setKeystore("jettykeystore");
> sslSocketConnector.setKeyPassword("password");
> sslSocketConnector.setPort(8443);
> srv.setConnectors(new Connector[] { connector, sslSocketConnector });
>
> Here is how it would like
>
>   @Override
>   public void start() {
>     Preconditions.checkState(srv == null,
>             "Running HTTP Server found in source: " + getName()
>             + " before I started one."
>             + "Will not attempt to start.");
>     srv = new Server();
>     SocketConnector connector = new SocketConnector();
>     connector.setPort(port);
>     connector.setHost(host);
>
>         SslSocketConnector sslSocketConnector = new SslSocketConnector();
>         sslSocketConnector.setKeystore("jettykeystore");
>         sslSocketConnector.setKeyPassword("password");
>         sslSocketConnector.setPort(8443);
>
>     srv.setConnectors(new Connector[] { connector, sslSocketConnector });
>     try {
>       org.mortbay.jetty.servlet.Context root =
>               new org.mortbay.jetty.servlet.Context(
>               srv, "/", org.mortbay.jetty.servlet.Context.SESSIONS);
>       root.addServlet(new ServletHolder(new FlumeHTTPServlet()), "/");
>       srv.start();
> ........
>
> To submit a patch, would need to refine the code a bit and add test cases.
> Shall take a while. HTH !
>
>
> On Mon, Jun 24, 2013 at 8:24 PM, shushuai zhu <sszhu@yahoo.com> wrote:
>
> Ashish, thanks again. Could you elaborate a little more what I should do?
> I am relatively new to Flume (just started using it a couple of weeks ago)
> and also new to open source code base.
>
> Shushuai
>    *From:* Ashish <paliwalashish@gmail.com>
> *To:* user@flume.apache.org; shushuai zhu <sszhu@yahoo.com>
> *Sent:* Friday, June 21, 2013 10:38 PM
>
> *Subject:* Re: How to use HTTPS with HTTPSource?
>
> HTTP source has not changed much :) One way to patch in additional SSL
> stuff in jetty code. The second more tedious way is to use NIO framework
> with SSL code like Netty or MINA. We are working on MINA M1 release and
> perhaps after that shall take a tab at writing an HTTP source based on that.
>
> HTH !
>
>
> On Sat, Jun 22, 2013 at 3:07 AM, shushuai zhu <sszhu@yahoo.com> wrote:
>
> Ashish,
>
> Thanks for the pointer. So I will create my own source, say HTTPSSource,
> which extends the HTPSource to add the https connection, then use the
> custom HTTPSSource in flume.conf.
>
> Is this the right source code you referred (I am using flume 1.3.1)?
>
>
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=tree;f=flume-ng-core/src/main/java/org/apache/flume/source/http;h=fa5477033ca23c075fec6af2c391b01d1d49f69a;hb=trunk
>
>
> Shushuai
>
>    *From:* Ashish <paliwalashish@gmail.com>
> *To:* user@flume.apache.org; shushuai zhu <sszhu@yahoo.com>
> *Sent:* Friday, June 21, 2013 5:09 AM
> *Subject:* Re: How to use HTTPS with HTTPSource?
>
> AFAIK, HTTPSource doesn't support HTTPS. HTTP Source uses embedded jetty,
> so it would extending HTTP source to add SSL configuration for this.
>
> This link should help
> http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/tree/examples/embedded/src/main/java/org/eclipse/jetty/embedded/ManyConnectors.java
>
> It should be just few lines of code change.
>
> Please note I am referring to source code in master branch.
>
>
> On Thu, Jun 20, 2013 at 1:40 AM, shushuai zhu <sszhu@yahoo.com> wrote:
>
> Hi,
>
> I am using HTTP to transfer data from remote machine to HTTPSource of
> Flume agent (the flume conf. file looks like below). Is HTTPS supported?
> Anyone has examples to use HTTPS to transfer the data?
>
> agent1.sources = r1
> agent1.sources.r1.type = org.apache.flume.source.http.HTTPSource
> agent1.sources.r1.port = 41415
> agent1.sources.r1.channels = ch1
> agent1.sources.r1.handler = org.apache.flume.source.http.JSONHandler
> Thanks.
>
> Shushuai
>
>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
>
>


-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

Mime
View raw message