flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Chavez <pcha...@ntent.com>
Subject RE: Flume Exception - http source
Date Thu, 12 Nov 2015 22:33:23 GMT
HTTP Source is a listener. It doesn’t actively pull from a remote endpoint.

The address in the config is an address on the flume server to bind to.

From: Timothy Garza [mailto:timothy.garza@collinsongroup.com]
Sent: Thursday, November 12, 2015 2:09 PM
To: user@flume.apache.org
Subject: RE: Flume Exception - http source

Thanks Hari

Re.
“HTTP Source does not pull anything from an address, instead it binds to the address specified
in the config”

Can you clarify this a bit more - it seems to contradict ☺. An address is specified in the
Config.


Kind Regards

Timothy Garza
Database and BI Developer
Collinson Technology Services
Skype: timothy.garza.cts
collinsongroup.com<http://www.collinsongroup.com/>


[Collinson Group]<http://www.collinsongroup.com/>



From: Hari Shreedharan [mailto:hshreedharan@cloudera.com]
Sent: 12 November 2015 22:02
To: user@flume.apache.org<mailto:user@flume.apache.org>
Subject: Re: Flume Exception - http source

HTTP Source does not pull anything from an address, instead it binds to the address specified
in the config. It can listen on that port and then accept data from applications.

Thanks,
Hari Shreedharan


On Nov 12, 2015, at 1:59 PM, Timothy Garza <timothy.garza@collinsongroup.com<mailto:timothy.garza@collinsongroup.com>>
wrote:

Hi Flumers

I’m new to Flume so please go easy on me, but this is a placeholder for those that follow.

I’m seeing the following Exception when configuring a http source for Flume v1.6.0, pointing
at an externally hosted web API I know I can connect to, and GET/POST to otherwise

15/11/12 21:23:34 ERROR lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner:
{ source:org.apache.flume.source.http.HTTPSource{name:<source-name>,state:IDLE} } -
Exception follows.
java.lang.IllegalStateException: Running HTTP Server found in source: <source-name>
before I started one.Will not attempt to start.  “<--Exception text!!!”
        at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
        at org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:176)
        at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
        at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Here are my config lines for the Flume source:
flume-ng agent
-Dflume.root.logger=WARN,console

--conf-file
<Flume-Agent>.sources.<source-name>.type = http
<Flume-Agent>.sources.<source-name>.port = 443
<Flume-Agent>.sources.<source-name>.bind = https://<url<https://%3curl/>
to a perfectly healthy web API> <-- serving JSON data
<Flume-Agent>.sources.<source-name>.handler = org.apache.flume.source.http.JSONHandler
<Flume-Agent>.sources.<source-name>.selector.type = replicating <-- multiple
sinks

Now I’ve seen others have had to recompile the Flume Jar to resolve this but is that really
what’s going on here?
https://community.cloudera.com/t5/Data-Ingestion-Integration/Flume-TwitterSource-language-filter/td-p/23519
https://github.com/cloudera/cdh-twitter-example/issues/10

Apart from the Exceptions, what other logging will help diagnose this?



The Collinson Group Limited; Registered number: 2577557, Registered in England & Wales;
Registered Office: Cutlers Exchange, 123 Houndsditch, London, EC3A 7BU.


This e-mail may contain privileged and confidential information and/or copyright material
and is intended for the use of the addressee only. If you receive this e-mail by mistake please
advise the sender immediately by using the reply facility in your e-mail software and delete
this e-mail from your computer system. You may not deliver, copy or disclose its contents
to anyone else. Any unauthorised use may be unlawful. Any views expressed in this e-mail are
those of the individual sender and may not necessarily reflect the views of The Collinson
Group Ltd and/or its subsidiaries or any other associated company (collectively “Collinson
Group”).

As communications via the Internet are not secure Collinson Group cannot accept any liability
if this e-mail is accessed by third parties during the course of transmission or is modified
or amended in any way following despatch. Collinson Group cannot guarantee that any attachment
to this email does not contain a virus, therefore it is strongly recommended that you carry
out your own virus check before opening any attachment, as we cannot accept liability for
any damage sustained as a result of software virus infection. Senders of messages shall be
taken to consent to the monitoring and recording of e-mails addressed to members of the Company.

Mime
View raw message