camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claus Ibsen (JIRA)" <>
Subject [jira] [Commented] (CAMEL-5433) camel-hdfs - Allow to startup async as hdfs file system has hardcoded 15 min retry in case no connection
Date Mon, 09 Jul 2012 09:16:35 GMT


Claus Ibsen commented on CAMEL-5433:

And example of thread dump during that retry logic
   java.lang.Thread.State: RUNNABLE
	at Method)
	- locked <7fb6998a8> (a$2)
	- locked <7fb699898> (a java.util.Collections$UnmodifiableSet)
	- locked <7fb699690> (a
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(
	- locked <7fb679d80> (a org.apache.hadoop.ipc.Client$Connection)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(
	- locked <7fb679d80> (a org.apache.hadoop.ipc.Client$Connection)
	at org.apache.hadoop.ipc.Client$Connection.access$2000(
	at org.apache.hadoop.ipc.Client.getConnection(
	at org.apache.hadoop.ipc.RPC$Invoker.invoke(
	at $Proxy87.getProtocolVersion(Unknown Source)
	at org.apache.hadoop.ipc.RPC.getProxy(
	at org.apache.hadoop.ipc.RPC.getProxy(
	at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(
	at org.apache.hadoop.hdfs.DFSClient.<init>(
	at org.apache.hadoop.hdfs.DFSClient.<init>(
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(
	at org.apache.hadoop.fs.FileSystem.createFileSystem(
	at org.apache.hadoop.fs.FileSystem.access$200(
	at org.apache.hadoop.fs.FileSystem$Cache.get(
	at org.apache.hadoop.fs.FileSystem.get(
	at org.apache.camel.component.hdfs.HdfsInfo.<init>(
	at org.apache.camel.component.hdfs.HdfsOutputStream.createOutputStream(
	at org.apache.camel.component.hdfs.HdfsProducer.doStart(
	at org.apache.camel.util.ServiceHelper.startService(
	at org.apache.camel.impl.ProducerCache.doGetProducer(
> camel-hdfs - Allow to startup async as hdfs file system has hardcoded 15 min retry in
case no connection
> --------------------------------------------------------------------------------------------------------
>                 Key: CAMEL-5433
>                 URL:
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-hdfs
>    Affects Versions: 2.10.0
>            Reporter: Claus Ibsen
>            Assignee: Claus Ibsen
>             Fix For: 2.10.1, 2.11
> When start a Camel app with camel-hdfs (either as producer or consumer), then getting
a hdfs file system object may have Hadoop block for up till 15 minutes where it tries to obtain
a connection to the cluster.
> This makes the app appear as blocking and people dont want their apps to block for 15
min on startup.
> We should let the hadoop hdfs connection happing in an async thread, so the app can startup.
Configurable so you can turn back the old behavior.
> Hadoop dosn't allow to configure a lower timeout etc this as its hardcoded.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message