camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Baptista <Daniel.Bapti...@performgroup.com>
Subject RE: HDFSOutputStream not closing
Date Mon, 17 Oct 2011 12:27:39 GMT
Hi Claus, thanks for the response.

I can raise a ticket and will come up with a unit test. Just for completeness the I am running
windows XP and Java 1.6.0_14.
I am trying to put a file into HDFS with Hadoop version 0.20 (1 name node 2 data nodes) where
the cluster runs on GNU/Linux. 

Cheers, Dan.

-----Original Message-----
From: Claus Ibsen [mailto:claus.ibsen@gmail.com] 
Sent: 17 October 2011 13:06
To: users@camel.apache.org
Subject: Re: HDFSOutputStream not closing

On Mon, Oct 17, 2011 at 12:23 PM, Daniel Baptista
<Daniel.Baptista@performgroup.com> wrote:
> Hi All,
>
> I have a route that is supposed to take a file from a directory and marshal it through
to HDFS via the camel-hdfs component. Everything is working except the file never gets renamed
from .opened (and it's empty, but I guess this is because the stream is open still). I am
not specifying a split strategy and appending to files is not an issue for what I am doing
as I always have the whole file in the Exchange.
>
> I have tried stepping through the code but I can't see where the stream is ever closed
except when a split strategy is defined, I have tried MESSSAGES:0 but this results in an empty
seq0 file and a seq1.opened which never closes (at least it is never renamed).
>
> So I  guess my question is how do I just put a file into HDFS?
>
> I am using version 2.8.1 and an example endpoint URL is hdfs://10.1.10.159?blockSize=67108864&replication=2
>

Do you mind creating a JIRA ticket and if possible attach an unit test
which reproduces the issue.
Also details about your JDK, OS version used would be great. As File
Systems can vary per OS.


> Thanks in advance,
>
> Dan.
>



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/

Mime
View raw message