apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashwin Chandra Putta <ashwinchand...@gmail.com>
Subject Re: HDFS Write File Operator struggling to start
Date Thu, 04 Aug 2016 21:42:55 GMT
Raja,

When do you see this issue? Does it happen while the application is
running? Does this happen while restarting a failed application? Or does
this happen while starting a new application?

Regards,
Ashwin.

On Thu, Aug 4, 2016 at 11:25 AM, Samba Surabhi <sasurabh@outlook.com> wrote:

> If it is the issue with file size, you can rotate the output file.
>
> writer.setAlwaysWriteToTmp(true);
>
> writer.setRotationWindows(240);
>
> Thanks,
>
> Samba Surabhi.
>
>
> ------------------------------
> From: Raja.Aravapalli@target.com
> To: users@apex.apache.org
> Subject: HDFS Write File Operator struggling to start
> Date: Thu, 4 Aug 2016 14:49:16 +0000
>
>
>
> Hi
>
> I have a HDFS file write operator in my DAG, which is failing to start a
> new operator and keep on trying to start one!!
>
> It created approx. 800 temporary files in the destination hdfs directory!!
> How can I fix this issue…. And debug the root cause…. ?
>
> All I can see in container log is, File corrupted message…!!
>
>
> Can someone please help me fix this ?
>
>
> Regards,
> Raja.
>



-- 

Regards,
Ashwin.

Mime
View raw message