apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashwin Chandra Putta <ashwinchand...@gmail.com>
Subject Re: HDFS Write File Operator struggling to start
Date Fri, 05 Aug 2016 22:45:04 GMT
Can you check in the logs the first time the issue occurred for what
triggered it? Look for the first container that failed.

Regards,
Ashwin

On Fri, Aug 5, 2016 at 12:58 PM, Raja.Aravapalli <Raja.Aravapalli@target.com
> wrote:

>
> Hi Ashwin,
>
> It happens when the application is running!!
>
>
> Regards,
> Raja.
>
> From: Ashwin Chandra Putta <ashwinchandrap@gmail.com>
> Reply-To: "users@apex.apache.org" <users@apex.apache.org>
> Date: Thursday, August 4, 2016 at 4:42 PM
> To: "users@apex.apache.org" <users@apex.apache.org>
> Subject: Re: HDFS Write File Operator struggling to start
>
> Raja,
>
> When do you see this issue? Does it happen while the application is
> running? Does this happen while restarting a failed application? Or does
> this happen while starting a new application?
>
> Regards,
> Ashwin.
>
> On Thu, Aug 4, 2016 at 11:25 AM, Samba Surabhi <sasurabh@outlook.com>
> wrote:
>
>> If it is the issue with file size, you can rotate the output file.
>>
>> writer.setAlwaysWriteToTmp(true);
>>
>> writer.setRotationWindows(240);
>>
>> Thanks,
>>
>> Samba Surabhi.
>>
>>
>> ------------------------------
>> From: Raja.Aravapalli@target.com
>> To: users@apex.apache.org
>> Subject: HDFS Write File Operator struggling to start
>> Date: Thu, 4 Aug 2016 14:49:16 +0000
>>
>>
>>
>> Hi
>>
>> I have a HDFS file write operator in my DAG, which is failing to start a
>> new operator and keep on trying to start one!!
>>
>> It created approx. 800 temporary files in the destination hdfs
>> directory!! How can I fix this issue…. And debug the root cause…. ?
>>
>> All I can see in container log is, File corrupted message…!!
>>
>>
>> Can someone please help me fix this ?
>>
>>
>> Regards,
>> Raja.
>>
>
>
>
> --
>
> Regards,
> Ashwin.
>



-- 

Regards,
Ashwin.

Mime
View raw message