tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yanbin.zhang (Jira)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-4262) Integer overflow in unordered.output.buffer.size-mb
Date Tue, 19 Jan 2021 03:41:00 GMT

    [ https://issues.apache.org/jira/browse/TEZ-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17267626#comment-17267626
] 

yanbin.zhang commented on TEZ-4262:
-----------------------------------

@jonathan Turner eagles ,yes ,you are right .Our tez version is 0.9.0。Thank you very much
for your reply.

> Integer overflow in unordered.output.buffer.size-mb 
> ----------------------------------------------------
>
>                 Key: TEZ-4262
>                 URL: https://issues.apache.org/jira/browse/TEZ-4262
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: yanbin.zhang
>            Assignee: yanbin.zhang
>            Priority: Critical
>         Attachments: TEZ-4262.test.patch
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The value of the tez.runtime.unordered.output.buffer.size-mb parameter exceeds the range
of the int type。
> Exception thrown is given below:
> 2020-12-30 14:57:37,762 [INFO] [Dispatcher thread \{Central}] |HistoryEventHandler.criticalEvents|:
[HISTORY][DAG:dag_1609310178038_0003_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1609310178038_0003_1_00_000160_0,
creationTime=1609311453532, allocationTime=1609311455291, startTime=1609311457280, finishTime=1609311457744,
timeTaken=464, status=FAILED, taskFailureType=NON_FATAL, errorEnum=FRAMEWORK_ERROR, diagnostics=Error:
Error while running task ( failure ) : attempt_1609310178038_0003_1_00_000160_0:java.lang.IllegalArgumentException2020-12-30
14:57:37,762 [INFO] [Dispatcher thread \{Central}] |HistoryEventHandler.criticalEvents|: [HISTORY][DAG:dag_1609310178038_0003_1][Event:TASK_ATTEMPT_FINISHED]:
vertexName=Map 1, taskAttemptId=attempt_1609310178038_0003_1_00_000160_0, creationTime=1609311453532,
allocationTime=1609311455291, startTime=1609311457280, finishTime=1609311457744, timeTaken=464,
status=FAILED, taskFailureType=NON_FATAL, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Error
while running task ( failure ) : attempt_1609310178038_0003_1_00_000160_0:java.lang.IllegalArgumentException
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:108) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:177)
at org.apache.tez.runtime.common.resources.MemoryDistributor.requestMemory(MemoryDistributor.java:110)
at org.apache.tez.runtime.api.impl.TezTaskContextImpl.requestInitialMemory(TezTaskContextImpl.java:214)
at org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput.initialize(UnorderedPartitionedKVOutput.java:76)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable._callInternal(LogicalIOProcessorRuntimeTask.java:550)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:533)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:518)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message