hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4
Date Fri, 06 Aug 2021 11:53:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-15327?focusedWorklogId=635090&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635090
]

ASF GitHub Bot logged work on HADOOP-15327:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 06/Aug/21 11:52
            Start Date: 06/Aug/21 11:52
    Worklog Time Spent: 10m 
      Work Description: szilard-nemeth commented on a change in pull request #3259:
URL: https://github.com/apache/hadoop/pull/3259#discussion_r684174702



##########
File path: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
##########
@@ -785,37 +833,54 @@ private void removeJobShuffleInfo(JobID jobId) throws IOException {
     }
   }
 
-  static class TimeoutHandler extends IdleStateAwareChannelHandler {
+  @VisibleForTesting
+  public void setUseOutboundExceptionHandler(boolean useHandler) {
+    this.useOutboundExceptionHandler = useHandler;
+  }
 
+  static class TimeoutHandler extends IdleStateHandler {
+    private final int connectionKeepAliveTimeOut;
     private boolean enabledTimeout;
 
+    public TimeoutHandler(int connectionKeepAliveTimeOut) {
+      //disable reader timeout
+      //set writer timeout to configured timeout value
+      //disable all idle timeout
+      super(0, connectionKeepAliveTimeOut, 0);

Review comment:
       Good catch, modified to: 
   super(0, connectionKeepAliveTimeOut, 0, TimeUnit.SECONDS);




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 635090)
    Time Spent: 1h 40m  (was: 1.5h)

> Upgrade MR ShuffleHandler to use Netty4
> ---------------------------------------
>
>                 Key: HADOOP-15327
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15327
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Xiaoyu Yao
>            Assignee: Szilard Nemeth
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HADOOP-15327.001.patch, HADOOP-15327.002.patch, HADOOP-15327.003.patch,
HADOOP-15327.004.patch, HADOOP-15327.005.patch, HADOOP-15327.005.patch, getMapOutputInfo_BlockingOperationException_awaitUninterruptibly.log,
testfailure-testMapFileAccess-emptyresponse.zip, testfailure-testReduceFromPartialMem.zip
>
>          Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> This way, we can remove the dependencies on the netty3 (jboss.netty)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message