hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16692) LLAP: Keep alive connection in shuffle handler should not be closed until entire data is flushed out
Date Wed, 17 May 2017 12:03:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013902#comment-16013902
] 

Hive QA commented on HIVE-16692:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12868503/HIVE-16692.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10718 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal] (batchId=9)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable] (batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] (batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5303/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5303/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5303/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12868503 - PreCommit-HIVE-Build

> LLAP: Keep alive connection in shuffle handler should not be closed until entire data
is flushed out
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-16692
>                 URL: https://issues.apache.org/jira/browse/HIVE-16692
>             Project: Hive
>          Issue Type: Bug
>          Components: llap
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>            Priority: Minor
>         Attachments: HIVE-16692.1.patch
>
>
> In corner cases with keep-alive enabled, it is possible that the headers are written
out in the response and downstream was able to read the headers.  
> But possible that the mapOutput construction took a lot longer time (due to disk or any
other issue) in server side. In the mean time, keep alive timeout can kick in and close the
connection from server side. In such cases, there is a possibility that downstream can get
"connection reset". Ideally keep alive should kick in only after flushing entire response
downstream.
> e.g error msg in client side
> {noformat}
> java.net.SocketException: Connection reset
>         at java.net.SocketInputStream.read(SocketInputStream.java:209) ~[?:1.8.0_112]
>         at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_112]
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_112]
>         at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_112]
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_112]
>         at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704) ~[?:1.8.0_112]
>         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) ~[?:1.8.0_112]
>         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:675) ~[?:1.8.0_112]
>         at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
~[?:1.8.0_112]
>         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
~[?:1.8.0_112]
>         at org.apache.tez.http.HttpConnection.getInputStream(HttpConnection.java:260)
~[tez-runtime-library-0.8.4.2.6.1.0-11.jar:0.8.4.2.6.1.0-11]
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.setupConnection(Fetcher.java:460)
~[tez-runtime-library-0.8.4.2.6.1.0-11.jar:0.8.4.2.6.1.0-11]
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:492)
~[tez-runtime-library-0.8.4.2.6.1.0-11.jar:0.8.4.2.6.1.0-11]
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:417)
~[tez-runtime-library-0.8.4.2.6.1.0-11.jar:0.8.4.2.6.1.0-11]
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:215)
~[tez-runtime-library-0.8.4.2.6.1.0-11.jar:0.8.4.2.6.1.0-11]
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:73)
~[tez-runtime-library-0.8.4.2.6.1.0-11.jar:0.8.4.2.6.1.0-11]
>         at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) ~[tez-common-0.8.4.2.6.1.0-11.jar:0.8.4.2.6.1.0-11]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_112]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_112]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_112]
>         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
> {noformat}
> This corner case handling was not pulled in earlier from MR handler fixes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message