hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15947) Enhance Templeton service job operations reliability
Date Tue, 28 Feb 2017 10:02:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887707#comment-15887707
] 

Hive QA commented on HIVE-15947:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12855067/HIVE-15947.3.patch

{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10289 tests executed
*Failed tests:*
{noformat}
TestConcurrentJobRequestsBase - did not produce a TEST-*.xml file (likely timed out) (batchId=171)
TestMockAnswerHelper - did not produce a TEST-*.xml file (likely timed out) (batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table]
(batchId=147)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=223)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3834/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3834/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3834/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12855067 - PreCommit-HIVE-Build

> Enhance Templeton service job operations reliability
> ----------------------------------------------------
>
>                 Key: HIVE-15947
>                 URL: https://issues.apache.org/jira/browse/HIVE-15947
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Subramanyam Pattipaka
>            Assignee: Subramanyam Pattipaka
>         Attachments: HIVE-15947.2.patch, HIVE-15947.3.patch, HIVE-15947.patch
>
>
> Currently Templeton service doesn't restrict number of job operation requests. It simply
accepts and tries to run all operations. If more number of concurrent job submit requests
comes then the time to submit job operations can increase significantly. Templetonused hdfs
to store staging file for job. If HDFS storage can't respond to large number of requests and
throttles then the job submission can take very large times in order of minutes.
> This behavior may not be suitable for all applications and client applications  may be
looking for predictable and low response for successful request or send throttle response
to client to wait for some time before re-requesting job operation.
> In this JIRA, I am trying to address following job operations 
> 1) Submit new Job
> 2) Get Job Status
> 3) List jobs
> These three operations has different complexity due to variance in use of cluster resources
like YARN/HDFS.
> The idea is to introduce a new config templeton.job.submit.exec.max-procs which controls
maximum number of concurrent active job submissions within Templeton and use this config to
control better response times. If a new job submission request sees that there are already
templeton.job.submit.exec.max-procs jobs getting submitted concurrently then the request will
fail with Http error 503 with reason 
>    β€œToo many concurrent job submission requests received. Please wait for some time
before retrying.”
>  
> The client is expected to catch this response and retry after waiting for some time.
The default value for the config templeton.job.submit.exec.max-procs is set to β€˜0’. This
means by default job submission requests are always accepted. The behavior needs to be enabled
based on requirements.
> We can have similar behavior for Status and List operations with configs templeton.job.status.exec.max-procs
and templeton.list.job.exec.max-procs respectively.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message