ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-11837) Simple insert takes up 6 slots with tez: mapreduce.input.fileinputformat.split.minsize=1 should be changed
Date Wed, 10 Jun 2015 17:16:01 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580829#comment-14580829
] 

Hudson commented on AMBARI-11837:
---------------------------------

SUCCESS: Integrated in Ambari-trunk-Commit #2877 (See [https://builds.apache.org/job/Ambari-trunk-Commit/2877/])
AMBARI-11837. Simple insert takes up 6 slots with tez: mapreduce.input.fileinputformat.split.minsize=1
should be changed.(vbrodetskyi) (vbrodetskyi: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=fd4290e054f10b55c218d6fee75ab523be7ea39a)
* ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml


> Simple insert takes up 6 slots with tez: mapreduce.input.fileinputformat.split.minsize=1
should be changed
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-11837
>                 URL: https://issues.apache.org/jira/browse/AMBARI-11837
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>            Reporter: Vitaly Brodetskyi
>            Assignee: Vitaly Brodetskyi
>            Priority: Critical
>             Fix For: 2.1.0
>
>         Attachments: AMBARI-11837.patch
>
>
> {noformat}
> create table test (f1 int, f2 int);
> insert into table test values (10,10);
> {noformat}
> This ends up creating 6 containers (HiveInput creates 6 splits) in hive1 queue in cn041
cluster.  Accidentally one could use couple of JDBC programs to use up all available slots
with simple inserts.
> Appears that "mapreduce.input.fileinputformat.split.minsize" is set to 1. Setting it
to a higher value "mapreduce.input.fileinputformat.split.minsize=1000000" fixes the problem
and launches 1 container. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message