ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vitalyi Brodetskyi" <vbrodets...@hortonworks.com>
Subject Review Request 35297: Simple insert takes up 6 slots with tez: mapreduce.input.fileinputformat.split.minsize=1 should be changed
Date Wed, 10 Jun 2015 14:16:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35297/
-----------------------------------------------------------

Review request for Ambari and Andrew Onischuk.


Bugs: AMBARI-11837
    https://issues.apache.org/jira/browse/AMBARI-11837


Repository: ambari


Description
-------

{noformat}
create table test (f1 int, f2 int);
insert into table test values (10,10);
{noformat}

This ends up creating 6 containers (HiveInput creates 6 splits) in hive1 queue in cn041 cluster.
 Accidentally one could use couple of JDBC programs to use up all available slots with simple
inserts.

Appears that "mapreduce.input.fileinputformat.split.minsize" is set to 1. Setting it to a
higher value "mapreduce.input.fileinputformat.split.minsize=1000000" fixes the problem and
launches 1 container.


Diffs
-----

  ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
f4e4b25 

Diff: https://reviews.apache.org/r/35297/diff/


Testing
-------

mvn clean test


Thanks,

Vitalyi Brodetskyi


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message