ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xabriel J Collazo Mojica (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-21730) Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
Date Tue, 15 Aug 2017 19:09:00 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xabriel J Collazo Mojica updated AMBARI-21730:
----------------------------------------------
    Description: 
Ambari 2.5+ sets the following Hive Interactive flag by default to:

{{hive.llap.execution.mode=only}}

This mode breaks compatibility with StorageHander based tables. For example, HBase tables
like these fail:

CREATE TABLE hbase_table_1(key int, value string) 
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
    TBLPROPERTIES ("hbase.table.name" = "xyz", "hbase.mapred.output.outputtable" = "xyz");

With a stack trace like:

0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE hbase_table_1 SELECT
* FROM hbase_table_1;
    INFO  : Compiling command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
    ....
    INFO  : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
    INFO  : Dag submit failed due to There is conflicting local resource (guava-14.0.1.jar)
between dag local resource and vertex Map 1 local resource. 
    Resource of dag : resource { scheme: "hdfs" host: "intel68bvt1.fyre.ibm.com" port: 8020
file: "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
} size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
    Resource of vertex: resource { scheme: "hdfs" host: "intel68bvt1.fyre.ibm.com" port: 8020
file: "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
}

*When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is not seen anymore.*

  was:
Ambari 2.5+ sets the following Hive Interactive flag by default to:

{{hive.llap.execution.mode=only}}

This mode breaks compatibility with StorageHander based tables. For example, HBase tables
like these fail:

CREATE TABLE hbase_table_1(key int, value string) 
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
    TBLPROPERTIES ("hbase.table.name" = "xyz", "hbase.mapred.output.outputtable" = "xyz");

With a stack trace like:

0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE hbase_table_1 SELECT
* FROM hbase_table_1;
    INFO  : Compiling command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
    ....
    INFO  : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
    INFO  : Dag submit failed due to There is conflicting local resource (guava-14.0.1.jar)
between dag local resource and vertex Map 1 local resource. 
    Resource of dag : resource { scheme: "hdfs" host: "intel68bvt1.fyre.ibm.com" port: 8020
file: "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
} size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
    Resource of vertex: resource { scheme: "hdfs" host: "intel68bvt1.fyre.ibm.com" port: 8020
file: "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
}

When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is not seen anymore.


> Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.
> ------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-21730
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21730
>             Project: Ambari
>          Issue Type: Bug
>    Affects Versions: 2.5.0, 2.5.1
>            Reporter: Xabriel J Collazo Mojica
>
> Ambari 2.5+ sets the following Hive Interactive flag by default to:
> {{hive.llap.execution.mode=only}}
> This mode breaks compatibility with StorageHander based tables. For example, HBase tables
like these fail:
> CREATE TABLE hbase_table_1(key int, value string) 
>     STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>     WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
>     TBLPROPERTIES ("hbase.table.name" = "xyz", "hbase.mapred.output.outputtable" = "xyz");
> With a stack trace like:
> 0: jdbc:hive2://intel68bvt2.fyre.ibm.com:2181> INSERT OVERWRITE TABLE hbase_table_1
SELECT * FROM hbase_table_1;
>     INFO  : Compiling command(queryId=hive_20170814172300_b01cbd64-f790-468b-855c-2ff8224f4fc0):
INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM hbase_table_1
>     ....
>     INFO  : Dag name: INSERT OVERWRITE TABLE hbase...hbase_table_1(Stage-3)
>     INFO  : Dag submit failed due to There is conflicting local resource (guava-14.0.1.jar)
between dag local resource and vertex Map 1 local resource. 
>     Resource of dag : resource { scheme: "hdfs" host: "intel68bvt1.fyre.ibm.com" port:
8020 file: "/tmp/hive/hive/0ccc7843-8d48-474a-888a-96bea2fcc1cd/hive_2017-08-14_17-23-00_028_5196182473716985960-6/hive/_tez_scratch_dir/guava-14.0.1.jar"
} size: 2189117 timestamp: 1502756584191 type: FILE visibility: PRIVATE
>     Resource of vertex: resource { scheme: "hdfs" host: "intel68bvt1.fyre.ibm.com" port:
8020 file: "/tmp/hive/hive/_tez_session_dir/de9aa10e-39a4-4d76-a0d8-55493fc6ba00/guava-14.0.1.jar"
}
> *When this flag is set to: {{hive.llap.execution.mode=auto}}, the issue is not seen anymore.*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message