hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexandre Linte (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13830) Hive on spark driver crash with Spark 1.6.1
Date Tue, 28 Jun 2016 09:19:57 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352653#comment-15352653 ] 

Alexandre Linte commented on HIVE-13830:
----------------------------------------

Hi,
I upgraded Hive to Hive 2.1.0. Now I have the following errors:
- HQL commands
{noformat}
hive (shfs3453)> SELECT COUNT(year) FROM chicagocrimes GROUP BY year;
SELECT COUNT(year) FROM chicagocrimes GROUP BY year
FAILED: SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client.
{noformat}
- Client logs
{noformat}
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.conf.HiveConfUsing the default value passed in for log id: c10f51a3-a72d-40c7-9ff6-26e5fb3732da
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.session.SessionStateUpdating thread name to c10f51a3-a72d-40c7-9ff6-26e5fb3732da main
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.DriverCompiling command(queryId=shfs3453_20160628110208_f0b51237-d391-472d-abe8-f2dd2457a9ed): SELECT COUNT(year) FROM chicagocrimes GROUP BY year
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerStarting Semantic Analysis
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerCompleted phase 1 of Semantic Analysis
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerGet metadata for source tables
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerGet metadata for subqueries
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerGet metadata for destination tables
Jun 28 11:02:08 hive-cli INFO - hive.ql.ContextNew scratch dir is hdfs://sandbox/tmp/hive/shfs3453/c10f51a3-a72d-40c7-9ff6-26e5fb3732da/hive_2016-06-28_11-02-08_399_7245611464735028300-1
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerCompleted getting MetaData in Semantic Analysis
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerGet metadata for source tables
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerGet metadata for subqueries
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerGet metadata for destination tables
Jun 28 11:02:08 hive-cli INFO - hive.ql.ContextNew scratch dir is hdfs://sandbox/tmp/hive/shfs3453/c10f51a3-a72d-40c7-9ff6-26e5fb3732da/hive_2016-06-28_11-02-08_399_7245611464735028300-1
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.common.FileUtilsCreating directory if it doesn't exist: hdfs://sandbox/tmp/hive/shfs3453/c10f51a3-a72d-40c7-9ff6-26e5fb3732da/hive_2016-06-28_11-02-08_399_7245611464735028300-1/-mr-10001/.hive-staging_hive_2016-06-28_11-02-08_399_7245611464735028300-1
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.parse.CalcitePlannerCBO Succeeded; optimized logical plan.
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.ppd.OpProcFactoryProcessing for FS(6)
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.ppd.OpProcFactoryProcessing for SEL(5)
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.ppd.OpProcFactoryProcessing for GBY(4)
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.ppd.OpProcFactoryProcessing for RS(3)
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.ppd.OpProcFactoryProcessing for GBY(2)
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.ppd.OpProcFactoryProcessing for SEL(1)
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.ppd.OpProcFactoryProcessing for TS(0)
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcFactoryRS 3 oldColExprMap: {KEY._col0=Column[_col0], VALUE._col0=Column[_col1]}
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcFactoryRS 3 newColExprMap: {KEY._col0=Column[_col0], VALUE._col0=Column[_col1]}
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryloading spark properties from:spark-defaults.conf
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.default.parallelism -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.kryoserializer.buffer -> 100m).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.executor.memory -> 4g).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.driver.memory -> 2g).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.kryo.classesToRegister -> org.apache.hadoop.hive.ql.io.HiveKey,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.eventLog.compress -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.akka.frameSize -> 100).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.kryoserializer.buffer.max -> 1500m).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.sql.hive.metastore.jars -> builtin).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.yarn.scheduler.heartbeat.interval-ms -> 3000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.broadcast.compress -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.eventLog.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.executor.extraClassPath -> /opt/application/Spark-Cassandra/current/jar/spark-cassandra-connector.jar).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.broadcast.blockSize -> 4096).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.master -> yarn-client).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.io.compression.snappy.blockSize -> 32k).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.executor.cores -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.io.compression.codec -> org.apache.spark.io.SnappyCompressionCodec).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.kryo.referenceTracking -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.serializer -> org.apache.spark.serializer.KryoSerializer).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.executor.instances -> 4).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.executor.extraJavaOptions -> -Djava.library.path=/opt/application/Hadoop/current/lib/native/).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.driver.maxResultSize -> 1200m).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.sql.hive.metastore.version -> 1.2.1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from spark-defaults.conf (spark.eventLog.dir -> hdfs:///Products/SPARK/logs/).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.leaderport -> 3888).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.localizer.client.thread-count -> 5).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.storefile.refresh.period -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload RPC property from hive configuration (hive.spark.client.connect.timeout -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.admin.address -> 0.0.0.0:8047).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.property.clientPort -> 2181).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.dfs.client.read.shortcircuit.buffer.size -> 131072).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.failover-retries-on-socket-timeouts -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.thrift.framed.max_frame_size_in_mb -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.address -> 0.0.0.0:0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.ha.automatic-failover.embedded -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-state-store.parent-path -> /yarn/rmstore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.metrics.exposeOperationTimes -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.local.dir -> /tmp/hbase-shfs3453/local/).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.nonsecure-mode.user-pattern -> ^[_.A-Za-z0-9][-@_.A-Za-z0-9]{0,255}?[$]?$).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.client-am.ipc.max-retries -> 3).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.keytab.file -> /opt/application/Hbase/current/keytabs/hbase.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.info.port -> 16030).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.fs.state-store.retry-interval-ms -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (zookeeper.znode.parent -> /hbase).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.coprocessor.regionserver.classes -> org.apache.hadoop.hbase.security.access.AccessController).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.client.best-effort -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.disk-health-checker.interval-ms -> 120000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-timeout-ms -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.aux-services.mapreduce_shuffle.class -> org.apache.hadoop.mapred.ShuffleHandler).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.health-checker.script.timeout-ms -> 1200000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.client.retry-interval-ms -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.coprocessor.abortonerror -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.bytes.per.checksum -> 16384).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.ha.automatic-failover.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.logroll.errors.tolerated -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.data.umask.enable -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.windows-container.memory-limit.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.root-dir -> /sharedcache).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.resource-tracker.address.rm1 -> resourcemanager01.bigdata.fr:8031).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.resource-tracker.address.rm2 -> resourcemanager02.bigdata.fr:8031).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.webapp.address -> resourcemanager01.bigdata.fr:8188).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.status.publisher.class -> org.apache.hadoop.hbase.master.ClusterStatusPublisher$MulticastPublisher).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.balancer.period -> 300000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.readonly -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.cluster.distributed -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.staging-dir -> /Products/MR/staging).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.env-whitelist -> JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,HADOOP_YARN_HOME).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.max.filesize -> 10737418240).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.sleep-delay-before-sigkill.ms -> 250).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.address -> resourcemanager01.bigdata.fr:10200).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.recovery.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.dns.interface -> default).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.work-preserving-recovery.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-num-retries -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.ttl-ms -> 604800000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.generic-application-history.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.am.liveness-monitor.expiry-interval-ms -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.ipc.client.fallback-to-simple-auth-allowed -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.table.lock.enable -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.property.initLimit -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb -> 1024).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.compactionThreshold -> 3).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.webapp.address -> 0.0.0.0:8042).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.webapp.https.address -> resourcemanager01.bigdata.fr:8190).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.keytab.file -> /opt/application/Hbase/current/keytabs/hbase.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.admin.address -> resourcemanager01.bigdata.fr:8033).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.http.filter.initializers -> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.resource-tracker.address -> resourcemanager01.bigdata.fr:8031).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.kerberos.principal -> hbase/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.log-aggregation.retain-seconds -> 43200).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.coprocessor.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.metrics.showTableName -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.nodemanager-client-async.thread-pool-max-size -> 500).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.resource.cpu-vcores -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.status.multicast.address.port -> 16100).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.nested-level -> 3).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.property.maxClientCnxns -> 300).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.thrift.http -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.aux-services -> mapreduce_shuffle).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.client.job.max-retries -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs -> 86400).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.fail-fast -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.info.port -> 16010).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.nodemanager-connect.retry-interval-ms -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.container-metrics.unregister-delay-ms -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.write.buffer -> 2097152).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.info.port.auto -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.application-client-protocol.poll-interval-ms -> 200).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.http.max.threads -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.security.qop -> auth).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.command-opts -> -Xmx3800m -Xss512k -XX:+UseParallelGC -XX:ParallelGCThreads=4 -verbose:gc -XX:+PrintTenuringDistribution -XX:+PrintGCDetails -Xloggc:GC.log).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.fail-fast -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.generic-application-history.fs-history-store.uri -> /Products/YARN/timeline/).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.pmem-check-enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.scheduler.minimum-allocation-mb -> 1024).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.http-authentication.type -> kerberos).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.useMulti -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.shuffle.log.backups -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.memstore.block.multiplier -> 4).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.max-completed-applications -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload RPC property from hive configuration (hive.spark.client.rpc.threads -> 8).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.acl.enable -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nm.liveness-monitor.expiry-interval-ms -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.state-store.max-completed-applications -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.log-aggregation-enable -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.ipc.client.tcpnodelay -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.principal -> rm/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.storescanner.parallel.seek.enable -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.htablepool.size.max -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.snapshot.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.store.in-memory.staleness-period-mins -> 10080).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.info.bindAddress -> 0.0.0.0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.scheduler.monitor.policies -> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.http-authentication.simple.anonymous.allowed -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.local-cache.max-files-per-directory -> 8192).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.admin.acl -> rm,yarn,hdfs).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.cluster-id -> sandbox-RMS).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.containerlauncher.threadpool-initial-size -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.coprocessor.user.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.job.task.listener.thread-count -> 30).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.security.visibility.mutations.checkauth -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.scheduler.client.thread-count -> 50).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.ha.rm-ids -> rm1,rm2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.resourcemanager.minimum.version -> NONE).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.cgroups.mount -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.process-kill-wait.ms -> 2000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rootdir -> hdfs://sandbox/Products/HBASE/warehouse/).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-auth -> @/opt/application/Hadoop/current/etc/hadoop/rmhazk-auth.txt).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.server.versionfile.writeattempts -> 3).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.server.compactchecker.interval.multiplier -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.ipc.server.callqueue.scan.ratio -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.address.rm2 -> resourcemanager02.bigdata.fr:8032).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.address.rm1 -> resourcemanager01.bigdata.fr:8032).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload RPC property from hive configuration (hive.spark.client.rpc.max.size -> 52428800).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.job.committer.cancel-timeout -> 60000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.scheduler.address -> resourcemanager01.bigdata.fr:8030).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.principal -> nm/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.server.scanner.max.result.size -> 104857600).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.logcleaner.plugins -> org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.shuffle.log.separate -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds -> -1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.http.policy -> HTTP_ONLY).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.retries.number -> 35).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.authentication.kerberos.principal -> HTTP/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.catalog.timeout -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.majorcompaction -> 604800000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.localizer.fetch.thread-count -> 4).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.kerberos.principal -> HTTP/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.generic-application-history.max-applications -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.keytab -> /opt/application/Hadoop/current/keytabs/yarn.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.thrift.compact -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.online.schema.update.enable -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload spark property from hive configuration (spark.master -> yarn-client).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.proxy-user-privileges.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.checksum.verify -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.docker-container-executor.exec-name -> /usr/bin/docker).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.lease.recovery.dfs.timeout -> 64000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.fs.tmp.dir -> /user/shfs3453/hbase-staging).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.minWorkerThreads -> 16).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.delayed.delegation-token.removal-interval-ms -> 30000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.catalog.timeout -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.store-class -> org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.log.level -> INFO).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.coprocessor.master.classes -> org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.delete.thread-count -> 4).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.scheduler.maximum-allocation-vcores -> 24).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.data.umask -> 000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.resource.mb -> 4096).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user -> nobody).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.uploader.server.address -> 0.0.0.0:8046).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.infoserver.redirect -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.bin.path -> /opt/application/Hadoop/current/bin/yarn).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.coordinated.state.manager.class -> org.apache.hadoop.hbase.coordination.ZkCoordinatedStateManager).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.preclose.flush.size -> 5242880).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.kerberos.principal -> hbase/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.localityCheck.threadPoolSize -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.localizer.cache.target-size-mb -> 10240).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.port -> 16020).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.status.multicast.address.ip -> 226.1.1.3).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.defaults.for.version -> 1.1.2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.remote-app-log-dir -> /Products/YARN/logs).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.bulkload.retries.number -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.hard-kill-timeout-ms -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.scheduler.maximum-allocation-mb -> 65536).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.hostname -> resourcemanager01.bigdata.fr).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms -> 300000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.region.replica.replication.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.container.liveness-monitor.interval-ms -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.vmem-pmem-ratio -> 2.1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rpc.shortoperation.timeout -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.hostname -> 0.0.0.0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.cleaner.resource-sleep-ms -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.status.listener.class -> org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.keyvalue.maxsize -> 10485760).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.pause -> 100).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload RPC property from hive configuration (hive.spark.client.server.connect.timeout -> 90000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.column.max.version -> 1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.nodes.include-path -> /opt/application/Hadoop/current/etc/hadoop/slaves).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.status.published -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.webapp.cross-origin.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.nodemanager.minimum.version -> NONE).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.port -> 8080).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rpc.timeout -> 60000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.resources-handler.class -> org.apache.hadoop.yarn.server.nodemanager.util.DefaultLCEResourcesHandler).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.ttl-enable -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.checksum.algo.impl -> org.apache.hadoop.yarn.sharedcache.ChecksumSHA256Impl).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.hfilecleaner.plugins -> org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.cleaner.initial-delay-mins -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.webapp.address -> 0.0.0.0:8788).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.keytab -> /opt/application/Hadoop/current/keytabs/yarn.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-acl -> @/opt/application/Hadoop/current/etc/hadoop/rmhazk-acl.txt).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.hlog.reader.impl -> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.generic-application-history.store-class -> org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage -> 100).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.hlog.writer.impl -> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.client.job.retry-interval -> 2000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.ha.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.replication.rpc.codec -> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.admin.address.rm1 -> resourcemanager01.bigdata.fr:8033).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.admin.address.rm2 -> resourcemanager02.bigdata.fr:8033).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.memstore.flush.size -> 134217728).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.dns.interface -> default).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.log-aggregation.retain-check-interval-seconds -> -1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.msginterval -> 3000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.majorcompaction.jitter -> 0.50).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (zookeeper.znode.rootserver -> root-region-server).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.disk-health-checker.min-healthy-disks -> 0.25).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.max-cached-nodemanagers-proxies -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.threads.min -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.resource-tracker.client.thread-count -> 50).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.store.class -> org.apache.hadoop.yarn.server.sharedcachemanager.store.InMemorySCMStore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.client.thread-count -> 50).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.group -> hadoop).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.resource.memory-mb -> -33088).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.dns.nameserver -> default).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.scheduler.address.rm1 -> resourcemanager01.bigdata.fr:8030).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.scheduler.address.rm2 -> resourcemanager02.bigdata.fr:8030).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.flusher.count -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.max.total.tasks -> 100).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.kerberos.principal -> stargate/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.handler.count -> 30).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.security.authorization -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.admin.client.thread-count -> 1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.logroll.period -> 3600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.log.retain-seconds -> 10800).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.cgroups.hierarchy -> /hadoop-yarn).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.security.exec.permission.checks -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.container.log.limit.kb -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.admin.thread-count -> 1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regions.slop -> 0.2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.threads.max -> 100).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.shuffle.log.limit.kb -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.snapshot.restore.take.failsafe.snapshot -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.blockingStoreFiles -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts -> 3).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.maxQueuedRequests -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.keytab.file -> /etc/krb5/spnego.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.nm.uploader.thread-count -> 20).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.webapp.address -> resourcemanager01.bigdata.fr:8088).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.memstore.mslab.enabled -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.remote-app-log-dir-suffix -> logs).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.dns.nameserver -> default).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.security.authentication -> kerberos).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-state-store.root-node.acl -> setindatabag).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.recovery.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.admin-env -> MALLOC_ARENA_MAX=$MALLOC_ARENA_MAX).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.handler-thread-count -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.hostname -> 0.0.0.0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rootdir.perms -> 700).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.container-monitor.interval-ms -> 3000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.support.proxyuser -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.store.class -> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rs.cacheblocksonwrite -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.client.max-retries -> 30).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.log-dirs -> file:///mnt/hd0/hadoop/yarn/log/).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.fs.state-store.uri -> /tmp/hadoop-shfs3453/yarn/system/rmstore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.keytab -> /opt/application/Hadoop/current/keytabs/yarn.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.security.visibility.mutations.checkauths -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.http-authentication.kerberos.principal -> HTTP/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.leveldb-timeline-store.read-cache-size -> 104857600).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.ipc.server.callqueue.read.ratio -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.authentication.kerberos.keytab -> /etc/krb5/spnego.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.job.committer.commit-window -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.cells.scanned.per.heartbeat.check -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.connect.max-wait.ms -> 900000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.auth.key.update.interval -> 86400000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.config.read.zookeeper.config -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.client.max-retries -> 3).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.defaults.for.version.skip -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.webapp.https.address -> 0.0.0.0:8090).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.ipc.rpc.class -> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.localizer.cache.cleanup.interval-ms -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.amlauncher.thread-count -> 50).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.connect.retry-interval.ms -> 30000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.scheduler.minimum-allocation-vcores -> 1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.dynamic.jars.dir -> hdfs://sandbox/Products/HBASE/warehouse//lib).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline_service.webapp.address -> resourcemanager01.bigdata.fr:8188).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.nodemanager-connect-retries -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.region.split.policy -> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-address -> namenode01.bigdata.fr:2181,namenode02.bigdata.fr:2181,datanode01.bigdata.fr:2181,).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.store.in-memory.initial-delay-mins -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.resource.percentage-physical-cpu-limit -> 100).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.webapp.cross-origin.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.handler.abort.on.error.percent -> 0.5).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload RPC property from hive configuration (hive.spark.client.secret.bits -> 256).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.system-metrics-publisher.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.auth.token.max.lifetime -> 604800000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.scanner.timeout.period -> 60000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.port -> 16000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.task.container.log.backups -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.uploader.server.thread-count -> 50).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.scheduler.class -> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.keytab.file -> /opt/application/Hbase/current/keytabs/hbase.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.nodemanager-connect.max-wait-ms -> 180000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.nodes.exclude-path -> /opt/application/Hadoop/current/etc/hadoop/exclude).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.fs.state-store.retry-policy-spec -> 2000, 500).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.nodemanagers.heartbeat-interval-ms -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.fs.state-store.num-retries -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.compaction.kv.max -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.failover-proxy-provider -> org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.quorum -> namenode01.bigdata.fr:2181,namenode02.bigdata.fr:2181,datanode01.bigdata.fr:2181,).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.snapshot.restore.failsafe.name -> hbase-failsafe-{snapshot.name}-{restore.timestamp}).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.container-executor.class -> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.recovery.dir -> /tmp/hadoop-shfs3453/yarn-nm-recovery).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.scanner.caching -> 2147483647).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.coprocessor.region.classes -> org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.visibility.VisibilityController,org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.scanner.max.result.size -> 2097152).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.nm.uploader.replication.factor -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.table.max.rowsize -> 1073741824).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.bulkload.staging.dir -> /tmp/hbase-staging).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.lease.recovery.timeout -> 900000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.compaction.max -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.server.thread.wakefrequency -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.thrift.framed -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.recovery.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.webapp.address.rm2 -> resourcemanager02.bigdata.fr:8088).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.webapp.address.rm1 -> resourcemanager01.bigdata.fr:8088).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.thrift.maxWorkerThreads -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.dispatcher.drain-events.timeout -> 300000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.superuser -> hdfs).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.time.to.purge.deletes -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs -> 86400).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.client.failover-retries -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.client-server.address -> 0.0.0.0:8045).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.container-manager.thread-count -> 20).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.http.staticuser.user -> dr.stack).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.tmp.dir -> /tmp/hbase-shfs3453).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.app-checker.class -> org.apache.hadoop.yarn.server.sharedcachemanager.RemoteAppChecker).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.container.log.backups -> 0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.property.syncLimit -> 5).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.leveldb-timeline-store.path -> /mnt/hd0/hadoop/yarn/timeline).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms -> 10000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.distributed.log.replay -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.support.proxyuser -> true).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.address -> resourcemanager01.bigdata.fr:8032).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.windows-container.cpu-limit.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.localizer.address -> 0.0.0.0:8040).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.logcleaner.ttl -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.regionSplitLimit -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.client-server.thread-count -> 50).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.health-checker.interval-ms -> 600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.master.loadbalancer.class -> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.info.bindAddress -> 0.0.0.0).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.scheduler.monitor.enable -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.zk-retry-interval-ms -> 1000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (zookeeper.znode.acl.parent -> acl).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.configuration.provider-class -> org.apache.hadoop.yarn.LocalConfigurationProvider).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.local-dirs -> file:///mnt/hd0/hadoop/yarn/local/).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.leveldb-state-store.path -> /tmp/hadoop-shfs3453/yarn/system/rmstore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.max.perregion.tasks -> 1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.http-authentication.kerberos.keytab -> /opt/application/Hadoop/current/keytabs/hadoop.keytab).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.state-store-class -> org.apache.hadoop.yarn.server.timeline.recovery.LeveldbTimelineStateStore).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.peerport -> 2888).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.regionserver.optionalcacheflushinterval -> 3600000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.client.max.perserver.tasks -> 5).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.leveldb-state-store.path -> /tmp/hadoop-shfs3453/yarn/timeline).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.filter.classes -> org.apache.hadoop.hbase.rest.filter.GzipFilter).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.delete.debug-delay-sec -> 60000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rest.authentication.type -> kerberos).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.rpc.protection -> auth-conf).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.log-aggregation.compression-type -> none).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.vmem-check-enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.enabled -> false).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.zookeeper.property.dataDir -> /tmp/hbase-shfs3453/zookeeper).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.checksum.algorithm -> CRC32).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.cleaner.period-mins -> 1440).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.timeline-service.principal -> tl/_HOST@REALM.FR).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hstore.blockingWaitTime -> 90000).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.hregion.percolumnfamilyflush.size.lower.bound -> 16777216).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.ha.automatic-failover.zk-base-path -> /yarn/leader-election).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.ipc.server.callqueue.handler.factor -> 0.1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.sharedcache.store.in-memory.check-period-mins -> 720).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload HBase configuration (hbase.storescanner.parallel.seek.threads -> 10).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.nodemanager.aux-services.mapreduce.shuffle.class -> org.apache.hadoop.mapred.ShuffleHandler).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.resourcemanager.am.max-attempts -> 2).
Jun 28 11:02:08 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactoryload yarn property from hive configuration in yarn-client mode (yarn.app.mapreduce.am.resource.cpu-vcores -> 1).
Jun 28 11:02:08 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplLoading spark defaults: file:/opt/application/Hive/apache-hive-2.1.0-bin/conf/spark-defaults.conf
Jun 28 11:02:08 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplRunning client driver with argv: kinit hiveserver2/sparkclient01.bigdata.fr@REALM.FR -k -t /opt/application/Hive/current/keytabs/hiveserver2.keytab; /opt/application/Spark/current/bin/spark-submit --properties-file /tmp/spark-submit.1914700103794483224.properties --class org.apache.hive.spark.client.RemoteDriver /opt/application/Hive/apache-hive-2.1.0-bin/lib/hive-exec-2.1.0.jar --remote-host sparkclient01.bigdata.fr --remote-port 64022 --conf hive.spark.client.connect.timeout=1000 --conf hive.spark.client.server.connect.timeout=90000 --conf hive.spark.client.channel.log.level=null --conf hive.spark.client.rpc.max.size=52428800 --conf hive.spark.client.rpc.threads=8 --conf hive.spark.client.secret.bits=256 --conf hive.spark.client.rpc.server.address=null
Jun 28 11:02:08 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplkinit: Client 'hiveserver2/sparkclient01.bigdata.fr@REALM.FR' not found in Kerberos database while getting initial credentials
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplWarning: Ignoring non-spark config property: hive.spark.client.server.connect.timeout=90000
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplWarning: Ignoring non-spark config property: hive.spark.client.rpc.threads=8
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplWarning: Ignoring non-spark config property: hive.spark.client.connect.timeout=1000
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplWarning: Ignoring non-spark config property: hive.spark.client.secret.bits=256
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplWarning: Ignoring non-spark config property: hive.spark.client.rpc.max.size=52428800
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImplException in thread "main" java.lang.NoSuchFieldError: SPARK_RPC_SERVER_ADDRESS
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.hive.spark.client.rpc.RpcConfiguration.<clinit>(RpcConfiguration.java:45)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:134)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:516)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at java.lang.reflect.Method.invoke(Method.java:606)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
Jun 28 11:02:10 hive-cli INFO - org.apache.hive.spark.client.SparkClientImpl    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Jun 28 11:02:10 hive-cli ERROR - org.apache.hive.spark.client.SparkClientImplError while waiting for client to connect.
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Cancel client '9f709de7-d5c2-4a16-9346-40568e651542'. Error: Child process exited before connecting back with error log kinit: Client 'hiveserver2/sparkclient01.bigdata.fr@REALM.FR' not found in Kerberos database while getting initial credentials
Warning: Ignoring non-spark config property: hive.spark.client.server.connect.timeout=90000
Warning: Ignoring non-spark config property: hive.spark.client.rpc.threads=8
Warning: Ignoring non-spark config property: hive.spark.client.connect.timeout=1000
Warning: Ignoring non-spark config property: hive.spark.client.secret.bits=256
Warning: Ignoring non-spark config property: hive.spark.client.rpc.max.size=52428800
Exception in thread "main" java.lang.NoSuchFieldError: SPARK_RPC_SERVER_ADDRESS
        at org.apache.hive.spark.client.rpc.RpcConfiguration.<clinit>(RpcConfiguration.java:45)
        at org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:134)
        at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:516)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

        at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
        at org.apache.hive.spark.client.SparkClientImpl.<init>(SparkClientImpl.java:104)
        at org.apache.hive.spark.client.SparkClientFactory.createClient(SparkClientFactory.java:80)
        at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.createRemoteClient(RemoteHiveSparkClient.java:99)
        at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.<init>(RemoteHiveSparkClient.java:95)
        at org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactory.createHiveSparkClient(HiveSparkClientFactory.java:67)
        at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:62)
        at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionManagerImpl.getSession(SparkSessionManagerImpl.java:114)
        at org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.getSparkSession(SparkUtilities.java:136)
        at org.apache.hadoop.hive.ql.optimizer.spark.SetSparkReducerParallelism.process(SetSparkReducerParallelism.java:117)
        at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:158)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
        at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.runJoinOptimizations(SparkCompiler.java:178)
        at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.optimizeOperatorPlan(SparkCompiler.java:116)
        at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:134)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10857)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:239)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:437)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1158)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1253)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Cancel client '9f709de7-d5c2-4a16-9346-40568e651542'. Error: Child process exited before connecting back with error log kinit: Client 'hiveserver2/sparkclient01.bigdata.fr@REALM.FR' not found in Kerberos database while getting initial credentials
Warning: Ignoring non-spark config property: hive.spark.client.server.connect.timeout=90000
Warning: Ignoring non-spark config property: hive.spark.client.rpc.threads=8
Warning: Ignoring non-spark config property: hive.spark.client.connect.timeout=1000
Warning: Ignoring non-spark config property: hive.spark.client.secret.bits=256
Warning: Ignoring non-spark config property: hive.spark.client.rpc.max.size=52428800
Exception in thread "main" java.lang.NoSuchFieldError: SPARK_RPC_SERVER_ADDRESS
        at org.apache.hive.spark.client.rpc.RpcConfiguration.<clinit>(RpcConfiguration.java:45)
        at org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:134)
        at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:516)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

        at org.apache.hive.spark.client.rpc.RpcServer.cancelClient(RpcServer.java:179)
        at org.apache.hive.spark.client.SparkClientImpl$3.run(SparkClientImpl.java:465)
        at java.lang.Thread.run(Thread.java:745)
Jun 28 11:02:10 hive-cli WARN - org.apache.hive.spark.client.SparkClientImplChild process exited with code 1
Jun 28 11:02:10 hive-cli ERROR - org.apache.hadoop.hive.ql.DriverFAILED: SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client.
org.apache.hadoop.hive.ql.parse.SemanticException: Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client.
        at org.apache.hadoop.hive.ql.optimizer.spark.SetSparkReducerParallelism.process(SetSparkReducerParallelism.java:121)
        at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:158)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
        at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.runJoinOptimizations(SparkCompiler.java:178)
        at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.optimizeOperatorPlan(SparkCompiler.java:116)
        at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:134)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10857)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:239)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:437)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1158)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1253)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Jun 28 11:02:10 hive-cli INFO - org.apache.hadoop.hive.ql.DriverCompleted compiling command(queryId=shfs3453_20160628110208_f0b51237-d391-472d-abe8-f2dd2457a9ed); Time taken: 2.496 seconds
Jun 28 11:02:10 hive-cli INFO - org.apache.hadoop.hive.conf.HiveConfUsing the default value passed in for log id: c10f51a3-a72d-40c7-9ff6-26e5fb3732da
Jun 28 11:02:10 hive-cli INFO - org.apache.hadoop.hive.ql.session.SessionStateResetting thread name to  main
{noformat}
It's very strange that Hive tries to do a kinit on the machine, with a wrong principal.

> Hive on spark driver crash with Spark 1.6.1
> -------------------------------------------
>
>                 Key: HIVE-13830
>                 URL: https://issues.apache.org/jira/browse/HIVE-13830
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark, spark-branch
>    Affects Versions: 2.0.0, 2.1.0
>         Environment: Hadoop 2.7.2, Hive 2.1.0, Spark 1.6.1, Kerberos
>            Reporter: Alexandre Linte
>
> With Hive 1.2.1 I was able to use Hive on Spark successfully with the use of the spark-assembly "spark-assembly-1.4.1-hadoop2.7.1.jar". 
> Today with Hive 2.0.0, I'm unable to use Hive on Spark whether it be with the spark-assembly "spark-assembly-1.4.1-hadoop2.7.1.jar" or the spark-assembly "spark-assembly-1.6.1-hadoop2.7.2.jar".
> My configuration is the following:
>   * spark-default.conf available in HIVE_DIR/conf
>   * spark assembly available in HIVE_DIR/lib
> I gathered several logs below:
> - HQL commands
> {noformat}
> $ hive -v --database shfs3453
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/opt/application/Hive/apache-hive-2.0.0-bin/lib/hive-jdbc-2.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/opt/application/Hive/apache-hive-2.0.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/opt/application/Spark/spark-1.6.1/assembly/target/scala-2.10/spark-assembly-1.6.1-hadoop2.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/opt/application/Hadoop/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Logging initialized using configuration in file:/opt/application/Hive/apache-hive-2.0.0-bin/conf/hive-log4j2.properties
> use shfs3453
> OK
> Time taken: 1.425 seconds
> Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
> hive (shfs3453)> set hive.execution.engine=spark;
> set hive.execution.engine=spark
> hive (shfs3453)> set spark.master=yarn-client;
> set spark.master=yarn-client
> hive (shfs3453)> CREATE TABLE chicagoCrimes2 (ID BIGINT, CaseNumber STRING, Day STRING, Block STRING, IUCR INT, PrimaryType STRING, Description STRING, LocationDescription STRING, Arrest BOOLEAN, Domestic BOOLEAN, Beat INT, District INT, Ward INT, CommunityArea INT, FBICode INT, XCoordinate BIGINT, YCoordinate BIGINT, Year INT, UpdatedOn STRING, Latitude FLOAT, Longitude FLOAT, Location STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
> CREATE TABLE chicagoCrimes2 (ID BIGINT, CaseNumber STRING, Day STRING, Block STRING, IUCR INT, PrimaryType STRING, Description STRING, LocationDescription STRING, Arrest BOOLEAN, Domestic BOOLEAN, Beat INT, District INT, Ward INT, CommunityArea INT, FBICode INT, XCoordinate BIGINT, YCoordinate BIGINT, Year INT, UpdatedOn STRING, Latitude FLOAT, Longitude FLOAT, Location STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE
> OK
> Time taken: 0.408 seconds
> hive (shfs3453)> INSERT OVERWRITE TABLE chicagocrimes2 SELECT * FROM chicagocrimes WHERE Description = 'FIRST DEGREE MURDER';
> INSERT OVERWRITE TABLE chicagocrimes2 SELECT * FROM chicagocrimes WHERE Description = 'FIRST DEGREE MURDER'
> Query ID = shfs3453_20160524092714_41c89aec-2c6f-49e9-98c7-d227ca144f73
> Total jobs = 1
> Launching Job 1 out of 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
>   set mapreduce.job.reduces=<number>
> Starting Spark Job = 79484279-8e75-4b13-8e71-7de463f4d51e
> Status: SENT
> Failed to execute spark task, with exception 'java.lang.IllegalStateException(RPC channel is closed.)'
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
> {noformat}
> - Client logs
> {noformat}
> May 24 09:32:19 hive-cli WARN - org.apache.hive.spark.client.rpc.RpcDispatcherReceived error message:io.netty.handler.codec.DecoderException: java.lang.NoClassDefFoundError: org/apache/hive/spark/client/Job
>         at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:358)
>         at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:230)
>         at io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
>         at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
>         at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
>         at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
>         at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
>         at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
>         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
>         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoClassDefFoundError: org/apache/hive/spark/client/Job
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>         at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>         at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>         at java.lang.Class.forName0(Native Method)
>         at java.lang.Class.forName(Class.java:278)
>         at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
>         at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
>         at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670)
>         at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118)
>         at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
>         at org.apache.hive.spark.client.rpc.KryoMessageCodec.decode(KryoMessageCodec.java:97)
>         at io.netty.handler.codec.ByteToMessageCodec$1.decode(ByteToMessageCodec.java:42)
>         at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:327)
>         ... 15 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hive.spark.client.Job
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>         ... 39 more
> .
> May 24 09:32:19 hive-cli WARN - org.apache.hive.spark.client.SparkClientImplClient RPC channel closed unexpectedly.
> May 24 09:32:48 hive-cli INFO - org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitorJob hasn't been submitted after 61s. Aborting it.
> May 24 09:32:48 hive-cli ERROR - org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitorStatus: SENT
> May 24 09:32:48 hive-cli ERROR - org.apache.hadoop.hive.ql.exec.spark.SparkTaskFailed to execute spark task, with exception 'java.lang.IllegalStateException(RPC channel is closed.)'
> java.lang.IllegalStateException: RPC channel is closed.
>         at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
>         at org.apache.hive.spark.client.rpc.Rpc.call(Rpc.java:276)
>         at org.apache.hive.spark.client.rpc.Rpc.call(Rpc.java:259)
>         at org.apache.hive.spark.client.SparkClientImpl$ClientProtocol.cancel(SparkClientImpl.java:523)
>         at org.apache.hive.spark.client.SparkClientImpl.cancel(SparkClientImpl.java:187)
>         at org.apache.hive.spark.client.JobHandleImpl.cancel(JobHandleImpl.java:62)
>         at org.apache.hadoop.hive.ql.exec.spark.status.impl.RemoteSparkJobRef.cancelJob(RemoteSparkJobRef.java:54)
>         at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:128)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:158)
>         at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:101)
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1840)
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1584)
>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1361)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1184)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
>         at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
>         at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:778)
>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> May 24 09:32:48 hive-cli ERROR - org.apache.hadoop.hive.ql.exec.spark.SparkTaskFailed to execute spark task, with exception 'java.lang.IllegalStateException(RPC channel is closed.)'
> java.lang.IllegalStateException: RPC channel is closed.
>         at com.google.common.base.Preconditions.checkState(Preconditions.java:149) ~[guava-14.0.1.jar:?]
>         at org.apache.hive.spark.client.rpc.Rpc.call(Rpc.java:276) ~[hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hive.spark.client.rpc.Rpc.call(Rpc.java:259) ~[hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hive.spark.client.SparkClientImpl$ClientProtocol.cancel(SparkClientImpl.java:523) ~[hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hive.spark.client.SparkClientImpl.cancel(SparkClientImpl.java:187) ~[hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hive.spark.client.JobHandleImpl.cancel(JobHandleImpl.java:62) ~[hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.exec.spark.status.impl.RemoteSparkJobRef.cancelJob(RemoteSparkJobRef.java:54) ~[hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:128) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:158) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:101) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1840) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1584) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1361) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1184) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172) [hive-exec-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) [hive-cli-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) [hive-cli-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400) [hive-cli-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:778) [hive-cli-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717) [hive-cli-2.0.0.jar:2.0.0]
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645) [hive-cli-2.0.0.jar:2.0.0]
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_85]
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_85]
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_85]
>         at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_85]
>         at org.apache.hadoop.util.RunJar.run(RunJar.java:221) [spark-assembly-1.6.1-hadoop2.7.2.jar:1.6.1]
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:136) [spark-assembly-1.6.1-hadoop2.7.2.jar:1.6.1]
> May 24 09:32:48 hive-cli ERROR - org.apache.hadoop.hive.ql.DriverFAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
> May 24 09:32:48 hive-cli INFO - org.apache.hadoop.hive.ql.DriverCompleted executing command(queryId=shfs3453_20160524092714_41c89aec-2c6f-49e9-98c7-d227ca144f73); Time taken: 65.543 seconds
> {noformat}
> - Yarn logs (executor)
> {noformat}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/mnt/hd4/hadoop/yarn/local/usercache/shfs3453/filecache/18/spark-assembly-1.6.1-hadoop2.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/opt/application/Hadoop/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 16/05/24 09:32:14 INFO executor.CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT]
> 16/05/24 09:32:15 INFO spark.SecurityManager: Changing view acls to: shfs3453
> 16/05/24 09:32:15 INFO spark.SecurityManager: Changing modify acls to: shfs3453
> 16/05/24 09:32:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(shfs3453); users with modify permissions: Set(shfs3453)
> 16/05/24 09:32:15 INFO spark.SecurityManager: Changing view acls to: shfs3453
> 16/05/24 09:32:15 INFO spark.SecurityManager: Changing modify acls to: shfs3453
> 16/05/24 09:32:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(shfs3453); users with modify permissions: Set(shfs3453)
> 16/05/24 09:32:16 INFO slf4j.Slf4jLogger: Slf4jLogger started
> 16/05/24 09:32:16 INFO Remoting: Starting remoting
> 16/05/24 09:32:16 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutorActorSystem@datanode05.bigdata.fr:37444]
> 16/05/24 09:32:16 INFO util.Utils: Successfully started service 'sparkExecutorActorSystem' on port 37444.
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd8/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-8206d302-c8c7-4c79-974f-e349586c64f3
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd3/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-c3f44368-0b14-4bbf-b216-9fb332fd5174
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd2/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-008678f2-f592-4026-8342-9f03432539bc
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd1/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-44aad90f-755c-4294-b294-355b929b43e7
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd9/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-0862fb2d-aa29-48eb-abde-e4b8fa6943af
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd10/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-10811a27-1cb0-44cb-a842-db98638d67b5
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd4/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-b6f7eb6f-bb1e-4f18-80a8-d0e0de4c0c5e
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd5/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-c0949d83-1c03-457d-8c09-b61ae02c9567
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd6/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-c33365d4-9ff0-41a9-9684-19fbd3a8c8ec
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-94c7ebea-3df2-4004-911d-03e46f02909d
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd11/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-316bc7e1-1cb6-4a68-bf64-f4169d433e4e
> 16/05/24 09:32:17 INFO storage.DiskBlockManager: Created local directory at /mnt/hd7/hadoop/yarn/local/usercache/shfs3453/appcache/application_1463151644662_0023/blockmgr-d4b6e343-2f03-4558-acc6-027b896491ee
> 16/05/24 09:32:17 INFO storage.MemoryStore: MemoryStore started with capacity 2.7 GB
> 16/05/24 09:32:17 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@192.168.200.208:55898
> 16/05/24 09:32:17 INFO executor.CoarseGrainedExecutorBackend: Successfully registered with driver
> 16/05/24 09:32:17 INFO executor.Executor: Starting executor ID 4 on host datanode05.bigdata.fr
> 16/05/24 09:32:17 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 49034.
> 16/05/24 09:32:17 INFO netty.NettyBlockTransferService: Server created on 49034
> 16/05/24 09:32:17 INFO storage.BlockManagerMaster: Trying to register BlockManager
> 16/05/24 09:32:17 INFO storage.BlockManagerMaster: Registered BlockManager
> 16/05/24 09:32:19 INFO executor.CoarseGrainedExecutorBackend: Driver commanded a shutdown
> 16/05/24 09:32:20 INFO storage.MemoryStore: MemoryStore cleared
> 16/05/24 09:32:20 INFO storage.BlockManager: BlockManager stopped
> 16/05/24 09:32:20 WARN executor.CoarseGrainedExecutorBackend: An unknown (hiveclient.bigdata.fr:55898) driver disconnected.
> 16/05/24 09:32:20 ERROR executor.CoarseGrainedExecutorBackend: Driver 192.168.200.208:55898 disassociated! Shutting down.
> 16/05/24 09:32:20 INFO util.ShutdownHookManager: Shutdown hook called
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message