impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 孙清孟 <sqm2...@gmail.com>
Subject impala is not parallelized
Date Thu, 03 Aug 2017 02:55:04 GMT
This is my impala cluster:


  <http://192.168.200.101:7180/cmf/services/14/instances#sort>
Role Type <http://192.168.200.101:7180/cmf/services/14/instances#sort>
State <http://192.168.200.101:7180/cmf/services/14/instances#sort>
Host <http://192.168.200.101:7180/cmf/services/14/instances#sort>
Commission State
<http://192.168.200.101:7180/cmf/services/14/instances#sort>
Role Group <http://192.168.200.101:7180/cmf/services/14/instances#sort>
Impala Catalog Server
<http://192.168.200.101:7180/cmf/services/14/instances/48/status> Started
with Outdated Configuration cdha0.embed.com
<http://192.168.200.101:7180/cmf/hardware/hosts/1/status> Commissioned Impala
Catalog Server Default Group
Impala Daemon
<http://192.168.200.101:7180/cmf/services/14/instances/50/status> Started
cdha2.embed.com <http://192.168.200.101:7180/cmf/hardware/hosts/3/status>
Commissioned Impala Daemon Default Group
Impala Daemon
<http://192.168.200.101:7180/cmf/services/14/instances/52/status> Started
cdha1.embed.com <http://192.168.200.101:7180/cmf/hardware/hosts/2/status>
Commissioned Impala Daemon Default Group
Impala Daemon
<http://192.168.200.101:7180/cmf/services/14/instances/49/status> Started
with Outdated Configuration cdha3.embed.com
<http://192.168.200.101:7180/cmf/hardware/hosts/5/status> Commissioned Impala
Daemon Default Group
Impala Daemon
<http://192.168.200.101:7180/cmf/services/14/instances/51/status> Started
cdha4.embed.com <http://192.168.200.101:7180/cmf/hardware/hosts/4/status>
Commissioned Impala Daemon Default Group
Impala StateStore
<http://192.168.200.101:7180/cmf/services/14/instances/53/status> Started
cdha0.embed.com <http://192.168.200.101:7180/cmf/hardware/hosts/1/status>
Commissioned Impala StateStore Default Group


When I run a SQL:

insert into table cdr_partition_true partition(ym = '2014-11') select
        call_1,
        call_2,
        type_1,
        own_1,
        own_2,
        hdfs_id,
        a_imsi,
        p_imsi,
        a_imei,
        p_imei,
        CAST(unix_timestamp(start_time) AS INT),
        CAST(unix_timestamp(end_time) AS INT),
        time,
        a_LAC,
        a_CI,
        p_LAC,
        p_CIfrom cdr_partition_cwang



The EXPLAIN, it says only one host:

----------------
Estimated Per-Host Requirements: Memory=2.80GB VCores=1
WARNING: The following tables are missing relevant table and/or column
statistics.
default.cdr_partition_cwang

WRITE TO HDFS [default.cdr_partition_true, OVERWRITE=false,
PARTITION-KEYS=('2014-11')]
|  partitions=1
|  hosts=1 per-host-mem=1.00GB
|
00:SCAN HDFS [default.cdr_partition_cwang, RANDOM]
   partitions=1/1 files=1 size=2.00GB
   table stats: unavailable
   column stats: unavailable
   hosts=1 per-host-mem=1.80GB
   tuple-ids=0 row-size=128B cardinality=unavailable
----------------

And instance is 1  -> Average Fragment F00.num instances: 1

Is this means my work only was performed  on only one impala node?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message