kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Kim <bbuil...@gmail.com>
Subject Re: Configure Impala for Kudu on Separate Cluster
Date Tue, 15 Aug 2017 05:36:36 GMT
Hi Todd,

I tried to create a Kudu table using impala shell, and I got this error.

create table my_first_table
(
  id bigint,
  name string,
  primary key(id)
)
partition by hash partitions 16
stored as kudu;
Query: create table my_first_table
(
  id bigint,
  name string,
  primary key(id)
)
partition by hash partitions 16
stored as kudu
ERROR: ImpalaRuntimeException: Error creating Kudu table
'impala::default.my_first_table'
CAUSED BY: NonRecoverableException: Too many attempts:
KuduRpc(method=ListTables, tablet=null, attempt=101,
DeadlineTracker(timeout=180000, elapsed=178226), Traces: [0ms] querying
master, [1ms] Sub rpc: ConnectToMaster sending RPC to server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [2ms] Sub rpc:
ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [5ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[21ms] querying master, [22ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [24ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [26ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[41ms] querying master, [41ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [43ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [46ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[62ms] querying master, [62ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [64ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [66ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[81ms] querying master, [81ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [84ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [86ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[122ms] querying master, [122ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [123ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [124ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[161ms] querying master, [162ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [163ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [166ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[302ms] querying master, [302ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [304ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [306ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[382ms] querying master, [382ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [384ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [386ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[622ms] querying master, [622ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [623ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [626ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[701ms] querying master, [702ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [703ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [706ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[2062ms] querying master, [2062ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [2065ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [2068ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[4661ms] querying master, [4662ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [4663ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [4666ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[6961ms] querying master, [6962ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [6964ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [6967ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[9102ms] querying master, [9102ms] Sub rpc: ConnectToMaster sending RPC to
server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [9104ms] Sub
rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [9106ms] delaying RPC due to Service unavailable: Master
config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader.
Exceptions received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed,
[12001ms] querying master, [12002ms] Sub rpc: ConnectToMaster sending RPC
to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051, [12003ms]
Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [12006ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [15822ms] querying master, [15822ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[15824ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [15826ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [16002ms] querying master, [16002ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[16004ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [16006ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [18381ms] querying master, [18381ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[18383ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [18386ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [18821ms] querying master, [18822ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[18823ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [18826ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [19262ms] querying master, [19262ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[19263ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [19267ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [23082ms] querying master, [23082ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[23083ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [23086ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [26062ms] querying master, [26063ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[26064ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [26067ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [29721ms] querying master, [29722ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[29723ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [29726ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [30141ms] querying master, [30143ms] Sub rpc: ConnectToMaster
sending RPC to server master-prod-dc1-datanode151.pdc1i.gradientx.com:7051,
[30144ms] Sub rpc: ConnectToMaster received from server
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051 response Network
error: [Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051]
Connection closed, [30147ms] delaying RPC due to Service unavailable:
Master config (prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no
leader. Exceptions received: org.apache.kudu.client.RecoverableException:
[Peer master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection
closed, [33361ms] trace too long, truncated)
CAUSED BY: NoLeaderFoundException: Master config (
prod-dc1-datanode151.pdc1i.gradientx.com:7051) has no leader. Exceptions
received: org.apache.kudu.client.RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed
CAUSED BY: RecoverableException: [Peer
master-prod-dc1-datanode151.pdc1i.gradientx.com:7051] Connection closed

I also added to Impala Command Line Argument Advanced Configuration Snippet
(Safety Valve) the setting:
-kudu_master_hosts=prod-dc1-datanode151.pdc1i.gradientx.com:7051

And to Impala Service Environment Advanced Configuration Snippet (Safety
Valve) the setting:
IMPALA_KUDU=1

I don't think these are correct because it's not working.

Cheers,
Ben


On Mon, Aug 14, 2017 at 9:58 PM Todd Lipcon <todd@cloudera.com> wrote:

> Hi Ben,
>
> What error are you getting? The Impala shell shouldn't be relevant here --
> it's just submitting queries to the Impala daemons themselves.
>
> This deployment scenario hasn't been tested much as far as I know, so I
> wouldn't be surprised if there are issues with scheduling fragments given
> lack of locality, etc, but I would expect it to at least "basically work".
> (I've done it once or twice to copy data from one cluster to another)
>
> -Todd
>
> On Mon, Aug 14, 2017 at 7:59 PM, Benjamin Kim <bbuild11@gmail.com> wrote:
>
>> Can someone help me with configuring Impala using Cloudera Manager for
>> Kudu 1.4.0 on CDH 5.12.0? I cannot get it to connect using impala shell.
>>
>> Cheers,
>> Ben
>>
>>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
View raw message