kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geetika Gupta <geetika.gu...@knoldus.in>
Subject Issue in loading data in Kudu using Impala
Date Sun, 06 May 2018 12:20:34 GMT
Hi, community

We are trying to load data in kudu using Impala, but we are encountering
the following error:

Status: Memory limit exceeded: Error occurred on backend slave3:22000 by
fragment eb410bbc56495c0e:50f46d0800000000 Memory left in process limit:
-438.06 MB Query(eb410bbc56495c0e:50f46d0800000000): Reservation=0
ReservationLimit=30.18 GB OtherMemory=35.95 GB Total=35.95 GB Peak=35.95 GB
Fragment eb410bbc56495c0e:50f46d0800000000: Reservation=0 OtherMemory=35.95
GB Total=35.95 GB Peak=35.95 GB SORT_NODE (id=1): Total=34.62 GB Peak=34.62
GB HDFS_SCAN_NODE (id=0): Total=1.33 GB Peak=1.37 GB KrpcDataStreamSender
(dst_id=2): Total=736.00 B Peak=736.00 B CodeGen: Total=11.72 KB Peak=1.41
MBProcess: memory limit exceeded. Limit=37.72 GB Total=38.15 GB Peak=38.20
GB Buffer Pool: Free Buffers: Total=0 Buffer Pool: Clean Pages: Total=0
Buffer Pool: Unused Reservation: Total=0 Data Stream Service Queue:
Limit=1.89 GB Total=0 Peak=0 Data Stream Manager Early RPCs: Total=0 Peak=0
TCMalloc Overhead: Total=75.40 MB RequestPool=default-pool: Total=35.95 GB
Peak=35.95 GB Query(eb410bbc56495c0e:50f46d0800000000): Reservation=0
ReservationLimit=30.18 GB OtherMemory=35.95 GB Total=35.95 GB Peak=35.95 GB
Untracked Memory: Total=2.13 GB

The quey that we executed is as follows:

* insert** into lineitem select * from PARQUETIMPALA500.LINEITEM order by
L_ORDERKEY limit 750000000 offset 536870913*

We are loading data from a parquet table and we are performing the queries
on* 500 GB* of* TPCH* data.

We also tried the following query:
* insert into lineitem select * from PARQUETIMPALA500.LINEITEM*

but in this case, the impalad process got aborted after 23hrs of executing
the query.

We are executing the queries on 8 node cluster with the following
configuration
Cluster : 8 Node Cluster (48 GB RAM , 8 CPU Core and 2 TB hard-disk each,
Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz

-- 
Regards,
Geetika Gupta

Mime
View raw message