impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bin Wang <>
Subject Memory limit exceed even with very simple count query
Date Wed, 05 Apr 2017 16:52:47 GMT

I'm using Impala on production for a while. But since yesterday, some
queries reports memory limit exceeded. Then I try a very simple count
query, it still have memory limit exceeded.

The query is:

select count(0) from adhoc_data_fast.log where day>='2017-04-04' and

And the response in the Impala shell is:

Query submitted at: 2017-04-06 00:41:00 (Coordinator:
Query progress can be monitored at:

Memory limit exceeded
GzipDecompressor failed to allocate 54525952000 bytes.

I have many nodes and each of them have lots of memory avaliable (~ 60 GB).
And the query failed very fast after I execute it and the nodes have almost
no memory usage.

The table "adhoc_data_fast.log" is an AVRO table and is encoded with gzip
and is partitioned by the field "day". And each partition has no more than
one billion rows.

My Impala version is:

hdfs@szq7:/home/ubuntu$ impalad --version
impalad version 2.7.0-cdh5.9.1 RELEASE (build
Built on Wed Jan 11 13:39:25 PST 2017

Any one can help for this? Thanks very much!

View raw message