cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cogumelos Maravilha <cogumelosmaravi...@sapo.pt>
Subject Extract big data to file
Date Wed, 08 Feb 2017 10:34:23 GMT
Hi list,

My database stores data from Kafka. Using C* 3.0.10

In my cluster I'm using:
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}

The result of extract one day of data uncompressed is around 360G.

I've find these approaches:

echo "SELECT kafka from red where datetimestamp >= '2017-02-02 00:00:00'
and datetimestamp < '2017-02-02 15:00:01';" | cqlsh 100.100.221.146 9042
> result.txt
Here by default I get 100 rows.

Using CAPTURE result.csv with paging off I always get the error out of
memory. With paging on I need to put something heavy in the top of the
Enter key. Crazy thing need to enable paging to get ride of out of
memory! I've take a look to the result file and is empty, perhaps is
cooking the result in memory to in the end past to disk.

Is there another approach like this on ACID databases:
copy (select kafka from red where datetimestamp >= '2017-02-02 00:00:00'
and datetimestamp < '2017-02-02 15:00:01') to 'result.csv' WITH CSV HEADER;

Thanks in advance.


Mime
View raw message