incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tatu Saloranta <tsalora...@gmail.com>
Subject Re: Heap sudden jump during import
Date Tue, 06 Apr 2010 17:55:21 GMT
On Tue, Apr 6, 2010 at 12:15 AM, JKnight JKnight <beuknight@gmail.com> wrote:
> When import, all data in json file will load in memory. So that, you can not
> import large data.
> You need to export large sstable file to many small json files, and run
> import.

Why would you ever read the whole file in memory? JSON is very easily
streamable. Or does the whole data set need to be validated or
something (I assume not, if file splitting could be used). Perhaps it
is just an implementation flaw in importer tool.

-+ Tatu +-

Mime
View raw message