cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Reynolds <blreynol...@gmail.com>
Subject Bulk Loading with Composite Column Slow?
Date Mon, 16 Jul 2012 14:00:02 GMT
Hi

I'm using Cassandra 1.1.0 from Datastax and attempting to load a
ColumnFamily with a single column with a Composite name.  It seems to load
ok, but is much slower than similar code without the composite column.

I tried building the Composite outside the while loop and just copying it
before adding components in each iteration but that didn't seem to help.

thanks
bri

Code based on bulk loading tutorial:

        List<AbstractType<?>> compositeList = new
ArrayList<AbstractType<?>>();
        compositeList.add(UTF8Type.instance);
        compositeList.add(UTF8Type.instance);

        SSTableSimpleUnsortedWriter usersWriter = new
SSTableSimpleUnsortedWriter(
                directory,
                new RandomPartitioner(),
                keyspace,
                columnFamily,
                CompositeType.getInstance(compositeList),
                null,
                32);

        String line;
        int lineNumber = 1;
        CsvEntry entry = new CsvEntry();
        // There is no reason not to use the same timestamp for every
column in that example.
        long timestamp = System.currentTimeMillis() * 1000;
        while ((line = reader.readLine()) != null)
        {
            if (entry.parse(line, lineNumber))
            {
                if ( (lineNumber % 10000) == 0) {
                    System.out.println("Line "+lineNumber);
                }
                CompositeType.Builder builder = new CompositeType.Builder(
CompositeType.getInstance(compositeList) );
                String siteKey = entry.key;
                usersWriter.newRow(bytes( siteKey ));
                builder.add(bytes(entry.part1));
                builder.add(bytes(entry.part2));
                usersWriter.addColumn(builder.build(), bytes(entry.date),
timestamp);
            }
            lineNumber++;
        }

Mime
View raw message