cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: bulk loading
Date Mon, 21 Jun 2010 10:31:52 GMT
You should be using the thrift API, or a wrapper around the thrift API. It looks like you're
using internal cassandra classes. 

There is a Java wrapper called Hector, and there was another talked about on the mail list
recently.

There is also a bulk import / export tool see
http://wiki.apache.org/cassandra/Operations

Aaron

On 21 Jun 2010, at 05:50, Torsten Curdt wrote:

> I am trying to get the bulk loading example to work for simple CF.
> 
>    List<ColumnFamily> columnFamilies = new LinkedList<ColumnFamily>();
> 
>    while(...) {
>        String[] fields = ...
> 
>        ColumnFamily columnFamily = ColumnFamily.create(keyspace, family);
> 
>        long now = System.currentTimeMillis();
> 
>        for (int i = 0; i < fields.length; i++) {
>            columnFamily.addColumn(new QueryPath(family, null,
>                ("col" + i).getBytes(UTF8)),
>                fields[i].getBytes(UTF8),
>                now);
>        }
>        columnFamilies.add(columnFamily);
> 
>        count += 1;
> 
>        if (count % 1000 == 0) {
>            Message message = createMessage(keyspace, key.toString(),
> family, columnFamilies);
>            for (InetAddress endpoint:
> StorageService.instance.getNaturalEndpoints(keyspace, key.toString()))
> {
>                MessagingService.instance.sendOneWay(message, endpoint);
>            }
>            columnFamilies.clear();
>        }
>    }
> 
> First I tried with my one "cassandra -f" instance then I saw this
> requires a separate IP. (Why?)
> But even with a separate IPs
> "StorageService.instance.getNaturalEndpoints" does not return an
> endpoint.
> 
> Any suggestions?
> 
> cheers
> --
> Torsten


Mime
View raw message