cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takayuki Tsunakawa" <tsunakawa.ta...@jp.fujitsu.com>
Subject Re: [Q] MapReduce behavior and Cassandra's scalability for petabytes of data
Date Fri, 22 Oct 2010 08:30:15 GMT
Hello, Aaron,

Thank you for much info (especially pointers that seem interesting).

> So you would not have 1,000 tasks sent to each of the 1,000 cassandra
nodes.

Yes, I meant one map task would be sent to each task tracker, resulting in
1,000 concurrent map tasks in the cluster. ColumnFamilyInputFormat cannot
identify the nodes that actually hold some data, so the job tracker will
send the map tasks to all of the 1,000 nodes. This is wasteful and
time-consuming if only 200 nodes hold some data for a keyspace.

> When the task runs on the cassandra node it will iterate through all of
the rows in the specified ColumnFamily with keys in the Token range the Node
is responsible for.

I hope the ColumnFamilyInputFormat will allow us to set KeyRange to select
rows passed to map.

I'll read the web pages you gave me. Thank you.
All, any other advice and comment is appreciated.

Regards,
Takayuki Tsunakawa

----- Original Message ----- 
From: aaron morton
To: user@cassandra.apache.org
Sent: Friday, October 22, 2010 4:05 PM
Subject: Re: [Q] MapReduce behavior and Cassandra's scalability for
petabytes of data


For plain old log analysis the Cloudera Hadoop distribution may be a better
match. Flume is designed to help with streaming data into HDFS, the LZo
compression extensions would help with the data size and PIG would make the
analysis easier (IMHO).
http://www.cloudera.com/hadoop/
http://www.cloudera.com/blog/2010/09/using-flume-to-collect-apache-2-web-server-logs/
http://www.cloudera.com/blog/2009/11/hadoop-at-twitter-part-1-splittable-lzo-compression/


I'll try to answer your questions, others please jump in if I'm wrong.


1. Data in a keyspace will be distributed to all nodes in the cassandra
cluster. AFAIK the Job Tracker should only send one task to each task
tracker, and normally you would have a task tracker running on each
cassandra node. The task tracker can then throttle how may concurrent tasks
can run. So you would not have 1,000 tasks sent to each of the 1,000
cassandra nodes.


When the task runs on the cassandra node it will iterate through all of the
rows in the specified ColumnFamily with keys in the Token range the Node is
responsible for. If cassandra is using the RandomPartitioner, data will be
spear around the cluster. So, for example, a Map-Reduce job that only wants
to read the last weeks data may have to read from every node. Obviously this
depends on how the data is broken up between rows / columns.




2. Some of the other people from riptano.com or rackspace may be able to
help with Cassandra's outer limits. There is a 400 node cluster planned
http://www.riptano.com/blog/riptano-and-digital-reasoning-form-partnership


Hope that helps.
Aaron

Mime
View raw message