incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Kumar <kumarami...@gmail.com>
Subject reading sstables stored in hdfs
Date Sat, 23 Mar 2013 19:43:14 GMT
I am starting some work on an input-format that would let us read
sstables stored in HDFS, I wonder if anyone has worked on something
similar before. I did come across

http://techblog.netflix.com/2012/02/aegisthus-bulk-data-pipeline-out-of.html

However it's not open sourced/available yet.

I am writing for a sanity check before I go too deep into this.

I have a few questions -hoping someone here would be able to help.

So far, I have been able to read sstables stored on the local file
system using the SSTableScanner and the SSTableReader. I am wondering
what would be a good way to proceed -having a custom implementation of
RandomAccessFile like the (RandomAccessReader and the
CompressedRandomAccessReader), that would use hadoop's  File System
API?


I did search for, but could have missed -Is there some documentation
on the binary format of the data, index, and stats files? That might
make it simpler for me to prototype without having to go through the
Cassandra Internals. I am currently working of our production
deployment that is 1.1.0.

Any guidance if you want to give (I am new to Cassandra Internals).

Many thanks
Amit

Mime
View raw message