Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of cko2223@gmail.com designates
 209.85.160.66 as permitted sender)
Subject: ETL Tools to transfer data from Cassandra into other relational
 databases
From: "cko2223@gmail.com" <cko2223@gmail.com>
Content-Type: text/plain;
	charset=us-ascii
Message-Id: <525E1900-3A1A-4552-99CB-E6487E0950C4@gmail.com>
Date: Fri, 14 Dec 2012 14:19:15 +1100
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)

We will use Cassandra as logging storage in one of our web application. The a=
pplication only insert rows into Cassandra but never update or delete any ro=
ws. The CF is expected to grow by about 0.5 million rows per day.
=20
We need to transfer the data in Cassandra to another relational database dai=
ly. Due to the large size of the CF, instead of truncating the relational ta=
ble and reloading all rows into it each time, we plan to run a job to select=
 the "delta" rows since the last run and insert them into the relational dat=
abase.
=20
We know we can use Java, Pig or Hive to extract the delta rows to a flat fil=
e and load the data into the target relational table. We are particularly in=
terested in a process that can extract delta rows without scanning the entir=
e CF.
=20
Has anyone used any other ETL tools to do this kind of delta extraction from=
 Cassandra? We appreciate any comments and experience.
=20
Thanks,
Chin