cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lohfink <clohf...@blackbirdit.com>
Subject Re: do Cassandra generate a event or log containing key value of column when a column expires due to TTL
Date Fri, 22 Aug 2014 14:24:22 GMT
Few options I can think of, probably some better ideas out there.  These mostly depending on
size of data and how frequently updated.

1) a map reduce or spark job to filter out non-empty rows
2) add some logging and do a custom build of cassandra (ie in "removeDeletedCF" of ColumnFamilyStore)
and grep log files to get a report that can be consumed by your system.  Potentially lotta
redundant data and may be long after actual expiration before shows up.
3) when you insert your TTLed columns you also write to a wide row, with timestamp of expiration
and use that to drive the report... there will be a bit to that though and a MR job still
might be good idea for doing processing.

---
Chris

On Aug 22, 2014, at 6:02 AM, Gaurav Bhatnagar <gbhatnagar@gmail.com> wrote:

> Hi,
>     I have stored following data structure in cassandra
> 
> RowKey: 119551747098
> 
> 
> => (name=c:per:@batchId, value=ad1, timestamp=1408345109805011, ttl=1436489)
> 
> => (name=c:per:@currency, value=USD, timestamp=1408345109805009, ttl=1436489)
> 
> => (name=c:per:@decimalValue, value=2, timestamp=1408345109805003, ttl=1436489)
> 
> 
> 
> here Rowkey 119551747098 is a numeric number containing serial number of data
> 
> 
> 
> These columns get expired when ttl value for that column is reached.
> 
> I what to generate an audit trail which contains value of RowKey along with column name
and value when they get deleted due to ttl expiration.
> 
> 
> 
> I want this audit trail for reconciliation purpose so that I can know which all RowKeys
have got deleted from the system.
> 
> 
> 
> Is there any way in cassandra through which I can print value of RowKeys which get deleted
due TTL expiration.
> 
> 
> 
> Regards,
> 
> Gaurav
> 


Mime
View raw message