hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3434) ability to increment a counter without reading original value from storage
Date Mon, 10 Jan 2011 23:15:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979841#action_12979841
] 

stack commented on HBASE-3434:
------------------------------

(No worries regards blue-skying Dhruba... go for it).

If we did not want to alter the fundamentals of HBase, Collections might be done as a Coprocessor
instance.  Coprocessors have hooks pre/post Get as well as on flush/compact.  You'd mark the
region to load KeyCollectionCoprocessor.  The KCCP would work on ColumnFamilies marked as
KeyCollections either of type increment or type List.  On flush, we'd write out the aggregating
Tombstone (We want to avoid data bloat if we can -- just don't put it into the FS beyond the
WAL).  On Get, we'd aggregate until we hit a tombstone writing back a new tombstone record
if "too many" deltas have gone in since the last tombstone.

Having all this KeyCollection code cohere inside a Coprocessor is a nice way of keeping the
code all together rather than spread about the server.

Otherwise, we could make a HCD as carrying Collections only and then have the server do extra
processing on Get, flush, compact if the HCD is of this type.  If the marking was done as
a special KV Type, then we could have Collections live in the same family as plain KVs, if
we wanted to do such a thing (I don't think we do especially if the write-rate of increments
is high).






> ability to increment a counter without reading original value from storage
> --------------------------------------------------------------------------
>
>                 Key: HBASE-3434
>                 URL: https://issues.apache.org/jira/browse/HBASE-3434
>             Project: HBase
>          Issue Type: Improvement
>          Components: client, regionserver
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> There are a bunch of applications that do read-modify-write operations on HBase constructs,
e.g  a counter; The counter value has to be read in from hdfs before it can be incremented.
 We have an application where the number of increments on a counter far outnumbers the number
of times the counter is used or read. For these type of applications, it will be very beneficial
to not have to read in the counter from disk before it can be incremented.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message