hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Coprocessors and batch processing
Date Thu, 11 Aug 2011 05:46:23 GMT
Here's another coprocessor question...

>From the client we batch operations in order to reduce the number of round trips.
Currently there is no way (that I can find) to make use of those batches in coprocessors.

This is an issue when, for example, sets of puts and gets are (partially) forwarded to another
table by the coprocessor.
Right now this would need to use many single puts/deletes/gets from the various {pre|post}{put|delete|get}
hooks.

There is no useful demarcation; other than maybe waiting a few miliseconds, which is awkward.


Of course this forwarding could be done directly from the client, put then what's the point
of coprocessors?

I guess there could either be a {pre|post}Multi on RegionObserver (although HRegionServer.multi
does a lot of munging).
Or maybe a general {pre|post}Request with no arguments - in which case it would be at least
possible to write code in the coprocessor
to collect the puts/deletes/etc through the normal single prePut/preDelete/etc hooks and then
batch-process them in postRequest().

-- Lars

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message