hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature
Date Mon, 23 Feb 2015 22:41:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14334019#comment-14334019

Jonathan Lawlor commented on HBASE-13071:

Those performance improvements look great [~eshcar]! I agree with [~stack] that this configuration
should be on by default.

I have been thinking about this asynchronous scanner a bit and was wondering how it deals
with the case where the user has specified a maximum result size via Scan#setMaxResultSize():
	* Does the prefetch mechanism attempt to prefetch a certain number of rows and loop until
that number of rows is retrieved (i.e. does the prefetch only stop making prefetch RPCs once
it has accumulated the expected number of rows)?
	* If the prefetch does stop when the maxResultSize limit has been exceeded, when would the
next prefetch be initiated? When next() is called and the cache is observed to have less than
half the specified caching value? 
	** If that's the case, I was thinking that it may be possible that a table with very large
rows could potentially create memory issues on the client (and this may be a good reason to
make this async behavior configurable rather that go all-in on async scanners). 
	** The scenario that I have in mind is the case where the client is not able to hold X rows
in memory where X is half of the specified scanner caching (i.e. the case where (half_caching_limit
* size_of_row) exceeds the avalaible memory on the client). The synchronous scanners currently
guard the client from out of memory exceptions by considering the remaining result size after
each next RPC -- if the maxResultSize limit is exceeded the scanner stops making RPCs, regardless
of how many rows the scanner has received from RPCs. However, in the case of asynch scanners
(if they work how I am thinking they do) each call to next() may trigger a prefetch, and each
prefetch would receive only Y rows, where Y << half_of_caching. The prefetches would
be triggered on each call to next() until we have accumulated half_of_caching rows in our
scanner cache and the client may OOM before that limit can be reached
	*** In such a case, it may be best to instead use a synch scanner. Alternatively, we could
keep track of the size of results in our cache and allow the user to specifiy maxSizeOfCache
so that prefetches aren't continuosly fired off on calls to next() when scanning a table with
large rows
	*** This also raises the question about whether or not it would be worthwhile to have an
asynchronous scanner that used the size of the Results (in memory) as the deciding factor
for prefetching -- i.e. If the size of the results in the cache is less than half of maxResultSize
then perform a prefetch for another chunk of maxResultSize worth of values... just a thought

My concern above is definitely a corner case issue, but I thought I'd raise it for discussion.

I like the idea that you pointed out towards the end about ramping the prefetch size up from
a small initial size to the actual prefetch size. Looks like it would definitely help ease
the initial latency jump seen for large prefetches. 

Really nice results :)

> Hbase Streaming Scan Feature
> ----------------------------
>                 Key: HBASE-13071
>                 URL: https://issues.apache.org/jira/browse/HBASE-13071
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Eshcar Hillel
>         Attachments: HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf
> A scan operation iterates over all rows of a table or a subrange of the table. The synchronous
nature in which the data is served at the client side hinders the speed the application traverses
the data: it increases the overall processing time, and may cause a great variance in the
times the application waits for the next piece of data.
> The scanner next() method at the client side invokes an RPC to the regionserver and then
stores the results in a cache. The application can specify how many rows will be transmitted
per RPC; by default this is set to 100 rows. 
> The cache can be considered as a producer-consumer queue, where the hbase client pushes
the data to the queue and the application consumes it. Currently this queue is synchronous,
i.e., blocking. More specifically, when the application consumed all the data from the cache
--- so the cache is empty --- the hbase client retrieves additional data from the server and
re-fills the cache with new data. During this time the application is blocked.
> Under the assumption that the application processing time can be balanced by the time
it takes to retrieve the data, an asynchronous approach can reduce the time the application
is waiting for data.
> We attach a design document.
> We also have a patch that is based on a private branch, and some evaluation results of
this code.

This message was sent by Atlassian JIRA

View raw message