hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature
Date Sun, 01 Mar 2015 10:41:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342129#comment-14342129
] 

Hadoop QA commented on HBASE-13071:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12701690/HBASE-13071_trunk_1.patch
  against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
  ATTACHMENT ID: 12701690

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 4 new or modified
tests.
    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions
(2.4.1 2.5.2 2.6.0)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

                {color:red}-1 checkstyle{color}.  The applied patch generated 1946 checkstyle
errors (more than the master's current 1937 errors).

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +                                    ClusterConnection connection, RpcRetryingCallerFactory
rpcCallerFactory,
+    super(configuration,scan,name,connection,rpcCallerFactory,rpcControllerFactory,pool,replicaCallTimeoutMicroSecondScan);
+  public ClientSimpleScanner(Configuration configuration, Scan scan, TableName name, ClusterConnection
connection,
+                             RpcRetryingCallerFactory rpcCallerFactory, RpcControllerFactory
rpcControllerFactory,
+                             ExecutorService pool, int replicaCallTimeoutMicroSecondScan)
throws IOException {
+    super(configuration,scan,name,connection,rpcCallerFactory,rpcControllerFactory,pool,replicaCallTimeoutMicroSecondScan);
+  public static final String HBASE_CLIENT_SCANNER_ASYNC_PREFETCH = "hbase.client.scanner.async.prefetch";

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations
                  org.apache.hadoop.hbase.client.TestOperation

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/checkstyle-aggregate.html

                Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13021//console

This message is automatically generated.

> Hbase Streaming Scan Feature
> ----------------------------
>
>                 Key: HBASE-13071
>                 URL: https://issues.apache.org/jira/browse/HBASE-13071
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.98.11
>            Reporter: Eshcar Hillel
>         Attachments: HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch, HBaseStreamingScanDesign.pdf,
HbaseStreamingScanEvaluation.pdf
>
>
> A scan operation iterates over all rows of a table or a subrange of the table. The synchronous
nature in which the data is served at the client side hinders the speed the application traverses
the data: it increases the overall processing time, and may cause a great variance in the
times the application waits for the next piece of data.
> The scanner next() method at the client side invokes an RPC to the regionserver and then
stores the results in a cache. The application can specify how many rows will be transmitted
per RPC; by default this is set to 100 rows. 
> The cache can be considered as a producer-consumer queue, where the hbase client pushes
the data to the queue and the application consumes it. Currently this queue is synchronous,
i.e., blocking. More specifically, when the application consumed all the data from the cache
--- so the cache is empty --- the hbase client retrieves additional data from the server and
re-fills the cache with new data. During this time the application is blocked.
> Under the assumption that the application processing time can be balanced by the time
it takes to retrieve the data, an asynchronous approach can reduce the time the application
is waiting for data.
> We attach a design document.
> We also have a patch that is based on a private branch, and some evaluation results of
this code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message