hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11558) Caching set on Scan object gets lost when using TableMapReduceUtil in 0.95+
Date Tue, 29 Jul 2014 18:36:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078141#comment-14078141
] 

Hadoop QA commented on HBASE-11558:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12658453/HBASE_11558_v2.patch
  against trunk revision .
  ATTACHMENT ID: 12658453

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified
tests.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +      "\022\024\n\014more_results\030\003 \001(\010\022\013\n\003ttl\030\004 \001(\r\022\030\n\007r"
+
+              new java.lang.String[] { "Column", "Attribute", "StartRow", "StopRow", "Filter",
"TimeRange", "MaxVersions", "CacheBlocks", "BatchSize", "MaxResultSize", "StoreLimit", "StoreOffset",
"LoadColumnFamiliesOnDemand", "Small", "Reversed", "Consistency", "Caching", });

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
                  org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10215//console

This message is automatically generated.

> Caching set on Scan object gets lost when using TableMapReduceUtil in 0.95+
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-11558
>                 URL: https://issues.apache.org/jira/browse/HBASE-11558
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce, Scanners
>            Reporter: Ishan Chhabra
>            Assignee: Ishan Chhabra
>             Fix For: 0.99.0, 0.96.3, 0.98.5, 2.0.0
>
>         Attachments: HBASE_11558-0.96.patch, HBASE_11558-0.96_v2.patch, HBASE_11558-0.98.patch,
HBASE_11558-0.98_v2.patch, HBASE_11558.patch, HBASE_11558_v2.patch, HBASE_11558_v2.patch
>
>
> 0.94 and before, if one sets caching on the Scan object in the Job by calling scan.setCaching(int)
and passes it to TableMapReduceUtil, it is correctly read and used by the mappers during a
mapreduce job. This is because Scan.write respects and serializes caching, which is used internally
by TableMapReduceUtil to serialize and transfer the scan object to the mappers.
> 0.95+, after the move to protobuf, ProtobufUtil.toScan does not respect caching anymore
as ClientProtos.Scan does not have the field caching. Caching is passed via the ScanRequest
object to the server and so is not needed in the Scan object. However, this breaks application
code that relies on the earlier behavior. This will lead to sudden degradation in Scan performance
0.96+ for users relying on the old behavior.
> There are 2 options here:
> 1. Add caching to Scan object, adding an extra int to the payload for the Scan object
which is really not needed in the general case.
> 2. Document and preach that TableMapReduceUtil.setScannerCaching must be called by the
client.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message