hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5878) Use getVisibleLength public api from HdfsDataInputStream from Hadoop-2.
Date Wed, 14 Jan 2015 07:37:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276606#comment-14276606

Hadoop QA commented on HBASE-5878:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  against master branch at commit 9b7f36b8cf521bcc01ac6476349a9d2f34be8bb3.
  ATTACHMENT ID: 12692160

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 checkstyle{color}.  The applied patch does not increase the total number
of checkstyle errors

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

     {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s): 	at org.apache.curator.test.TestingZooKeeperMain.runFromConfig(TestingZooKeeperMain.java:73)
	at org.apache.curator.test.TestingZooKeeperServer$1.run(TestingZooKeeperServer.java:134)
	at org.apache.oozie.test.MiniHCatServer$1.run(MiniHCatServer.java:137)
	at org.apache.oozie.test.XTestCase$MiniClusterShutdownMonitor.run(XTestCase.java:1071)
	at org.apache.oozie.test.XTestCase.waitFor(XTestCase.java:692)
	at org.apache.oozie.test.XTestCase.sleep(XTestCase.java:710)
	at org.apache.oozie.service.TestZKXLogStreamingService.testDisableLogOverWS(TestZKXLogStreamingService.java:88)

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//artifact/patchprocess/checkstyle-aggregate.html

  Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12458//console

This message is automatically generated.

> Use getVisibleLength public api from HdfsDataInputStream from Hadoop-2.
> -----------------------------------------------------------------------
>                 Key: HBASE-5878
>                 URL: https://issues.apache.org/jira/browse/HBASE-5878
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>            Reporter: Uma Maheswara Rao G
>            Assignee: Ashish Singhi
>             Fix For: 1.0.0, 2.0.0
>         Attachments: HBASE-5878-v2.patch, HBASE-5878-v3.patch, HBASE-5878-v4.patch, HBASE-5878.patch
> SequencFileLogReader: 
> Currently Hbase using getFileLength api from DFSInputStream class by reflection. DFSInputStream
is not exposed as public. So, this may change in future. Now HDFS exposed HdfsDataInputStream
as public API.
> We can make use of it, when we are not able to find the getFileLength api from DFSInputStream
as a else condition. So, that we will not have any sudden surprise like we are facing today.
> Also,  it is just logging one warn message and proceeding if it throws any exception
while getting the length. I think we can re-throw the exception because there is no point
in continuing with dataloss.
> {code}
> long adjust = 0;
>           try {
>             Field fIn = FilterInputStream.class.getDeclaredField("in");
>             fIn.setAccessible(true);
>             Object realIn = fIn.get(this.in);
>             // In hadoop 0.22, DFSInputStream is a standalone class.  Before this,
>             // it was an inner class of DFSClient.
>             if (realIn.getClass().getName().endsWith("DFSInputStream")) {
>               Method getFileLength = realIn.getClass().
>                 getDeclaredMethod("getFileLength", new Class<?> []{});
>               getFileLength.setAccessible(true);
>               long realLength = ((Long)getFileLength.
>                 invoke(realIn, new Object []{})).longValue();
>               assert(realLength >= this.length);
>               adjust = realLength - this.length;
>             } else {
>               LOG.info("Input stream class: " + realIn.getClass().getName() +
>                   ", not adjusting length");
>             }
>           } catch(Exception e) {
>             SequenceFileLogReader.LOG.warn(
>               "Error while trying to get accurate file length.  " +
>               "Truncation / data loss may occur if RegionServers die.", e);
>           }
>           return adjust + super.getPos();
> {code}

This message was sent by Atlassian JIRA

View raw message