hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-5878) Use getVisibleLength public api from HdfsDataInputStream from Hadoop-2.
Date Wed, 25 Apr 2012 19:22:18 GMT
Uma Maheswara Rao G created HBASE-5878:
------------------------------------------

             Summary: Use getVisibleLength public api from HdfsDataInputStream from Hadoop-2.
                 Key: HBASE-5878
                 URL: https://issues.apache.org/jira/browse/HBASE-5878
             Project: HBase
          Issue Type: Bug
          Components: wal
            Reporter: Uma Maheswara Rao G
            Assignee: Uma Maheswara Rao G


SequencFileLogReader: 

Currently Hbase using getFileLength api from DFSInputStream class by reflection. DFSInputStream
is not exposed as public. So, this may change in future. Now HDFS exposed HdfsDataInputStream
as public API.
We can make use of it, when we are not able to find the getFileLength api from DFSInputStream
as a else condition. So, that we will not have any sudden surprise like we are facing today.

Also,  it is just logging one warn message and proceeding if it throws any exception while
getting the length. I think we can re-throw the exception because there is no point in continuing
with dataloss.


{code}
long adjust = 0;

          try {
            Field fIn = FilterInputStream.class.getDeclaredField("in");
            fIn.setAccessible(true);
            Object realIn = fIn.get(this.in);
            // In hadoop 0.22, DFSInputStream is a standalone class.  Before this,
            // it was an inner class of DFSClient.
            if (realIn.getClass().getName().endsWith("DFSInputStream")) {
              Method getFileLength = realIn.getClass().
                getDeclaredMethod("getFileLength", new Class<?> []{});
              getFileLength.setAccessible(true);
              long realLength = ((Long)getFileLength.
                invoke(realIn, new Object []{})).longValue();
              assert(realLength >= this.length);
              adjust = realLength - this.length;
            } else {
              LOG.info("Input stream class: " + realIn.getClass().getName() +
                  ", not adjusting length");
            }
          } catch(Exception e) {
            SequenceFileLogReader.LOG.warn(
              "Error while trying to get accurate file length.  " +
              "Truncation / data loss may occur if RegionServers die.", e);
          }

          return adjust + super.getPos();
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message