hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12419) "Partial cell read caused by EOF" ERRORs on replication source during replication
Date Tue, 04 Nov 2014 02:19:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195584#comment-14195584
] 

Andrew Purtell commented on HBASE-12419:
----------------------------------------

This comes from BaseDecoder#rethrowEofException
{code}
  private void rethrowEofException(IOException ioEx) throws IOException {
    boolean isEof = false;
    try {
      isEof = this.in.available() == 0;
    } catch (Throwable t) {
      LOG.trace("Error getting available for error message - ignoring", t);
    }
    if (!isEof) throw ioEx;
    LOG.error("Partial cell read caused by EOF: " + ioEx);
    EOFException eofEx = new EOFException("Partial cell read");
    eofEx.initCause(ioEx);
    throw eofEx;
  }
{code}

which is called if we fail to parse a cell while advancing the reader through the WAL:
{code}
  public boolean advance() throws IOException {
    if (!this.hasNext) return this.hasNext;
    if (this.in.available() == 0) {
      this.hasNext = false;
      return this.hasNext;
    }
    try {
      this.current = parseCell();
    } catch (IOException ioEx) {
      rethrowEofException(ioEx);
    }
    return this.hasNext;
  }
{code}


> "Partial cell read caused by EOF" ERRORs on replication source during replication
> ---------------------------------------------------------------------------------
>
>                 Key: HBASE-12419
>                 URL: https://issues.apache.org/jira/browse/HBASE-12419
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.7
>            Reporter: Andrew Purtell
>             Fix For: 2.0.0, 0.98.8, 0.99.2
>
>
> We are seeing exceptions like these on the replication sources when replication is active:
> {noformat}
> 2014-11-04 01:20:19,738 ERROR [regionserver8120-EventThread.replicationSource,1] codec.BaseDecoder:
Partial cell read caused by EOF: java.io.IOException: Premature EOF from inputStream
> {noformat}
> HBase 0.98.8-SNAPSHOT, Hadoop 2.4.1.
> Happens both with and without short circuit reads on the source cluster.
> I'm able to reproduce this reliably:
> # Set up two clusters. Can be single slave.
> # Enable replication in configuration
> # Use LoadTestTool -init_only on both clusters
> # On source cluster via shell: alter 'cluster_test',{NAME=>'test_cf',REPLICATION_SCOPE=>1}
> # On source cluster via shell: add_peer 'remote:port:/hbase'
> # On source cluster, LoadTestTool -skip_init -write 1:1024:10 -num_keys 1000000
> # Wait for LoadTestTool to complete
> # Use the shell to verify 1M rows are in 'cluster_test' on the target cluster.
> All 1M rows will replicate without data loss, but I'll see 5-15 instances of "Partial
cell read caused by EOF" messages logged from codec.BaseDecoder at ERROR level on the replication
source. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message