hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ramkrishna.S.Vasudevan" <ramkrishna.vasude...@huawei.com>
Subject Handling EOFexception while splitlog
Date Tue, 21 Feb 2012 14:52:12 GMT
Hi Devs

 

We ran into one issue while splitting HLogs due to EOFException. (0.90.6
version).

 

Due to some reason the DNs were not able to connect to NN (network
fluctuation) and the master was splitting the logs.

While parsing the hlog, we get  the length and we expect it might be 0. (No
problem here).

 

But in this scenario the DFSClient throws an EOFException as all DNs are not
able to connect to NN and due to that the while reading we get an
EOFException.

 

In this specific case we just return and the master considers the split to
be successful.  This leads to data loss.

 

May be  this can be fixed from HDFS side, I would like to know can we throw
an IOException in this case and make the split log to be retried as we have
a retry logic now.

 

I was going through HBASE-2643 as part of which EOFException was handled and
dint find this scenario in that. 

Please provide your suggestions.

 

Regards

Ram


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message