accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Wall <mjw...@gmail.com>
Subject Re: Question on missing RFiles
Date Fri, 11 May 2018 22:11:25 GMT
Adam,

Do you have GC logs?  Can you see if those missing RFiles were removed by
the GC process?  That could indicate you somehow got old metadata info
replayed.  Also, the rfiles increment so compare the current rfile names in
the srv.dir directory vs what is in the metadata table.  Are the existing
files after files in the metadata.  Finally, pick a few of the missing
files and grep all your master and tserver logs to see if you can learn
anything.  This sounds ungood.

Mike

On Fri, May 11, 2018 at 6:06 PM Christopher <ctubbsii@apache.org> wrote:

> This is strange. I've only ever seen this when HDFS has reported problems,
> such as missing blocks, or another obvious failure. What is your durability
> settings (were WALs turned on)?
>
> On Fri, May 11, 2018 at 12:45 PM Adam J. Shook <adamjshook@gmail.com>
> wrote:
>
>> Hello all,
>>
>> On one of our clusters, there are a good number of missing RFiles from
>> HDFS, however HDFS is not/has not reported any missing blocks.  We were
>> experiencing issues with HDFS; some flapping DataNode processes that needed
>> more heap.
>>
>> I don't anticipate I can do much besides create a bunch of empty RFiles
>> (open to suggestions).  My question is, Is it possible that Accumulo could
>> have written the metadata for these RFiles but failed to write it to HDFS?
>> In which case it would have been re-tried later and the data was persisted
>> to a different RFile?  Or is it an 'RFile is in Accumulo metadata if and
>> only if it is in HDFS' situation?
>>
>> Accumulo 1.8.1 on HDFS 2.6.0.
>>
>> Thank you,
>> --Adam
>>
>

Mime
View raw message