accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: What happens when tablet is 'lost'?
Date Thu, 09 Aug 2012 13:16:08 GMT
On Thu, Aug 9, 2012 at 9:01 AM, Hugh Xedni <hughxedni@gmail.com> wrote:
> Keith,
>
> From your response...
>
> "A tablets persistent data is composed of a set of files in HDFS and a set
> of write ahead logs."
> "As long as the other tablet server can read the write ahead logs and HDFS
> files, no data is lost."
>
> Question.
> I understand when a write arrives at a TabletServer it is written to a
> Write-Ahead Log and then stored in the MemTable until flusing to HDFS via
> minor compaction.
> 1) Are write ahead logs even used when data is bulk inserted into hdfs then
> handed to Accumulo?

No write ahead logs are used directly.  Bulk import moves a file into
the tables dir in HDFS. Then an entry is written to the metadata table
for the tablet.  This write to the metadata table ends up in the write
ahead log of the tablet server serving the metadata tablet.

> 2) If not, how is data replication achieved when using bulk insert methods?

Bulk import just moves file already in hdfs.  So the file retains the
hdfs replication it had before bulk import.

>
> --
> Thank you,
>
> Hugh Xedni
>
>
>
> On Thu, Jul 26, 2012 at 12:31 PM, David Medinets <david.medinets@gmail.com>
> wrote:
>>
>> Thanks, Keith. That explanation is just what I wanted.
>
>
>
>
>

Mime
View raw message