accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-4751) Some WALs don't replicate due to lacking a createdTime entry
Date Mon, 04 Dec 2017 18:24:00 GMT


Josh Elser commented on ACCUMULO-4751:

bq. My initial finding is that some entries coming out of the StatusUtil don't have a createdTime.
Whereas something like [1] writes a metadata entry to a WAL and the replication status is
updated with a createdTime. I am guessing that, in this case, there is no ingest for a particular
table during the lifetime of the WAL, so the work to replicate the WAL to a particular table
(via the key extent) is never created. However, something else is writing some information
that doesn't have a createdTime.

Yeah, I do recall this distinction. I remember having a separation between when a WAL is "created"
in memory (more like the TServer decides it's going to use that file), but there aren't any
records in that file yet. I could also see replication missing some part of the lifecycle
of the WAL itself as this is pretty obtuse (not explicitly documented). LMK if you need to
knock heads together.

> Some WALs don't replicate due to lacking a createdTime entry
> ------------------------------------------------------------
>                 Key: ACCUMULO-4751
>                 URL:
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.7.3, 1.8.1
>            Reporter: Adam J Shook
>            Assignee: Adam J Shook
> From what I can tell, the below error is thrown when no data for a particular table is
written to a WAL, but the file is closed.  This would be because the {{Status}} entry from
the {{StatusUtil}} for {{fileClosed}} is pre-built and therefore does not have a {{createdTime}}.
 This prevents a WAL from being replicated until a {{createdTime}} entry is added manually.
> From the Accumulo master:
> {code}
> Status record ([begin: 0 end: 0 infiniteEnd: true closed: true]) for hdfs://namenode:9000/accumulo/wal/
in table 7l was written to metadata table which lacked createdTime
> {code}
> There are two solutions I have in mind:
> 1. Update the {{StatusUtil}} such that every returned {{Status}} object sets the {{createdTime}}
to {{System.currentTimeMillis}} if not explicitly given.
> 2. Update the Accumulo Master to set the {{createdTime}} to the WAL's modification time
in HDFS if the WAL is closed but there is no {{createdTime}}.

This message was sent by Atlassian JIRA

View raw message