accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: New Accumulo Blog Post
Date Thu, 03 Nov 2016 02:34:27 GMT
I'm aware of at least one person who has patched Accumulo to allow
customizing the HDFS volume on which the WALs are stored. This reminds me
that I need to check on the status of that patch. I'm hoping it'll be
contributed soon.

I'm also curious if it'd make a difference writing to HDFS with the data
nodes mounted with sync, instead of doing a separate sync call.

On Wed, Nov 2, 2016 at 9:49 PM <dlmarion@comcast.net> wrote:

> Regarding #2 – I think there are two options here:
>
>
>
> 1. Modify Accumulo to take advantage of HDFS Heterogeneous Storage
>
> 2. Modify Accumulo WAL code to support volumes
>
>
>
> *From:* Jeff Kubina [mailto:jeff.kubina@gmail.com]
> *Sent:* Wednesday, November 02, 2016 9:02 PM
> *To:* user@accumulo.apache.org
> *Subject:* Re: New Accumulo Blog Post
>
>
>
> Thanks for the blog post, very interesting read. Some questions ...
>
>
>
> 1. Are the operations "Writes mutation to tablet servers’ WAL/Sync or
> flush tablet servers’ WAL" and "Adds mutations to sorted in memory map of
> each tablet." performed by threads in parallel?
>
>
>
> 2. Could the latency of hsync-ing the WALs be overcome by modifying
> Accumulo to write them to a separate SSD-only HDFS? To maintain data
> locality it would require two datanode processes (one for the HDDs and one
> for the SSD), running on the same node, which is not hard to do.
>
>
>

Mime
View raw message