accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Write data to a file from inside of an iterator
Date Sun, 06 Nov 2016 23:45:20 GMT
(/me smiles at Dylan)

Do it client-side. There is no safe way to do this inside of Accumulo
itself.

On Nov 5, 2016 13:29, "Yamini Joshi" <yamini.1691@gmail.com> wrote:

> Hello all
>
> As per https://github.com/apache/accumulo/blob/master/docs/src/
> main/asciidoc/chapters/iterator_design.txt
> "
> Implementations of Iterator might be tempted to open BatchWriters inside
> of an Iterator as a means
> to implement triggers for writing additional data outside of their client
> application. The lifecycle of an Iterator
> is *not* managed in such a way that guarantees that this is safe nor
> efficient. Specifically, there
> is no way to guarantee that the internal ThreadPool inside of the
> BatchWriter is closed (and the thread(s)
> are reaped) without calling the close() method. `close`'ing and recreating
> a `BatchWriter` after every
> Key-Value pair is also prohibitively performance limiting to be considered
> an option."
>
> If I need to write a subset of records generated from an iterator to a
> file/table, I can't use a batch writer inside of an iterator? Is there any
> other way to go about it?
>
> Best regards,
> Yamini Joshi
>

Mime
View raw message