accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: Re: Accumulo 1.7 InputFormat Iterator Question
Date Wed, 17 Aug 2016 22:28:20 GMT
I'm not sure it's really necessary to add the ability to clear. The way I
see it, you create the configuration, then execute, create a new
configuration, then execute, etc. I'm not sure what the use case is for
clearing.... if you didn't want it there, why did you add it only to remove
it? But it seems to me that clearing indicates a potential misuse, or at
least, anti-best-practices. Granted, I don't do a lot of MapReduce these
days, maybe I'm missing some significant use case.

Russ mentioned the method isn't idempotent. That's true. Additionally, the
Scanner will throw an exception if the name and/or priority is already in
use. We don't do any up-front deduplication in the job config.

On Wed, Aug 17, 2016 at 5:45 PM Josh Elser <josh.elser@gmail.com> wrote:

> Sounds like something we should be addressing (the ability to clear
> iterators from a Job's configuration)...
>
> -------- Original Message --------
> Subject:        Re: Accumulo 1.7 InputFormat Iterator Question
> Date:   Wed, 17 Aug 2016 21:31:01 +0000
> From:   Russ Weeks <rweeks@newbrightidea.com>
> Reply-To:       user@accumulo.apache.org
> To:     user@accumulo.apache.org
>
>
>
> Hi, Jamie,
>
> Try the static method AccumuloInputFormat.addIterator(job, new
> IteratorSetting(...)).
>
> Note that the method isn't idempotent. To clear the iterators on a job
> you can
> call
> job.getConfiguration.unset("AccumuloInputFormat.ScanOpts.Iterators") (but
> that isn't officially part of the public API)
>
> -Russ
>
> On Wed, Aug 17, 2016 at 2:26 PM Jamie Johnson <jej2003@gmail.com
> <mailto:jej2003@gmail.com>> wrote:
>
>      I am upgrading from Accumulo 1.6 to 1.7 and I am trying to
>      understand how iterators are supposed to be set in 1.7 for an input
>      format.  In my situation, if a particular property is set an
>      additional iterator needs to be added to do some additional
>      checking.  Previously I had done this in the
>      AbstractRecordReader.setupIterators() method but this has been
>      deprecated.  I had attempted to put them in
>      AbstractRecordReader.contextIterators(), but this isn't always
>      called.  This change has made me question if I was ever doing this
>      according to best practices and now wonder what the correct way to
>      do this is.  Any pointers would be greatly appreciated.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message