accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billie J Rinaldi <billie.j.rina...@ugov.gov>
Subject Re: org.apache.accumulo.core.iterators.Combiner: key scope?
Date Mon, 19 Mar 2012 19:50:39 GMT
Another thing to consider is what to do with the differing column qualifiers.  Throw them away,
returning a blank column qualifier on the single Key returned?  What if we want to combine
column qualifiers and ignore Values instead?  Should we try to pass the qualifiers into a
reduce method with the Values?  That would be a more general approach, but I'm not sure how
to create an API that wouldn't be messy.

Billie


----- Original Message -----
> From: "Jason Trost" <jason.trost@gmail.com>
> To: accumulo-user@incubator.apache.org
> Sent: Thursday, March 15, 2012 7:33:48 AM
> Subject: org.apache.accumulo.core.iterators.Combiner: key scope?
> I found myself needing a combiner that will sum the values of a row
> where each key has the same row and column family (but col qual
> differs).
> 
> I was looking through the Combiner class and I was wondering if there
> would be any issues with making this line of the code configurable.
> (line 70 in org.apache.accumulo.core.iterators.Combiner)
> 
> private boolean _hasNext() {
> return source.hasTop() && !source.getTopKey().isDeleted() &&
> topKey.equals(source.getTopKey(),
> PartialKey.ROW_COLFAM_COLQUAL_COLVIS);
> }
> 
> Specifically I was thinking it would be useful to be able to configure
> the partial key field using one of the following values (from
> org.apache.accumulo.core.data.PartialKey):
> ROW
> ROW_COLFAM
> ROW_COLFAM_COLQUAL
> ROW_COLFAM_COLQUAL_COLVIS
> 
> I see the main value here in using Combiners at scan time to perform
> various rollups and counts.
> 
> I am guessing there may be some security implications of doing this?
> Maybe the labels of aggregations based on any partial key not
> including colvis would need to be combined.
> 
> Thoughts on this?
> 
> Thanks,
> 
> --Jason

Mime
View raw message