accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yamini Joshi <yamini.1...@gmail.com>
Subject Re: Modify Keys within iterator
Date Fri, 30 Sep 2016 18:05:43 GMT
If I give it an empty range, it gives me the output of simple scan(without
the iterator applied even though the iterator is working). I guess it's bad
to modify keys within an iterator.

Best regards,
Yamini Joshi

On Fri, Sep 30, 2016 at 12:51 PM, Dan Blum <dblum@bbn.com> wrote:

> What happens if you just use an empty range?
>
>
>
> What I suspect is happening is that the scanner code sees a key beyond the
> range’s end key and stops. In general you can’t transform rows in keys for
> this reason, and you might have issues even if you don’t transform the rows
> if the keys end up out of order – see the comments in TransformingIterator.
>
>
>
> *From:* Yamini Joshi [mailto:yamini.1691@gmail.com]
> *Sent:* Friday, September 30, 2016 1:37 PM
> *To:* user@accumulo.apache.org
> *Subject:* Re: Modify Keys within iterator
>
>
>
> I am using pyaccumulo. Here's the code snippet:
>
> rowIds=['r2','r10']
>
> hashFilter = KeyModifyIterator(priority=10)
> iterator.append(hashFilter)
>
> *for *entry *in *self.dbconn.batch_scan(table , scanranges=(Range(srow=row, erow=row)
*for *row *in *rowIDs),iterators=[hashFilter]):
>     *print *entry
>
>
>
>
> Best regards,
> Yamini Joshi
>
>
>
> On Fri, Sep 30, 2016 at 12:31 PM, Dan Blum <dblum@bbn.com> wrote:
>
> What code are you using to test the iterator, where you see no output?
>
>
>
> *From:* Yamini Joshi [mailto:yamini.1691@gmail.com]
> *Sent:* Friday, September 30, 2016 1:26 PM
> *To:* user@accumulo.apache.org
> *Subject:* Modify Keys within iterator
>
>
>
> Hello Everyone!
>
> I am trying to write an iterator to modify keys within a table (at scan).
> My use case is to select a few records that match a certain criterion and
> then modify them within the iterator(using the following class) for some
> other succeeding iterator/combiner. The problem is that this iterator does
> return any records/keys. I added some primitive prints and found that the
> keys (this.key) is changed but the output of iterator is nothing. I'd
> appreciate if someone could give me any insight. I'm sure I'm making a
> teeny tiny mistake somewhere.
>
> Schema:
>
>          row                   colF
> colQ                 ts                Val
>
> I/P:    r_1                   f                             f_1
>                            v1
>
>          r_1                   fx                            f_1
>                             v1
>
> O/P:  f_1                   r
> r_1                                      v1
>
>          f_1                   fx
> fx                                        v1
>
>
>
>
>
> public class KeyModifyIterator implements SortedKeyValueIterator<Key,Value>
> {
>
>   private SortedKeyValueIterator<Key,Value> source;
>   private Key key;
>   private Value value;
>
>   @Override
>   public void init(SortedKeyValueIterator<Key,Value> source,
> Map<String,String> options, IteratorEnvironment env) throws IOException {
>     this.source = source;
>   }
>
>   @Override
>   public boolean hasTop() {
>     return key != null;
>   }
>
>   @Override
>   public void next() throws IOException {
>         if (source.hasTop()) {
>             ByteSequence currentRow = source.getTopKey().getRowData();
>             ByteSequence currentColf = source.getTopKey().
> getColumnFamilyData();
>             ByteSequence currentColq = source.getTopKey().
> getColumnQualifierData();
>             long ts = source.getTopKey().getTimestamp();
>             String v = source.getTopValue().toString();
>             System.out.println("Key = " + currentRow.toString() + " Cf = "
> + currentColf.toString() + " Cq = " + currentColq.toString()  + " val = " +
> v.toString());
>
>             if (currentColf.toString().equals("fx")){
>                 System.out.println("Updating fx" );
>                 this.key = new Key(currentColq.toArray(),
> currentColf.toArray(), currentColf.toArray(), new byte[0], ts);
>                 this.value = new Value (v.getBytes(UTF_8));
>             }
>             else{
>                 System.out.println("Updating other" );
>                 this.key = new Key(currentColq.toArray(),
> "r".getBytes(UTF_8), currentRow.toArray(), new byte[0], ts);
>                 this.value = new Value (v.getBytes(UTF_8));
>                 System.out.println(this.key.toString());
>             }
>
>
>
>             source.next();
>
>
>           } else {
>             this.key = null;
>             this.value = null;
>       }
>   }
>
>   @Override
>   public void seek(Range range, Collection<ByteSequence> columnFamilies,
> boolean inclusive) throws IOException {
>     source.seek(range, columnFamilies, inclusive);
>     next();
>   }
>
>   @Override
>   public Key getTopKey() {
>     return key;
>   }
>
>   @Override
>   public Value getTopValue() {
>     return value;
>   }
>
>   @Override
>   public SortedKeyValueIterator<Key,Value> deepCopy(IteratorEnvironment
> env) {
>     return null;
>   }
>
>
> }
>
>
> Best regards,
> Yamini Joshi
>
>
>

Mime
View raw message