accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yamini Joshi <yamini.1...@gmail.com>
Subject Re: Modify Keys within iterator
Date Fri, 30 Sep 2016 18:11:54 GMT
Alright. Thanks :)

Best regards,
Yamini Joshi

On Fri, Sep 30, 2016 at 1:10 PM, Brian Loss <bfloss@praxiseng.com> wrote:

> That’s true for the row. For the other parts of the key, it can be done
> under the right circumstances.
>
> On Sep 30, 2016, at 2:05 PM, Yamini Joshi <yamini.1691@gmail.com> wrote:
>
> If I give it an empty range, it gives me the output of simple scan(without
> the iterator applied even though the iterator is working). I guess it's bad
> to modify keys within an iterator.
>
> Best regards,
> Yamini Joshi
>
> On Fri, Sep 30, 2016 at 12:51 PM, Dan Blum <dblum@bbn.com> wrote:
>
>> What happens if you just use an empty range?
>>
>>
>>
>> What I suspect is happening is that the scanner code sees a key beyond
>> the range’s end key and stops. In general you can’t transform rows in keys
>> for this reason, and you might have issues even if you don’t transform the
>> rows if the keys end up out of order – see the comments in
>> TransformingIterator.
>>
>>
>>
>> *From:* Yamini Joshi [mailto:yamini.1691@gmail.com]
>> *Sent:* Friday, September 30, 2016 1:37 PM
>> *To:* user@accumulo.apache.org
>> *Subject:* Re: Modify Keys within iterator
>>
>>
>>
>> I am using pyaccumulo. Here's the code snippet:
>>
>> rowIds=['r2','r10']
>>
>> hashFilter = KeyModifyIterator(priority=10)
>> iterator.append(hashFilter)
>>
>> *for *entry *in *self.dbconn.batch_scan(table , scanranges=(Range(srow=row, erow=row)
*for *row *in *rowIDs),iterators=[hashFilter]):
>>     *print *entry
>>
>>
>>
>>
>> Best regards,
>> Yamini Joshi
>>
>>
>>
>> On Fri, Sep 30, 2016 at 12:31 PM, Dan Blum <dblum@bbn.com> wrote:
>>
>> What code are you using to test the iterator, where you see no output?
>>
>>
>>
>> *From:* Yamini Joshi [mailto:yamini.1691@gmail.com]
>> *Sent:* Friday, September 30, 2016 1:26 PM
>> *To:* user@accumulo.apache.org
>> *Subject:* Modify Keys within iterator
>>
>>
>>
>> Hello Everyone!
>>
>> I am trying to write an iterator to modify keys within a table (at scan).
>> My use case is to select a few records that match a certain criterion and
>> then modify them within the iterator(using the following class) for some
>> other succeeding iterator/combiner. The problem is that this iterator does
>> return any records/keys. I added some primitive prints and found that the
>> keys (this.key) is changed but the output of iterator is nothing. I'd
>> appreciate if someone could give me any insight. I'm sure I'm making a
>> teeny tiny mistake somewhere.
>>
>> Schema:
>>
>>          row                   colF
>> colQ                 ts                Val
>>
>> I/P:    r_1                   f                             f_1
>>                            v1
>>
>>          r_1                   fx                            f_1
>>                             v1
>>
>> O/P:  f_1                   r
>> r_1                                      v1
>>
>>          f_1                   fx
>> fx                                        v1
>>
>>
>>
>>
>>
>> public class KeyModifyIterator implements SortedKeyValueIterator<Key,Value>
>> {
>>
>>   private SortedKeyValueIterator<Key,Value> source;
>>   private Key key;
>>   private Value value;
>>
>>   @Override
>>   public void init(SortedKeyValueIterator<Key,Value> source,
>> Map<String,String> options, IteratorEnvironment env) throws IOException {
>>     this.source = source;
>>   }
>>
>>   @Override
>>   public boolean hasTop() {
>>     return key != null;
>>   }
>>
>>   @Override
>>   public void next() throws IOException {
>>         if (source.hasTop()) {
>>             ByteSequence currentRow = source.getTopKey().getRowData();
>>             ByteSequence currentColf = source.getTopKey().getColumnFa
>> milyData();
>>             ByteSequence currentColq = source.getTopKey().getColumnQu
>> alifierData();
>>             long ts = source.getTopKey().getTimestamp();
>>             String v = source.getTopValue().toString();
>>             System.out.println("Key = " + currentRow.toString() + " Cf =
>> " + currentColf.toString() + " Cq = " + currentColq.toString()  + " val = "
>> + v.toString());
>>
>>             if (currentColf.toString().equals("fx")){
>>                 System.out.println("Updating fx" );
>>                 this.key = new Key(currentColq.toArray(),
>> currentColf.toArray(), currentColf.toArray(), new byte[0], ts);
>>                 this.value = new Value (v.getBytes(UTF_8));
>>             }
>>             else{
>>                 System.out.println("Updating other" );
>>                 this.key = new Key(currentColq.toArray(),
>> "r".getBytes(UTF_8), currentRow.toArray(), new byte[0], ts);
>>                 this.value = new Value (v.getBytes(UTF_8));
>>                 System.out.println(this.key.toString());
>>             }
>>
>>
>>
>>             source.next();
>>
>>
>>           } else {
>>             this.key = null;
>>             this.value = null;
>>       }
>>   }
>>
>>   @Override
>>   public void seek(Range range, Collection<ByteSequence> columnFamilies,
>> boolean inclusive) throws IOException {
>>     source.seek(range, columnFamilies, inclusive);
>>     next();
>>   }
>>
>>   @Override
>>   public Key getTopKey() {
>>     return key;
>>   }
>>
>>   @Override
>>   public Value getTopValue() {
>>     return value;
>>   }
>>
>>   @Override
>>   public SortedKeyValueIterator<Key,Value> deepCopy(IteratorEnvironment
>> env) {
>>     return null;
>>   }
>>
>>
>> }
>>
>>
>> Best regards,
>> Yamini Joshi
>>
>>
>>
>
>
>

Mime
View raw message