accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wells <awe...@clearedgeit.com>
Subject Re: Change column family
Date Wed, 27 May 2015 12:08:36 GMT
to implement that iterator.

looks like you will only need to override replaceColumnFamily

and this looks to return the new ColumnFamily via the argument. So
manipulate the Text object provided.

On Wed, May 27, 2015 at 8:06 AM, Andrew Wells <awells@clearedgeit.com>
wrote:

> Looks like you want to override these methods:
>
> protected  Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceColumnFamily
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceColumnFamily%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text%29>*
> (Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
> org.apache.hadoop.io.Text newColFam)
>           Make a new key with all parts (including delete flag) coming
> from originalKey but use newColFam as the column family.  protected  Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceColumnQualifier
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceColumnQualifier%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text%29>*
> (Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
> org.apache.hadoop.io.Text newColQual)
>           Make a new key with all parts (including delete flag) coming
> from originalKey but use newColQual as the column qualifier.  protected
> Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceColumnVisibility
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceColumnVisibility%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text%29>*
> (Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
> org.apache.hadoop.io.Text newColVis)
>           Make a new key with all parts (including delete flag) coming
> from originalKey but use newColVis as the column visibility.  protected
> Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceKeyParts
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceKeyParts%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text,%20org.apache.hadoop.io.Text%29>*
> (Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
> org.apache.hadoop.io.Text newColQual, org.apache.hadoop.io.Text newColVis)
>           Make a new key with a column qualifier, and column visibility.  protected
>  Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceKeyParts
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceKeyParts%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text,%20org.apache.hadoop.io.Text,%20org.apache.hadoop.io.Text%29>*
> (Key
> <http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
> org.apache.hadoop.io.Text newColFam, org.apache.hadoop.io.Text newColQual,
> org.apache.hadoop.io.Text newColVis)
>           Make a new key with a column family, column qualifier, and
> column visibility.
>
>
>
> On Wed, May 27, 2015 at 7:40 AM, shweta.agrawal <shweta.agrawal@orkash.com
> > wrote:
>
>>  Thanks for all the suggestion.
>>
>> I read about TransformingIterator and started implementing it,  I
>> extended this class and tried to override its abstract method. But I am not
>> able to get where and what to write to change column family?
>>
>> So please provide your suggestions.
>>
>> Thanks
>> Shweta
>>
>>
>>
>> On Tuesday 26 May 2015 08:33 PM, Adam Fuchs wrote:
>>
>> This can also be done with a row-doesn't-fit-into-memory constraint. You
>> won't need to hold the second column in-memory if your iterator tree deep
>> copies, filters, transforms and merges. Exhibit A:
>>
>>  [HeapIterator-derivative]
>>    |_________________________
>>    |                         \
>> [transform-graph1-to-graph2]  \
>>    |                           \
>> [column-family-graph1]     [all-but-column-family-graph1]
>>
>>  With this design, you can subclass the HeapIterator, deep copy the
>> source in the init method, wrap one in a custom transform iterator, and
>> create a appropriate seek method. This is probably more on the advanced
>> side of Accumulo programming, but can be done.
>>
>>  Adam
>>
>>
>> On Tue, May 26, 2015 at 8:59 AM, Eric Newton <eric.newton@gmail.com>
>> wrote:
>>
>>>        Short answer: no.
>>>
>>>  Long answer: maybe.
>>>
>>>  You can write an iterator which will transform:
>>>
>>> row, cf1, cq, vis -> value
>>>
>>>  into:
>>>
>>> row, cf2, cq, vis -> value
>>>
>>>  And if you can do this while maintaining sort order, you can get your
>>> new ColumnFamily transformed during scans and compactions.
>>>
>>>  But this bit about maintaining the sort order is more complex than it
>>> sounds.
>>>
>>>  If you have the following:
>>>
>>>  row, a, cq, vis -> value
>>> row, aa, cq, vis -> value
>>>
>>>
>>>  And you want to transform cf "a" into cf "b":
>>>
>>>  row, aa, cq, vis -> value
>>> row, b, cq, vis -> value
>>>
>>>
>>>  Your iterator needs to hold the second column in memory, after
>>> transforming the first column.  Tablet server memory for holding Key/Values
>>> is not infinite.
>>>
>>>  -Eric
>>>
>>> On Tue, May 26, 2015 at 8:44 AM, shweta.agrawal <
>>> shweta.agrawal@orkash.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to ask, is it possible in accumulo to change the column family
>>>> without changing the whole data.
>>>>
>>>> Suppose my column family is graph1, now i want to rename this column
>>>> family as graph2.
>>>> Is it possible?
>>>>
>>>> Thanks
>>>> Shweta
>>>>
>>>
>>>
>>
>>
>
>
> --
> *Andrew George Wells*
> *Software Engineer*
> *awells@clearedgeit.com <awells@clearedgeit.com>*
>
>


-- 
*Andrew George Wells*
*Software Engineer*
*awells@clearedgeit.com <awells@clearedgeit.com>*

Mime
View raw message