accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wells <awe...@clearedgeit.com>
Subject Re: Change column family
Date Wed, 27 May 2015 12:06:49 GMT
Looks like you want to override these methods:

protected  Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceColumnFamily
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceColumnFamily%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text%29>*
(Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
org.apache.hadoop.io.Text newColFam)
          Make a new key with all parts (including delete flag) coming from
originalKey but use newColFam as the column family.  protected  Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceColumnQualifier
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceColumnQualifier%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text%29>*
(Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
org.apache.hadoop.io.Text newColQual)
          Make a new key with all parts (including delete flag) coming from
originalKey but use newColQual as the column qualifier.  protected  Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceColumnVisibility
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceColumnVisibility%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text%29>*
(Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
org.apache.hadoop.io.Text newColVis)
          Make a new key with all parts (including delete flag) coming from
originalKey but use newColVis as the column visibility.  protected  Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceKeyParts
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceKeyParts%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text,%20org.apache.hadoop.io.Text%29>*
(Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
org.apache.hadoop.io.Text newColQual, org.apache.hadoop.io.Text newColVis)
          Make a new key with a column qualifier, and column
visibility.  protected
 Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
*replaceKeyParts
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/iterators/user/TransformingIterator.html#replaceKeyParts%28org.apache.accumulo.core.data.Key,%20org.apache.hadoop.io.Text,%20org.apache.hadoop.io.Text,%20org.apache.hadoop.io.Text%29>*
(Key
<http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/data/Key.html>
originalKey,
org.apache.hadoop.io.Text newColFam, org.apache.hadoop.io.Text newColQual,
org.apache.hadoop.io.Text newColVis)
          Make a new key with a column family, column qualifier, and column
visibility.



On Wed, May 27, 2015 at 7:40 AM, shweta.agrawal <shweta.agrawal@orkash.com>
wrote:

>  Thanks for all the suggestion.
>
> I read about TransformingIterator and started implementing it,  I extended
> this class and tried to override its abstract method. But I am not able to
> get where and what to write to change column family?
>
> So please provide your suggestions.
>
> Thanks
> Shweta
>
>
>
> On Tuesday 26 May 2015 08:33 PM, Adam Fuchs wrote:
>
> This can also be done with a row-doesn't-fit-into-memory constraint. You
> won't need to hold the second column in-memory if your iterator tree deep
> copies, filters, transforms and merges. Exhibit A:
>
>  [HeapIterator-derivative]
>    |_________________________
>    |                         \
> [transform-graph1-to-graph2]  \
>    |                           \
> [column-family-graph1]     [all-but-column-family-graph1]
>
>  With this design, you can subclass the HeapIterator, deep copy the
> source in the init method, wrap one in a custom transform iterator, and
> create a appropriate seek method. This is probably more on the advanced
> side of Accumulo programming, but can be done.
>
>  Adam
>
>
> On Tue, May 26, 2015 at 8:59 AM, Eric Newton <eric.newton@gmail.com>
> wrote:
>
>>        Short answer: no.
>>
>>  Long answer: maybe.
>>
>>  You can write an iterator which will transform:
>>
>> row, cf1, cq, vis -> value
>>
>>  into:
>>
>> row, cf2, cq, vis -> value
>>
>>  And if you can do this while maintaining sort order, you can get your
>> new ColumnFamily transformed during scans and compactions.
>>
>>  But this bit about maintaining the sort order is more complex than it
>> sounds.
>>
>>  If you have the following:
>>
>>  row, a, cq, vis -> value
>> row, aa, cq, vis -> value
>>
>>
>>  And you want to transform cf "a" into cf "b":
>>
>>  row, aa, cq, vis -> value
>> row, b, cq, vis -> value
>>
>>
>>  Your iterator needs to hold the second column in memory, after
>> transforming the first column.  Tablet server memory for holding Key/Values
>> is not infinite.
>>
>>  -Eric
>>
>> On Tue, May 26, 2015 at 8:44 AM, shweta.agrawal <
>> shweta.agrawal@orkash.com> wrote:
>>
>>> Hi,
>>>
>>> I want to ask, is it possible in accumulo to change the column family
>>> without changing the whole data.
>>>
>>> Suppose my column family is graph1, now i want to rename this column
>>> family as graph2.
>>> Is it possible?
>>>
>>> Thanks
>>> Shweta
>>>
>>
>>
>
>


-- 
*Andrew George Wells*
*Software Engineer*
*awells@clearedgeit.com <awells@clearedgeit.com>*

Mime
View raw message