hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Access cell tags from HBase shell
Date Fri, 06 May 2016 05:39:51 GMT
>>Then how can I revert them to a recognizable form?
I think for that I don't think we have any APIs. May be for now you may
have to parse the tag expression and map every oridinal to the visibility
label string.

Regards
Ram

On Thu, May 5, 2016 at 9:09 PM, <benedict.whittamsmith@thomsonreuters.com>
wrote:

> Yes, cell.getTagsLength is != 0. Good.
>
>
>
> I’m running HBase locally in pseudo distributed mode so I only have one
> hbase-site.xml to edit (?). But I must force the issue in the java client
> by setting the configuration programmatically so:
>
>
>
>
> config.set("hbase.client.rpc.codec", "org.apache.hadoop.hbase.codec.KeyValueCodecWithTags");
>
>
>
> (If I don’t do this I get cell.getTagsLength = 0 even though the property
> is set in hbase-site.xml)
>
>
>
> I guess I could now use cell.getTagsArray to “see” the tags. (Though maybe
> both these steps rule out the use of the shell.)
>
>
>
> Then how can I revert them to a recognizable form?
>
>
>
> Thanks for your help,
>
>
>
> Ben
>
>
>
>
>
> *From:* ramkrishna vasudevan [mailto:ramkrishna.s.vasudevan@gmail.com]
> *Sent:* 05 May 2016 08:00
> *To:* Whittam Smith, Benedict (TR Technology & Ops)
> *Cc:* Anoop John; user@hbase.apache.org
>
> *Subject:* Re: Access cell tags from HBase shell
>
>
>
> Have you set the property in the hbase-site.xml on the client side also?
>
> Can you try to retrieve per cell from the REsult and check if the
> cell.getTagsLength is != 0?
>
>
>
> Regards
>
> Ram
>
>
>
> On Thu, May 5, 2016 at 1:41 AM, <benedict.whittamsmith@thomsonreuters.com>
> wrote:
>
> Thanks guys. I've given it a go with the Java client but without success.
>
> I assume I set this property in hbase-site.xml:
>
> <property>
>       <name>hbase.client.rpc.codec</name>
>       <value>org.apache.hadoop.hbase.codec.KeyValueCodecWithTags</value>
> </property>
>
> Then I successfully set the labels. But the scan returns nothing new:
>
> scan = new Scan();
> ResultScanner rs = table.getScanner(scan);
> try {
>   for (Result r = rs.next(); r != null; r = rs.next()) {
>   System.out.println(r);
>   } ...
>
> Where am I doing wrong/not noticing?
>
> Many thanks,
>
> Ben
>
> > -----Original Message-----
> > From: Anoop John [mailto:anoop.hbase@gmail.com]
>
> > Sent: 03 May 2016 18:35
> > To: user@hbase.apache.org
> > Subject: Re: Access cell tags from HBase shell
> >
> > You have to config the Codec KeyValueCodecWithTags at client side.
> > Server also will use same Codec to talk with this client.  Ya just
> > check with a java client 1st and then experiment with shell.
> >
> > -Anoop-
> >
> > On Tue, May 3, 2016 at 9:43 PM, ramkrishna vasudevan
> > <ramkrishna.s.vasudevan@gmail.com> wrote:
> > > Hi Benedict
> > >
> > > As super user you should be able to get back the tags as ordinals and
> > make
> > > sure you set the codec KeyValueCodecWithTags.
> > > But I am not sure if it is possible to do it from the hBase shell.
> > Can you
> > > try from a java client ?
> > >
> > > I did not do the hands on of late on this but I can do it if you face
> > any
> > > difficulties and revert back if needed.
> > >
> > > Regards
> > > Ram
> > >
> > > On Tue, May 3, 2016 at 8:19 PM,
> > <benedict.whittamsmith@thomsonreuters.com>
> > > wrote:
> > >
> > >> Hi Anoop,
> > >>
> > >> Can I still get the labels back (as ordinals, as a super user, and
> > using
> > >> the KeyValueCodecWithTags codec) using the HBase shell?
> > >>
> > >> If so, what are the steps I need to take (i.e. doesn't seem to be
> > working
> > >> for me, but then I've likely made a mistake setting the codec).
> > >>
> > >> Thanks,
> > >>
> > >> Ben
> > >>
> > >> > -----Original Message-----
> > >> > From: Anoop John [mailto:anoop.hbase@gmail.com]
> > >> > Sent: 15 September 2015 14:28
> > >> > To: user@hbase.apache.org
> > >> > Subject: Re: Access cell tags from HBase shell
> > >> >
> > >> > We are not returning back the cell labels back to client.  So what
> > I
> > >> > will
> > >> > recommend you to test is by having a predicate in scan and test
> > you see
> > >> > only the relevant data back.
> > >> > But there is way to return cells (all*) with out any vis check and
> > >> > cells in
> > >> > client will have the vis label tag also in it. This is by issuing
> > the
> > >> > scan
> > >> > as a super user.  And also set the codec as KeyValueCodecWithTags.
> > >> > But one thing we wont be storing the vis label with Cells as
> > string..
> > >> > We
> > >> > will optimize..  We will store them as ordinals and & and |
> > condition
> > >> > also
> > >> > we will optimize. So even if you read back the vis label tags back
> > in
> > >> > client it will be hard to parse it and understand..  Any thing
> > more you
> > >> > would like to know, pls let me know..  Will be happy to help.
> > >> >
> > >> > BTW  once you test and if start to use the feature pls let me
> > know..
> > >> > Will
> > >> > be great to hear the usage cases and feedback.
> > >> >
> > >> > -Anoop-
> > >> >
> > >> >
> > >> > On Fri, Sep 11, 2015 at 5:35 AM, Suresh Subbiah
> > >> > <suresh.subbiah60@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi Anoop,
> > >> > >
> > >> > > Thank you very much for the offer to help.
> > >> > >
> > >> > > I have been thinking some more about what it is that we need
to
> > do
> > >> > and have
> > >> > > realized that we don't need custom cell tags.
> > >> > > We we will only be using visibility labels. This is basically
> > for
> > >> > testing
> > >> > > purpose and to understand exactly how data looks.
> > >> > >
> > >> > > How do we see visibility labels that are applied to a particular
> > >> > cell? For
> > >> > > ex, if we want to know all the labels that have been applied
to
> > >> > > all cells, how do we do that? Or can that only be done by
> > applying a
> > >> > > predicate and then check to see if the pred passes?
> > >> > >
> > >> > > Is there a way to pass visibility labels to client is a test
> > mode ?
> > >> > >
> > >> > > Thanks
> > >> > > Suresh
> > >> > >
> > >> > >
> > >> > > On Thu, Sep 3, 2015 at 11:07 PM, Anoop John
> > <anoop.hbase@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > > Hi Suresh
> > >> > > >                 You wan to use ur own custom tags with cells?
> > The
> > >> > > features
> > >> > > > like cell level vis labels etc are also implemented by storing
> > them
> > >> > as
> > >> > > cell
> > >> > > > tags.  Yes as others said, the tags is by default a server
> > only
> > >> > thing.
> > >> > > > Means you can not pass tags from/to client along with cells.
> > There
> > >> > is
> > >> > > some
> > >> > > > security reasons why we had opted this path.  And there
were
> > no
> > >> > custom
> > >> > > tag
> > >> > > > needs by then. Pls let us know what you want to achieve.
> > There is
> > >> > ways
> > >> > > to
> > >> > > > pass tags to/from client. I can help you.
> > >> > > >
> > >> > > > -Anoop-
> > >> > > >
> > >> > > >
> > >> > > > On Tue, Sep 1, 2015 at 4:29 AM, Jerry He <jerryjch@gmail.com>
> > >> > wrote:
> > >> > > >
> > >> > > > > Hi, Suresh
> > >> > > > >
> > >> > > > > In you Java client program, you can 'label' the cells
in
> > your
> > >> > PUT.  You
> > >> > > > can
> > >> > > > > ask which labeled cells to be returned in your Get
and Scan,
> > but
> > >> > the
> > >> > > > labels
> > >> > > > > are not returned with the cells.
> > >> > > > > Yes, "labels on cells are only interpreted server side"
> > >> > > > >
> > >> > > > >
> > >> > > > > Jerry
> > >> > > > >
> > >> > > > > On Mon, Aug 31, 2015 at 1:27 PM, Suresh Subbiah <
> > >> > > > > suresh.subbiah60@gmail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Thank you very much Ted, Jean-Marc.
> > >> > > > > >
> > >> > > > > > I see that slide 4 in
> > >> > > > > > https://urldefense.proofpoint.com/v2/url?u=http-
> > >> > 3A__www.slideshare.net_HBaseCon_features-2Dsession-
> > >> > 2D2&d=CwIBaQ&c=4ZIZThykDLcoWk-
> > >> >
> > GVjSLm9hvvvzvGv0FLoWSRuCSs5Q&r=GQ6xvz2BG1vCgiGGeLHdL1qJLbLUqYG6W19eFBlz
> > >> > nzDGH3wjzyriGVJemENTKsgx&m=sLIg484DFLi0oSu5ylkGuIuB-
> > >> > re6sXaYY0fb9BreY2o&s=BhpulFRnZ_JNgAOPjb_MFtv0rnH9yaNXtQZE_g7y-
> > 28&e=
> > >> > states
> > >> > > > > > that "cells are only interpreted server side"
> > >> > > > > > However https://urldefense.proofpoint.com/v2/url?u=https-
> > >> > 3A__issues.apache.org_jira_browse_HBASE-
> > >> > 2D9056&d=CwIBaQ&c=4ZIZThykDLcoWk-
> > >> >
> > GVjSLm9hvvvzvGv0FLoWSRuCSs5Q&r=GQ6xvz2BG1vCgiGGeLHdL1qJLbLUqYG6W19eFBlz
> > >> > nzDGH3wjzyriGVJemENTKsgx&m=sLIg484DFLi0oSu5ylkGuIuB-
> > >> >
> > re6sXaYY0fb9BreY2o&s=u_ISxz2OpkFA6Y5cYXGcQqpG24S54zDi1WhuHfbq18A&e=  &
> > >> > > > > > https://urldefense.proofpoint.com/v2/url?u=https-
> > >> > 3A__issues.apache.org_jira_browse_HBASE-
> > >> > 2D9884&d=CwIBaQ&c=4ZIZThykDLcoWk-
> > >> >
> > GVjSLm9hvvvzvGv0FLoWSRuCSs5Q&r=GQ6xvz2BG1vCgiGGeLHdL1qJLbLUqYG6W19eFBlz
> > >> > nzDGH3wjzyriGVJemENTKsgx&m=sLIg484DFLi0oSu5ylkGuIuB-
> > >> >
> > re6sXaYY0fb9BreY2o&s=eiGA0TFAu3Or3cbLjX0wQJjwUGxldTOwU00qfZOgO8s&e=
> > >> > (likely others)
> > >> > > seem
> > >> > > > to
> > >> > > > > > indicate that tag values can be accessed/mutated
from
> > HBase
> > >> > client
> > >> > > > > > programs.
> > >> > > > > > I assumed that this mean that code has evolved
since the
> > slides
> > >> > were
> > >> > > > > > created. Or I could just be misunderstanding what
the
> > slide
> > >> > meant.
> > >> > > > > >
> > >> > > > > > I am hoping to write some kind of small java program
to do
> > this
> > >> > now
> > >> > > > > (since
> > >> > > > > > I am not familiar with REST or Thrift). Do you
think that
> > is a
> > >> > > > reasonable
> > >> > > > > > approach?
> > >> > > > > >
> > >> > > > > > Thanks
> > >> > > > > > Suresh
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Mon, Aug 31, 2015 at 2:59 PM, Ted Yu
> > <yuzhihong@gmail.com>
> > >> > wrote:
> > >> > > > > >
> > >> > > > > > > bq. retrieve the list of labels for a given
cell
> > >> > > > > > >
> > >> > > > > > > You're right, J-M.
> > >> > > > > > > There is no facility for the above in hbase
shell.
> > >> > > > > > >
> > >> > > > > > > On Mon, Aug 31, 2015 at 12:55 PM, Jean-Marc
Spaggiari <
> > >> > > > > > > jean-marc@spaggiari.org> wrote:
> > >> > > > > > >
> > >> > > > > > > > But I don't think you can retrieve the
list of labels
> > for a
> > >> > given
> > >> > > > > cell,
> > >> > > > > > > > right? Cells are only interpreted server
side and are
> > not
> > >> > > returned
> > >> > > > on
> > >> > > > > > the
> > >> > > > > > > > client side...
> > >> > > > > > > >
> > >> > > > > > > > 2015-08-31 15:52 GMT-04:00 Ted Yu
> > <yuzhihong@gmail.com>:
> > >> > > > > > > >
> > >> > > > > > > > > From the help message of put command,
you can see
> > the
> > >> > > following:
> > >> > > > > > > > >
> > >> > > > > > > > >   hbase> put 't1', 'r1', 'c1',
'value', ts1,
> > >> > > > > > > > {VISIBILITY=>'PRIVATE|SECRET'}
> > >> > > > > > > > >
> > >> > > > > > > > > Here is the group of related commands:
> > >> > > > > > > > >
> > >> > > > > > > > >   'visibility labels',
> > >> > > > > > > > >   :full_name => 'VISIBILITY
LABEL TOOLS',
> > >> > > > > > > > >   :comment => "NOTE: Above commands
are only
> > applicable
> > >> > if
> > >> > > > running
> > >> > > > > > with
> > >> > > > > > > > the
> > >> > > > > > > > > VisibilityController coprocessor",
> > >> > > > > > > > >   :commands => %w[
> > >> > > > > > > > >     add_labels
> > >> > > > > > > > >     list_labels
> > >> > > > > > > > >     set_auths
> > >> > > > > > > > >     get_auths
> > >> > > > > > > > >     clear_auths
> > >> > > > > > > > >     set_visibility
> > >> > > > > > > > >
> > >> > > > > > > > > FYI
> > >> > > > > > > > >
> > >> > > > > > > > > On Mon, Aug 31, 2015 at 12:25 PM,
Suresh Subbiah <
> > >> > > > > > > > > suresh.subbiah60@gmail.com
> > >> > > > > > > > > > wrote:
> > >> > > > > > > > >
> > >> > > > > > > > > > Hi,
> > >> > > > > > > > > >
> > >> > > > > > > > > > I am using HBase 0.98 with
HFiles of majorVersion
> > 3.
> > >> > > > > > > > > > Is it possible to get and
set cell tags from the
> > HBase
> > >> > shell
> > >> > > > > > utility?
> > >> > > > > > > > > > I have been looking at various
JIRAs and see some
> > >> > example
> > >> > > > client
> > >> > > > > > > > testing
> > >> > > > > > > > > > programs that seem to do this.
However it will be
> > >> > simpler if
> > >> > > I
> > >> > > > > > could
> > >> > > > > > > > > > get/set cell tags from HBase
shell.
> > >> > > > > > > > > >
> > >> > > > > > > > > > Thank you for your help.
> > >> > > > > > > > > >
> > >> > > > > > > > > > Best regards,
> > >> > > > > > > > > > Suresh
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message