hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vrushali Channapattan <vrushal...@gmail.com>
Subject Re: hbase (coprocessors & cell tags) used in hadoop-yarn
Date Mon, 04 Jan 2016 20:04:26 GMT
I see, thanks Anoop. We wanted to use cell tags for indicating the context
of information in cells in that cells for aggregation purpose. It is
referred to only in the coprocessor. We also use in the flush/compaction
processing to decide which cells to discard/what info to keep.

I will be on the lookout for Tag interface changes.

On Thu, Dec 24, 2015 at 7:19 AM, Anoop John <anoop.hbase@gmail.com> wrote:

> I can see in the patches that you are trying to use Cell creation with Tags
> and use of Tag APIs..  Only concern is Tag is Private audience marked. It
> was created to support per cell ACL/ visibility etc.
>
> As part of off heaping effort, we are planning to make some changes to Tag
> APIs.. (To make it interface impl itself).. This will happen in HBase
> trunk..   So later when you move to newer version need to change it.
>
> -Anoop-
>
>
> On Tue, Dec 22, 2015 at 12:37 PM, Vrushali Channapattan <
> vrushali.c@gmail.com> wrote:
>
> > A group of us in the hadoop community are working on Yarn's next gen
> > timeline service component
> https://issues.apache.org/jira/browse/YARN-2928
> >
> > that will be storing for application that runs on a hadoop cluster all of
> > the application stats, workflow metadata and container metrics
> information
> > in hbase tables (some plain hbase tables and some phoenix based ones).
> >
> > We have been thinking about validating some of the implementation
> > approaches we are taking with HBase. It would be great to get some
> feedback
> > on the code and design from the HBase dev perspective.
> >
> > Among other things, we are making use of cell tags in coprocessors for
> > summation, min and max operations on different versions of cells in a
> given
> > column during read as well flush and compaction operations.  Some
> relevant
> > subjiras that deal with hbase coprocessors
> > https://issues.apache.org/jira/browse/YARN-4062
> > https://issues.apache.org/jira/browse/YARN-3901
> >
> > We have the schema documented with example records in the code as well as
> > in pdf on the jira.
> >
> >
> >
> https://github.com/apache/hadoop/blob/feature-YARN-2928/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowRunTable.java#L34
> >
> >
> >
> https://github.com/apache/hadoop/blob/feature-YARN-2928/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/entity/EntityTable.java#L40
> >
> > Schema jira (pdf attachment that describes the schema)
> > https://issues.apache.org/jira/browse/YARN-3411
> >
> > Would appreciate any feedback/comments that you have and be glad to
> answer
> > any questions to clarify in depth further.
> >
> > thanks
> > Vrushali
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message