hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chia-Ping Tsai"<chia7...@apache.org>
Subject Re: [DISCUSS] Move Type out of KeyValue
Date Fri, 29 Sep 2017 05:20:01 GMT
Thanks for all comment.

The problem i want to resolve is the valid code should be exposed as IA.Public. Otherwise,
end user have to access the IA.Private class to build the custom cell.   

For example, I have a use case which plays a streaming role in our appliaction. It
applies the CellBuilder(HBASE-18519) to build custom cells. These cells have many same fields
so they are put in shared-memory for avoiding GC pause. Everything is wonderful. However,
we have to access the IA.Private class - KeyValue#Type - to get the valid code of Put.

I believe there are many use cases of custom cell, and consequently it is worth adding a way
to get the valid type via IA.Public class. Otherwise, it may imply that the custom cell is
based on a unstable way, because the related code can be changed at any time.
--
Chia-Ping

On 2017-09-29 00:49, Andrew Purtell <apurtell@apache.org> wrote: 
> I agree with Stack. Was typing up a reply to Anoop but let me move it down
> here.
> 
> The type code exposes some low level details of how our current stores are
> architected. But what if in the future you could swap out HStore implements
> Store with PStore implements Store, where HStore is backed by HFiles and
> PStore is backed by Parquet? Just as a hypothetical example. I know there
> would be larger issues if this were actually attempted. Bear with me. You
> can imagine some different new Store implementation that has some
> advantages but is not a design derived from the log structured merge tree
> if you like. Most values from a new Cell.Type based on KeyValue.Type
> wouldn't apply to cells from such a thing because they are particular to
> how LSMs work. I'm sure such a project if attempted would make a number of
> changes requiring a major version increment and low level details could be
> unwound from Cell then, but if we could avoid doing it in the first place,
> I think it would better for maintainability.
> 
> 
> On Thu, Sep 28, 2017 at 9:39 AM, Stack <stack@duboce.net> wrote:
> 
> > On Thu, Sep 28, 2017 at 2:25 AM, Chia-Ping Tsai <chia7712@apache.org>
> > wrote:
> >
> > > hi folks,
> > >
> > > User is allowed to create custom cell but the valid code of type -
> > > KeyValue#Type - is declared as IA.Private. As i see it, we should expose
> > > KeyValue#Type as Public Client. Three possible ways are shown below:
> > > 1) Change declaration of KeyValue#Type from IA.Private to IA.Public
> > > 2) Move KeyValue#Type into Cell.
> > > 3) Move KeyValue#Type to upper level
> > >
> > > Any suggestions?
> > >
> > >
> > What is the problem that we are trying to solve Chia-Ping? You want to make
> > Cells of a new Type?
> >
> > My first reaction is that KV#Type is particular to the KV implementation.
> > Any new Cell implementation should not have to adopt the KeyValue typing
> > mechanism.
> >
> > S
> >
> >
> >
> >
> > > --
> > > Chia-Ping
> > >
> > >
> >
> 
> 
> 
> -- 
> Best regards,
> Andrew
> 
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>    - A23, Crosstalk
> 

Mime
View raw message