hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <enis....@gmail.com>
Subject Re: State of protobufs in hbase-0.96
Date Wed, 28 Nov 2012 00:10:49 GMT
I had opened https://issues.apache.org/jira/browse/HBASE-7201 for some of
the discussions for HFile/Hlog. Converting HFile and HLog meta fields are
relatively easy, and if we are still going to keep KV as writable HFile is
mostly covered. However, I think we should convert HLogKey and WALEdit as
well.


On Tue, Nov 27, 2012 at 3:26 PM, Andrew Purtell <apurtell@apache.org> wrote:

> KeyValue is a special case. They're not really Writables either, in that
> readFields() and writeFields() are not used half the time for
> marshalling/unmarshalling.
>
>
> On Tue, Nov 27, 2012 at 1:46 PM, Jimmy Xiang <jxiang@cloudera.com> wrote:
>
> > We can't completely purge Writables since we are still using KV in HFile.
> >
> > Are we going to convert the file format too?  If so, we need some
> > migration strategy.
> >
> > Thanks,
> > Jimmy
> >
> > On Tue, Nov 27, 2012 at 1:41 PM, Gary Helmling <ghelmling@gmail.com>
> > wrote:
> > > Yep, the initial conversion work (HBASE-5448) only deprecated
> > > CoprocessorProtocol, but in subsequent discussion I think the universal
> > > consensus was to remove it completely for 0.96 and go PB-only.
>  Otherwise
> > > we can't completely purge Writables for 0.96.
> > >
> > >
> > >
> > > On Tue, Nov 27, 2012 at 1:37 PM, Andrew Purtell <apurtell@apache.org>
> > wrote:
> > >
> > >> We agreed to remove Writables from coprocessors and go PB-only for
> 0.96.
> > >> This way, to avoid building up any transitional cruft.
> > >>
> > >>
> > >> On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lhofhansl@yahoo.com>
> > >> wrote:
> > >>
> > >> > I thought for coprocessors we had decided to only deprecate the old
> > >> > protocol. I'm +1 removing it, though, because it let's us
> potentially
> > get
> > >> > rid of all Writables.
> > >> >
> > >> >
> > >> >
> > >> > ________________________________
> > >> >  From: Gary Helmling <ghelmling@gmail.com>
> > >> > To: "dev@hbase.apache.org" <dev@hbase.apache.org>; lars hofhansl
<
> > >> > lhofhansl@yahoo.com>
> > >> > Sent: Tuesday, November 27, 2012 11:45 AM
> > >> > Subject: Re: State of protobufs in hbase-0.96
> > >> >
> > >> >
> > >> > For coprocessor endpoints, we have
> > >> > https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella
> issue
> > >> for
> > >> > removing the Writable based CoprocessorProtocol support.
> > >> >
> > >> > There are still a few subtasks to work through to get there, but
> once
> > >> done
> > >> > we can pull out CoprocessorProtocol, Exec, ExecResult,
> > >> > HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
> > >> > supporting code.
> > >> >
> > >> >
> > >> >
> > >> > On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <
> lhofhansl@yahoo.com>
> > >> > wrote:
> > >> >
> > >> > HBASE-7215 has a rant about the current state of the protobuf work
> in
> > >> > 0.96... Continuing here...
> > >> > >
> > >> > >HBASE-7215 came about because I simply wanted to add a field to
> > Mutation
> > >> > (Put, Delete, etc) for HBASE-5954.
> > >> > >
> > >> > >Should be easy now with protobufs, right? Nope!
> > >> > >
> > >> > >- Both Put and Delete (and Result, Action, MultiAction,
> > MultiResponse,
> > >> > MultiPut, MultiResponse, etc, etc, etc) are still implementing
> > Writable
> > >> and
> > >> > are still used that way.
> > >> > >- After I dug in I found that Writable is literally all over the
> > place
> > >> > still.
> > >> > >- In some cases we even serialize an Object as Writable inside
a
> > >> protobuf
> > >> > message.
> > >> > >- HBaseObjectWritable is still around and still used all over
the
> > place
> > >> > >
> > >> > >So what we have now has Writables and Protobuf code, worse than
it
> > was
> > >> > before (lots of extra code, two ways to serialize stuff, and still
> no
> > >> wire
> > >> > compatibility).
> > >> > >
> > >> > >HBASE-7215 will fix some of the stuff, but there's a lot more
to
> do.
> > >> > >
> > >> > >-- Lars
> > >> > >
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>    - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message