Mailing-List: contact cassandra-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cassandra-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of schumi.han@gmail.com
 designates 209.85.146.177 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=jR/OWx2p5y9k1VtLT7QQf49DqkjjLvGyvBDvLZZ/JeTrnJa+BVjfQEtHlvekPN/4Ed
         ax7u3JDHSaUOwDN8E7NVSOj5Yg+eC51RB96GA3H2NII0Z7j4DXhZbaiOhkxnxjRZK+hd
         yKzzvoYjmddHb00JD0r0cTCMRPKGkt1t0mbzM=
MIME-Version: 1.0
In-Reply-To: <a06de5520903281606n1b774dc8i4fb3ee475b052ce5@mail.gmail.com>
References: <e06563880903280811o6588e1edu51e8433591dc478e@mail.gmail.com>
	 <a06de5520903280934jae6b13btff59ca50e1038fd9@mail.gmail.com>
	 <e06563880903280948v3dd7bbd4t877d9eed94797926@mail.gmail.com>
	 <a06de5520903281039u47582baek63ed22d92ff6ebdf@mail.gmail.com>
	 <a06de5520903281049m71a213f4vc0d0ecce8f061eed@mail.gmail.com>
	 <37187DFB-5305-412B-A0B0-31DC7C83AE69@gmail.com>
	 <a06de5520903281606n1b774dc8i4fb3ee475b052ce5@mail.gmail.com>
Date: Sun, 29 Mar 2009 11:57:00 +0800
Message-ID: <4e777ed10903282057x2eb541e4v4af525695560a208@mail.gmail.com>
Subject: Re: Message classes
From: Zhu Han <schumi.han@gmail.com>
To: cassandra-dev@incubator.apache.org
Content-Type: multipart/alternative; boundary=001636457194416ce8046639f542

--001636457194416ce8046639f542
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Basically,  use a string to represent a verb handler in the message is not
very space efficient. UTF encoding is not very compact according to my
understanding.  But it's not a big issue at least now.

best regards,
hanzhu


On Sun, Mar 29, 2009 at 7:06 AM, Avinash Lakshman <
avinash.lakshman@gmail.com> wrote:

> For debugging purposes we dynamic register verb handlers with the servers
> to
> get certain kinds of information that would be of interest. I do not know
> how we could do the same with enums. Maybe there is some esoteric way, that
> I am not aware of, to achieve this. Basically we stick in an entry to the
> map of registered verb handlers where the key is a string.
> Avinash
>
> On Sat, Mar 28, 2009 at 11:47 AM, Jonathan ellis <jbellis@gmail.com>
> wrote:
>
> > My problem is I am severely allergic to boilerplate code. :)  But it does
> > sound like the thrift idea isn't going to work.
> >
> > What about the tangential idea of using enums for handler type instead of
> > strings?
> >
> > -Jonathan
> >
> >
> > On Mar 28, 2009, at 12:49 PM, Avinash Lakshman <
> avinash.lakshman@gmail.com>
> > wrote:
> >
> >  Given that we need super control over the serialization process in some
> >> conditions and do not any in most other situations I am not sure what is
> >> the
> >> best solution here. Maybe a hybrid and that would be crazy.
>  Fundamentally
> >> I
> >> think this is a triviality and this mechanism is really brain dead
> simple.
> >> Your point about tests is very very well taken. In our defense if we had
> >> religiously written tests I do not think we would have been where we are
> >> w.r.t to where the system is now :). We have a slew of tests that we do
> >> have
> >> but they are super well written but rather ad-hoc and I do not mean the
> >> main() that you see in most other places.
> >> Avinash
> >>
> >> On Sat, Mar 28, 2009 at 10:39 AM, Avinash Lakshman <
> >> avinash.lakshman@gmail.com> wrote:
> >>
> >>  For eg Thrift will definitely not help in the messages that we use for
> >>> the
> >>> membership protocol. Because there we need to control how big the
> >>> serialized
> >>> messages are - we make sure we serialize a part of the object such that
> >>> it
> >>> fits into an ethernet MTU packet. We do this so that we don't get
> bitten
> >>> by
> >>> UDP fragmentation. I don't think you could do operations like that in
> >>> Thrift
> >>> based serialization mechanism. We need more control over the
> >>> serialization
> >>> mechanism.
> >>>
> >>> So I don't know if this is something that is insanely important in any
> >>> capacity in my opinion. I am sure there are bunch of other reasons I
> can
> >>> come up with - we went through this exercise 2 year back. Of course if
> >>> you
> >>> want to investigate the efficacy I can't stop you from doing so :).
> >>>
> >>> Avinash
> >>>
> >>>
> >>> On Sat, Mar 28, 2009 at 9:48 AM, Jonathan Ellis <jbellis@gmail.com>
> >>> wrote:
> >>>
> >>>  One point of clarification -- I don't understand why looking up by
> >>>> string is better than using an enum, for instance.  java will autobox
> >>>> enums for use in a hashmap lookup.
> >>>>
> >>>> On Sat, Mar 28, 2009 at 10:34 AM, Avinash Lakshman
> >>>> <avinash.lakshman@gmail.com> wrote:
> >>>>
> >>>>> Why is it ad-hoc? And it uses a factory pattern and I don't think it
> >>>>>
> >>>> hard
> >>>>
> >>>>> once you get a hang of it. Consumers of the system are not even going
> >>>>> to
> >>>>> know about these details. Personally I am never a fan of fixing
> >>>>> anything
> >>>>> that is not broken - in this case it has been working really well for
> >>>>>
> >>>> us.
> >>>>
> >>>>> This is now just a matter of what one might prefer. Thrift is
> something
> >>>>>
> >>>> that
> >>>>
> >>>>> I would not like to see anywhere apart from the entry point. With
> >>>>>
> >>>> regards to
> >>>>
> >>>>> the using the string to lookup the handlers it was done because if
> you
> >>>>>
> >>>> don't
> >>>>
> >>>>> do that then you will have to resort to RPC style instead of message
> >>>>>
> >>>> passing
> >>>>
> >>>>> or find the handlers based on the kind of messages i.e if-else
> >>>>>
> >>>> branching. We
> >>>>
> >>>>> use non-blocking I/O for all our internal messaging and Thrift using
> >>>>> blocking I/O. There is big difference in throughput and also Thrift
> >>>>> non-blocking I/O from what I hear is horrendous in performance and
> >>>>> stability. My friend you don't have my vote for this :).
> >>>>> Avinash
> >>>>>
> >>>>> On Sat, Mar 28, 2009 at 8:11 AM, Jonathan Ellis <jbellis@gmail.com>
> >>>>>
> >>>> wrote:
> >>>>
> >>>>>
> >>>>>  we have a Message class that mostly represents a bunch of bytes (but
> >>>>>> sometimes does not, which in some cases causes bugs) and a bunch of
> >>>>>> other *Message classes that are not Message subclasses but generate
> >>>>>> Message objects (so you have the amusingly redundant Message message
> =
> >>>>>> readResponseMessage.makeReadResponseMessage() in places).
> >>>>>>
> >>>>>> I think we can replace these ad-hoc and tedious-to-write Message
> >>>>>> factories with generated thrift code.  Thrift is good at this and
> >>>>>> efficient (currently our message identifiers are inefficient strings
> >>>>>> like "ROW-READ-VERB-HANDLER").
> >>>>>>
> >>>>>> Any objections to investigating replacing the hand-coded messages
> with
> >>>>>> thrift?
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>
>

--001636457194416ce8046639f542--