Mailing-List: contact avro-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: avro-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <4AFB0EB4.7080701@apache.org>
References: <a3fa9dad0911102047q5a487a68n623061a616527590@mail.gmail.com>
	<4AFAE958.1000201@apache.org>
 <a3fa9dad0911110849w6b1f1ef0sc7e77a423aa79d32@mail.gmail.com>
	<4AFB0EB4.7080701@apache.org>
From: Philip Zeyliger <philip@cloudera.com>
Date: Thu, 12 Nov 2009 22:43:49 -0800
Message-ID: <15da8a100911122243g5df48930r7278ec005bfe1276@mail.gmail.com>
Subject: Re: Why Paranamer?
To: avro-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=000e0cd22d90a846a804783afcb6

--000e0cd22d90a846a804783afcb6
Content-Type: text/plain; charset=ISO-8859-1

Makes sense to me.  I think it may be useful to check in the .avpr files
that are induced on the way, to let folks start trying to use different
clients for certain operations.

-- Philip

On Wed, Nov 11, 2009 at 11:21 AM, Doug Cutting <cutting@apache.org> wrote:

> Justin Santa Barbara wrote:
>
>> What about Philip's point on existing Hadoop interfaces?  Any plans for
>> how
>> we'll generate the Protocol object for these?
>>
>
> I'm hoping to use reflection initially for this.  That's the motivation for
> my renewed interest in AVRO-80.
>
> https://issues.apache.org/jira/browse/AVRO-80
>
> My rationale is that I don't want to assume we'll move Hadoop onto Avro
> overnight.  So I'd like to move things in a way that's easy to maintain in a
> branch/patch.  If we can get reflection to work, then we only need to update
> two places per Hadoop protocol: where it calls RPC.getProxy() and
> RPC.getServer().  Then we can start looking at performance.  We cannot
> commit Avro-based Hadoop RPC until performance is adequate, and we don't
> want to have to maintain a patch that changes many central data structures
> in Hadoop while we're testing and improving performance, since that might
> take time.
>
> Once we've committed Hadoop to using Avro, then we can consider,
> protocol-by-protocol, replacing Hadoop's Writable objects with Avro
> generated objects.  Until then, the protocol will be defined implicitly by
> Java through reflection.
>
> Note that if performance is inadequate due to reflection itself, rather
> than the client/server implementations, we might resort to byte-code
> modification to accelerate it.
>
> https://issues.apache.org/jira/browse/AVRO-143
>
> This would also be a temporary approach.  Longer-term we should move Hadoop
> to use protocols declared in .avpr files and generated classes. But I don't
> think that's practical in the short-term.
>
> My current short-term goal is to try to get Avro's reflection to the point
> where it can implement NamenodeProtocol.
>
> Does this make sense?
>
> Doug
>

--000e0cd22d90a846a804783afcb6--