Return-Path: X-Original-To: apmail-avro-user-archive@www.apache.org Delivered-To: apmail-avro-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 287C044B8 for ; Wed, 8 Jun 2011 16:54:16 +0000 (UTC) Received: (qmail 4534 invoked by uid 500); 8 Jun 2011 16:54:16 -0000 Delivered-To: apmail-avro-user-archive@avro.apache.org Received: (qmail 4500 invoked by uid 500); 8 Jun 2011 16:54:15 -0000 Mailing-List: contact user-help@avro.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@avro.apache.org Delivered-To: mailing list user@avro.apache.org Received: (qmail 4492 invoked by uid 99); 8 Jun 2011 16:54:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Jun 2011 16:54:15 +0000 X-ASF-Spam-Status: No, hits=4.4 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HK_RANDOM_ENVFROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of teddyyyy123@gmail.com designates 209.85.213.171 as permitted sender) Received: from [209.85.213.171] (HELO mail-yx0-f171.google.com) (209.85.213.171) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Jun 2011 16:54:09 +0000 Received: by yxe1 with SMTP id 1so314681yxe.30 for ; Wed, 08 Jun 2011 09:53:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=QbSJnp4nIFV+hb1mJqtJbxFCaOiJE6B7ROizSfEDvpY=; b=iaFjezXjGDuooYdD18v35gWCodwN7fyRNFGQgQqgNagy3PqBtRvRwKeiRPmETk/ZP9 lGJ4qTBqsAMRZ8L4TMyJghDwv5Ct61z8dtunFL3rpEE6xlXsT0VfdWCcDszz747UJFcU /hN8jgZ66rZkxTA0MyoYVEqso1cv+V8rs+8+c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=TcsQoUyTPBdcEeZn/hoOTpGDfO+CLNaP2wtbZh+FRIZtXREONpBVyPiuMe2x2+HLLE V8K9h/7SjGCX/FMZ+ucFyriAJ0ZYNNF/xPtsruF486rtLd8jjeFaYtK+9aS4wYHx2KuJ B7I8E3mKvb8gV9pdCJpCw8r74kBYYYwwfce/4= MIME-Version: 1.0 Received: by 10.236.29.106 with SMTP id h70mr1634634yha.214.1307552027789; Wed, 08 Jun 2011 09:53:47 -0700 (PDT) Received: by 10.236.202.5 with HTTP; Wed, 8 Jun 2011 09:53:46 -0700 (PDT) Received: by 10.236.202.5 with HTTP; Wed, 8 Jun 2011 09:53:46 -0700 (PDT) In-Reply-To: <4DEF89BB.8010600@apache.org> References: <4DEF89BB.8010600@apache.org> Date: Wed, 8 Jun 2011 09:53:46 -0700 Message-ID: Subject: Re: thread pool for Socket server? From: Yang To: user@avro.apache.org Content-Type: multipart/alternative; boundary=00151773da722449cd04a5362ee8 --00151773da722449cd04a5362ee8 Content-Type: text/plain; charset=ISO-8859-1 Thanks Doug I'll try betty and contribute the result Yang On Jun 8, 2011 7:40 AM, "Doug Cutting" wrote: > A thread pool might indeed provide an improvement. However the Netty > and Jetty-based servers are probably better starting points for reliable > performance. > > Also, please use SaslSocketServer instead of SocketSerer, with the > anonymous mechanism if no security is required. I'd like to standardize > on SASL for non-HTTP, high-performance Avro RPC. > > It would be great if someone could contribute a Netty-based > implementation of Avro's SASL profile. That would provide a > high-performance client and server that can support authentication > and/or encryption. > > Doug > > On 06/08/2011 03:58 AM, Yang wrote: >> currently SocketServer basically does this: >> >> while (true) { >> new Connection(channel.accept()); >> >> } >> >> >> >> so it faithfully accepts any incoming connection and creates a worker >> thread processing the request. >> >> I think this is prone to DOS attacks, or ungraceful failure in the case >> of heavy traffic. >> >> if we use a limited thread pool inside Connection() implementation, we >> can reject connections if we are too heavily loaded. >> >> thought it's basically a trivial thing to implement, but considering >> this is a very common use case, is it worthwhile to put >> such an implementation into the avro source? >> >> Thanks >> Yang --00151773da722449cd04a5362ee8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Thanks Doug
I'll try betty and contribute the result
Yang

On Jun 8, 2011 7:40 AM, "Doug Cutting"= <cutting@apache.org> wrote= :
> A thread pool might indeed provide an improv= ement. However the Netty
> and Jetty-based servers are probably better starting points for reliab= le
> performance.
>
> Also, please use SaslSocketServer = instead of SocketSerer, with the
> anonymous mechanism if no security= is required. I'd like to standardize
> on SASL for non-HTTP, high-performance Avro RPC.
>
> It w= ould be great if someone could contribute a Netty-based
> implementat= ion of Avro's SASL profile. That would provide a
> high-performa= nce client and server that can support authentication
> and/or encryption.
>
> Doug
>
> On 06/08/201= 1 03:58 AM, Yang wrote:
>> currently SocketServer basically does t= his:
>>
>> while (true) {
>> ne= w Connection(channel.accept());
>>
>> }
>>
>>
>>
>= ;> so it faithfully accepts any incoming connection and creates a worker=
>> thread processing the request.
>>
>> I thin= k this is prone to DOS attacks, or ungraceful failure in the case
>> of heavy traffic.
>>
>> if we use a limited thr= ead pool inside Connection() implementation, we
>> can reject conn= ections if we are too heavily loaded.
>>
>> thought it&#= 39;s basically a trivial thing to implement, but considering
>> this is a very common use case, is it worthwhile to put
>>= ; such an implementation into the avro source?
>>
>> Tha= nks
>> Yang
--00151773da722449cd04a5362ee8--