Return-Path: X-Original-To: apmail-manifoldcf-user-archive@www.apache.org Delivered-To: apmail-manifoldcf-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 82A3810CDC for ; Thu, 29 Jan 2015 06:08:07 +0000 (UTC) Received: (qmail 4151 invoked by uid 500); 29 Jan 2015 06:08:08 -0000 Delivered-To: apmail-manifoldcf-user-archive@manifoldcf.apache.org Received: (qmail 4118 invoked by uid 500); 29 Jan 2015 06:08:07 -0000 Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@manifoldcf.apache.org Delivered-To: mailing list user@manifoldcf.apache.org Received: (qmail 4108 invoked by uid 99); 29 Jan 2015 06:08:07 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Jan 2015 06:08:07 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of abjitu@gmail.com designates 209.85.217.173 as permitted sender) Received: from [209.85.217.173] (HELO mail-lb0-f173.google.com) (209.85.217.173) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Jan 2015 06:08:02 +0000 Received: by mail-lb0-f173.google.com with SMTP id p9so24577832lbv.4 for ; Wed, 28 Jan 2015 22:06:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=8l9igZPSuM0M9CaI/5whbLm7tL2ncK7Lw6mJVCQ6Vw0=; b=K9oDmBz9yHSkTsdltFT3lmjd0i9G0MvEkgPadBmHV8bgaquzIhP99BrADWIamz14+a +9uQdNKy9PDVGbvPEWV45wJDYWLpD0S5FlHdc+SgL57oZpVvd84HATOt0Gjl43ZT4HOC exlTZGILbpfPPwElUJoASAusg11uXghsnyqwfamOSLjRp2/9YwAa2kNIDE0ESVz9mKIL kAE6WnXaXZjnm3sSVHSfDAu4XC2ZybqyI/pbwA8Zxxn1hTLx/5N7msk1GiQSfmdSRY8P bu5HNgQSXuJ8iwOMf/ZFXtNxksZeYPDEsGZW3pM2oSMA7ZUuBsMa+o6Yr+HctAyhL6M9 vBDg== MIME-Version: 1.0 X-Received: by 10.112.202.195 with SMTP id kk3mr12334975lbc.53.1422511616260; Wed, 28 Jan 2015 22:06:56 -0800 (PST) Received: by 10.25.39.132 with HTTP; Wed, 28 Jan 2015 22:06:56 -0800 (PST) In-Reply-To: References: Date: Thu, 29 Jan 2015 11:36:56 +0530 Message-ID: Subject: Re: Scheduling Info From: Jitu To: "user@manifoldcf.apache.org" Content-Type: multipart/alternative; boundary=001a11c36b04937fcb050dc44be7 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c36b04937fcb050dc44be7 Content-Type: text/plain; charset=UTF-8 Hi Karl, Thanks for your quick reply. Please correct me if i am wrong. For 1000 jobs assuming every repository connector is different from other and max thread per repository connector is 10 then will there be 10,000 threads spanned for repository connections? how many threads will be spanned for 1000 jobs? Thanks, Jitu On Wed, Jan 28, 2015 at 6:31 PM, Karl Wright wrote: > Yes, if I understand the user's usage correctly, all 1000 jobs are all > active at the same time. > > Note that you can scale MCF in several different ways if you need to (and > I am not clear at all at the moment that your case would require that). > You can increase thread count somewhat (I wouldn't go beyond 100). You can > add more agents processes. But frankly 90 jobs sounds pretty small to me. > > Karl > > > On Wed, Jan 28, 2015 at 6:38 AM, Jitu wrote: > >> Hi Karl, >> Assuming all these 1000 jobs are running concurrently. >> >> Thanks, >> Jitu >> >> >> On Wed, Jan 28, 2015 at 5:07 PM, Jitu wrote: >> >>> Hi Karl, >>> >>> Thanks for the reply. these thousands of jobs run on single machine? >>> wont there be any issue with number of threads spanned. will there be any >>> issue with throughput of the jobs. >>> >>> Thanks, >>> Jitu >>> >>> On Wed, Jan 28, 2015 at 4:29 PM, Karl Wright wrote: >>> >>>> Hi Jitu, >>>> >>>> We have people with hundreds of connections and thousands of jobs. >>>> They work fine. Why do you think that 90 jobs would be a problem? >>>> >>>> Karl >>>> >>>> >>>> >>>> On Wed, Jan 28, 2015 at 5:09 AM, Jitu wrote: >>>> >>>>> Hi Karl, >>>>> >>>>> We have a requirement where We have around 30 clients and each client >>>>> has Google drive, wiki and box connectors. if i run all the clients job in >>>>> one machine say 30 * 3 = 90 jobs running concurrently may not scale >>>>> assuming all clients scheduled at same time. >>>>> >>>>> please advice on how to handle many jobs concurrently. >>>>> >>>>> Thanks, >>>>> Jitu >>>>> >>>> >>>> >>> >> > --001a11c36b04937fcb050dc44be7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Karl,

Thanks for your quick= reply. Please correct me if i am wrong. For 1000 jobs assuming every repos= itory connector is different from other and max thread per repository conne= ctor is 10 then will there be 10,000 threads spanned for repository connect= ions? how many threads will be spanned for 1000 jobs?

T= hanks,
Jitu

On Wed, Jan 28, 2015 at 6:31 PM, Karl Wright <daddywri@gmail= .com> wrote:
Yes, if I understand the user's usage correctly, all 1000 = jobs are all active at the same time.

Note that you can scale = MCF in several different ways if you need to (and I am not clear at all at = the moment that your case would require that).=C2=A0 You can increase threa= d count somewhat (I wouldn't go beyond 100).=C2=A0 You can add more age= nts processes.=C2=A0 But frankly 90 jobs sounds pretty small to me.

Karl



On Wed, Jan 28, 2015 at 5:07 PM, Jitu <abj= itu@gmail.com> wrote:
Hi Karl,

Thanks for the reply. thes= e thousands of jobs run on single machine? wont there be any issue with num= ber of threads spanned. will there be any issue with throughput of the jobs= .

Thanks,
Jitu

On Wed, Jan 28, 2015 at 4:29 PM, Kar= l Wright <daddywri@gmail.com> wrote:
Hi Jitu,

We have people with hun= dreds of connections and thousands of jobs.=C2=A0 They work fine.=C2=A0 Why= do you think that 90 jobs would be a problem?

Karl


On Wed, Jan 28, 2015 at 5:09 AM, Jitu <abjitu@gmail= .com> wrote:
Hi Karl,

We have a requirement where We have arou= nd 30 clients and each client has Google drive, wiki and box connectors. if= i run all the clients job in one machine say 30 * 3 =3D 90 jobs running co= ncurrently may not scale assuming all clients scheduled at same time.

please advice on how to handle many jobs concurrently.=

Thanks,
Jitu





--001a11c36b04937fcb050dc44be7--