manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jitu <abj...@gmail.com>
Subject Re: Scheduling Info
Date Thu, 29 Jan 2015 06:06:56 GMT
Hi Karl,

Thanks for your quick reply. Please correct me if i am wrong. For 1000 jobs
assuming every repository connector is different from other and max thread
per repository connector is 10 then will there be 10,000 threads spanned
for repository connections? how many threads will be spanned for 1000 jobs?

Thanks,
Jitu

On Wed, Jan 28, 2015 at 6:31 PM, Karl Wright <daddywri@gmail.com> wrote:

> Yes, if I understand the user's usage correctly, all 1000 jobs are all
> active at the same time.
>
> Note that you can scale MCF in several different ways if you need to (and
> I am not clear at all at the moment that your case would require that).
> You can increase thread count somewhat (I wouldn't go beyond 100).  You can
> add more agents processes.  But frankly 90 jobs sounds pretty small to me.
>
> Karl
>
>
> On Wed, Jan 28, 2015 at 6:38 AM, Jitu <abjitu@gmail.com> wrote:
>
>> Hi Karl,
>>           Assuming all these 1000 jobs are running concurrently.
>>
>> Thanks,
>> Jitu
>>
>>
>> On Wed, Jan 28, 2015 at 5:07 PM, Jitu <abjitu@gmail.com> wrote:
>>
>>> Hi Karl,
>>>
>>> Thanks for the reply. these thousands of jobs run on single machine?
>>> wont there be any issue with number of threads spanned. will there be any
>>> issue with throughput of the jobs.
>>>
>>> Thanks,
>>> Jitu
>>>
>>> On Wed, Jan 28, 2015 at 4:29 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>
>>>> Hi Jitu,
>>>>
>>>> We have people with hundreds of connections and thousands of jobs.
>>>> They work fine.  Why do you think that 90 jobs would be a problem?
>>>>
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Wed, Jan 28, 2015 at 5:09 AM, Jitu <abjitu@gmail.com> wrote:
>>>>
>>>>> Hi Karl,
>>>>>
>>>>> We have a requirement where We have around 30 clients and each client
>>>>> has Google drive, wiki and box connectors. if i run all the clients job
in
>>>>> one machine say 30 * 3 = 90 jobs running concurrently may not scale
>>>>> assuming all clients scheduled at same time.
>>>>>
>>>>> please advice on how to handle many jobs concurrently.
>>>>>
>>>>> Thanks,
>>>>> Jitu
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message