manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ameya Aware <ameya.aw...@gmail.com>
Subject Re: Performance on shared drive documents
Date Wed, 30 Jul 2014 15:18:37 GMT
So how can improve the performance of crawling for shared drives?


Thanks,
Ameya


On Thu, Jul 24, 2014 at 2:53 PM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Ameya,
>
> ManifoldCF can only crawl as fast as the underlying repository will
> support.  The performance in the case of a network drive is obviously going
> to be slower than that of a local drive.
>
> Karl
>
>
>
> On Thu, Jul 24, 2014 at 2:49 PM, Ameya Aware <ameya.aware@gmail.com>
> wrote:
>
>> Hi Karl,
>>
>> Currently document security is the not the main aim.
>>
>> I just want all the files to be crawled as quickly as possible.
>>
>> So do you want to say that my shared drive would never get crawled as
>> fast as my C drive?
>>
>>
>> On Thu, Jul 24, 2014 at 2:44 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> Hi Ameya,
>>>
>>> The filesystem connector does not support document security.  Are you
>>> sure this is what you want to do?
>>>
>>> Either way, the same logic applies, since it is the same underlying
>>> technology.
>>>
>>> Karl
>>>
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <ameya.aware@gmail.com>
>>> wrote:
>>>
>>>> i apologize for my mistake.
>>>>
>>>> I wanted to say that i am not using windows share connector. I am
>>>> using filesystem connector only where in i provide the location of shared
>>>> drive in repository paths.
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <daddywri@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ameya,
>>>>>
>>>>> Yes, I figured that you would be using the share connector to crawl
>>>>> shared drives.  That's what I am talking about, though.  The share
>>>>> connector is only as fast as the server you are connected to, and there
may
>>>>> be indirections also if the path contains DFS links.  Regardless, though,
>>>>> Windows file servers are easily overloaded, in my experience, so try
not to
>>>>> overload them unnecessarily.  When a file server gets overloaded, it
often
>>>>> fails with a timeout (which you can see in the simple history report,
or in
>>>>> the manifoldcf log).  That's a sign that you need to throttle more, and
the
>>>>> best way to do that is to limit further the number of simultaneous
>>>>> connections.
>>>>>
>>>>> Thanks,
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <ameya.aware@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Actually i am using windows share connector. I am using filesystem
>>>>>> connector only where in i provide the location of shared drive in
>>>>>> repository paths.
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <daddywri@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ameya,
>>>>>>>
>>>>>>> Windows shared drive access performance is not great, I'm afraid.
>>>>>>> However, it is possible that you are overwhelming the server
you are
>>>>>>> talking to.  Try reducing the maximum number of connections for
the shared
>>>>>>> drive repository connection to between 2 and 5; that sometimes
helps.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <ameya.aware@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> When i am crawling full of my C drive the job is running
pretty
>>>>>>>> much faster.
>>>>>>>>
>>>>>>>> But at the same time when i m crawling documents kept on
shared
>>>>>>>> drive, it is taking too much longer time than to crawl C
drive.
>>>>>>>>
>>>>>>>> are documents at shared drive supposed to take to much time??
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ameya
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message