manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Performance on shared drive documents
Date Thu, 24 Jul 2014 18:53:31 GMT
Hi Ameya,

ManifoldCF can only crawl as fast as the underlying repository will
support.  The performance in the case of a network drive is obviously going
to be slower than that of a local drive.

Karl



On Thu, Jul 24, 2014 at 2:49 PM, Ameya Aware <ameya.aware@gmail.com> wrote:

> Hi Karl,
>
> Currently document security is the not the main aim.
>
> I just want all the files to be crawled as quickly as possible.
>
> So do you want to say that my shared drive would never get crawled as fast
> as my C drive?
>
>
> On Thu, Jul 24, 2014 at 2:44 PM, Karl Wright <daddywri@gmail.com> wrote:
>
>> Hi Ameya,
>>
>> The filesystem connector does not support document security.  Are you
>> sure this is what you want to do?
>>
>> Either way, the same logic applies, since it is the same underlying
>> technology.
>>
>> Karl
>>
>>
>>
>> On Thu, Jul 24, 2014 at 2:40 PM, Ameya Aware <ameya.aware@gmail.com>
>> wrote:
>>
>>> i apologize for my mistake.
>>>
>>> I wanted to say that i am not using windows share connector. I am using
>>> filesystem connector only where in i provide the location of shared drive
>>> in repository paths.
>>>
>>>
>>> On Thu, Jul 24, 2014 at 2:36 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>
>>>> Hi Ameya,
>>>>
>>>> Yes, I figured that you would be using the share connector to crawl
>>>> shared drives.  That's what I am talking about, though.  The share
>>>> connector is only as fast as the server you are connected to, and there may
>>>> be indirections also if the path contains DFS links.  Regardless, though,
>>>> Windows file servers are easily overloaded, in my experience, so try not
to
>>>> overload them unnecessarily.  When a file server gets overloaded, it often
>>>> fails with a timeout (which you can see in the simple history report, or
in
>>>> the manifoldcf log).  That's a sign that you need to throttle more, and the
>>>> best way to do that is to limit further the number of simultaneous
>>>> connections.
>>>>
>>>> Thanks,
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Thu, Jul 24, 2014 at 2:32 PM, Ameya Aware <ameya.aware@gmail.com>
>>>> wrote:
>>>>
>>>>> Actually i am using windows share connector. I am using filesystem
>>>>> connector only where in i provide the location of shared drive in
>>>>> repository paths.
>>>>>
>>>>>
>>>>> On Thu, Jul 24, 2014 at 2:13 PM, Karl Wright <daddywri@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ameya,
>>>>>>
>>>>>> Windows shared drive access performance is not great, I'm afraid.
>>>>>> However, it is possible that you are overwhelming the server you
are
>>>>>> talking to.  Try reducing the maximum number of connections for the
shared
>>>>>> drive repository connection to between 2 and 5; that sometimes helps.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 24, 2014 at 2:11 PM, Ameya Aware <ameya.aware@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> When i am crawling full of my C drive the job is running pretty
much
>>>>>>> faster.
>>>>>>>
>>>>>>> But at the same time when i m crawling documents kept on shared
>>>>>>> drive, it is taking too much longer time than to crawl C drive.
>>>>>>>
>>>>>>> are documents at shared drive supposed to take to much time??
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ameya
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message