Mailing-List: contact dev-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@manifoldcf.apache.org
Date: Sun, 23 Nov 2014 16:57:12 +0000 (UTC)
From: "Karl Wright (JIRA)" <jira@apache.org>
To: dev@manifoldcf.apache.org
Message-ID: <JIRA.12757045.1416594786000.2327.1416761832925@Atlassian.JIRA>
In-Reply-To: <JIRA.12757045.1416594786000@Atlassian.JIRA>
References: <JIRA.12757045.1416594786000@Atlassian.JIRA>
 <JIRA.12757045.1416594786361@arcas>
Subject: [jira] [Resolved] (CONNECTORS-1112) Heavy contention over
 repository connections
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CONNECTORS-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Karl Wright resolved CONNECTORS-1112.
-------------------------------------
    Resolution: Fixed

> Heavy contention over repository connections
> --------------------------------------------
>
>                 Key: CONNECTORS-1112
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1112
>             Project: ManifoldCF
>          Issue Type: Improvement
>          Components: Framework core, JCIFS connector
>    Affects Versions: ManifoldCF 1.7.2
>            Reporter: Aeham Abushwashi
>            Assignee: Karl Wright
>             Fix For: ManifoldCF 1.8, ManifoldCF 2.0
>
>         Attachments: CONNECTORS-1112.patch
>
>
> On a recent test using (a slightly customised version of) the JCIFS repository connector, I noticed the Stuffer thread spends more than 50% of its time calling out to IRepositoryConnectorPool#grab (between wait() and talking to ZK). 
> I wondered if the StufferThread#run method could benefit from caching repository connections along the lines of CONNECTORS-1094 so I patched it up (see attachement) which took the time down a bit. Upon closer inspection I saw the calls to RepositoryConnectorPool#grab and RepositoryConnectorPool#releaseMultiple were constantly blocked waiting on a lock owned by the Idle cleanup thread. It turns out the Idle cleanup thread does some relatively expensive work ** at least for the JCIFS connector ** once it's acquired a lock on ConnectorPool#poolHash. It reads a config property in SharedDriveConnector#setThreadContext and that involves a trip to ZK and some  XML parsing which results in loading and processing JAR files. I subsequently monitored other threads and found that many can be impacted for prolonged periods of time as they try to acquire and release repository connections.
> Caching the values of config properties almost eliminates the time the Idle cleanup thread spends under lock and is easy enough to implement for the JCIFS connector. It would be great if this could be done in a way that is more generic to prevent a slight inefficiency in the code of one connector slowing down the entire system.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)