manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <>
Subject [jira] [Updated] (CONNECTORS-1112) Heavy contention over repository connections
Date Sat, 22 Nov 2014 14:04:34 GMT


Karl Wright updated CONNECTORS-1112:
    Affects Version/s: ManifoldCF 1.7.2
        Fix Version/s: ManifoldCF 2.0
                       ManifoldCF 1.8
             Assignee: Karl Wright
           Issue Type: Improvement  (was: Bug)

> Heavy contention over repository connections
> --------------------------------------------
>                 Key: CONNECTORS-1112
>                 URL:
>             Project: ManifoldCF
>          Issue Type: Improvement
>          Components: Framework core, JCIFS connector
>    Affects Versions: ManifoldCF 1.7.2
>            Reporter: Aeham Abushwashi
>            Assignee: Karl Wright
>             Fix For: ManifoldCF 1.8, ManifoldCF 2.0
>         Attachments: CONNECTORS-1112.patch
> On a recent test using (a slightly customised version of) the JCIFS repository connector,
I noticed the Stuffer thread spends more than 50% of its time calling out to IRepositoryConnectorPool#grab
(between wait() and talking to ZK). 
> I wondered if the StufferThread#run method could benefit from caching repository connections
along the lines of CONNECTORS-1094 so I patched it up (see attachement) which took the time
down a bit. Upon closer inspection I saw the calls to RepositoryConnectorPool#grab and RepositoryConnectorPool#releaseMultiple
were constantly blocked waiting on a lock owned by the Idle cleanup thread. It turns out the
Idle cleanup thread does some relatively expensive work ** at least for the JCIFS connector
** once it's acquired a lock on ConnectorPool#poolHash. It reads a config property in SharedDriveConnector#setThreadContext
and that involves a trip to ZK and some  XML parsing which results in loading and processing
JAR files. I subsequently monitored other threads and found that many can be impacted for
prolonged periods of time as they try to acquire and release repository connections.
> Caching the values of config properties almost eliminates the time the Idle cleanup thread
spends under lock and is easy enough to implement for the JCIFS connector. It would be great
if this could be done in a way that is more generic to prevent a slight inefficiency in the
code of one connector slowing down the entire system.

This message was sent by Atlassian JIRA

View raw message