ant-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Johannes Rudolph (JIRA)" <>
Subject [jira] Created: (IVY-973) [PATCH] SFTP repositories become very slow after a while
Date Wed, 19 Nov 2008 10:09:44 GMT
[PATCH] SFTP repositories become very slow after a while

                 Key: IVY-973
             Project: Ivy
          Issue Type: Improvement
          Components: Core
    Affects Versions: 2.0-RC2
            Reporter: Johannes Rudolph
         Attachments: faster-sftp.patch

If many versions of a module live in a folder of an SFTP repository each resolve for dynamic
versions becomes dreadfully slow because a roundtrip to the server is made to the server for
every version which lies on the server. 

The call sequence goes like that:

org.apache.ivy.plugins.resolver.util.ResolverHelper.findAll calls
  1.  org.apache.ivy.plugins.resolver.util.ResolverHelper.listTokenValues which calls
        SFTPRepository.list with the directory of the module as parameter (1. roundtrip)
  2. for every found file matching the pattern SFTPRepository.getResource(path).exists()
        In turn SFTPResource tries to initialize itself and calls SFTPRepository.resolveResource
which does the second roundtrip to the server - for every version of the file

Example: Having 20 versions of a module in a repository which is a roundtrip of 150ms away,
leads to a lag of 20 * 150ms = 3s . Multiply that with any module having a dynamic version.

The solution is to cache all the resources in SFTPRepository. You can use the information
retrieved when calling SFTPRepository.list to fill the cache in advance.

The disadvantage is that you may have stale data. IMO this is mostly no problem because every
call to list updates the cache and files already available in the repository won't hopefully
change any metadata in the time between.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message