commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NET-455) FTPClient listFiles(FTPFileFilter) can be faster...
Date Fri, 23 Mar 2012 20:21:28 GMT

    [ https://issues.apache.org/jira/browse/NET-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237019#comment-13237019
] 

Nicolas commented on NET-455:
-----------------------------

Thanks for your help !
The two cases where observed separately, and not in a row, so no caching possible. I did many
runs like ABAABBA (but each time I stopped the JVM and restart my soft with the other filter
method) and so I could reproduce the case each time.

However, I looked the source code of FTPClient, and you are right, no difference.
Maybe I have some sort of answer, but I'm really not sure :

I call 
{code} files = ftpClient.listFiles("./", ftpFileFilter); {code}
So it calls the function 
{code}initiateListParsing(String parserKey, String pathname) {code}
with the pathname not null, whereas in the "fast" case, the pathname is null.

So I do "LIST -a ./" instead of a "LIST -a" (if I'm right).

I'll dig into it on monday, trying null path or getFiles("./"), I'll come back with the results!

 
                
> FTPClient listFiles(FTPFileFilter) can be faster...
> ---------------------------------------------------
>
>                 Key: NET-455
>                 URL: https://issues.apache.org/jira/browse/NET-455
>             Project: Commons Net
>          Issue Type: Improvement
>          Components: FTP
>    Affects Versions: 3.0.1
>         Environment: all
>            Reporter: Nicolas
>            Priority: Minor
>              Labels: ftpclient, listFiles
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> I tested two ways of sorting files on a remote FTP server with many files ( > 110
000). Both of these use the FtpFileFilter class (with a test on the extension filename...
very simple).
> The first one is the conventionnal => 
> {code}
> files = ftpClient.listFiles("./", ftpFileFilter);
> {code}
> It takes about 90 seconds ...
> the second one is ... nearly as simple as the first one : 
> {code}
> FTPFile []  allFilesListed = ftpClient.listFiles();
> files = FTPFileFilterWithExtension.fastFilter(ftpFileFilter, allFilesListed);
> {code}
> The function FTPFileFilterWithExtension.fastFilter() call the same ftpFileFilter :
> {code}
> public static FTPFile [] fastFilter(FTPFileFilter filter, FTPFile ftpFileArray []) {
>     ArrayList<FTPFile> listOfFTPFiles = new ArrayList<FTPFile>();
>     for (FTPFile ftpf : ftpFileArray) {
>         boolean accepted = filter.accept(ftpf);
>         if (accepted) {
>             listOfFTPFiles.add(ftpf);
>         }
>     }
>     FTPFile resultArray [] = new FTPFile[listOfFTPFiles.size()];
>     return listOfFTPFiles.toArray(resultArray);
> }
> {code}
> The second method is EIGHT times faster than the first one ... and I don't really understand
why...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message