Return-Path: Delivered-To: apmail-commons-user-archive@www.apache.org Received: (qmail 9236 invoked from network); 14 May 2010 10:52:05 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 14 May 2010 10:52:05 -0000 Received: (qmail 81339 invoked by uid 500); 14 May 2010 10:52:04 -0000 Delivered-To: apmail-commons-user-archive@commons.apache.org Received: (qmail 80835 invoked by uid 500); 14 May 2010 10:51:59 -0000 Mailing-List: contact user-help@commons.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Commons Users List" Delivered-To: mailing list user@commons.apache.org Received: (qmail 80827 invoked by uid 99); 14 May 2010 10:51:58 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 May 2010 10:51:58 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mario@ops.co.at designates 195.242.168.21 as permitted sender) Received: from [195.242.168.21] (HELO exchange.ops.co.at) (195.242.168.21) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 May 2010 10:51:51 +0000 Received: from ex1.opshq.ops.at ([172.27.1.21]) by ex1 ([172.27.1.21]) with mapi; Fri, 14 May 2010 12:51:30 +0200 From: Mario Ivankovits To: 'Kirill Safonov' , 'Commons Users List' Date: Fri, 14 May 2010 12:51:30 +0200 Subject: AW: potential problem: FtpFileObject caches children when not attached Thread-Topic: potential problem: FtpFileObject caches children when not attached Thread-Index: Acrx5NEO/les8PAaT06jMPHZvZfRnABbkpAA Message-ID: References: <4beac458.0c3fdf0a.0e4f.19af@mx.google.com> In-Reply-To: <4beac458.0c3fdf0a.0e4f.19af@mx.google.com> Accept-Language: de-DE Content-Language: de-DE X-MS-Has-Attach: yes X-MS-TNEF-Correlator: acceptlanguage: de-DE Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_0025_01CAF364.2B6FB220" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org ------=_NextPart_000_0025_01CAF364.2B6FB220 Content-Type: multipart/alternative; boundary="----=_NextPart_001_0026_01CAF364.2B6FB220" ------=_NextPart_001_0026_01CAF364.2B6FB220 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi! =20 Hmmm =85 I tried to reproduce it like this: =20 FileObject fos =3D VFS.getManager().resolveFile("ftp://XYZ.com/public/ test with space", = fops); fos.getType(); fos.getChildren(); =20 fos.refresh(); fos.getType(); fos.getChildren(); =20 After the refresh() I just get one ftp list =85 can you please outline = with code how to reproduce the problem. Thanks! =20 Ciao, Mario =20 =20 Von: Kirill Safonov [mailto:kirill.safonov@gmail.com]=20 Gesendet: Mittwoch, 12. Mai 2010 17:07 An: 'Commons Users List' Cc: Mario Ivankovits Betreff: potential problem: FtpFileObject caches children when not = attached =20 Hello, =20 While digging a bit into FTP performance, I=92ve noticed the following: =20 When FtpFileObject.refresh() is called, FtpFileObject.children field is reset to null in FtpFileObject.doDetach(), and = AbstractFileObject.attached flag is set to false (this is OK) But, if then FtpFileObject.getChildFile() is called (this may happen = when getType() is invoked for one of the child files =96 see below), current implementation will ignore =91attached=92 state and silently fill = children cache by calling doGetChildren(). =20 The result is that FtpFileObject remains in inconsistent state: it is = not attached with AbstractFileObject.type field =3D null, child names cache (AbstractFileObject.children) =3D null, but has children cache bulit. =20 I=92m not currently sure which consequences it may cause. =20 What I=92ve found up to date is that when I load list of children I see = that two FTP LIST commands are issued:=20 - first one is to list the children themselves - then parent file is refreshed since CacheStrategy is = ON_RESOLVE (this is actually another issue I=92d like to discuss, since it gives a performance drop clearing the children cache right after is was build) - and then another LIST call that comes from child.getType(). = If there was no inconsistency I described above, this LIST would have been called for each of the children. =20 Hope all this makes sense, Regards, Kirill ------=_NextPart_001_0026_01CAF364.2B6FB220 Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Hi!

 

Hmmm =85 I tried to = reproduce it like this:

 

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 FileObject fos =3D VFS.getManager().resolveFile("ftp://XYZ.com/public/ test = with space", fops);

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = fos.getType();

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = fos.getChildren();

 

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = fos.refresh();

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = fos.getType();

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = fos.getChildren();

 

After the refresh() I = just get one ftp list =85 can you please outline with code how to reproduce the = problem.

Thanks!

 

Ciao,

Mario

 

 

Von:<= /b> = Kirill Safonov [mailto:kirill.safonov@gmail.com]
Gesendet: Mittwoch, 12. Mai 2010 17:07
An: 'Commons Users List'
Cc: Mario Ivankovits
Betreff: potential problem: FtpFileObject caches children when = not attached

 

Hello,

 

While digging a bit = into FTP performance, I=92ve noticed the following:

 

When = FtpFileObject.refresh() is called, FtpFileObject.children field is reset to null in FtpFileObject.doDetach(), and AbstractFileObject.attached flag is set to = false (this is OK)

But, if then FtpFileObject.getChildFile() is called (this may happen when getType() = is invoked for one of the child files =96 see below), current = implementation will ignore =91attached=92 state and silently fill children cache by calling doGetChildren().

 

The result is that = FtpFileObject remains in inconsistent state: it is not attached with = AbstractFileObject.type field =3D null, child names cache (AbstractFileObject.children) =  =3D null, but has children cache bulit.

 

I=92m not currently = sure which consequences it may cause.

 

What I=92ve found up = to date is that when I load list of children I see that two FTP LIST commands are = issued:

-          first one is to list the children = themselves

-          then parent file is refreshed since = CacheStrategy is ON_RESOLVE (this is actually another issue I=92d like to discuss, since = it gives a performance drop clearing the children cache right after is was = build)

-          and then another LIST call that comes from child.getType(). If there was no inconsistency I described above, this = LIST would have been called for each of the children.

 

Hope all this makes = sense,

 Regards,

 Kirill

------=_NextPart_001_0026_01CAF364.2B6FB220-- ------=_NextPart_000_0025_01CAF364.2B6FB220 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIILVTCCAj0w ggGmAhEAzbp/VvDf5LxU/iKss3KqVTANBgkqhkiG9w0BAQIFADBfMQswCQYDVQQGEwJVUzEXMBUG A1UEChMOVmVyaVNpZ24sIEluYy4xNzA1BgNVBAsTLkNsYXNzIDEgUHVibGljIFByaW1hcnkgQ2Vy dGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNOTYwMTI5MDAwMDAwWhcNMjgwODAxMjM1OTU5WjBfMQsw CQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xNzA1BgNVBAsTLkNsYXNzIDEgUHVi bGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwgZ8wDQYJKoZIhvcNAQEBBQADgY0A MIGJAoGBAOUZv22jVmEtmUhx9mfeuY3rt56GgAqRDvo4Ja9GiILlc6igmyRdDR/MZW4MsNBWhBiH mgabEKFz37RYOWtuwfYV1aioP6oSBo0xrH+wNNePNGeICc0UEeJORVZpH3gCgNrcR5EpuzbJY1zF 4Ncth3uhtzKwezC6Ki8xqu6jZ9rbAgMBAAEwDQYJKoZIhvcNAQECBQADgYEATD+4i8Zo3+5DMw5d 6abLB4RNejP/khv0Nq3YlSI2aBFsfELM85wuxAc/FLAPT/+Qknb54rxK6Y/NoIAK98Up8YIiXbix 3YEjo3slFUYweRb46gVLlH8dwhzI47f0EEA8E8NfH1PoSOSGtHuhNbB7Jbq4046rPzidADQAmPPR cZQwggRGMIIDr6ADAgECAhBm/Ufjwhnk6JrNmd31OsskMA0GCSqGSIb3DQEBBQUAMF8xCzAJBgNV BAYTAlVTMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjE3MDUGA1UECxMuQ2xhc3MgMSBQdWJsaWMg UHJpbWFyeSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAeFw0wNTEwMjgwMDAwMDBaFw0xNTEwMjcy MzU5NTlaMIHdMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsT FlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczov L3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0 ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0g RzIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDJ36zn6vj4AxTEAJLVwX42wjzvfHIV y8CrjD0clc5vHhAsPwDtlybmtsfmrUMdP6SHR0dMPlT4bPjH/LGevTBwvJexAwXqlfGtQMVEeksF ovJg/Nc6ZWLv/xB7ola7xU5wLdaiHzztsELoXo1XIaymmdkR6dIaB8B0R0IL/MU06v3muiTRHQgV N6LXc88BQS9jsjo/vqUabvTJSls9laYVuzUCGfnU77yPDnF2WbtLtj7W/FoW9NYOifJJ/mwM7RXp 2Yh1nHnOYCfdua11zi9zlXpAOoV1SbC432i8q80TgoURUKPgPAuuwApTzdcwb4UyRhvkSRDCbOKv H3n/27S1AgMBAAGjgf8wgfwwEgYDVR0TAQH/BAgwBgEB/wIBADBEBgNVHSAEPTA7MDkGC2CGSAGG +EUBBxcBMCowKAYIKwYBBQUHAgEWHGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEwCwYDVR0P BAQDAgEGMBEGCWCGSAGG+EIBAQQEAwIBBjAuBgNVHREEJzAlpCMwITEfMB0GA1UEAxMWUHJpdmF0 ZUxhYmVsMy0yMDQ4LTE1NTAdBgNVHQ4EFgQUEX1eGX08BN9qbNaiiho/Mdg7lFIwMQYDVR0fBCow KDAmoCSgIoYgaHR0cDovL2NybC52ZXJpc2lnbi5jb20vcGNhMS5jcmwwDQYJKoZIhvcNAQEFBQAD gYEAPKPaAmM6xJOqq3LT3K1QOB4MnhZKiLfu69n/D42VoNa7+moLrmGE2GhHie9PrLIfSUGbSTN2 k4uebrlDHGC9wtyKLYfBRcARcgQaayQqbG/n/AcTKdB3OiPn9cGFaBm/xgFUIBmuNYLMYjxhCcb0 1euwD6afM4Wa03GOUI+Z3WIwggTGMIIDrqADAgECAhAHULRUWQLA4CdD6xGPyARQMA0GCSqGSIb3 DQEBBQUAMIHdMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsT FlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczov L3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0 ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0g RzIwHhcNMDkxMDE0MDAwMDAwWhcNMTAxMDE0MjM1OTU5WjCCARMxFzAVBgNVBAoTDlZlcmlTaWdu LCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMUYwRAYDVQQLEz13d3cudmVy aXNpZ24uY29tL3JlcG9zaXRvcnkvUlBBIEluY29ycC4gYnkgUmVmLixMSUFCLkxURChjKTk4MR4w HAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNDAyBgNVBAsTK0RpZ2l0YWwgSUQgQ2xhc3Mg MSAtIE1pY3Jvc29mdCBGdWxsIFNlcnZpY2UxGTAXBgNVBAMUEE1hcmlvIEl2YW5rb3ZpdHMxHjAc BgkqhkiG9w0BCQEWD21hcmlvQG9wcy5jby5hdDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA qAXN5vxqFOWBBcurhtZHuqdQB9Fl8V4HeMHxXC2SmqbJNCAqbVESDjZPNWL1EE6pwqL+pBfr1iHE h8NkyfkflJ3thzGF6LrlTU4IdriCW5KLPu9Gwtpme9TBVLxH4YxLiKLBbsItnrqMfMDmycnKENua z9PVFsi+cQuzeupkxW8CAwEAAaOBzDCByTAJBgNVHRMEAjAAMEQGA1UdIAQ9MDswOQYLYIZIAYb4 RQEHFwEwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYTALBgNVHQ8E BAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMCMEoGA1UdHwRDMEEwP6A9oDuGOWh0 dHA6Ly9JbmRDMURpZ2l0YWxJRC1jcmwudmVyaXNpZ24uY29tL0luZEMxRGlnaXRhbElELmNybDAN BgkqhkiG9w0BAQUFAAOCAQEAfF6IHFF5nz9UzexmSw7P1YpFSMLqjuiUp67B7DG+mcTj8rSPNWSn Ye3PqpstzRY+VYmLsvT0Vntq9AvCzpWWrctkiGddxOCjAh3eSImqPT0Pe3NX2tPvO43cw/H4rVjv SLbv+hCX2IUvEVbCSLVFxU9MsoNxbf0MXMAdjoOKxXYTTT/eIKmcLvdgcxwaGd+e15l1ic5ZF1gS pHBXNVD0sXrazwvW5ygm83tivwxXUIYfTSMzR3LZ+lQ3esfRJKpLdusCgO/pM7nD/XKnhT1ikdus 2O7XxBdeZVmFa1f+du70JqR2KtJnx2YcAYn8cpUB6EN5UvTWR4sUZC8lTBGWVjGCBMQwggTAAgEB MIHyMIHdMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZl cmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczovL3d3 dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQx NzA1BgNVBAMTLlZlcmlTaWduIENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzIC EAdQtFRZAsDgJ0PrEY/IBFAwCQYFKw4DAhoFAKCCAycwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEH ATAcBgkqhkiG9w0BCQUxDxcNMTAwNTE0MTA1MTI4WjAjBgkqhkiG9w0BCQQxFgQUmgpjym6SAmhb dP7T5bqIGOMuRL8wgbcGCSqGSIb3DQEJDzGBqTCBpjALBglghkgBZQMEASowCwYJYIZIAWUDBAEW MAoGCCqGSIb3DQMHMAsGCWCGSAFlAwQBAjAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAw BwYFKw4DAgcwDQYIKoZIhvcNAwICASgwBwYFKw4DAhowCwYJYIZIAWUDBAIDMAsGCWCGSAFlAwQC AjALBglghkgBZQMEAgEwCgYIKoZIhvcNAgUwggEDBgkrBgEEAYI3EAQxgfUwgfIwgd0xCzAJBgNV BAYTAlVTMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3Qg TmV0d29yazE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNv bS9ycGEgKGMpMDUxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuVmVy aVNpZ24gQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHMgIQB1C0VFkCwOAnQ+sR j8gEUDCCAQUGCyqGSIb3DQEJEAILMYH1oIHyMIHdMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVy aVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOzA5BgNVBAsTMlRl cm1zIG9mIHVzZSBhdCBodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MR4wHAYDVQQL ExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENsYXNzIDEgSW5kaXZp ZHVhbCBTdWJzY3JpYmVyIENBIC0gRzICEAdQtFRZAsDgJ0PrEY/IBFAwDQYJKoZIhvcNAQEBBQAE gYAcBVsBSSAZ/fUqUeaeeaC/6dAXr9zw+a2nA73b3DpTtthjDzIAE3rb1pfKArYN4ZgIVUq8Rimv zDpav/YPRxURijT/0UN8ciLRVI3zgnxVy+SKzonN1xLshl1N8tGrTRqlB7z0cCHx257LrIfM0qrt izICrT2iWUiv+oXAcxRRCQAAAAAAAA== ------=_NextPart_000_0025_01CAF364.2B6FB220--