lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: SimplePostToolTest very slow
Date Sat, 14 Sep 2013 07:31:38 GMT
Hallo Shai,

 

the main problem with any security manager is: To check if a connection is allowed, it has
to resolve DNS and look the IP up in the policy.

 

As Robert said before: The fix is easy: To emulate a broken URL use one of the non-routeable
local IPv6 prefixes. We do this in Solr tests already to emulate broken cloud slaves, see
the abstract solr cloud test class. We should use one of these URLs instead of example.com.
I disagree with changing the security policy at all, because it helps to fix broken tests
(like this one *g*) and prevents external hackers to overtake my own test instance, because
Solr runs on 0.0.0.0, so while running tests you can easily connect to solr from internet.
The security manager prevents this, too.

 

Uwe

 

-----

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

 <http://www.thetaphi.de/> http://www.thetaphi.de

eMail: uwe@thetaphi.de

 

From: Shai Erera [mailto:serera@gmail.com] 
Sent: Saturday, September 14, 2013 6:32 AM
To: dev@lucene.apache.org
Subject: Re: SimplePostToolTest very slow

 

OK, I think I've made some progress -- configuration issue, but not proxies, it seems to be
our security manager/policy.

I printed system properties and env in setUp and ran only a single test (testIsOn) to compare
the output. I didn't notice any proxy settings, but I did notice that from Ant we're using
our own security manager and policy. So when I ran the test from eclipse using -Djava.security.manager=org.apache.lucene.util.TestSecurityManager
-Djava.security.policy=D:\dev\lucene\lucene-trunk\lucene\tools\junit4\tests.policy, it ran
for 23s too (I only ran a single test method).

I don't think it's related to TestSecurityManager, since it only checks that System.exit isn't
called from tests.
Looking at tests.policy, there are a bunch of socket connection related lines .. I'm guess
it's somewhere there, though I don't know this stuff.

I will try to look into it more later.

Shai

 

On Sat, Sep 14, 2013 at 6:59 AM, Shai Erera <serera@gmail.com> wrote:

I still don't see why you'd get different timings between Eclipse and
Ant if you're running with the same VM -- it should be pretty
consistent (either it caches dns lookups or it doesn't).

 

I agree, it's suspicious. I searched for URL performance differences between eclipse and Ant
and hit this page: http://ant.apache.org/manual/proxy.html. It suggests Ant uses different
proxy settings by default for its own tasks as well as 3rd party tasks that use java.net.URL.
I tried running with -autoproxy but from Ant each test method still takes ~23s vs Eclipse
which is ~0.1s.

Will be interesting to identify the differences .. I think it's a configuration issue, as
Eclipse needs to make URL connections for e.g. its marketplace, so maybe it comes pre-configured
somehow. I'm now curious, I'll dig :).

Shai

 

On Sat, Sep 14, 2013 at 12:33 AM, Chris Hostetter <hossman_lucene@fucit.org> wrote:


: If you want to experiment, a really trivial test is below -- on my system,

: there is a ~5 second gap between each pair of "INIT" and "H1" timestamps,
: but no other odd gaps in subsequent timestamps -- suggesting no caching of
: DNS per hostname, but that the URL class doesn't "re-lookup" on subsequent
: hashCode() calls.

Or maybe i could actually cut & paste the test this time...


import java.net.URL;
import org.apache.lucene.util.LuceneTestCase;
public class TestURLDNSAbsurdity extends LuceneTestCase {

  public void testHowSlowCanYouGo() throws Exception {
    go("1");
    go("2");
  }
  public void go(String s) throws Exception {
    System.out.println(s + " PRE: " + System.currentTimeMillis());
    URL url = new URL("http://example.com/");
    System.out.println(s + "INIT: " + System.currentTimeMillis());
    int h1 = url.hashCode();
    System.out.println(s + "  H1: " + System.currentTimeMillis());
    int h2 = url.hashCode();
    System.out.println(s + "  H2: " + System.currentTimeMillis());
    boolean e1 = url.equals(this);
    System.out.println(s + "  E1: " + System.currentTimeMillis());
    boolean e2 = url.equals(this);
    System.out.println(s + "  E2: " + System.currentTimeMillis());
    assertEquals(h1,h2);
    assertEquals(e1,e2);
  }
}

        ...

   [junit4] Started J0 PID(31843@frisbee).
   [junit4] Suite: org.apache.solr.util.TestURLDNSAbsurdity
   [junit4]   1> 1 PRE: 1379107971313
   [junit4]   1> 1INIT: 1379107971314
   [junit4]   1> 1  H1: 1379107976449
   [junit4]   1> 1  H2: 1379107976449
   [junit4]   1> 1  E1: 1379107976449
   [junit4]   1> 1  E2: 1379107976449
   [junit4]   1> 2 PRE: 1379107976450
   [junit4]   1> 2INIT: 1379107976450
   [junit4]   1> 2  H1: 1379107981510
   [junit4]   1> 2  H2: 1379107981510
   [junit4]   1> 2  E1: 1379107981510
   [junit4]   1> 2  E2: 1379107981510
   [junit4] OK      10.3s | TestURLDNSAbsurdity.testHowSlowCanYouGo





-Hoss

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

 

 


Mime
View raw message