any23-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HansBrende <...@git.apache.org>
Subject [GitHub] any23 issue #58: ANY23-324 Changed default html parser from NekoHTML to Jsou...
Date Tue, 23 Jan 2018 23:10:58 GMT
Github user HansBrende commented on the issue:

    https://github.com/apache/any23/pull/58
  
    @lewismc Then I did the same test again, with the following modification: instead of alternating
between the two parsers, I ran the same parser over and over again.
    
    NekoHTML code:
    
    ```
    public static void main(String[] args) throws IOException {
        String url = "https://en.wikipedia.org/wiki/List_of_compositions_by_Johann_Sebastian_Bach";
        String encoding = "utf-8";
        byte[] webpage = IOUtils.toString(new URL(url)).getBytes(encoding);
    
        TagSoupParsingConfiguration config = TagSoupParser.legacyConfig();
        long totalTime = 0;
        for (int i = 0; i < 6000; i++) {
            ByteArrayInputStream input = new ByteArrayInputStream(webpage);
    
            long start = System.currentTimeMillis();
            config.parse(input, url, encoding);
            long duration = System.currentTimeMillis() - start;
    
            long timeToAdd = i < 1000 ? 0 : duration; //SKIP FIRST 1000 ITERATIONS
            totalTime += timeToAdd;
    
            if (i % 100 == 0) {
                System.out.println(i);
            }
        }
    
        System.out.println("total time neko:  " + totalTime + " ms");
    
    }
    ```
    RESULT:
    <pre>
    total time neko:  522608 ms
    </pre>
    
    ---------------------------------------------
    
    Jsoup code (same as before, except I changed `config` to `JsoupConfig.instance`):
    ```
    public static void main(String[] args) throws IOException {
        String url = "https://en.wikipedia.org/wiki/List_of_compositions_by_Johann_Sebastian_Bach";
        String encoding = "utf-8";
        byte[] webpage = IOUtils.toString(new URL(url)).getBytes(encoding);
    
        TagSoupParsingConfiguration config = JsoupConfig.instance;
        long totalTime = 0;
        for (int i = 0; i < 6000; i++) {
            ByteArrayInputStream input = new ByteArrayInputStream(webpage);
    
            long start = System.currentTimeMillis();
            config.parse(input, url, encoding);
            long duration = System.currentTimeMillis() - start;
    
            long timeToAdd = i < 1000 ? 0 : duration; //SKIP FIRST 1000 ITERATIONS
            totalTime += timeToAdd;
    
            if (i % 100 == 0) {
                System.out.println(i);
            }
        }
    
        System.out.println("total time jsoup: " + totalTime + " ms");
    
    }
    ```
    
    <pre>
    total time jsoup: 437072 ms
    </pre>
    
    Obviously, this test isn't perfect, but I hope it helps!


---

Mime
View raw message