lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1931) Schema Browser does not scale with large indexes
Date Thu, 29 Dec 2011 12:51:30 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177153#comment-13177153
] 

Erick Erickson commented on SOLR-1931:
--------------------------------------

bq: What if one is looking at a completely optimized index?

I wondered about that myself, and I suspect this would work just as you indicate, optimizing
the index *would* get you the exact unique counts for all the fields. Which conveniently leaves
it up to the user to decide just how necessary getting exact information is....

Here's a code snippet from Muir (thanks!!!) that we should preserve, 4.x only.
    new ReaderUtil.Gather(reader) {
      @Override
      protected void add(int base, IndexReader r) throws IOException {
        System.out.println("segment: " + r.toString());
        FieldsEnum e = r.fields().iterator();
        String field;
        while ((field = e.next()) != null) {
          System.out.println("\t" + field + ": " +
e.terms().getUniqueTermCount());
        }
      }
    }.run();

segment: _34(4.0):C1802000/89498
        body: 4886489
        date: 136729
        datenum: 631685
        group100: 100
        group100K: 100000
        group10K: 10000
        group1M: 999999
        groupblock: 180200
        groupend: 1
        id: 1802000
        timesecnum: 73524
        title: 139038
        titleTokenized: 73144
segment: _67(4.0):C1802000/89561
        body: 4985143
                
> Schema Browser does not scale with large indexes
> ------------------------------------------------
>
>                 Key: SOLR-1931
>                 URL: https://issues.apache.org/jira/browse/SOLR-1931
>             Project: Solr
>          Issue Type: Improvement
>          Components: web gui
>    Affects Versions: 1.4
>            Reporter: Lance Norskog
>            Priority: Minor
>
> The Schema  Browser JSP by default causes the Luke handler to "scan the world". In large
indexes this make the UI useless.
> On an index with 64m documents & 8gb of disk space, the Schema Browser took 6 minutes
to open and hogged all disk I/O, making Solr useless.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message