lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ronald Wood <rw...@smarsh.com>
Subject Re: Is it safe to upgrade an existing field to docvalues?
Date Wed, 24 Aug 2016 14:13:45 GMT
Yes, Shawn, our indexes meet the requirements for atomic updates. 

We actually depend on atomic updates since our users can alter metadata about any of our indexed
records. We don’t have to incur the cost of a full re-index of a record for every change.
This is especially critical when a user does a bulk update of the status of 1 million records.
☺

- Ronald S. Wood


On 8/24/16, 09:56, "Shawn Heisey" <apache@elyograg.org> wrote:

    On 8/23/2016 2:01 PM, Ronald Wood wrote:
    > In general, is there a way to migrate existing indexes (we have petabytes of data)
by enabling docvalues and incrementally re-indexing? We expect the latter would take a month
using an atomic update process.
    
    One way to handle it is to build a new index with an updated
    configuration, then switch to the new index.  Since you're not running
    SolrCloud, you can switch by swapping the cores.  If you were running
    SolrCloud, you'd need to alias the old name to the new collection, which
    might involve deleting the old collection first.  Swapping cores in
    cloud mode will break things.
    
    The other replies you've gotten are interesting.  The approach using
    Atomic Updates will only work if your index meets the requirements for
    Atomic Updates.
    
    https://wiki.apache.org/solr/Atomic_Updates#Caveats_and_Limitations
    
    You've already said it would take a month using atomic update ... which
    might mean you've already thought about whether or not your index meets
    the requirements.
    
    Toke's tool looks quite interesting, and would probably do the job a lot
    faster than any other method.
    
    Thanks,
    Shawn
    
    


Mime
View raw message