lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Vaillancourt <...@elementspace.com>
Subject Re: solrcloud shards backup/restoration
Date Fri, 06 Sep 2013 23:39:31 GMT
I wouldn't say I love this idea, but wouldn't it be safe to LVM snapshot
the Solr index? I think this may even work on a live server, depending on
some file I/O details. Has anyone tried this?

An in-Solr solution sounds more elegant, but considering the tlog concern
Shalin mentioned, I think this may work as an interim solution.

Cheers!

Tim


On 6 September 2013 15:41, Aditya Sakhuja <aditya.sakhuja@gmail.com> wrote:

> Thanks Shalin and Mark for your responses. I am on the same page about the
> conventions for taking the backup. However, I am less sure about the
> restoration of the index. Lets say we have 3 shards across 3 solrcloud
> servers.
>
> 1.> I am assuming we should take a backup from each of the shard leaders to
> get a complete collection. do you think that will get the complete index (
> not worrying about what is not hard committed at the time of backup ). ?
>
> 2.> How do we go about restoring the index in a fresh solrcloud cluster ?
> From the structure of the snapshot I took, I did not see any
> replication.properties or index.properties  which I see normally on a
> healthy solrcloud cluster nodes.
> if I have the snapshot named snapshot.20130905 does the snapshot.20130905/*
> go into data/index ?
>
> Thanks
> Aditya
>
>
>
> On Fri, Sep 6, 2013 at 7:28 AM, Mark Miller <markrmiller@gmail.com> wrote:
>
> > Phone typing. The end should not say "don't hard commit" - it should say
> > "do a hard commit and take a snapshot".
> >
> > Mark
> >
> > Sent from my iPhone
> >
> > On Sep 6, 2013, at 7:26 AM, Mark Miller <markrmiller@gmail.com> wrote:
> >
> > > I don't know that it's too bad though - its always been the case that
> if
> > you do a backup while indexing, it's just going to get up to the last
> hard
> > commit. With SolrCloud that will still be the case. So just make sure you
> > do a hard commit right before taking the backup - yes, it might miss a
> few
> > docs in the tran log, but if you are taking a back up while indexing, you
> > don't have great precision in any case - you will roughly get a snapshot
> > for around that time - even without SolrCloud, if you are worried about
> > precision and getting every update into that backup, you want to stop
> > indexing and commit first. But if you just want a rough snapshot for
> around
> > that time, in both cases you can still just don't hard commit and take a
> > snapshot.
> > >
> > > Mark
> > >
> > > Sent from my iPhone
> > >
> > > On Sep 6, 2013, at 1:13 AM, Shalin Shekhar Mangar <
> > shalinmangar@gmail.com> wrote:
> > >
> > >> The replication handler's backup command was built for pre-SolrCloud.
> > >> It takes a snapshot of the index but it is unaware of the transaction
> > >> log which is a key component in SolrCloud. Hence unless you stop
> > >> updates, commit your changes and then take a backup, you will likely
> > >> miss some updates.
> > >>
> > >> That being said, I'm curious to see how peer sync behaves when you try
> > >> to restore from a snapshot. When you say that you haven't been
> > >> successful in restoring, what exactly is the behaviour you observed?
> > >>
> > >> On Fri, Sep 6, 2013 at 5:14 AM, Aditya Sakhuja <
> > aditya.sakhuja@gmail.com> wrote:
> > >>> Hello,
> > >>>
> > >>> I was looking for a good backup / recovery solution for the solrcloud
> > >>> indexes. I am more looking for restoring the indexes from the index
> > >>> snapshot, which can be taken using the replicationHandler's backup
> > command.
> > >>>
> > >>> I am looking for something that works with solrcloud 4.3 eventually,
> > but
> > >>> still relevant if you tested with a previous version.
> > >>>
> > >>> I haven't been successful in have the restored index replicate across
> > the
> > >>> new replicas, after I restart all the nodes, with one node having the
> > >>> restored index.
> > >>>
> > >>> Is restoring the indexes on all the nodes the best way to do it ?
> > >>> --
> > >>> Regards,
> > >>> -Aditya Sakhuja
> > >>
> > >>
> > >>
> > >> --
> > >> Regards,
> > >> Shalin Shekhar Mangar.
> >
>
>
>
> --
> Regards,
> -Aditya Sakhuja
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message