lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Unload collection in SolrCloud
Date Thu, 22 May 2014 15:30:31 GMT
If we manage to extend the "lazy core" loading from stand-alone to
"lazy collection" loading in SolrCloud would that satisfy the
use-case? It still doesn't allow manual unloading of the collection,
but the large collection would "age out" if it was truly not used all
that much. That said, I don't know if there's active work in this
direction right now.

Best,
Erick

On Thu, May 22, 2014 at 5:35 AM, Saumitra Srivastav
<saumitra.srivastav7@gmail.com> wrote:
> Yes, that's what I am doing.
>
> IMO in addition to search, Solr satisfies the needs of lot of analytics
> applications as well, and on-demand loading is a common use case in
> analytics(to keep TCO low), so it would be nice to keep this supported.
>
>
> Regards,
> Saumitra
>
>
>
> On Thu, May 22, 2014 at 5:37 PM, Shalin Shekhar Mangar [via Lucene] <
> ml-node+s472066n4137630h17@n3.nabble.com> wrote:
>
>> Ah, I see. So if I understand it correctly, you are sharing the cluster
>> with other collections which are more frequently used and you want to keep
>> resources available for them so you keep your collection dormant most of
>> the time until requested.
>>
>> No, we don't have such an API. It'd be cool to have a lazy loaded
>> collection though. Thank you for describing the use-case because the way
>> that we're moving towards (ZK as truth etc.), the core admin APIs will
>> gradually be phased out and satisfying your use-case would become
>> impossible. Let me think more on this.
>>
>>
>> On Thu, May 22, 2014 at 4:57 PM, Saumitra Srivastav <
>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=0>>
>> wrote:
>>
>> > I don't want to delete the collection/shards. I just want to unload all
>> > shards/replica of the collection temporarily.
>> >
>> > Let me explain my use case.
>> >
>> > I have a collection alias say *collectionA* which consists of n
>> > collections(n<=5) each with 8 shards and 2 replica over a 16 machine
>> > cluster.
>> > *collectionA* is quite big in size and used very rarely, so we keep all
>> > shards/replica of *collectionA* unloaded most of the time. Only when
>> user
>> > request to use it, we load it in memory. To load/unload shards/replica
>> of
>> > aliased *collectionA*, we use CLUSTERSTATUS api to get list of all
>> > shards/replicas in aliased collection and then use CORE ADMIN api to
>> > load/unload them.
>> >
>> > As you can see there is lot of manual work involved, so I want to know
>> if
>> > there is an API to load/unload ALL shards/replicas of a collection?
>> >
>> >
>> > Regards,
>> > Saumitra
>> >
>> >
>> > On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] <
>> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=1>>
>> wrote:
>> >
>> > > You can use the delete Collection API.
>> > >
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
>> > >
>> > >
>> > >
>> > > On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
>> > > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137608&i=0>>
>>
>> > > wrote:
>> > >
>> > > > Guys, any suggestions for this??
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > View this message in context:
>> > > >
>> > >
>> >
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
>> > > > Sent from the Solr - User mailing list archive at Nabble.com.
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Regards,
>> > > Shalin Shekhar Mangar.
>> > >
>> > >
>> > > ------------------------------
>> > >  If you reply to this email, your message will be added to the
>> discussion
>> > > below:
>> > >
>> > >
>> >
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html
>> > >  To start a new topic under Solr - User, email
>> > > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=2>
>> > > To unsubscribe from Unload collection in SolrCloud, click here<
>> >
>> >
>> > > .
>> > > NAML<
>> >
>> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
>>
>> > >
>> > >
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> >
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html
>>
>> > Sent from the Solr - User mailing list archive at Nabble.com.
>> >
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>>
>> ------------------------------
>>  If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137630.html
>>  To start a new topic under Solr - User, email
>> ml-node+s472066n472068h95@n3.nabble.com
>> To unsubscribe from Unload collection in SolrCloud, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM=>
>> .
>> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137639.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message