lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aman Tandon <amantandon...@gmail.com>
Subject Re: deleting large amount data from solr cloud
Date Sat, 12 Apr 2014 05:22:56 GMT
Vinay please share your experience after trying this solution.


On Sat, Apr 12, 2014 at 4:12 AM, Vinay Pothnis <pothnis@gmail.com> wrote:

> The query is something like this:
>
>
> *curl -H 'Content-Type: text/xml' --data '<delete><query>param1:(val1 OR
> val2) AND -param2:(val3 OR val4) AND date_param:[1383955200000 TO
> 1385164800000]</query></delete>'
> 'http://host:port/solr/coll-name1/update?commit=true'*
>
> Trying to restrict the number of documents deleted via the date parameter.
>
> Had not tried the "distrib=false" option. I could give that a try. Thanks
> for the link! I will check on the cache sizes and autowarm values. Will try
> and disable the caches when I am deleting and give that a try.
>
> Thanks Erick and Shawn for your inputs!
>
> -Vinay
>
>
>
> On 11 April 2014 15:28, Shawn Heisey <solr@elyograg.org> wrote:
>
> > On 4/10/2014 7:25 PM, Vinay Pothnis wrote:
> >
> >> When we tried to delete the data through a query - say 1 day/month's
> worth
> >> of data. But after deleting just 1 month's worth of data, the master
> node
> >> is going out of memory - heap space.
> >>
> >> Wondering is there any way to incrementally delete the data without
> >> affecting the cluster adversely.
> >>
> >
> > I'm curious about the actual query being used here.  Can you share it, or
> > a redacted version of it?  Perhaps there might be a clue there?
> >
> > Is this a fully distributed delete request?  One thing you might try,
> > assuming Solr even supports it, is sending the same delete request
> directly
> > to each shard core with distrib=false.
> >
> > Here's a very incomplete list about how you can reduce Solr heap
> > requirements:
> >
> > http://wiki.apache.org/solr/SolrPerformanceProblems#
> > Reducing_heap_requirements
> >
> > Thanks,
> > Shawn
> >
> >
>



-- 
With Regards
Aman Tandon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message