lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinay Pothnis <poth...@gmail.com>
Subject Re: deleting large amount data from solr cloud
Date Mon, 14 Apr 2014 04:09:14 GMT
Aman,
Yes - Will do!

Furkan,
How do you mean by 'bulk delete'?

-Thanks
Vinay


On 12 April 2014 14:49, Furkan KAMACI <furkankamaci@gmail.com> wrote:

> Hi;
>
> Do you get any problems when you index your data? On the other hand
> deleting as bulks and reducing the size of documents may help you not to
> hit OOM.
>
> Thanks;
> Furkan KAMACI
>
>
> 2014-04-12 8:22 GMT+03:00 Aman Tandon <amantandon.10@gmail.com>:
>
> > Vinay please share your experience after trying this solution.
> >
> >
> > On Sat, Apr 12, 2014 at 4:12 AM, Vinay Pothnis <pothnis@gmail.com>
> wrote:
> >
> > > The query is something like this:
> > >
> > >
> > > *curl -H 'Content-Type: text/xml' --data '<delete><query>param1:(val1
> OR
> > > val2) AND -param2:(val3 OR val4) AND date_param:[1383955200000 TO
> > > 1385164800000]</query></delete>'
> > > 'http://host:port/solr/coll-name1/update?commit=true'*
> > >
> > > Trying to restrict the number of documents deleted via the date
> > parameter.
> > >
> > > Had not tried the "distrib=false" option. I could give that a try.
> Thanks
> > > for the link! I will check on the cache sizes and autowarm values. Will
> > try
> > > and disable the caches when I am deleting and give that a try.
> > >
> > > Thanks Erick and Shawn for your inputs!
> > >
> > > -Vinay
> > >
> > >
> > >
> > > On 11 April 2014 15:28, Shawn Heisey <solr@elyograg.org> wrote:
> > >
> > > > On 4/10/2014 7:25 PM, Vinay Pothnis wrote:
> > > >
> > > >> When we tried to delete the data through a query - say 1 day/month's
> > > worth
> > > >> of data. But after deleting just 1 month's worth of data, the master
> > > node
> > > >> is going out of memory - heap space.
> > > >>
> > > >> Wondering is there any way to incrementally delete the data without
> > > >> affecting the cluster adversely.
> > > >>
> > > >
> > > > I'm curious about the actual query being used here.  Can you share
> it,
> > or
> > > > a redacted version of it?  Perhaps there might be a clue there?
> > > >
> > > > Is this a fully distributed delete request?  One thing you might try,
> > > > assuming Solr even supports it, is sending the same delete request
> > > directly
> > > > to each shard core with distrib=false.
> > > >
> > > > Here's a very incomplete list about how you can reduce Solr heap
> > > > requirements:
> > > >
> > > > http://wiki.apache.org/solr/SolrPerformanceProblems#
> > > > Reducing_heap_requirements
> > > >
> > > > Thanks,
> > > > Shawn
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > With Regards
> > Aman Tandon
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message