lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: High CPU usage after import
Date Tue, 15 Apr 2014 02:49:24 GMT
So are you sending the MP3 files to Solr? That's actually generally a
bad practice, it places the load for analyzing all the files on Solr.
Yes, SolrCell makes this possible, and it's great for small data sets.
What I'd actually recommend is that you parse the files on a SolrJ
client using Tika and just send what you care about to Solr. It scales
much better that way.

Not a real solution to your specific problem, but still. See:
http://searchhub.org/2012/02/14/indexing-with-solrj/

FWIW,
Erick

On Mon, Apr 14, 2014 at 2:26 AM, Александр Вандышев
<a-wonder-v@rambler.ru> wrote:
> This does not happen in all the files. Maybe they're broken.
>
>
> Thanks for participating. Unfortunately it's not that. Set stored = "false"
> for
> all fields, nothing has changed.Re: High CPU usage after import
>
> Are you storing the data? That is, the raw binary of the MP3? B/c when
> stored="true", Solr will try to compress the data, perhaps that's
> what's driving the CPU utilization?
>
> Easy test: set stored="false" for everything..
>
> FWIW,
> Erick
>
> On Fri, Apr 11, 2014 at 5:23 AM, Александр Вандышев
> <a-wonder-v@rambler.ru> wrote:
>>
>> I realized what the problem was. One of the Solr threads freezes when
>> importing
>> MP3 files. When there are many such files Solr loads all processors. Is
>> there a
>> way to free thread?
>>
>> Re: High CPU usage after import That could mean that the code is hung
>> somehow.
>> Or, maybe Solr is just
>> working on the commit. Unless you have an explicit commit, the automatic
>> commit will occur some time after the extract request. How much data are
>> we
>> talking about?
>>
>> What does the Solr log say? Compare that to the case where CPU usage does
>> settle down.
>>
>> -- Jack Krupansky
>>
>> -----Original Message-----
>> From: Александр Вандышев
>> Sent: Thursday, April 3, 2014 3:24 AM
>> To: Solr User
>> Subject: High CPU usage after import
>>
>> Thanks for the answer. I meant that the CPU does not free after the end of
>> import.Tomtcat or Solr continue use it in max level.
>>
>> .
>>
>> Вт. 01 апр. 2014 20:09:24 пользователь Jack Krupansky
>> (jack@basetechnology.com)
>> написал:
>>
>>
>> Some document types can consume significant CPU resources, such as large
>> PDF
>> files.
>>
>> -- Jack Krupansky
>>
>> -----Original Message-----
>> From: Александр Вандышев
>> Sent: Tuesday, April 1, 2014 9:28 AM
>> To: Solr User
>> Subject: High CPU usage after import
>>
>> I use a update/extract handler for indexing a large number of files. If
>> during
>> indexing a CPU loads was not maximum at the end of import loading
>> decreases.
>> If

Mime
View raw message