Return-Path: 
 <dev-return-132913-apmail-lucene-dev-archive=lucene.apache.org@lucene.apache.org>
X-Original-To: apmail-lucene-dev-archive@www.apache.org
Delivered-To: apmail-lucene-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id EE3CBF783
	for <apmail-lucene-dev-archive@www.apache.org>;
 Thu, 25 Apr 2013 21:08:20 +0000 (UTC)
Received: (qmail 51891 invoked by uid 500); 25 Apr 2013 21:08:18 -0000
Delivered-To: apmail-lucene-dev-archive@lucene.apache.org
Received: (qmail 51826 invoked by uid 500); 25 Apr 2013 21:08:18 -0000
Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@lucene.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@lucene.apache.org>
List-Post: <mailto:dev@lucene.apache.org>
List-Id: <dev.lucene.apache.org>
Reply-To: dev@lucene.apache.org
Delivered-To: mailing list dev@lucene.apache.org
Received: (qmail 51779 invoked by uid 99); 25 Apr 2013 21:08:18 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Apr 2013 21:08:18 +0000
Date: Thu, 25 Apr 2013 21:08:18 +0000 (UTC)
From: =?utf-8?Q?Jan_H=C3=B8ydahl_=28JIRA=29?= <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <JIRA.12498247.1297347819944.226695.1366924098233@arcas>
In-Reply-To: <JIRA.12498247.1297347819944@arcas>
References: <JIRA.12498247.1297347819944@arcas>
Subject: [jira] [Commented] (SOLR-2356) indexing using DataImportHandler
 does not use entire CPU capacities
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/SOLR-2356?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D13642=
219#comment-13642219 ]=20

Jan H=C3=B8ydahl commented on SOLR-2356:
-----------------------------------

In my opinion, DIH should be completely redesigned as a standalone webapp. =
It is a major design flaw that it is a RequestHandler within a Solr Core/co=
llection.

As a standalone web app it could easily be deplyed on its own, talk to mult=
iple collections and be parallellized.
               =20
> indexing using DataImportHandler does not use entire CPU capacities
> -------------------------------------------------------------------
>
>                 Key: SOLR-2356
>                 URL: https://issues.apache.org/jira/browse/SOLR-2356
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>    Affects Versions: 4.0-ALPHA
>         Environment: intel xeon processor (4 cores), Debian Linux Lenny, =
OpenJDK 64bits server v1.6.0
>            Reporter: colby
>            Priority: Minor
>              Labels: test
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When I use a DataImportHandler to index a large number of documents (~35M=
), cpu usage doesn't go over than 100% cpu (i.e. just one core).
> When I configure 4 threads for the <entity> tag, the cpu usage is splitte=
d to 25% per core but never use 400% of cpu (i.e 100% of the 4 cores)
> I use solr embedded with jetty server.
> Is there a way to tune this feature in order to use all cores and improve=
 indexing performances ?
> Because for the moment, an extra script (PHP) gives better indexing perfo=
rmances than DIH.
> thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org