Mailing-List: contact connectors-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: connectors-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of daddywri@gmail.com designates
 209.85.217.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALUFAGCpav5=wHA4TLirLe01yTzPdsg4iJF6vBC0mGoNOfwJBg@mail.gmail.com>
References: <4F995979.5080303@usit.uio.no>
	<CALUFAGCpav5=wHA4TLirLe01yTzPdsg4iJF6vBC0mGoNOfwJBg@mail.gmail.com>
Date: Thu, 26 Apr 2012 15:14:53 -0400
Message-ID: 
 <CALUFAGCzKKXOu7oFfJBzmRG9RqYCCfiG26kY5K8xSGF6Q7Z8+Q@mail.gmail.com>
Subject: Re: Ingestion API socket timeout exception waiting for response code
From: Karl Wright <daddywri@gmail.com>
To: connectors-user@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Erlend,

I had some time today and was able to verify that everything worked
fine against what I have currently on my laptop, which is Solr 3.2.
The second job run looks like this:

04-26-2012 15:11:44.154 	job end 	1335467343879(test)		0 	1 =09
04-26-2012 15:11:34.159 	document deletion (solr)
	file:/C:/testcrawl/there.txt	200 	0 	117
04-26-2012 15:11:24.690 	read document 	C:\testcrawl	OK 	0 	1 =09
04-26-2012 15:11:24.494 	job start 	1335467343879(test)		0 	1

So it appears that either something changed in Solr, or SSL support is
broken, or your network is not permitting a valid HTTP response for
some reason.

Karl


On Thu, Apr 26, 2012 at 11:10 AM, Karl Wright <daddywri@gmail.com> wrote:
> Hi Erlend,
>
> Can you try the following:
>
> (1) Make a fresh Solr checkout of 3.6 or whatever Solr version you are
> using, and build it
> (2) Start it
> (3) Run a simple filesystem crawl using a Solr connection that is
> created with the default values
> (4) Delete a file in your filesystem that was crawled
> (5) Crawl again
>
> Does the deletion happen OK?
>
> AFAIK, nothing has changed in the Solr connector that should affect
> the ability to delete. =A0This test will confirm that it is still
> working.
>
> Thanks,
> Karl
>
>
> On Thu, Apr 26, 2012 at 10:19 AM, Erlend Gar=E5sen
> <e.f.garasen@usit.uio.no> wrote:
>> It seems that MCF cannot delete documents from Solr. A timeout occurs, a=
nd
>> the job stops after a while.
>>
>> This is what I can see from the log:
>> =A0WARN 2012-04-20 18:24:30,373 (Worker thread '16') - Service interrupt=
ion
>> reported for job 1327930125433 connection 'Web crawler': Ingestion API
>> socket timeout exception waiting for response code: Read timed out;
>> ingestion will be retried again later
>>
>> If I take a further look in Simple History, it seems that this error is
>> related to document deletion.
>>
>> I have tried to delete the document manually by using curl from the same
>> server MCF is installed on in case we have some access restrictions, but
>> Curr succeeded.
>>
>> We do not have any problems with adding, the timeout only occurs while
>> deleting documents.
>>
>> I have checked our Solr configuration. MCF does use the correct path for
>> document deletion, i.e. /update.
>>
>> The correct realm, username and password for our Solr server are entered
>> correctly and the SSL certificate is valid as well.
>>
>> Erlend
>>
>> --
>> Erlend Gar=E5sen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31=
050