Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@manifoldcf.apache.org
Received-SPF: pass (athena.apache.org: domain of daddywri@gmail.com designates
 209.85.160.177 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1412164093.69966.YahooMailNeo@web141503.mail.bf1.yahoo.com>
References: <-4537991995572748453@unknownmsgid>
	<1412164093.69966.YahooMailNeo@web141503.mail.bf1.yahoo.com>
Date: Wed, 1 Oct 2014 08:05:46 -0400
Message-ID: 
 <CALUFAGBDLE4d-qD1GxzzHXbTdM+90mTdDq0p4n+2b6fmxwt4zA@mail.gmail.com>
Subject: Re: Wiki connector stuck crawling namespaces other than default
From: Karl Wright <daddywri@gmail.com>
To: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>,
 Kambiz Niktabar <niktabar@yahoo.com>
Content-Type: multipart/alternative; boundary=089e0122a5c2ef0f2205045b51d7

--089e0122a5c2ef0f2205045b51d7
Content-Type: text/plain; charset=UTF-8

Hi Kambiz,

The debugging output indicates that your namespace name is "404".  That
doesn't sound correct to me.

>>>>>>
GET
/wiki/api.php?format=xml&action=query&list=allpages&apnamespace=404&apfrom=Africa%3ATetianCarbonates&aplimit=500
HTTP/1.1
<<<<<<

I've gone back and looked at the code and can find no way that the
namespace would be corrupted.  But maybe this is actually correct.  Can you
send along a screen shot of the view page for the job?

Also, the wiki connector seeds documents in batches of 500 at a time.  It
uses the last title fetched in order to be able to find the next batch of
500.  So if there are a lot of documents, it will take a while to seed them
all.  In your log I see signs that this is what is happening.  Have a look
at all the GET requests and note the apfrom parameter.


Thanks,
Karl

--089e0122a5c2ef0f2205045b51d7
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div>Hi Kambiz,<br><br></div>The debugging =
output indicates that your namespace name is &quot;404&quot;.=C2=A0 That do=
esn&#39;t sound correct to me.<br><br>&gt;&gt;&gt;&gt;&gt;&gt;<br>GET /wiki=
/api.php?format=3Dxml&amp;action=3Dquery&amp;list=3Dallpages&amp;apnamespac=
e=3D404&amp;apfrom=3DAfrica%3ATetianCarbonates&amp;aplimit=3D500 HTTP/1.1<b=
r>&lt;&lt;&lt;&lt;&lt;&lt;<br><br></div>I&#39;ve gone back and looked at th=
e code and can find no way that the namespace would be corrupted.=C2=A0 But=
 maybe this is actually correct.=C2=A0 Can you send along a screen shot of =
the view page for the job?<br><br></div><div>Also, the wiki connector seeds=
 documents in batches of 500 at a time.=C2=A0 It uses the last title fetche=
d in order to be able to find the next batch of 500.=C2=A0 So if there are =
a lot of documents, it will take a while to seed them all.=C2=A0 In your lo=
g I see signs that this is what is happening.=C2=A0 Have a look at all the =
GET requests and note the apfrom parameter.<br><br></div><br></div>Thanks,<=
br>Karl<br><br><div><div><div><br></div></div></div></div>

--089e0122a5c2ef0f2205045b51d7--