Mailing-List: contact connectors-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: connectors-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of daddywri@gmail.com designates
 209.85.216.47 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=wdBLzoMc6nSSxCzoAL6tQgiaLXj0vf6VuNE2cgF8nvX/XoroTC0mzxZBGolMm4IBce
         VcY3hNKvweV+j2H+O04ZW/azALggSmms+W3c1W7JiwZ87tX+03cPa62oka/qZeXSvwbe
         6u1DDsqVaaLZOWyV0Av12gQkTvY230lrVX39A=
MIME-Version: 1.0
In-Reply-To: <008001cbe2ca$ac1e2900$045a7b00$@efendi.ca>
References: <003101cbe289$9a9e8cb0$cfdba610$@efendi.ca>
	<AANLkTiknsvt4gXNPWZsyysXttdOEwuT8VqT166UhCLnE@mail.gmail.com>
	<004201cbe28f$e360e550$aa22aff0$@efendi.ca>
	<004301cbe290$6df4f850$49dee8f0$@efendi.ca>
	<AANLkTinHDiv4R3eWbiwQXJz-8=F_aKyH_rNDKSk9PiA0@mail.gmail.com>
	<004901cbe295$960c15d0$c2244170$@efendi.ca>
	<004d01cbe296$8a170f90$9e452eb0$@efendi.ca>
	<AANLkTinM95TRgwqS=txQ5U_Msv8KyPBRHRz37uXvetfZ@mail.gmail.com>
	<006e01cbe2b9$80ba6f00$822f4d00$@efendi.ca>
	<007f01cbe2c9$1c461550$54d23ff0$@efendi.ca>
	<008001cbe2ca$ac1e2900$045a7b00$@efendi.ca>
Date: Tue, 15 Mar 2011 02:57:22 -0400
Message-ID: <AANLkTinmVVRS5qc6ikhhyYuN_y+Ca3kzfvKFYzJYHBMh@mail.gmail.com>
Subject: Re: SOLR
From: Karl Wright <daddywri@gmail.com>
To: connectors-user@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1

No, all retrieval is being done by ManifoldCF.  Solr Cell does not retrieve it.

The RSS connector retrieves content from an RSS feed, which is
basically a bunch of references.  The feed itself is not indexed, but
the documents it refers to are.  If those documents, when you bring
them up in a browser, have login and navigation information, you may
well see these in the index.

The RSS connector can be configured to just index the document's
description or content information from the feed, and not the document
itself, but that requires you to change one of the settings for the
job.

Karl

On Tue, Mar 15, 2011 at 12:37 AM, Fuad Efendi <fuad@efendi.ca> wrote:
> Hi Karl,
>
> My only guess is we submit URI of a document to SOLR Cell, and Solr Cell
> retrieves it from Internet (using probably HttpClient and "may be" using own
> Robot signature?)
> Even in case of RSS...
> Only this can explain why I have "navigation" and "login" in SOLR index...
>
> Am I right?
>
>
> Thanks
>
>
>
> -----Original Message-----
> From: Fuad Efendi [mailto:fuad@efendi.ca]
> Sent: March-15-11 12:26 AM
> To: connectors-user@incubator.apache.org
> Subject: RE: SOLR
>
> UPDATE:
> SOLR 1.4.1 (june-2010) works fine with ManifoldCF trunk.
> SOLR trunk doesn't work, and I suspect bugs in TIKA...
>
> But it is strange :)
>
> I am looking at SOLR, each document contains huge array of "links",
> including many links to Yahoo login... something weird (it doesn't look like
> RSS)... but searchable.
>
>
> -----Original Message-----
> From: Fuad Efendi [mailto:fuad@efendi.ca]
> Sent: March-14-11 10:50 PM
> To: 'connectors-user@incubator.apache.org'
> Subject: RE: SOLR
>
>
> I just noticed:
> Currently, default for ManifoldCF is /update/extract, which corresponds to
> SOLR Cell request handler.
>
> So...
> It is EXTREMELY generic...
> http://wiki.apache.org/solr/ExtractingRequestHandler
>
>
>