Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of
 tobias.wunderlich@igd-r.fraunhofer.de designates 217.64.175.212 as permitted
 sender)
From: "Wunderlich, Tobias" <tobias.wunderlich@igd-r.fraunhofer.de>
To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org>
Subject: Creating custom Filter / Tokenizer / Request Handler for
 integration of NER-Framework
Thread-Topic: Creating custom Filter / Tokenizer / Request Handler for
 integration of NER-Framework
Thread-Index: Ac05qgw0vMoQoCyKSBOqNWywSLwnGA==
Date: Thu, 24 May 2012 12:36:59 +0000
Message-ID: 
 <A967EAEB85AE4D4CB8C6DF4E02A1E3071C249E2C@EXMBS1.ad.igd.fraunhofer.de>
Accept-Language: de-DE, en-US
Content-Language: de-DE
Content-Type: multipart/alternative;
	boundary="_000_A967EAEB85AE4D4CB8C6DF4E02A1E3071C249E2CEXMBS1adigdfrau_"
MIME-Version: 1.0

--_000_A967EAEB85AE4D4CB8C6DF4E02A1E3071C249E2CEXMBS1adigdfrau_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hey Guys,

I am recently working on a project to integrate a Named-Entity-Recognition-=
Framework (NER) in an existing searchplatform based on Solr. The Platform u=
ses ManifoldCF to automatically gather the content from various repositorie=
s. The NER-Framework creates Annotations/Metadata from given content which =
I then want to integrate into the search-platform as metadata to use for fa=
ceting. Since MCF handles all content gathering, I need a way to integrate =
the NER-Framework directly into Solr. The Goal is to get all Annotations pe=
r document into a multivalued field.  My first thought was to create a cust=
om filter, which just takes the content and gives back only the Annotations=
.  But as I understand it, a filter only processes predetermined Tokens, wh=
ich is useless for my purpose, since the NER-Framework needs to process the=
 whole content of a document. What about a custom Tokenizer? Would it be po=
ssible to process the whole text and give back only the Annotations as Toke=
ns? A third thought was to manipulate the ExtractRequestHandler (Solr Cell)=
 used by MCF to somehow add the Annotations as Metadata when the content an=
d metadata is distributed to the different fields.

I hope my problem description is sufficient. Does anybody have any thoughts=
 on that subject?

Best regards,
Tobias


--_000_A967EAEB85AE4D4CB8C6DF4E02A1E3071C249E2CEXMBS1adigdfrau_--