Return-Path: Delivered-To: apmail-portals-jetspeed-dev-archive@www.apache.org Received: (qmail 71427 invoked from network); 29 Mar 2011 03:13:05 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 29 Mar 2011 03:13:05 -0000 Received: (qmail 26446 invoked by uid 500); 29 Mar 2011 03:13:05 -0000 Delivered-To: apmail-portals-jetspeed-dev-archive@portals.apache.org Received: (qmail 26418 invoked by uid 500); 29 Mar 2011 03:13:04 -0000 Mailing-List: contact jetspeed-dev-help@portals.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Jetspeed Developers List" Delivered-To: mailing list jetspeed-dev@portals.apache.org Received: (qmail 26401 invoked by uid 99); 29 Mar 2011 03:13:03 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Mar 2011 03:13:03 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Mar 2011 03:12:53 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 08DEB23888E4; Tue, 29 Mar 2011 03:12:32 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r1086467 [1/2] - in /portals/jetspeed-2/portal/trunk/components/jetspeed-search: ./ src/main/java/org/apache/jetspeed/search/ src/main/java/org/apache/jetspeed/search/lucene/ src/main/java/org/apache/jetspeed/search/solr/ src/test/java/org/... Date: Tue, 29 Mar 2011 03:12:31 -0000 To: jetspeed-dev@portals.apache.org From: woonsan@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20110329031232.08DEB23888E4@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: woonsan Date: Tue Mar 29 03:12:30 2011 New Revision: 1086467 URL: http://svn.apache.org/viewvc?rev=1086467&view=rev Log: JS2-1246: Adding solr search engine impl with embedded test env TODO: Fix unit test Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/SearchResultsImpl.java - copied, changed from r1086461, portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchResultsImpl.java portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/solr/ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/solr/SolrSearchEngineImpl.java portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/java/org/apache/jetspeed/search/solr/ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/java/org/apache/jetspeed/search/solr/TestSolrPortletRegistrySearch.java portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/admin-extra.html portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/elevate.xml portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/mapping-ISOLatin1Accent.txt portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/protwords.txt portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/schema.xml portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/scripts.conf portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/solrconfig.xml portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/spellings.txt portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/stopwords.txt portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/synonyms.txt portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/xslt/ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/xslt/example.xsl portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/xslt/example_atom.xsl portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/xslt/example_rss.xsl portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/xslt/luke.xsl Removed: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchResultsImpl.java Modified: portals/jetspeed-2/portal/trunk/components/jetspeed-search/pom.xml portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchEngineImpl.java Modified: portals/jetspeed-2/portal/trunk/components/jetspeed-search/pom.xml URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/pom.xml?rev=1086467&r1=1086466&r2=1086467&view=diff ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/pom.xml (original) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/pom.xml Tue Mar 29 03:12:30 2011 @@ -97,4 +97,18 @@ + + + + org.apache.maven.plugins + maven-surefire-plugin + + + **/TestSolrPortletRegistrySearch.java + + + + + + Copied: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/SearchResultsImpl.java (from r1086461, portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchResultsImpl.java) URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/SearchResultsImpl.java?p2=portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/SearchResultsImpl.java&p1=portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchResultsImpl.java&r1=1086461&r2=1086467&rev=1086467&view=diff ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchResultsImpl.java (original) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/SearchResultsImpl.java Tue Mar 29 03:12:30 2011 @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.apache.jetspeed.search.lucene; +package org.apache.jetspeed.search; import java.util.Iterator; import java.util.List; Modified: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchEngineImpl.java URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchEngineImpl.java?rev=1086467&r1=1086466&r2=1086467&view=diff ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchEngineImpl.java (original) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/lucene/SearchEngineImpl.java Tue Mar 29 03:12:30 2011 @@ -36,6 +36,7 @@ import org.apache.jetspeed.search.Object import org.apache.jetspeed.search.ParsedObject; import org.apache.jetspeed.search.SearchEngine; import org.apache.jetspeed.search.SearchResults; +import org.apache.jetspeed.search.SearchResultsImpl; import org.apache.lucene.analysis.Analyzer; import org.apache.lucene.analysis.standard.StandardAnalyzer; import org.apache.lucene.document.Document; Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/solr/SolrSearchEngineImpl.java URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/solr/SolrSearchEngineImpl.java?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/solr/SolrSearchEngineImpl.java (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/main/java/org/apache/jetspeed/search/solr/SolrSearchEngineImpl.java Tue Mar 29 03:12:30 2011 @@ -0,0 +1,520 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.jetspeed.search.solr; + +import java.io.IOException; +import java.net.URL; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.apache.commons.collections.MultiMap; +import org.apache.commons.collections.map.MultiValueMap; +import org.apache.commons.lang.StringUtils; +import org.apache.jetspeed.search.BaseParsedObject; +import org.apache.jetspeed.search.HandlerFactory; +import org.apache.jetspeed.search.ObjectHandler; +import org.apache.jetspeed.search.ParsedObject; +import org.apache.jetspeed.search.SearchEngine; +import org.apache.jetspeed.search.SearchResults; +import org.apache.jetspeed.search.SearchResultsImpl; +import org.apache.solr.client.solrj.SolrQuery; +import org.apache.solr.client.solrj.SolrServer; +import org.apache.solr.client.solrj.response.QueryResponse; +import org.apache.solr.client.solrj.response.UpdateResponse; +import org.apache.solr.common.SolrDocument; +import org.apache.solr.common.SolrDocumentList; +import org.apache.solr.common.SolrInputDocument; +import org.apache.solr.common.SolrInputField; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @version $Id: SolrSearchEngineImpl.java 1086464 2011-03-29 02:08:39Z woonsan $ + */ +public class SolrSearchEngineImpl implements SearchEngine +{ + private final static Logger log = LoggerFactory.getLogger(SolrSearchEngineImpl.class); + + private SolrServer server; + private boolean optimizeAfterUpdate = true; + private HandlerFactory handlerFactory; + + public SolrSearchEngineImpl(SolrServer server, boolean optimzeAfterUpdate, HandlerFactory handlerFactory) + { + this.server = server; + this.optimizeAfterUpdate = optimzeAfterUpdate; + this.handlerFactory = handlerFactory; + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEnging#add(java.lang.Object) + */ + public boolean add(Object o) + { + Collection c = new ArrayList(1); + c.add(o); + + return add(c); + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEnging#add(java.util.Collection) + */ + public boolean add(Collection objects) + { + return removeIfExistsAndAdd(objects); + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEnging#remove(java.lang.Object) + */ + public boolean remove(Object o) + { + Collection c = new ArrayList(1); + c.add(o); + + return remove(c); + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEnging#remove(java.util.Collection) + */ + public synchronized boolean remove(Collection objects) + { + int deleteCount = 0; + + try + { + Iterator it = objects.iterator(); + while (it.hasNext()) + { + Object o = it.next(); + // Look up appropriate handler + ObjectHandler handler = handlerFactory.getHandler(o); + + // Parse the object + ParsedObject parsedObject = handler.parseObject(o); + + if (parsedObject.getKey() != null) + { + String queryString = new StringBuilder(40).append(ParsedObject.FIELDNAME_KEY).append(':').append(parsedObject.getKey()).toString(); + // Remove the document from search index + UpdateResponse rsp = server.deleteByQuery(queryString); + deleteCount += rsp.getResponse().size(); + } + } + + if (deleteCount > 0 && optimizeAfterUpdate) + { + server.optimize(); + } + } + catch (Exception e) + { + log.error("Exception during removing documents in the search index.", e); + } + + return deleteCount > 0; + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEnging#update(java.lang.Object) + */ + public boolean update(Object o) + { + Collection c = new ArrayList(1); + c.add(o); + + return update(c); + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEnging#update(java.util.Collection) + */ + public boolean update(Collection objects) + { + return removeIfExistsAndAdd(objects); + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEngine#search(java.lang.String) + */ + public SearchResults search(String queryString) + { + return search(queryString, ParsedObject.FIELDNAME_SYNTHETIC); + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEngine#search(java.lang.String, java.lang.String) + */ + public SearchResults search(String queryString, String defaultFieldName) + { + return search(queryString, defaultFieldName, 0); + } + + /* (non-Javadoc) + * @see org.apache.jetspeed.search.SearchEngine#search(java.lang.String, java.lang.String, int) + */ + public SearchResults search(String queryString, String defaultFieldName, int topHitsCount) + { + SearchResults results = null; + + try + { + SolrQuery query = new SolrQuery(); + query.setQuery(queryString); + QueryResponse rsp = server.query(query); + SolrDocumentList docList = rsp.getResults(); + List resultList = new ArrayList(); + + for (SolrDocument doc : docList) + { + ParsedObject result = new BaseParsedObject(); + + addFieldsToParsedObject(doc, result); + + Object type = doc.getFieldValue(ParsedObject.FIELDNAME_TYPE); + if(type != null) + { + result.setType(type.toString()); + } + + Object key = doc.getFieldValue(ParsedObject.FIELDNAME_KEY); + if(key != null) + { + result.setKey(key.toString()); + } + + Object description = doc.getFieldValue(ParsedObject.FIELDNAME_DESCRIPTION); + if(description != null) + { + result.setDescription(description.toString()); + } + + Object title = doc.getFieldValue(ParsedObject.FIELDNAME_TITLE); + if(title != null) + { + result.setTitle(title.toString()); + } + + Object content = doc.getFieldValue(ParsedObject.FIELDNAME_CONTENT); + if(content != null) + { + result.setContent(content.toString()); + } + + Object language = doc.getFieldValue(ParsedObject.FIELDNAME_LANGUAGE); + if (language != null) + { + result.setLanguage(language.toString()); + } + + Object classname = doc.getFieldValue(ParsedObject.FIELDNAME_CLASSNAME); + if (classname != null) + { + result.setClassName(classname.toString()); + } + + Object url = doc.getFieldValue(ParsedObject.FIELDNAME_URL); + if (url != null) + { + result.setURL(new URL(url.toString())); + } + + Collection keywords = doc.getFieldValues(ParsedObject.FIELDNAME_KEYWORDS); + if(keywords != null) + { + String[] keywordArray = new String[keywords.size()]; + int index = 0; + + for (Object keyword : keywords) + { + keywordArray[index++] = keyword.toString(); + } + + result.setKeywords(keywordArray); + } + + resultList.add(result); + } + + results = new SearchResultsImpl(resultList); + } + catch (Exception e) + { + log.error("Failed to search. ", e); + } + + return (results != null ? results : new SearchResultsImpl(new ArrayList())); + } + + public boolean optimize() + { + try + { + server.optimize(); + return true; + } + catch (Exception e) + { + if (log.isDebugEnabled()) { + log.error("Error while trying to optimize index. " + e, e); + } else { + log.error("Error while trying to optimize index. {}", e.toString()); + } + } + + return false; + } + + private synchronized boolean removeIfExistsAndAdd(Collection objects) + { + try + { + Collection docs = new ArrayList(); + + Iterator it = objects.iterator(); + + while (it.hasNext()) + { + Object o = it.next(); + // Look up appropriate handler + ObjectHandler handler = null; + try + { + handler = handlerFactory.getHandler(o); + } + catch (Exception e) + { + log.error("Failed to create hanlder for object " + o.getClass().getName()); + continue; + } + + // Parse the object + ParsedObject parsedObject = handler.parseObject(o); + + String key = parsedObject.getKey(); + // if there's an existing one with the same key, then remove it first. + if (parsedObject.getKey() != null) + { + SolrQuery query = new SolrQuery(); + String queryString = new StringBuilder(40).append(ParsedObject.FIELDNAME_KEY).append(':').append(key).toString(); + query.setQuery(ParsedObject.FIELDNAME_KEY + ":" + key); + QueryResponse rsp = server.query(query); + + if (rsp.getResults().size() > 0) + { + server.deleteByQuery(queryString); + } + } + + String type = parsedObject.getType(); + String title = parsedObject.getTitle(); + String description = parsedObject.getDescription(); + String content = parsedObject.getContent(); + String language = parsedObject.getLanguage(); + URL url = parsedObject.getURL(); + String className = parsedObject.getClassName(); + + // Create document + SolrInputDocument doc = new SolrInputDocument(); + + // Populate document from the parsed object + if (key != null) + { + doc.addField(ParsedObject.FIELDNAME_KEY, key, 1.0f); + } + if (type != null) + { + doc.addField(ParsedObject.FIELDNAME_TYPE, type, 1.0f); + } + if (title != null) + { + doc.addField(ParsedObject.FIELDNAME_TITLE, title, 1.0f); + } + if (description != null) + { + doc.addField(ParsedObject.FIELDNAME_DESCRIPTION, description, 1.0f); + } + if (content != null) + { + doc.addField(ParsedObject.FIELDNAME_CONTENT, content, 1.0f); + } + if (language != null) + { + doc.addField(ParsedObject.FIELDNAME_LANGUAGE, language, 1.0f); + } + if (url != null) + { + String urlString = url.toString(); + doc.addField(ParsedObject.FIELDNAME_URL, urlString, 1.0f); + } + if (className != null) + { + doc.addField(ParsedObject.FIELDNAME_CLASSNAME, className, 1.0f); + } + + String[] keywordArray = parsedObject.getKeywords(); + if(keywordArray != null) + { + for(int i=0; i syntheticField = new ArrayList(); + for (Map.Entry entry : doc.entrySet()) + { + SolrInputField field = entry.getValue(); + Object value = field.getFirstValue(); + + if (value != null) + { + syntheticField.add(value.toString()); + } + } + + doc.addField(ParsedObject.FIELDNAME_SYNTHETIC, StringUtils.join(syntheticField, ' '), 1.0f); + + docs.add(doc); + } + + if (objects.size() > 0 && optimizeAfterUpdate) + { + server.add(docs); + server.commit(); + + try + { + server.optimize(); + } + catch (IOException e) + { + log.error("Error while trying to optimize index.", e); + } + } + } + catch (Exception e) + { + log.error("Error while writing index.", e); + return false; + } + + return true; + } + + private void addFieldsToDocument(SolrInputDocument doc, Map fields) + { + if(fields != null) + { + Iterator keyIter = fields.keySet().iterator(); + while(keyIter.hasNext()) + { + Object key = keyIter.next(); + if(key != null) + { + Object values = fields.get(key); + if(values != null) + { + if(values instanceof Collection) + { + Iterator valueIter = ((Collection)values).iterator(); + while(valueIter.hasNext()) + { + Object value = valueIter.next(); + if(value != null) + { + doc.addField(key.toString(), value.toString(), 1.0f); + } + } + } + else + { + doc.addField(key.toString(), values.toString(), 1.0f); + } + } + } + } + } + } + + private void addFieldsToParsedObject(SolrDocument doc, ParsedObject o) + { + try + { + MultiMap multiKeywords = new MultiValueMap(); + MultiMap multiFields = new MultiValueMap(); + HashMap fieldMap = new HashMap(); + + Object classNameField = doc.getFieldValue(ParsedObject.FIELDNAME_CLASSNAME); + + if(classNameField != null) + { + String className = classNameField.toString(); + o.setClassName(className); + ObjectHandler handler = handlerFactory.getHandler(className); + + Set fields = handler.getFields(); + addFieldsToMap(doc, fields, multiFields); + addFieldsToMap(doc, fields, fieldMap); + + Set keywords = handler.getKeywords(); + addFieldsToMap(doc, keywords, multiKeywords); + } + + o.setKeywordsMap(multiKeywords); + o.setFields(multiFields); + o.setFields(fieldMap); + } + catch (Exception e) + { + log.error("Error trying to add fields to parsed object.", e); + } + } + + private void addFieldsToMap(SolrDocument doc, Set fieldNames, Map fields) + { + Iterator fieldIter = fieldNames.iterator(); + while(fieldIter.hasNext()) + { + String fieldName = (String)fieldIter.next(); + Collection values = doc.getFieldValues(fieldName); + + if (values != null) + { + for (Object value : values) + { + fields.put(fieldName, value.toString()); + } + } + } + } + +} Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/java/org/apache/jetspeed/search/solr/TestSolrPortletRegistrySearch.java URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/java/org/apache/jetspeed/search/solr/TestSolrPortletRegistrySearch.java?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/java/org/apache/jetspeed/search/solr/TestSolrPortletRegistrySearch.java (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/java/org/apache/jetspeed/search/solr/TestSolrPortletRegistrySearch.java Tue Mar 29 03:12:30 2011 @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.jetspeed.search.solr; + +import java.io.File; +import java.util.Arrays; +import java.util.HashMap; +import java.util.Map; + +import junit.framework.Test; +import junit.framework.TestSuite; + +import org.apache.commons.collections.MultiHashMap; +import org.apache.jetspeed.search.AbstractObjectHandler; +import org.apache.jetspeed.search.BaseParsedObject; +import org.apache.jetspeed.search.ParsedObject; +import org.apache.jetspeed.search.SearchEngine; +import org.apache.jetspeed.search.SearchResults; +import org.apache.jetspeed.search.handlers.HandlerFactoryImpl; +import org.apache.jetspeed.test.JetspeedTestCase; +import org.apache.solr.client.solrj.SolrServer; +import org.apache.solr.client.solrj.embedded.EmbeddedSolrServer; +import org.apache.solr.core.CoreContainer; + +/** + * TestSolrPortletRegistrySearch + * @version $Id: TestSolrPortletRegistrySearch.java 1086464 2011-03-29 02:08:39Z woonsan $ + */ +public class TestSolrPortletRegistrySearch extends JetspeedTestCase +{ + private SolrServer server; + private SearchEngine searchEngine; + + public TestSolrPortletRegistrySearch(String name) + { + super(name); + } + + /** + * Start the tests. + * + * @param args the arguments. Not used + */ + public static void main(String args[]) + { + junit.awtui.TestRunner.main( new String[] { TestSolrPortletRegistrySearch.class.getName() } ); + } + + /** + * Creates the test suite. + * + * @return a test suite (TestSuite) that includes all methods + * starting with "test" + */ + public static Test suite() + { + // All methods starting with "test" will be executed in the test suite. + return new TestSuite( TestSolrPortletRegistrySearch.class ); + } + + protected void setUp() throws Exception + { + super.setUp(); + HashMap mapping = new HashMap(); + mapping.put("java.util.HashMap", MapObjectHandler.class.getName()); + HandlerFactoryImpl hfi = new HandlerFactoryImpl(mapping); + + File targetDir = new File("target/test-classes"); + System.setProperty("solr.solr.home", targetDir.getCanonicalPath()); + CoreContainer.Initializer initializer = new CoreContainer.Initializer(); + CoreContainer coreContainer = initializer.initialize(); + server = new EmbeddedSolrServer(coreContainer, ""); + + searchEngine = new SolrSearchEngineImpl(server, true, hfi); + } + + protected void tearDown() throws Exception + { + super.tearDown(); + } + + public void testSimpleSearch() + { + Map paDemo = new HashMap(); + paDemo.put("keyPrefix", "PortletApplication::"); + paDemo.put("description", "demo portlet application"); + paDemo.put("title", "Demo"); + paDemo.put("name", "demo"); + paDemo.put("type", ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION); + + Map paRss = new HashMap(); + paRss.put("keyPrefix", "PortletApplication::"); + paRss.put("description", "rss portlet application"); + paRss.put("title", "RSS"); + paRss.put("name", "rss"); + paRss.put("type", ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION); + + searchEngine.add(paDemo); + searchEngine.add(paRss); + + SearchResults searchResults = searchEngine.search("demo"); + assertEquals(1, searchResults.size()); + + searchResults = searchEngine.search("rss"); + assertEquals(1, searchResults.size()); + + searchResults = searchEngine.search("application"); + assertEquals(2, searchResults.size()); + + // adding one more; the search engine is expected to have duplicate index. + searchEngine.add(paDemo); + + searchResults = searchEngine.search("demo"); + assertEquals(1, searchResults.size()); + + searchResults = searchEngine.search("application"); + assertEquals(2, searchResults.size()); + } + + public void testPortletSearch() + { + Map paDemo = new HashMap(); + paDemo.put("keyPrefix", "PortletApplication::"); + paDemo.put("description", "demo portlet application"); + paDemo.put("title", "Demo"); + paDemo.put("name", "demo"); + paDemo.put("type", ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION); + + Map helloPortlet = new HashMap(); + helloPortlet.put("keyPrefix", "PortletDefinition::"); + helloPortlet.put("description", "hello portlet definition"); + helloPortlet.put("title", "Hello World"); + helloPortlet.put("name", "hello"); + helloPortlet.put("type", ParsedObject.OBJECT_TYPE_PORTLET); + + Map guessPortlet = new HashMap(); + guessPortlet.put("keyPrefix", "PortletDefinition::"); + guessPortlet.put("description", "guess portlet definition"); + guessPortlet.put("title", "Guess - Pick A Number"); + guessPortlet.put("name", "guess"); + guessPortlet.put("type", ParsedObject.OBJECT_TYPE_PORTLET); + + searchEngine.add(paDemo); + searchEngine.add(Arrays.asList(helloPortlet, guessPortlet)); + + SearchResults searchResults = searchEngine.search("demo"); + assertEquals(1, searchResults.size()); + + searchResults = searchEngine.search("hello"); + assertEquals(1, searchResults.size()); + + searchResults = searchEngine.search("guess"); + assertEquals(1, searchResults.size()); + + searchResults = searchEngine.search("definition"); + assertEquals(2, searchResults.size()); + } + + public void testPortletSearchByRichQuery() + { + Map paDemo = new HashMap(); + paDemo.put("keyPrefix", "PortletApplication::"); + paDemo.put("description", "demo portlet application"); + paDemo.put("title", "Demo"); + paDemo.put("name", "demo"); + paDemo.put("type", ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION); + + Map helloPortlet = new HashMap(); + helloPortlet.put("keyPrefix", "PortletDefinition::"); + helloPortlet.put("description", "demo hello portlet definition"); + helloPortlet.put("title", "Hello World"); + helloPortlet.put("name", "hello"); + helloPortlet.put("type", ParsedObject.OBJECT_TYPE_PORTLET); + + Map guessPortlet = new HashMap(); + guessPortlet.put("keyPrefix", "PortletDefinition::"); + guessPortlet.put("description", "demo guess portlet definition"); + guessPortlet.put("title", "Guess - Pick A Number"); + guessPortlet.put("name", "guess"); + guessPortlet.put("type", ParsedObject.OBJECT_TYPE_PORTLET); + + searchEngine.add(paDemo); + searchEngine.add(Arrays.asList(helloPortlet, guessPortlet)); + + SearchResults searchResults = searchEngine.search("demo"); + assertEquals(3, searchResults.size()); + + String query = ParsedObject.FIELDNAME_TYPE + ":\"" + ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION + "\" AND ( demo )"; + + searchResults = searchEngine.search(query); + assertEquals(1, searchResults.size()); + + query = ParsedObject.FIELDNAME_TYPE + ":\"" + ParsedObject.OBJECT_TYPE_PORTLET + "\" " + + "AND NOT " + ParsedObject.FIELDNAME_TYPE + ":\"" + ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION + "\" " + + "AND ( demo )"; + + searchResults = searchEngine.search(query); + assertEquals(2, searchResults.size()); + + query = ParsedObject.FIELDNAME_TYPE + ":\"" + ParsedObject.OBJECT_TYPE_PORTLET + "\" " + + "AND NOT " + ParsedObject.FIELDNAME_TYPE + ":\"" + ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION + "\" " + + "AND ( hello )"; + + searchResults = searchEngine.search(query); + assertEquals(1, searchResults.size()); + + query = ParsedObject.FIELDNAME_TYPE + ":\"" + ParsedObject.OBJECT_TYPE_PORTLET + "\" " + + "AND NOT " + ParsedObject.FIELDNAME_TYPE + ":\"" + ParsedObject.OBJECT_TYPE_PORTLET_APPLICATION + "\" " + + "AND ( guess )"; + + searchResults = searchEngine.search(query); + assertEquals(1, searchResults.size()); + } + + public static class MapObjectHandler extends AbstractObjectHandler + { + private String keyPrefix; + + public ParsedObject parseObject(Object o) + { + BaseParsedObject result = null; + + if(o instanceof Map) + { + result = new BaseParsedObject(); + Map map = (Map) o; + + keyPrefix = map.get("keyPrefix"); + + result.setDescription(map.get("description")); + + result.setTitle(map.get("title")); + result.setKey(keyPrefix + map.get("name")); + result.setType(map.get("type")); + result.setClassName(map.get("class")); + + MultiHashMap fieldMap = new MultiHashMap(); + fieldMap.put(ParsedObject.ID, map.get("name")); + } + + return result; + } + } +} Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/admin-extra.html URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/admin-extra.html?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/admin-extra.html (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/admin-extra.html Tue Mar 29 03:12:30 2011 @@ -0,0 +1,31 @@ + + + Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/elevate.xml URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/elevate.xml?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/elevate.xml (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/elevate.xml Tue Mar 29 03:12:30 2011 @@ -0,0 +1,36 @@ + + + + + + + + + + + + + + + + + + Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/mapping-ISOLatin1Accent.txt URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/mapping-ISOLatin1Accent.txt?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/mapping-ISOLatin1Accent.txt (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/mapping-ISOLatin1Accent.txt Tue Mar 29 03:12:30 2011 @@ -0,0 +1,246 @@ +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Syntax: +# "source" => "target" +# "source".length() > 0 (source cannot be empty.) +# "target".length() >= 0 (target can be empty.) + +# example: +# "À" => "A" +# "\u00C0" => "A" +# "\u00C0" => "\u0041" +# "ß" => "ss" +# "\t" => " " +# "\n" => "" + +# À => A +"\u00C0" => "A" + +# Á => A +"\u00C1" => "A" + +#  => A +"\u00C2" => "A" + +# à => A +"\u00C3" => "A" + +# Ä => A +"\u00C4" => "A" + +# Å => A +"\u00C5" => "A" + +# Æ => AE +"\u00C6" => "AE" + +# Ç => C +"\u00C7" => "C" + +# È => E +"\u00C8" => "E" + +# É => E +"\u00C9" => "E" + +# Ê => E +"\u00CA" => "E" + +# Ë => E +"\u00CB" => "E" + +# Ì => I +"\u00CC" => "I" + +# Í => I +"\u00CD" => "I" + +# Î => I +"\u00CE" => "I" + +# Ï => I +"\u00CF" => "I" + +# IJ => IJ +"\u0132" => "IJ" + +# Ð => D +"\u00D0" => "D" + +# Ñ => N +"\u00D1" => "N" + +# Ò => O +"\u00D2" => "O" + +# Ó => O +"\u00D3" => "O" + +# Ô => O +"\u00D4" => "O" + +# Õ => O +"\u00D5" => "O" + +# Ö => O +"\u00D6" => "O" + +# Ø => O +"\u00D8" => "O" + +# Œ => OE +"\u0152" => "OE" + +# Þ +"\u00DE" => "TH" + +# Ù => U +"\u00D9" => "U" + +# Ú => U +"\u00DA" => "U" + +# Û => U +"\u00DB" => "U" + +# Ü => U +"\u00DC" => "U" + +# Ý => Y +"\u00DD" => "Y" + +# Ÿ => Y +"\u0178" => "Y" + +# à => a +"\u00E0" => "a" + +# á => a +"\u00E1" => "a" + +# â => a +"\u00E2" => "a" + +# ã => a +"\u00E3" => "a" + +# ä => a +"\u00E4" => "a" + +# å => a +"\u00E5" => "a" + +# æ => ae +"\u00E6" => "ae" + +# ç => c +"\u00E7" => "c" + +# è => e +"\u00E8" => "e" + +# é => e +"\u00E9" => "e" + +# ê => e +"\u00EA" => "e" + +# ë => e +"\u00EB" => "e" + +# ì => i +"\u00EC" => "i" + +# í => i +"\u00ED" => "i" + +# î => i +"\u00EE" => "i" + +# ï => i +"\u00EF" => "i" + +# ij => ij +"\u0133" => "ij" + +# ð => d +"\u00F0" => "d" + +# ñ => n +"\u00F1" => "n" + +# ò => o +"\u00F2" => "o" + +# ó => o +"\u00F3" => "o" + +# ô => o +"\u00F4" => "o" + +# õ => o +"\u00F5" => "o" + +# ö => o +"\u00F6" => "o" + +# ø => o +"\u00F8" => "o" + +# œ => oe +"\u0153" => "oe" + +# ß => ss +"\u00DF" => "ss" + +# þ => th +"\u00FE" => "th" + +# ù => u +"\u00F9" => "u" + +# ú => u +"\u00FA" => "u" + +# û => u +"\u00FB" => "u" + +# ü => u +"\u00FC" => "u" + +# ý => y +"\u00FD" => "y" + +# ÿ => y +"\u00FF" => "y" + +# ff => ff +"\uFB00" => "ff" + +# fi => fi +"\uFB01" => "fi" + +# fl => fl +"\uFB02" => "fl" + +# ffi => ffi +"\uFB03" => "ffi" + +# ffl => ffl +"\uFB04" => "ffl" + +# ſt => ft +"\uFB05" => "ft" + +# st => st +"\uFB06" => "st" Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/protwords.txt URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/protwords.txt?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/protwords.txt (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/protwords.txt Tue Mar 29 03:12:30 2011 @@ -0,0 +1,21 @@ +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#----------------------------------------------------------------------- +# Use a protected word file to protect against the stemmer reducing two +# unrelated words to the same base word. + +# Some non-words that normally won't be encountered, +# just to test that they won't be stemmed. +dontstems +zwhacky + Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/schema.xml URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/schema.xml?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/schema.xml (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/schema.xml Tue Mar 29 03:12:30 2011 @@ -0,0 +1,569 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + id + + + text + + + + + + + + + + + + + + + + + + + + + + + + + + + + Added: portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/scripts.conf URL: http://svn.apache.org/viewvc/portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/scripts.conf?rev=1086467&view=auto ============================================================================== --- portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/scripts.conf (added) +++ portals/jetspeed-2/portal/trunk/components/jetspeed-search/src/test/resources/conf/scripts.conf Tue Mar 29 03:12:30 2011 @@ -0,0 +1,24 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +user= +solr_hostname=localhost +solr_port=8983 +rsyncd_port=18983 +data_dir= +webapp_name=solr +master_host= +master_data_dir= +master_status_dir= --------------------------------------------------------------------- To unsubscribe, e-mail: jetspeed-dev-unsubscribe@portals.apache.org For additional commands, e-mail: jetspeed-dev-help@portals.apache.org