incubator-bloodhound-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Bloodhound" <bloodhound-...@incubator.apache.org>
Subject [Apache Bloodhound] Proposals/BEP-0004 added
Date Thu, 22 Nov 2012 10:32:55 GMT
Page "Proposals/BEP-0004" was added by andrej
Content:
-------8<------8<------8<------8<------8<------8<------8<------8<--------

= BEP 4 : Improved search architecture #overview

[[PageOutline]]

|| '''PEP''' || 4 ||
|| '''Title''' || Improved search architecture ||
|| '''Version''' ||  ||
|| '''Last-Modified''' ||  ||
|| '''Author''' || The Bloodhound project ||
|| '''Status''' || Draft ||
|| '''Type''' || Standards Trac ||
|| '''Content-Type''' || [wiki:PageTemplates/Proposals text/x-trac-wiki] ||
|| '''Created''' ||  ||
|| '''Post-History''' ||  ||

----

== Abstract #abstract
This document is meant to gather community consensus on the implementation of the improved
search functionality. Simultaneously, we want to merge search and custom query functionality
under the same umbrella. This is the place to discuss it and make proposals. Main discussion
must be placed in mail list with [BEP-0004] subject.

== Motivation ==

Nowadays modern issue tracking system should provide advanced free text search functionality
combine with flexible query mechanism. There are a lot of user requests asking for better
and customizable search.

The term project is very generic and may be confusing considering the context. Therefore in
this specification the word product is used instead.

== Rationale #rationale

=== Quick search box

Quick search box should be just frontend for adhoc query. For example, if user searches for
“bla” string, user receives Search Result view with the following query:  text~=”bla”.
User can refine query in Query Result view.

Search box should accept the following search types:
  - free-text search: “bla”
  - free-text + query: bla status!=closed
  - query only: status!=closed

Other nice to have features:
  - Did you mean...
  - Suggestions during typing

=== Query Result view

Query Result view represents consistent view of query result for different resources. Result
may be represented as
  - All resources result tab - default, common for all resources columns e.g. name and description
  - Resource result tabs - resource specific fields are shown e.g. id, status, summary for
ticket.
  - Query Result view can be instructed to limit result to specific resource type e.g. show
only tickets in result. In this case, resource tabs can be omitted.

User can refine search conditions either by editing query or by using Query Builder.
User can specify what fields should be selected and in what results order should be applied.
A new order type is introduced for free-text search: score

Query Result view should support facets (http://searchhub.org/2009/09/02/faceted-search-with-solr/)
 for example:
  - All resources result tab can show facets for resource type and product
  - Ticket result tab can show facets for products and ticket status

Other nice to have features in no particular order:
  - Possibility to save a query for specific user and sharing of saved queries
  - Search highlighting

=== Resource Query 

New Resource Query should provide the following functionality:
  - free-text search support
  - facet support
  - it is a superset of TracQuery functionality
  - basic query expressions AND, OR, NOT, search by specific field, search [FROM TO] - TBD
(can be similar to SQL or lucene/solr like)
  - search through different resources not only tickets: wiki, milestones, changesets and
other pluggable resources
  - search through all resource fields
  - search through attachments, history and comments
  - multi-product aware - apply security context etc
  - order by free-text score. Score calculation can be configured, for example if found in
summary: id: score:100, score*10, in keywords: score*5, in components: score*3 …

Nice to have features
  - Support search in attachments of specific types e.g word, exel etc.

=== Resource Query  macro

Similar to existing TracQuery, New macro must be introduced to enable access to Resource Query
from Wiki.

=== Query Builder

A lot improvements may be introduced here, TBD

== Proposal #proposal

=== Implementation steps

We can start with small step by implementing search box functionality similar to trac:wiki:AdvancedSearch
and then add support for query, wiki macro and query builder/wizard.

=== Possible free text platforms

A few possible solutions were discussed on the mail list:

==== Whoosh

Whoosh links:
  - Product page: http://pypi.python.org/pypi/Whoosh/
 - Discussion on implementing Trac search using Whoosh: https://groups.google.com/forum/?fromgroups=#!msg/trac-dev/sbU-g0C6kvk/1_juL29aAtQJ

==== !PyLucen

[http://lucene.apache.org/pylucene/ PyLucene] is not a Lucene port but a Python wrapper around
Java Lucene. !PyLucene embeds a Java VM with Lucene into a Python process. 

==== Java-based servers.
Solr or !ElasticSearch

== Rejected ideas

== Backwards Compatibility #backwards-compatibility

TracQuery implementation must be changed to use Resource Query for data retrieval. 

Existing TracSearch syntax will be mapped to Resource Query.

== Reference Implementation #reference-implementation

== Resources #resources

== References #references
Proposals/BEP-0003 - Multi-product architecture

=== Trac proposals on this subject
  - trac:wiki:AdvancedSearch
  - trac:wiki:SearchRefactoring

=== Plugins on this subject
  - http://trac-hacks.org/wiki/FullTextSearchPlugin from Logica
  - http://trac-hacks.org/wiki/TracAdvancedSearchPlugin - uses Solr, claims to be designed
with extensions points

=== Existing search and query implementation
  - TracQuery
  - TracSearch

== Copyright #copyright

Copyright © 2009-2012 The [http://www.apache.org Apache Software Foundation] [[BR]] 
Licensed under the [http://www.apache.org/licenses/LICENSE-2.0 Apache License, Version 2.0].

Apache Bloodhound, Apache, the Apache feather logo, and the Apache Bloodhound project logo
are trademarks of The Apache Software Foundation.
-------8<------8<------8<------8<------8<------8<------8<------8<--------

--
Page URL: <https://issues.apache.org/bloodhound/wiki/Proposals/BEP-0004>
Apache Bloodhound <https://issues.apache.org/bloodhound/>
The Apache Bloodhound (incubating) issue tracker

This is an automated message. Someone added your email address to be
notified of changes on 'Proposals/BEP-0004' page.
If it was not you, please report to .

Mime
View raw message