Mailing-List: contact bloodhound-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: bloodhound-dev@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <20121128151735.03AEB803AF@bloodhound-vm>
References: <20121128151735.03AEB803AF@bloodhound-vm>
Date: Wed, 28 Nov 2012 16:39:04 +0100
Message-ID: 
 <CAHM0gnggtNqfry9Jh516V4Vreko6FSqgW0DMu1ET5d3fAi0-Vg@mail.gmail.com>
Subject: Re: [Apache Bloodhound] Proposals/BEP-0004/ResourceQuery added
From: Andrej Golcov <andrej@digiverse.si>
To: bloodhound-dev@incubator.apache.org
Content-Type: multipart/alternative; boundary=f46d040838655d0bfb04cf8ff7ff

--f46d040838655d0bfb04cf8ff7ff
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hi all,

I moved search query requirements, query syntax and use cases into separate
page:
https://issues.apache.org/bloodhound/wiki/Proposals/BEP-0004/ResourceQuery

It is just first draft, so let's discuss how new search query should look
like and what features we want from it.

Regards, Andrej

On 28 November 2012 16:17, Apache Bloodhound <
bloodhound-dev@incubator.apache.org> wrote:

> Page "Proposals/BEP-0004/ResourceQuery" was added by andrej
> Content:
> -------8<------8<------8<------8<------8<------8<------8<------8<--------
> =3D Resource Query component
> [[PageOutline]]
> =3D=3D Introduction #introduction
>
> This page describes functionality of Resource Query component. Resource
> Query component is responsible for resource indexing and query execution.
> It is not responsible for representation of search results to user. For
> overview of search and query solution see [wiki:BEP-0004].
>
> Usually user will not access to Resource Query component directly but via
> UI frontends e.g. search page, widget or wiki macro. Consider below a
> simple search workflow:
>  1. User searches for =93bla status:closed=94 string in quick search box
>  1. Quick search forwards user to search page with URL
> =85?q=3Dbla20%status:closed
>  1. Search page calls Resource Query component and calls Resource Query
> component with query =94bla status:closed=94 and other parameters e.d. fi=
elds,
> sort etc.
>  1. Search page renders query results in appropriate way
>
> Resource Query component will provide a !ResourceQuery.query method with
> the following parameters:
>  * '''query''': query string e.g. =93bla status:closed=94 or a parsed
> representation of the query . For more information see [#query_syntax Que=
ry
> syntax].
>  * '''sort''': optional sorting
>  * '''boost''': optional list of fields with boost values e.g. {=93id=94:
> 1000, =93subject=94 :100, =93description=94:10}. Used only for score base=
d sorting.
>  * '''filters''': optional list of terms. Usually can be cached by
> underlying search framework. For example {=93type=94: =93wiki=94}
>  * '''fields''': list of fields to return
>  * optional paging fields: '''rows/start''' or '''page/pagesize''' fields
>  * '''facets''' - optional list of facet terms, can be field or expressio=
n.
>
> =3D=3D Resource Query is not a report tool #notreport
> As it was discussed on dev mailing list, search and query serve a
> different purpose than reports. Resource Query is not intended not provid=
e
> complex SQL like expressions linke JOIN, UNION etc. Resource Query will
> search through flattened resource representation. Query syntax should
> support issue tracker specifics such as search through attachments, relat=
ed
> tickets etc.
>
> =3D=3D Query Syntax #query_syntax
> Resource Query will accept [
> http://lucene.apache.org/core/old_versioned_docs/versions/2_9_1/querypars=
ersyntax.htmlLucene-like] syntax familiar to users of Solr, [
> http://packages.python.org/Whoosh/querylang.html Whoosh], Haystack, [
> http://code.google.com/p/unladen-swallow/issues/searchtips Google Code]
> and [
> http://confluence.jetbrains.com/display/YTD4/Search+and+Command+Attribute=
s#SearchandCommandAttributes-GeneralSearchAttributesYouTrack] with addition=
al functions/meta tags specific for Bloodhound/Trac
> e.g. related tickets, attachment etc.
>
> Default Resource Query operator is AND.
>
> Bloodhound should provide it=92s own query parser in order to be independ=
ent
> from underlying search platforms.
>
> =3D=3D=3D Issue-tracker specifics #tracker_specifics
> Resource Query should be able to search through Bloodhound specific
> fields/functions:
>  * comments
>  * attachments
>  * history
>  * related resources with different relation types: linked, duplicated,
> blocked, child/parent etc.
>
> Resource Query should support version changing, similar to WAS and CHANGE=
D
> operator in JIRA (
> https://confluence.atlassian.com/display/JIRA/Advanced+Searching#Advanced=
Searching-WAS,
> http://confluence.jetbrains.com/display/YTD4/Search+and+Command+Attribute=
s
> )
>
> Other functions or meta tags can be used in query. Meta tags can be marke=
d
> with specific character e.g. =93#=94 (similar to YouTrack special keyword=
s -
> http://confluence.jetbrains.com/display/YTD4/Search+and+Command+Attribute=
s#SearchandCommandAttributes-ShortcutKeywords
> ):
>  * #me - current user
>  * #my - assigned to me
>  * #currentProject
>  * #ticket, #wiki etc.
>  * date and time helper functions e.g. 2weeksago, 1yearago etc.
>
> Indexing and query syntax must be easily extended by plugins. Here is not
> a complete list of other possible meta tags that can be provided by
> additional plugins:
>  * #resolved/unresolved - status:(resolved OR closed)
>  * version aggregation e.g. earliestUnreleasedVersion
>  * #hasAttachment
>  * code:xxx ... - contains code in wiki format
>  * #duplicated
>  * #closed =3D status:closed
>  * #yesterday
>  * ...
>
> =3D=3D Use cases #usecases
> =3D=3D=3D User uses free text search or query  in quick search box
> #usecase_freesearch
> User inputs text  or query string in search box. The input can be directl=
y
> propagated to query parameter, for example:
>  * bla
>  * open issue
>  * bla =93open issue=94
>  * bla status:open
>  * status:open
>
> =3D=3D=3D Possibility to specify what fields to return #usecase_fields
> Search page or widget must specify fields parameter of ResourceQuery.quer=
y
>  method.
> {{{
> #!python
> resourceQuery.query(fields=3D("id" , "title", "status"),...)
> }}}
>
> =3D=3D=3D Boolean operators and grouping #usecase_grouping
> Resource Query must support AND, OR, NOT and grouping (default operator i=
s
> AND). Query string may look like:
>  * alpha AND NOT (beta OR gamma)
>  * =93render AND shading=94  - expression is equal to =93render shading=
=94
>  * title:x OR ( title:y AND message:z)
>
> =3D=3D=3D User can search using range expression #usecase_range
> Query string parameter should support inclusive and exclusive range
> expression, for example:
>  * date:[20050101 TO 20090715]
>  * title:{Aida TO Carmen}
>  * [0025 TO]
>  * {TO suffix}
>
> =3D=3D=3D Facets support #usecase_facets
> Query must support facets  (e.g. Resources: Tickets(10), Wiki (20)),
> Status (Open (22), Closed(33)) etc. Facets parameter should be used for
> this purposes.
>
> {{{
> #!python
> resourceQuery.query(facets=3D("type", "status"), ...)
> }}}
>
> =3D=3D=3D Flexible sorting #usecase_sorting
> Default sort order of text-search should be based on score and change
> date. Search page can set the following parameters for !ResourceQuery.que=
ry
> method:
> {{{
> #!python
> resourceQuery.query( sort =3D {"score":ASC, "change_date": DESC},
>   boost =3D {"id" : 1000, "subject" : 100, "description": 10},...)
> }}}
>
> =3D=3D=3D Paging support #usecase_paging
> Search page will represent query results in pages. For this purposes, it
> should use the following parameters of !ResourceQuery.query method.
> {{{
> #!python
> resourceQuery.query(start =3D 100, rows=3D50, ...)
> }}}
> or
> {{{
> #!python
> resourceQuery.query(pagesize=3D50, page=3D3, ...)
> }}}
>
> =3D=3D=3D Related ticket use case #usecase_ralated
> User queries tickets related to tickets that were reopened in last 14
> days. The query can be exprese with the following call:
> {{{
> #!python
> resourceQuery.query(
>   query=3D"changed.status_from:open changed_date:[1weekago TO]",
>   facets=3D("parent_ticket"), ...
> )
> }}}
>
> =3D=3D=3D Search in comments #usecase_comments
> {{{
> #!python
> resourceQuery.query(query=3D"attachment:bla", ...)
> }}}
>
> =3D=3D=3D Show in attachment  #usecase_comments
> {{{
> #!python
> resourceQuery.query(query=3D"attachment:bla", ...)
> }}}
>
> =3D=3D=3D Show tickets that were commented since yesterday.
>  #usecase_last_commented
> {{{
> #!python
> resourceQuery.query(query=3D"last_commented:[yesterday TO]", ...)
> }}}
>
> =3D=3D=3D Show all resources in current project that links to a ticket
> #usecase_project_linked
> {{{
> #!python
> resourceQuery.query(query=3D"#currentProject AND linked:#123", ...)
> }}}
> -------8<------8<------8<------8<------8<------8<------8<------8<--------
>
> --
> Page URL: <
> https://issues.apache.org/bloodhound/wiki/Proposals/BEP-0004/ResourceQuer=
y
> >
> Apache Bloodhound <https://issues.apache.org/bloodhound/>
> The Apache Bloodhound (incubating) issue tracker
>
> This is an automated message. Someone added your email address to be
> notified of changes on 'Proposals/BEP-0004/ResourceQuery' page.
> If it was not you, please report to .
>

--f46d040838655d0bfb04cf8ff7ff--