Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: Full text search - is it coming? If yes, approx when.
From: Martin Hewitt <martin@thenoi.se>
In-Reply-To: <AANLkTiknKs=OJsGuR8YVLXg=pwucMsbTZa9eWRL7QN2J@mail.gmail.com>
Date: Mon, 28 Mar 2011 16:16:50 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <E3746027-B752-4B3D-96DD-C79B878EA6BC@thenoi.se>
References: <AANLkTikj02TLqL6tqfLAnsHTPr2bctYhYA5OkoMshNKX@mail.gmail.com>
 <3D13B027-14D1-4EA6-9751-591173ED57D2@rgabostyle.com>
 <AANLkTim=7K4vBCPfzQgaUMevE-aJu4NpUx5qjeoNCMnt@mail.gmail.com>
 <AANLkTim2JUjfcR-ju24t_yY=p72dizsGJd+rXng6++sL@mail.gmail.com>
 <AF9AD1AC-0B41-4144-9828-DEF3BAAACC71@supercoders.com.au>
 <AANLkTin=iKNXK2jRAK-513eve9iwcCVanKKOzEjpUtU2@mail.gmail.com>
 <AANLkTindNch1JPcmgwshW4kkifop8Yxbc+oxE4XXu_Hg@mail.gmail.com>
 <4AEB3F68-AB89-4EED-A953-8FF1BC59CA0D@supercoders.com.au>
 <AANLkTinLpsFacey=cb2gm6XCSSAD6fv6N=67Oe4f1mTs@mail.gmail.com>
 <9F98111A-5E6B-479B-A770-A69467C980D4@supercoders.com.au>
 <AANLkTinbAHSLWeGqg53=Rwz-bm2N0RwHO-rT8Q3B2cj7@mail.gmail.com>
 <AANLkTiksch3=iTigX74c_1=7=U-_gnQ+14XgMY7nrPCq@mail.gmail.com>
 <AANLkTim5XyKndw5DYjQYhOC=-bXAbEzML1tH8sxA2q10@mail.gmail.com>
 <AANLkTimZVyH+qsRiyKUn2YRSO1XBJRi7-8wJr=F4_Vq1@mail.gmail.com>
 <AC2A56C0-4D4A-473B-A8A0-51D05C0B389B@supercoders.com.au>
 <AANLkTik4m_mRNefU4XYLc_ND_JVODiVyMWZDf5sfh_Da@mail.gmail.com>
  <AANLkTinEvD1tRhHdO_GwhchT_Xik3Kitw1C2t5YV4X32@mail.gmail.com>
 <A6B4E014-06C7-4953-9BC2-2797A7E02BE2@supercoders.com.au>
 <AANLkTiknKs=OJsGuR8YVLXg=pwucMsbTZa9eWRL7QN2J@mail.gmail.com>
To: user@couchdb.apache.org

Bit off-topic Robert, but CouchDB-lucene really is superb, we use it =
extensively and it's just brilliant, so thanks for contributing it to =
the Couch community.

Martin=20

On 28 Mar 2011, at 15:30, Robert Newson wrote:

> I am a CouchDB committer and author of couchdb-lucene. :)
>=20
> B.
>=20
> On 28 March 2011 10:44, Andrew Stuart (SuperCoders)
> <andrew.stuart@supercoders.com.au> wrote:
>> Hi Robert
>>=20
>> "there are no publicly known plans to build a native full-text =
indexing
>> feature for CouchDB."
>>=20
>> I don't know who is who around here as yet - are you commenting from =
inside
>> knowledge or as an end user/developer?
>>=20
>> Thanks
>>=20
>>=20
>> On 28/03/2011, at 8:24 PM, Robert Newson wrote:
>>=20
>> I have to dispute "There does not seem to be much understanding that
>> this could be a killer feature."
>>=20
>> Obviously full-text search is a killer feature, but it's trivially
>> available now via couchdb-lucene or elasticsearch.
>>=20
>> What people are asking for is native full-text search which, to me, =
is
>> essentially asking for an Erlang port of Lucene. We'd love this, but
>> it's a huge amount of work. Continually asking others to do
>> significant amounts of work is also wearying.
>>=20
>> To replace a Lucene-based solution and match its quality and breadth
>> is a huge chunk of work and is only necessary to satisfy people who,
>> for various reasons, don't want to use Java.
>>=20
>> To answer the original post, there are no publicly known plans to
>> build a native full-text indexing feature for CouchDB.
>>=20
>> B.
>>=20
>> On 28 March 2011 10:15, Olafur Arason <olafura@olafura.com> wrote:
>>>=20
>>> There does not seem to be much understanding that this could be a =
killer
>>> feature. People are now relying on Lucene which monitors the =
_changes
>>> feed.
>>>=20
>>> Cloudant has done it's own implementation which I gather through the
>>> information they have published makes a view out of all your word,
>>> they recommend java view because you can then reuse the lexer from
>>> Lucene. Then I think they are reusing the reader of the view to make
>>> their query. They have a similar syntax as Lucene for the query =
interface.
>>> They are still working on this and I think they don't have that much
>>> incentive to opensource it right away. But they have in past both
>>> opensourced there technology like BigCouch so I think it's more a
>>> matter of when rather then if.
>>>=20
>>> I think this is a good solution for a fulltext search. But I don't =
think
>>> that
>>> the java view does not have direct access to the data so it could be
>>> slow. But cloudant does clustering on view generation so that helps.
>>>=20
>>> But there is also general problem with the current view system where
>>> search technology could be used.
>>>=20
>>> The view are really good at sorting but people are using them to
>>> do key matches which they are not designed for. They beginkey and
>>> endkey are for sorting ranges and are not good for matching which
>>> most resources online are pointing to.
>>>=20
>>> For example when you do:
>>> beginkey =3D ["key11", "key21"]
>>> endkey =3D ["key19", "key21"]
>>>=20
>>> You get ["key11","key22"], ["key11", "key23"] ... ["key12","key21"],
>>> ["key12","key22"]...
>>> which makes sense when looking up sorting ranges but not using it to
>>> match keys. But you can have a range match lookup but only on the
>>> last key and never on two keys. So this would work:
>>>=20
>>> beginkey =3D ["key21", "key11"]
>>> endkey =3D ["key21", "key19"]
>>>=20
>>> The current view interface could be augmented to accept queries
>>> and could make them much more powerful then they currently are
>>> and just using the keys for sorting and selecting which values you
>>> want shown which they are designed to do and do really well.
>>>=20
>>> This would be a killer feature and could use the new infrastructure
>>> from Cloudant search.
>>>=20
>>> And don't tell me the Elastic or Lucene interface could do anything
>>> close to this :)
>>>=20
>>> Regards,
>>> Olafur Arason
>>>=20
>>> On Mon, Mar 28, 2011 at 04:31, Andrew Stuart (SuperCoders)
>>> <andrew.stuart@supercoders.com.au> wrote:
>>>>=20
>>>> It would be good to know if full text search is coming as a core =
feature
>>>> and
>>>> if yes, approximately when - does anyone know?
>>>>=20
>>>> Even an approximate timeframe would be good.
>>>>=20
>>>> thanks
>>>>=20
>>>=20
>> --
>> Message  protected by MailGuard: e-mail anti-virus, anti-spam and =
content
>> filtering.http://www.mailguard.com.au/mg
>> Click here to report this message as spam:
>> =
https://login.mailguard.com.au/report/1BZveI1wri/4izG2DWUCf9OUvbAh9DkfT/0
>>=20