Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id F1138200C04 for ; Tue, 24 Jan 2017 15:34:32 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id EFA69160B3D; Tue, 24 Jan 2017 14:34:32 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EED3D160B31 for ; Tue, 24 Jan 2017 15:34:31 +0100 (CET) Received: (qmail 83396 invoked by uid 500); 24 Jan 2017 14:34:31 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 83386 invoked by uid 99); 24 Jan 2017 14:34:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Jan 2017 14:34:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id ADF3F180661 for ; Tue, 24 Jan 2017 14:34:30 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.499 X-Spam-Level: ** X-Spam-Status: No, score=2.499 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gridgain-com.20150623.gappssmtp.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id LngfwgcOgXRN for ; Tue, 24 Jan 2017 14:34:28 +0000 (UTC) Received: from mail-io0-f171.google.com (mail-io0-f171.google.com [209.85.223.171]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 685B45F23A for ; Tue, 24 Jan 2017 14:34:27 +0000 (UTC) Received: by mail-io0-f171.google.com with SMTP id l66so136306657ioi.1 for ; Tue, 24 Jan 2017 06:34:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gridgain-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=WsHqQULts9hAqpQDFfQ21mIDPR40TScFDO8/7mQTkfA=; b=R/PMCZjuGwH3kdTYiMYPiu1/PuIwXwYApyu3uj7ovDb4pPRSljT3exPTZOUekEl3Qi zZ1GuY6qe3/CItmbOo3cBzXZxwiEJoEhoKVCFiAJuK4SMxAM9kurvgaYy2Q1wWojwqD9 VZPErPmH5p5GjbUc9Lu0xz7NapDUQHSR4W8XTCBFegrBaEj/pt0SUWsgNJjOitmsZhB+ ynJc3pM5tFbm9dWXPcQotYzRS4VLBMYyaQYmoICw7KII17Om6WvC8ats0L0iUNbUx7xA 2gT9jAULxcLSojyxu1n96SuQbXdAcGJY274Racbj3HY1qoH+jFmOhAwxq8zlofW0Oy+a JFZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=WsHqQULts9hAqpQDFfQ21mIDPR40TScFDO8/7mQTkfA=; b=cQwKkB1PLoqs6i6C0UEFHsZlbrxGxINVJqDscwO9jvA00uueUzgf3SrAG9h7OZJy2D Zkm5Hxd1eL/aP1k40cggtcPBY6pj85ASoUL4W8D1Z1lHIfVxKW4X2u671GwVebYuH9Bi M8lNx77ys+cw/VNjkCfkPk+FwH7pyKRYOjso02Gw+oySjHFLfkbunSjKt6VdH2qgF744 u8rfpuEPtFxkYmGcIfFMT4pRzhsgueOGX5sJqyyBxBlFGxe1+LOMT37Ckz3V3qwd7r5A aA8GOfp7aDMtaIG3wB0IhtXrlbs4FrIIpqXM+tijmxA+Ddl1lkS/qOxixxKL7CBIir/H z88g== X-Gm-Message-State: AIkVDXLnJY4DtARm127o6m8jSsRr9k7SHacqhDa/eRA/K+Rf+Tp7i5wjMQVhuFv3M4NW5ERcLeUlcwUMuf4z9l+i X-Received: by 10.107.48.200 with SMTP id w191mr30576836iow.68.1485268433242; Tue, 24 Jan 2017 06:33:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.171.132 with HTTP; Tue, 24 Jan 2017 06:33:12 -0800 (PST) In-Reply-To: <1485181865.207423454@f75.i.mail.ru> References: <1484831575.151575586@f26.i.mail.ru> <1485181865.207423454@f75.i.mail.ru> From: Andrey Mashenkov Date: Tue, 24 Jan 2017 17:33:12 +0300 Message-ID: Subject: Re: Re[4]: Faceted search To: user@ignite.apache.org, Mikhail Content-Type: multipart/alternative; boundary=001a11444da45ca57e0546d8019d archived-at: Tue, 24 Jan 2017 14:34:33 -0000 --001a11444da45ca57e0546d8019d Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Mikhail, With 1-st approach 1. Underlying storage can be changed without rewiring its code to add indexing support. 2. IndexSPI will updated in writeThrough(false) mode, however CacheStore won't. With 2-nd 1. You should bother about transaction handling (implementing CacheStore.sessionEnd() method). 2. If you use Lucene as CacheStore in readTrough(true), you should store whole entries in Lucene index to be able return entries via load\loadAll methods. This will significantly increase index size. On Mon, Jan 23, 2017 at 5:31 PM, Mikhail wrote: > Hi, Andrey. > > Thanks a lot. What are the current plans about updating th= e > version of Lucene to the latest one? It is really straightforward, I spen= d > about 3 hours to implement it. > I see 2 approaches: > 1.) Implement IndexingSPI using ElasticSearch to store and > get entities; > 2.) Store entities using ElasticSearch in CacheStore and > query them without usage of Ignite. > > What are the advantages of the first option? > > -- > Best Regards, > Mikhail > > =D0=A7=D0=B5=D1=82=D0=B2=D0=B5=D1=80=D0=B3, 19 =D1=8F=D0=BD=D0=B2=D0=B0= =D1=80=D1=8F 2017, 16:26 +03:00 =D0=BE=D1=82 Andrey Mashenkov < > andrey.mashenkov@gmail.com>: > > Hi Mikhail, > > You can try to implement your own IndexinSPI implementation and use SPI > queries, but it seems to be non trivial task. > Moreover Ignite already have Lucene 3.5.0 that is not supports Faceted > search. So, you will need to resolve version conflicts while implementing > IndexingSPI. > > On Thu, Jan 19, 2017 at 4:12 PM, Mikhail wrote: > > Hi, > > I need to implement something like this [1]. I think, I can'= t > implement it using just one SQL query (count the facets manually on every > query is not an option). Faceted search is brilliant feature of Lucene an= d > its family (ElasticSearch, Solr, etc). So, in order to implement it there > are two options: > > 1.) Use ElasticSearch (or other full text engine) as a Persistent Store. > 2.) Use existing Ignite Lucene index for faceted search. > > I don't like the first option, because I persist my entities in database. > Also I don't want to make ElasticSearch query for entities and then secon= d > Ignite query to get entities from caches. Therefore, it would be perfect = if > there were some mechanism in Ignite to override the "GridLuceneIndex" cla= ss > and implement necessary changes. Is there some way to do it? > > [1] http://proiot.ru/blog/posts/2016/05/15/lucene-fasetnyi-poisk/ > > -- > Best Regards, > Mikhail > > Wednesday, 12 January 2017, 12:19 +03:00 =D0=BE=D1=82 vkulichenko < > valentin.kulichenko@gmail.com>: > > > >Mikhail, > > > >From what I here, you can simply use SQL for this task. Is there > something in particular that doesn't work for you? > > > >-Val > > Hi, > > For my big data project I need to implement a catalog facet > search (like in modern online shops). I need to filter by criteria > (boolean, enum, range) and: > 1.) Show count of suitable items. > 2.) Hide filters that can't be applied mutually with the > specified filters. > 3.) Show (maybe approximate) count of items for each filter= , > that can be specified. > > All this I need to implement without retriving the whole > list of items (opposite - it would be trivial). > Could you imagine how to implement it using Ignite? > I can create Persistent Store which will write to Solr or > ElasticSearch, but I have already persist entities in database and use > Ignite for SQL and Fulltext queries. So I would prefer not to have 2 > different platforms for indexing content. > > -- > Best Regards, > Mikhail Fokanov > > > > > -- > =D0=A1 =D1=83=D0=B2=D0=B0=D0=B6=D0=B5=D0=BD=D0=B8=D0=B5=D0=BC, > =D0=9C=D0=B0=D1=88=D0=B5=D0=BD=D0=BA=D0=BE=D0=B2 =D0=90=D0=BD=D0=B4=D1=80= =D0=B5=D0=B9 =D0=92=D0=BB=D0=B0=D0=B4=D0=B8=D0=BC=D0=B8=D1=80=D0=BE=D0=B2= =D0=B8=D1=87 > =D0=A2=D0=B5=D0=BB. +7-921-932-61-82 > > Best regards, > Andrey V. Mashenkov > Cerr: +7-921-932-61-82 > > > --001a11444da45ca57e0546d8019d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Mikhail,

With 1-st approach=C2=A0
1. Underlying storag= e can be changed without rewiring its code to add indexing support.<= /div>
2. IndexSPI will updated in writ= eThrough(false) mode, however CacheStore won't.

With 2-nd
1. Y= ou should bother about transaction = handling (implementing CacheStore.sessionEnd() method).
2. If you use Lucene as CacheStore in readTr= ough(true), you should store whole entries in Lucene index to be able retur= n entries via load\loadAll methods. This will significantly increase index = size.



On Mon, Jan 23, 2017 at 5:31 PM, Mikha= il <wmaster@mail.ru> wrote:
Hi,=C2=A0Andrey.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0Thanks a lot. What are the current plans about updating the versi= on of Lucene to the latest one? It is really straightforward, I spend about= 3 hours to implement it.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0I see 2 approaches:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A01.) Implement IndexingSPI using ElasticSearch to store and get= entities;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02.) St= ore entities using ElasticSearch in=C2=A0CacheStore and query them without = usage of Ignite.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0What are the advantages of the first option?

--
Best Regard= s,
Mikhail

=D0=A7=D0=B5=D1=82=D0=B2=D0=B5=D1=80=D0=B3, 19 =D1=8F=D0=BD=D0=B2=D0=B0=D1= =80=D1=8F 2017, 16:26 +03:00 =D0=BE=D1=82 Andrey Mashenkov <andrey.mashenkov@gmail.= com>:

=09
=09
=09 =09
Hi Mikhail,

You can try to implement your own IndexinSPI implementation and= use SPI queries, but it=C2=A0seems to be non trivial task.
Moreover Ignite already have Lucene 3= .5.0 that is not supports Faceted search. So, you will need to resolve vers= ion conflicts while implementing IndexingSPI.

On Thu, Jan 19, 2017 at 4:12 PM, Mikhail <wmaster@mail.ru> wrote:
Hi,

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0I need to implement somethi= ng like this [1]. I think, I can't implement it using just one SQL quer= y (count the facets manually on every query is not an option). Faceted sear= ch is brilliant feature of Lucene and its family (ElasticSearch, Solr, etc)= .=C2=A0So, in order to implement it there are two options:

1.)=C2=A0Use ElasticSearch (or other full text engine) as a Persistent Stor= e.
2.) Use existing Ignite Lucene index for faceted search.

I don't like the first option, because I persist my entities in databas= e. Also I don't want to make ElasticSearch query for entities and then = second Ignite query to get entities from caches. Therefore, it would be per= fect if there were some mechanism in Ignite to override the "GridLucen= eIndex" class and implement necessary changes. Is there some way to do= it?

[1] http://proiot.ru/blog/posts/20= 16/05/15/lucene-fasetnyi-poisk/

--
Best Regards,
Mikhail

Wednesday, 12 January 2017, 12:19 +03:00 =D0=BE=D1=82 vkulichenko <valentin.kuli= chenko@gmail.com>:
>
>Mikhail,=C2=A0
>
>From what I here, you can simply use SQL for this task. Is there someth= ing in particular that doesn't work for you?=C2=A0
>
>-Val

Hi,

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 For my big data project I = need to implement a catalog facet search (like in modern online shops). I n= eed to filter by criteria (boolean, enum, range) and:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1.) Show count of suitable= =C2=A0items.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2.) Hide filters that can&= #39;t be applied=C2=A0mutually with the specified filters.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 3.) Show (maybe approximat= e) count of items for each filter, that can be specified.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 All this I need to impleme= nt without retriving the whole list of items (opposite - it would be trivia= l).
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Could you imagine how to i= mplement it using Ignite?
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0I can create Persiste= nt Store which will write to Solr or ElasticSearch, but I have already pers= ist entities in database and use Ignite for SQL and Fulltext queries. So I = would prefer not to have 2 different platforms for indexing content.

--
Best Regards,
Mikhail Fokanov



--
=D0=A1 =D1=83=D0=B2=D0=B0=D0=B6=D0=B5=D0=BD=D0= =B8=D0=B5=D0=BC,
=D0=9C=D0=B0=D1=88=D0=B5=D0=BD=D0=BA=D0=BE=D0=B2 =D0=90= =D0=BD=D0=B4=D1=80=D0=B5=D0=B9 =D0=92=D0=BB=D0=B0=D0=B4=D0=B8=D0=BC=D0=B8= =D1=80=D0=BE=D0=B2=D0=B8=D1=87
=D0=A2=D0=B5=D0=BB. +7-921-932-61-82

Best regard= s,
Andrey V. Mashenkov
Cerr: +7-921-932-61-82
=20 =20 =09
=09


--001a11444da45ca57e0546d8019d--