Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
MIME-Version: 1.0
In-Reply-To: <CAGsH7QH=6_vj4qt4Jk-9oGWB4UCyYgyNV566gOLLwOqwvTLPjA@mail.gmail.com>
References: <CAGsH7QH=6_vj4qt4Jk-9oGWB4UCyYgyNV566gOLLwOqwvTLPjA@mail.gmail.com>
From: Dmitriy Lyubimov <dlieu.7@gmail.com>
Date: Mon, 6 Jun 2016 10:02:27 -0700
Message-ID: <CAPud8Tq+MOzAowv4e4WS6Li35GUKOmKvgoYuwqXb5y_J58LKmw@mail.gmail.com>
Subject: Re: mahout tf-idf vs lucene tf-idf
To: "user@mahout.apache.org" <user@mahout.apache.org>
Content-Type: multipart/alternative; boundary=001a11409d8e908d1505349f09e6
archived-at: Mon, 06 Jun 2016 17:02:33 -0000

--001a11409d8e908d1505349f09e6
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

to add to Ted's reply, mahout has traditionally offered a bigram/trigram
analysis as a part of its tf-idf conversion (a step away from the bag of
words model so that directional statistically stable combinations of 2 or 3
words are reduced to their own term). However, this has not been ported to
spark/h20/flink engines, and is available as a mapreduce legacy algorithm
only.

On Sat, Jun 4, 2016 at 2:14 AM, forme book <forbookmail@gmail.com> wrote:

> Hi,
>
> I'm start to study text processing and I see that for evaluating two text
> is possible to obtaing vector model through TF-IDF technique.
>
> With Mahout is possible to create vectors from text with the use of
> lucene.vector, if I have not misheard takes a lucene index and then map a=
s
> a tf-idf,
>
> On the (Lucene side) has already by default this implementations, what I =
do
> struggle to understand what is the advantage of having lucene.vector in
> mahout when Lucene offer that feature out of the box ?
>
> Maybe I'm missing something big but what=E2=80=99s the Connection Between=
 then ?
>  could you please explain a possible user case ?
>
> Thanks for help
>
> Richard
>

--001a11409d8e908d1505349f09e6--