Mailing-List: contact mahout-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mahout-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of shashikant@gmail.com designates
 209.85.222.186 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=A5HJJ+BCUiTLjjBIIwFHG4IPOTlKMXWMRU82nVosPkOUMWblaiSQLr0lBQsJ+wbVeh
         itk5xFy/wuqwAphvpaZI0FLlcnkuXLYCd60vuwowut6uyeeyiGCDusLBLtMw+xr8WRKb
         umHuWN2nQ/n0IASRwPJg+wn4iE2uwjX2olTUU=
MIME-Version: 1.0
In-Reply-To: <c7d45fc70908110827y49296bb9ja647a9aad3ff4798@mail.gmail.com>
References: <610013F0-B082-42B5-8286-E9F09A3299A5@oobaloo.co.uk>
	<c7d45fc70908051427rf77b795ha8ad8d80ab88dde3@mail.gmail.com>
	<17469b150908100651l3e927acbm495668587db42d50@mail.gmail.com>
	<c7d45fc70908101002o5f0bd9fex4b8856683c933e40@mail.gmail.com>
	<17469b150908101111s4839fc2xbca7451a88b86ad4@mail.gmail.com>
	<c7d45fc70908101141h798b02ddqd164d2abb654bfe0@mail.gmail.com>
	<17469b150908110132o7acefac3g28c92ca67c5b066c@mail.gmail.com>
	<17469b150908110429g5c4cb4f5r566b58c0d336ba80@mail.gmail.com>
	<c7d45fc70908110827y49296bb9ja647a9aad3ff4798@mail.gmail.com>
From: Shashikant Kore <shashikant@gmail.com>
Date: Tue, 11 Aug 2009 23:24:20 +0530
Message-ID: <17469b150908111054t47b1633bl73c06dda40fba63b@mail.gmail.com>
Subject: Re: Methods for Naming Clusters
To: mahout-user@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Tue, Aug 11, 2009 at 8:57 PM, Ted Dunning<ted.dunning@gmail.com> wrote:
> If you expand the LLR equation and look at which terms are big, you will =
see
> k_11 * log(mumble) =A0as an important term for many words. =A0Usually, th=
is is
> about the same as tf.idf since mumble is about the same as the term
> frequency. =A0For a single document, tf.idf is a very close approximation=
 of
> LLR. =A0With many documents, the situation can change dramatically, howev=
er,
> and you can get cancellation effects that eliminate documents that would
> otherwise have high tf.idf. =A0These are generally the terms that lead to
> over-fitting with methods like naive bayes and are often not such great
> cluster descriptors.
>

Let me restate what I understood.

If a phrase is identified as prominent phrase by LLR and it also
happens to be the top-weighted feature in the centroid vector, it is
not a good candidate for cluster label.

Is this correct?

--shashi