Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Message-ID: <4BF93FFB.8030502@windwardsolutions.com>
Date: Sun, 23 May 2010 07:47:23 -0700
From: Jeff Eastman <jdog@windwardsolutions.com>
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US;
 rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4
MIME-Version: 1.0
To: user@mahout.apache.org
Subject: Re: Mahout LDA Parameter: maxIter
References: <AANLkTikpBQtZ23OMYeScd1SmSsLZ2_4uR0dfdzuDTZOb@mail.gmail.com>
 	<4BF7FB4C.1050003@windwardsolutions.com>
 <AANLkTil6ZugYVzKpqrGTS_NChf5EjpqA6FPOkaahG-pV@mail.gmail.com>
In-Reply-To: <AANLkTil6ZugYVzKpqrGTS_NChf5EjpqA6FPOkaahG-pV@mail.gmail.com>
Content-Type: multipart/mixed;
 boundary="------------040104020608040708070009"

--------------040104020608040708070009
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Yes, but it takes another 80 iterations to get there and the results, on 
Reuters at least, don't seem to improve that much.

On 5/22/10 5:01 PM, Robin Anil wrote:
> David's rule of thumb was to let the iterations go until relative change in
> LL becomes around 10^-4
>
> Robin
>
> On Sat, May 22, 2010 at 9:12 PM, Jeff Eastman<jdog@windwardsolutions.com>wrote:
>
>    
>> I suggest you try running with a trunk checkout and upgrading to Hadoop
>> 0.20.2. Mahout is still in motion and I've run LDA on Reuters on trunk in
>> the last few days. The maxIter parameter should not be an issue; you could
>> try removing it entirely and LDA will default to running to convergence
>> (about 100 iterations which can take some time). I've found the Reuters
>> results don't change too much after 20. Even with a clean trunk checkout
>> Reuters will only use a single node and the iterations should take about 5
>> mins each. If you want to run on a multi-node cluster, install the patch in
>> MAHOUT-397 (
>>
>>
>> https://issues.apache.org/jira/browse/MAHOUT-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel)
>> and use the same arguments as in examples/bin/build-reuters.sh. Even on a
>> 3-node cluster this brings the iteration time down to about a minute and a
>> half which is worth doing.
>>
>> Hope this helps,
>> Jeff
>>
>> http://www.windwardsolutions.com
>>
>>
>>
>>
>> On 5/22/10 5:40 AM, 杨杰 wrote:
>>
>>      
>>> Hi, everyone
>>>
>>> I'm trying mahout now. When running LDA on reuter corpus
>>> (
>>> http://lucene.grantingersoll.com/2010/02/16/trijug-intro-to-mahout-slides-and-demo-examples/
>>> ),
>>> A parameter refuses to work. This parameter is "maxIter", without
>>> which, i cannot decide the iteration to run~
>>>
>>> My CMD is:
>>> bin/mahout.hadoop lda --input mahout/seq-sparse-tf/vectors --output
>>> mahout/seq-sparse-tf/lda-out5 --numWords 34000 --numTopics 20
>>> --maxIter 1
>>>
>>> But got a exception:
>>> 10/05/22 20:32:11 ERROR lda.LDADriver: Exception
>>> org.apache.commons.cli2.OptionException: Unexpected 2 while processing
>>> Options
>>>         at
>>> org.apache.commons.cli2.commandline.Parser.parse(Parser.java:100)
>>>         at
>>> org.apache.mahout.clustering.lda.LDADriver.main(LDADriver.java:115)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>         at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>         at
>>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>>>         at
>>> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>>>         at
>>> org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:172)
>>> ...
>>>
>>> What's the problem? I'm using version 0.3&   Hadoop 0.20.0.
>>>
>>> Thank you!
>>>
>>>
>>>
>>>
>>>        
>>
>>      
>    


--------------040104020608040708070009--