From user-return-20573-apmail-mahout-user-archive=mahout.apache.org@mahout.apache.org Fri Jul 11 09:16:25 2014 Return-Path: X-Original-To: apmail-mahout-user-archive@www.apache.org Delivered-To: apmail-mahout-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BC55A10CAD for ; Fri, 11 Jul 2014 09:16:25 +0000 (UTC) Received: (qmail 43587 invoked by uid 500); 11 Jul 2014 09:16:24 -0000 Delivered-To: apmail-mahout-user-archive@mahout.apache.org Received: (qmail 43526 invoked by uid 500); 11 Jul 2014 09:16:24 -0000 Mailing-List: contact user-help@mahout.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@mahout.apache.org Delivered-To: mailing list user@mahout.apache.org Received: (qmail 43513 invoked by uid 99); 11 Jul 2014 09:16:23 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Jul 2014 09:16:23 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of alexott@gmail.com designates 209.85.217.173 as permitted sender) Received: from [209.85.217.173] (HELO mail-lb0-f173.google.com) (209.85.217.173) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Jul 2014 09:16:19 +0000 Received: by mail-lb0-f173.google.com with SMTP id v6so400636lbi.4 for ; Fri, 11 Jul 2014 02:15:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ZSrtI4mavtt85fEPnG6IeJT1K/DqBQCZbaVFDoBAQBQ=; b=xgOKyk+n7RpSrEIBY8q8CP/hGJ07PYIP0IGboJs6zbPikXs7MLsKSVtaZTrBjZbiIU XubE9cmtOSAF6IDE23p3rP10UzNbl714sWYjd7SBZI6DHYrzj9rxN6JrtwedLv1Zoq6V bx8oHR1FoyzQyRIUmyl7iLpdgpwwDQngnDAMnpi7eiG8CXD1hVweaiOqyBVbpyumfF6Y qX//ibtih4B3es4Mp6UU1RBxsj3fzbSolztzyISwodmGxGjD3n96bVGdJrLT9U3af9HV ZlWzx81zzUQ+Q/s2IGgIN8bjxqfhpgxGswjqKrPtmCF/807IKx+oPPfR7vErTVAPBy9H /++A== MIME-Version: 1.0 X-Received: by 10.112.161.71 with SMTP id xq7mr2633916lbb.57.1405070157640; Fri, 11 Jul 2014 02:15:57 -0700 (PDT) Received: by 10.112.138.167 with HTTP; Fri, 11 Jul 2014 02:15:57 -0700 (PDT) In-Reply-To: References: Date: Fri, 11 Jul 2014 11:15:57 +0200 Message-ID: Subject: Re: Error running Mahout NB Classifier in Train step - java.lang.OutOfMemoryError: Java heap space From: Alex Ott To: user@mahout.apache.org Content-Type: multipart/alternative; boundary=001a11c260d0a17b8c04fde763e4 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c260d0a17b8c04fde763e4 Content-Type: text/plain; charset=UTF-8 Hi You may need to increase the memory for map/reduce tasks that run in cluster. By default they have only 200M, and you can adjust the value by putting following in conf/mapred-site.xml mapred.child.java.opts -Xmx1024m On Fri, Jul 11, 2014 at 8:05 AM, prabhat wrote: > Hi, > > I am getting following error while training a model. All Mapreduce runs > fine without error on HDP Ambari1.5(as per logs). But, I assume, while > creating models, it gives following error. Please help > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > at > > org.apache.mahout.math.map.OpenIntDoubleHashMap.rehash(OpenIntDoubleHashMap.java:491) > at > > org.apache.mahout.math.map.OpenIntDoubleHashMap.put(OpenIntDoubleHashMap.java:444) > at > > org.apache.mahout.math.RandomAccessSparseVector.setQuick(RandomAccessSparseVector.java:127) > at > org.apache.mahout.math.VectorWritable.readFields(VectorWritable.java:120) > at > > org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2178) > at > org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2306) > at > > org.apache.mahout.common.iterator.sequencefile.SequenceFileIterator.computeNext(SequenceFileIterator.java:101) > at > > org.apache.mahout.common.iterator.sequencefile.SequenceFileIterator.computeNext(SequenceFileIterator.java:40) > at > > com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) > at > > com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) > at > com.google.common.collect.Iterators$5.hasNext(Iterators.java:543) > at > > com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43) > at > > org.apache.mahout.classifier.naivebayes.BayesUtils.readModelFromDir(BayesUtils.java:81) > at > > org.apache.mahout.classifier.naivebayes.training.TrainNaiveBayesJob.run(TrainNaiveBayesJob.java:164) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > > org.apache.mahout.classifier.naivebayes.training.TrainNaiveBayesJob.main(TrainNaiveBayesJob.java:62) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) > at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145) > at > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:153) > at > org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > > > Prabhat Kumar Singh > -- With best wishes, Alex Ott http://alexott.net/ Twitter: alexott_en (English), alexott (Russian) Skype: alex.ott --001a11c260d0a17b8c04fde763e4--