hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Dóró <doroatt...@gmail.com>
Subject Re: getting EOFException
Date Wed, 14 May 2014 11:15:00 GMT
Ok, thanks.


2014-05-14 2:05 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:

> If your input size is big, there's no way to avoid the OOM.
>
> If your size is small and you use local mode, Please check your program.
>
> On Wed, May 14, 2014 at 7:57 AM, Attila Dóró <doroattila@gmail.com> wrote:
> > Hi,
> >
> > I did it, but I get the same exception.
> >
> >
> > 2014-05-11 5:41 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:
> >
> >> Hi,
> >>
> >> Please try to add following code into the hams-site.xml.
> >>
> >>   <property>
> >>     <name>bsp.child.java.opts</name>
> >>     <value>-Xmx2024m</value>
> >>   </property>
> >>
> >>
> >> On Sat, May 10, 2014 at 8:42 AM, Attila Dóró <doroattila@gmail.com>
> wrote:
> >> > Hi Edward,
> >> >
> >> > It works, thanks a lot! Can you give me a short explanation what was
> >> wrong
> >> > with my code? I actually used the example from
> >> > this<
> >>
> http://hadoop.apache.org/docs/r2.3.0/api/org/apache/hadoop/io/Writable.html
> >> >page
> >> > as a pattern.
> >> >
> >> > Unfortunately now I get ExecutionException. I get this:
> >> > *java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:
> >> Java
> >> > heap space*
> >> > I have 8GB memory, but I can adjust just ~5GB for my virtual machine,
> so
> >> I
> >> > increased the HAMA_HEAPSIZE to 5000 in hama-env.sh as you suggested
> >> > here<https://issues.apache.org/jira/browse/HAMA-581>,
> >> > but I still get this exception. Does anyone have any other idea?
> >> >
> >> >
> >> > 2014-05-07 1:48 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:
> >> >
> >> >> Hi Attila,
> >> >>
> >> >> Please try to change like below and see if it works for you:
> >> >>
> >> >>   @Override
> >> >>   public void readFields(DataInput in) throws IOException {
> >> >>     sourceVertexID.readFields(in);
> >> >>     edgeValue.readFields(in);
> >> >>     destinationVertexID.readFields(in);
> >> >>   }
> >> >>
> >> >>   @Override
> >> >>   public void write(DataOutput out) throws IOException {
> >> >>     sourceVertexID.write(out);
> >> >>     edgeValue.write(out);
> >> >>     destinationVertexID.write(out);
> >> >>   }
> >> >>
> >> >> On Sun, May 4, 2014 at 7:13 PM, Attila Dóró <doroattila@gmail.com>
> >> wrote:
> >> >> > Thanks for your response, but how do you mean I can't use object
> >> >> > references? The SSSP example uses a lot of object references too
> and
> >> it
> >> >> > works. I mean for example the START_VERTEX and startVertex are
both
> >> >> object
> >> >> > references, or the this is a reference too. Can't I use any of
> them?
> >> >> >
> >> >> > Here <http://pastebin.com/3Qetuz0Q> is the MyWritableClass.
I
> don't
> >> find
> >> >> > any mistakes in this class.
> >> >> >
> >> >> >
> >> >> > 2014-05-04 8:32 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:
> >> >> >
> >> >> >> I roughly read your code, and there's a bug, like
> >> >> >> https://issues.apache.org/jira/browse/HAMA-868.
> >> >> >>
> >> >> >> Modify you code to not use object references, and check your
> message
> >> >> >> writable class.
> >> >> >>
> >> >> >> On Sun, May 4, 2014 at 3:13 PM, Edward J. Yoon <
> >> edwardyoon@apache.org>
> >> >> >> wrote:
> >> >> >> > Hi,
> >> >> >> >
> >> >> >> > Please check whether your MyWritableClass is implemented
> correctly.
> >> >> >> >
> >> >> >> > On Sat, May 3, 2014 at 7:26 AM, Attila Dóró <
> doroattila@gmail.com>
> >> >> >> wrote:
> >> >> >> >> Hi,
> >> >> >> >>
> >> >> >> >> I try to write a graph algorithm, but I get EOFException
and I
> >> can't
> >> >> >> find
> >> >> >> >> where is the mistake.
> >> >> >> >> I use the SSSP<
> >> >> >>
> >> >>
> >>
> https://github.com/apache/hama/blob/trunk/examples/src/main/java/org/apache/hama/examples/SSSP.java
> >> >> >> >
> >> >> >> >> example
> >> >> >> >> as a base. I tried this example and it worked well.
I modified
> the
> >> >> >> compute
> >> >> >> >> function and some other lines in the SSSP, but I
didn't modify
> the
> >> >> >> >> SSSPTextReader class, which extends the VertexInputReader.
I
> know
> >> >> that
> >> >> >> the
> >> >> >> >> EOFException "signals that an end of file or end
of stream has
> >> been
> >> >> >> reached
> >> >> >> >> unexpectedly during input", but I didn't modify any
reader or
> >> input
> >> >> >> method,
> >> >> >> >> so I don't understand how can I get this exception.
> >> >> >> >>
> >> >> >> >> In the first superstep I get the proper output, but
after the
> >> first
> >> >> >> >> superstep I get this exception:
> >> >> >> >>
> >> >> >> >> java.io.EOFException
> >> >> >> >>     at
> java.io.DataInputStream.readFully(DataInputStream.java:180)
> >> >> >> >>     at
> java.io.DataInputStream.readFully(DataInputStream.java:152)
> >> >> >> >>     at
> >> >> >> >>
> >> >>
> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:157)
> >> >> >> >>     at
> >> >> >> >>
> >> >>
> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:118)
> >> >> >> >>     at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:361)
> >> >> >> >>     at
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
> >> >> >> >>     at
> >> >> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
> >> >> >> >>     at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
> >> >> >> >>     at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
> >> >> >> >>     at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
> >> >> >> >>     at
> >> >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >> >> >> >>     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >> >> >> >>     at
> >> >> >> >>
> >> >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> >> >> >> >>     at
> >> >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >> >> >> >>     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >> >> >> >>     at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >> >> >>     at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> >> >> >>     at java.lang.Thread.run(Thread.java:662)
> >> >> >> >>
> >> >> >> >> I have no idea what is the problem, so if somebody
could help
> me,
> >> I
> >> >> >> would
> >> >> >> >> really appreciate that.
> >> >> >> >>
> >> >> >> >> (Anyway, here <http://pastebin.com/3AHu8FnC>
is my code, but I
> >> >> haven't
> >> >> >> used
> >> >> >> >> comment so I think it's totally incomprehensible
for you)
> >> >> >> >>
> >> >> >> >> Thank you in advance for your help,
> >> >> >> >> Attila
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Best Regards, Edward J. Yoon
> >> >> >> > CEO at DataSayer Co., Ltd.
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> CEO at DataSayer Co., Ltd.
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> CEO at DataSayer Co., Ltd.
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> CEO at DataSayer Co., Ltd.
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> CEO at DataSayer Co., Ltd.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message