hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yingyi Bu <buyin...@gmail.com>
Subject Re: Error partitioning the input path
Date Fri, 27 Sep 2013 20:01:46 GMT
This is the property:

    <name>bsp.child.java.opts</name>
    <value>-Xmx4096m</value>

Yingyi



On Fri, Sep 27, 2013 at 3:56 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:

> That's strange ... I couldn't find any errors in tasks logs. How did
> you set the value of "bsp.child.java.opts" property?
>
> >  at org.apache.hama.examples.PageRank2.main(PageRank2.java:162)
>
> The original Hama PageRank example also fails?
>
> On Fri, Sep 27, 2013 at 7:06 PM, Yingyi Bu <buyingyi@gmail.com> wrote:
> > It fails again:
> >
> > 13/09/27 09:49:12 INFO mortbay.log: Logging to
> > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> > org.mortbay.log.Slf4jLog
> > 13/09/27 09:49:12 INFO bsp.FileInputFormat: Total input paths to process
> : 2
> > 13/09/27 09:49:12 WARN util.NativeCodeLoader: Unable to load
> native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > 13/09/27 09:49:12 WARN snappy.LoadSnappy: Snappy native library not
> loaded
> > 13/09/27 09:49:12 INFO bsp.FileInputFormat: Total input paths to process
> : 2
> > 13/09/27 09:49:13 INFO bsp.BSPJobClient: Running job:
> job_201309270937_0004
> > 13/09/27 09:49:16 INFO bsp.BSPJobClient: Current supersteps number: 0
> > 13/09/27 09:53:09 INFO bsp.BSPJobClient: Job failed.
> > 13/09/27 09:53:09 ERROR bsp.BSPJobClient: Error partitioning the input
> path.
> >
> > java.io.IOException: Runtime partition failed for the job.
> >         at
> org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465)
> >         at
> > org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333)
> >         at
> org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
> >         at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:229)
> >         at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
> >         at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:236)
> >         at org.apache.hama.examples.PageRank2.main(PageRank2.java:162)
> >
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:601)
> >         at
> >
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> >         at
> > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> >         at
> > org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:43)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:601)
> >         at org.apache.hama.util.RunJar.main(RunJar.java:146)
> >
> > I set the HAMA_LOG_DIR in conf/hama-env.sh to avoid logs going to NFS.
> > Just write a script to dump all tasklogs for the job.
> > Attached is the logs.  But I cannot find any clue from the logs.
> >
> > Any help is appreciated. Thanks!!
> >
> > Yingyi
> >
> >
> >
> > On Fri, Sep 27, 2013 at 2:44 AM, Edward J. Yoon <edwardyoon@apache.org>
> > wrote:
> >>
> >> Hi,
> >>
> >> The task logs are in ${HAMA_HOME}/logs/tasklogs/.
> >>
> >>
> >>
> >> On Fri, Sep 27, 2013 at 6:12 PM, Yingyi Bu <buyingyi@gmail.com> wrote:
> >> > All the groomsevers' tasklogs?
> >> >
> >> > Yingyi
> >> >
> >> >
> >> > On Fri, Sep 27, 2013 at 2:11 AM, Edward J. Yoon
> >> > <edwardyoon@apache.org>wrote:
> >> >
> >> >> Could you please check the tasklogs?
> >> >>
> >> >> On Fri, Sep 27, 2013 at 6:03 PM, Yingyi Bu <buyingyi@gmail.com>
> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I got a runtime partition exception as follows:
> >> >> >
> >> >> > 13/09/27 08:52:41 INFO mortbay.log: Logging to
> >> >> > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> >> >> > org.mortbay.log.Slf4jLog
> >> >> > 13/09/27 08:52:42 INFO bsp.FileInputFormat: Total input paths
to
> >> >> > process
> >> >> : 2
> >> >> > 13/09/27 08:52:42 WARN util.NativeCodeLoader: Unable to load
> >> >> native-hadoop
> >> >> > library for your platform... using builtin-java classes where
> >> >> > applicable
> >> >> > 13/09/27 08:52:42 WARN snappy.LoadSnappy: Snappy native library
not
> >> >> loaded
> >> >> > 13/09/27 08:52:42 INFO bsp.FileInputFormat: Total input paths
to
> >> >> > process
> >> >> : 2
> >> >> > 13/09/27 08:52:42 INFO bsp.BSPJobClient: Running job:
> >> >> job_201309270852_0002
> >> >> > 13/09/27 08:52:45 INFO bsp.BSPJobClient: Current supersteps
> number: 0
> >> >> > 13/09/27 08:58:10 INFO bsp.BSPJobClient: Current supersteps
> number: 1
> >> >> > 13/09/27 08:58:16 INFO bsp.BSPJobClient: Current supersteps
> number: 2
> >> >> > 13/09/27 09:00:04 INFO bsp.BSPJobClient: Job failed.
> >> >> > 13/09/27 09:00:04 ERROR bsp.BSPJobClient: Error partitioning the
> >> >> > input
> >> >> path.
> >> >> > java.io.IOException: Runtime partition failed for the job.
> >> >> >         at
> >> >> org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465)
> >> >> >         at
> >> >> >
> >> >> >
> org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333)
> >> >> >         at
> >> >> org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
> >> >> >         at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:229)
> >> >> >         at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
> >> >> >         at
> >> >> > org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:236)
> >> >> >         at
> >> >> > org.apache.hama.examples.PageRank2.main(PageRank2.java:164)
> >> >> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> >> > Method)
> >> >> >         at
> >> >> >
> >> >>
> >> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >> >         at
> >> >> >
> >> >>
> >> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >> >         at java.lang.reflect.Method.invoke(Method.java:601)
> >> >> >         at
> >> >> >
> >> >>
> >> >>
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> >> >> >         at
> >> >> > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> >> >> >         at
> >> >> > org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:43)
> >> >> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> >> > Method)
> >> >> >         at
> >> >> >
> >> >>
> >> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >> >         at
> >> >> >
> >> >>
> >> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >> >         at java.lang.reflect.Method.invoke(Method.java:601)
> >> >> >         at org.apache.hama.util.RunJar.main(RunJar.java:146)
> >> >> >
> >> >> > Does anyone know what's the possible reason of this exception?
> >> >> > Thanks!
> >> >> >
> >> >> > Yingyi
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >
> >
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message