incubator-hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Update of "HowToDebugBSPPrograms" by thomasjungblut
Date Mon, 24 Oct 2011 13:01:35 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The "HowToDebugBSPPrograms" page has been changed by thomasjungblut:
http://wiki.apache.org/hama/HowToDebugBSPPrograms

New page:
= How to Debug BSP Programs =

Debugging distributed programs is always difficult, because very few debuggers will let you
connect to a remote program that wasn't run with the proper command line arguments.

 1. Start by getting everything running (likely on a small input) in the local runner. 
    You do this by setting your BSP Master to "local" in your config. The local runner can
run 
    under the debugger and runs on your development machine.  A very quick and easy way to
set this 
    config variable is to include the following line just before you run the job:

    {{{conf.set("bsp.master.address", "local");}}}

    You may also want to do this to make the input and output files be in the local file system
rather than in the Hadoop 
    distributed file system (HDFS):

    {{{conf.set("fs.default.name", "local");}}}

    You can also set these configuration parameters in {{{hama-site.xml}}}.  The configuration
files should appear somewhere in your program's 
    class path when the program runs.


 2. Run the small input on a 1 node cluster. This will smoke out all of the issues that happen
with
    distribution and the "real" task runner, but you only have a single place to look at logs.
Besides the task logs, the most 
    useful ones are the grooms and bspmaster logs. Make sure you are logging at the INFO level
or you will 
    miss clues like the output of your tasks.

Mime
View raw message