hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Illecker <mar...@illecker.at>
Subject Re: HybridBSP (CPU and GPU) Task Integration
Date Fri, 23 Aug 2013 16:59:02 GMT
>
> What's the difference between launching `bsp task' and `gpu bsp task'?
> Will gpu bsp task fork and execute c/ c++ process?


The GPU bsp task can also be executed within a Java process.

In detail I want to run a Rootbeer Kernel (e.g., PiEstimationKernel [1])
within the bspGpu method.
A Rootbeer Kernel is written in Java and converted to CUDA. (the entry
point is the gpuMethod)
Finally there is a Java wrapper around the CUDA code, so it can be invoked
within the JVM.

So far there is no difference between a normal bsp task execution but I
want to use Hama Pipes to communicate via sockets.
The GPU bsp task should start like the default one but I will have to
establish the Pipes Server for communication.
And of course I need scheduling for theses GPU and CPU tasks.

I hope the following source will illustrate my scenario better:

public class MyHybridBSP extends
     HybridBSP<NullWritable, NullWritable, NullWritable, NullWritable,
Text> {

   @Override
   public void bsp(BSPPeer<NullWritable, NullWritable, NullWritable,
NullWritable, Text> peer)
       throws IOException, SyncException, InterruptedException {

     MyGPUKernel kernel = new MyGPUKernel();
     Rootbeer rootbeer = new Rootbeer();
     rootbeer.setThreadConfig(BLOCK_SIZE, GRID_SIZE, BLOCK_SIZE*GRID_SIZE);


     // Run GPU Kernels
     rootbeer.runAll(kernel);
   }

   @Override
   public void bspGpu(BSPPeer<NullWritable, NullWritable, NullWritable,
NullWritable, Text> peer)
       throws IOException, SyncException, InterruptedException {

     // process algorithm on CPU
   }

   class MyGPUKernel implements Kernel {
     public PiEstimatorKernel() { }

     public void gpuMethod() {
       // process algorithm on GPU

       // the following commands will need Hama Pipes
       HamaPeer.getConfiguration();
       HamaPeer.readNext(...,...);
       // and others....
     }
}

Thanks!

Martin

[1]
https://github.com/millecker/applications/blob/master/hama/rootbeer/piestimator/src/at/illecker/hama/rootbeer/examples/piestimator/gpu/PiEstimatorKernel.java

2013/8/23 Chia-Hung Lin <clin4j@googlemail.com>

> What's the difference between launching `bsp task' and `gpu bsp task'?
> Will gpu bsp task fork and execute c/ c++ process?
>
> It might be good to distinguish how gpu bsp task will be executed,
> then deciding how to launch such task.
>
> Basically for launching a bsp task, an external process is created.
> The logic to execute BSP.bsp() is at
>
>     BSPTask.java
>
> where the method
>
>     runBSP()
>
> is called with a BSP implementation class loaded at runtime
>
>     Class<?> workClass = job.getConfiguration().getClass("bsp.work.class",
>         BSP.class);
>
> and then the bsp method is executed
>
>     bsp.bsp(bspPeer);
>
>
>
>
>
>
> On 23 August 2013 21:45, Martin Illecker <martin@illecker.at> wrote:
> > Hi,
> >
> > I have created a HybridBSP [1] class which should combine the default BSP
> > (CPU) class with GPU methods [2].
> >
> > The abstract HybridBSP class extends the BSP class and adds bspGpu,
> > setupGpu and cleanupGpu method.
> >
> > public abstract class HybridBSP<K1, V1, K2, V2, M extends Writable>
> extends
> >     BSP<K1, V1, K2, V2, M> implements BSPGpuInterface<K1, V1, K2, V2, M>
> {
> >
> >   @Override
> >   public abstract void bspGpu(BSPPeer<K1, V1, K2, V2, M> peer)
> >       throws IOException, SyncException, InterruptedException;
> >
> >   @Override
> >   public void setupGpu(BSPPeer<K1, V1, K2, V2, M> peer) throws
> IOException,
> >       SyncException, InterruptedException {
> >   }
> >
> >   @Override
> >   public void cleanupGpu(BSPPeer<K1, V1, K2, V2, M> peer) throws
> IOException {
> >   }
> > }
> >
> >
> > Now I want to add a new scheduling technique which checks the conf
> property
> > (gpuBspTaskNum) and executes the bspGpu instead of default bsp method.
> >
> > e.g., bspTaskNum=3 and gpuBspTaskNum=1
> > The scheduler should run four bsp tasks simultaneously and execute three
> > times the bsp method and once the bspGpu. (both defined within one
> derived
> > HybridBSP class)
> >
> > Do I have to modify the taskrunner or create a new SimpleTaskScheduler?
> >
> > How can I integrate this into Hama?
> >
> > Thanks!
> >
> > Martin
> >
> > [1]
> >
> https://github.com/millecker/hama/blob/5d0e8b26abd6b63fa5afad09a2ba960bf9922868/core/src/main/java/org/apache/hama/bsp/gpu/HybridBSP.java
> > [2]
> >
> https://github.com/millecker/hama/blob/5d0e8b26abd6b63fa5afad09a2ba960bf9922868/core/src/main/java/org/apache/hama/bsp/gpu/BSPGpuInterface.java
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message