hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Viner <davevi...@pobox.com>
Subject Re: I am looking for a minimal Map-Reduce task
Date Sat, 09 Oct 2010 21:12:07 GMT
Perhaps you could do something simple like:

- use AWS Elastic Map Reduce
- start a JobFlow for Streaming, but leave it running when there are no
steps
- create a shell script that does something basic as the mapper ( touch
/tmp/foo)

Then run your job flow on 1 instance, and log into the instance, and look
for your distributed cache file in hdfs.

Would that work?

Dave Viner


On Sat, Oct 9, 2010 at 1:21 PM, Steve Lewis <lordjoe2000@gmail.com> wrote:

> For development purposes I need to run some code in a mapper and / or
> reducer ( imagine I am trying to verify that files in distributed cache are
> properly deployed)
>
> I am looking for code that does one step in a mapper and passes a single
> key-value pair to a reducer.
>
> In an ideal world there would be no input files (they are not needed and
> making them exist is not trivial)
>
> Any bright ideas or better yet - sample code
>
> --
> Steven M. Lewis PhD
> 4221 105th Ave Ne
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Institute for Systems Biology
> Seattle WA
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message