hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@maprtech.com>
Subject Re: How to simulate network delay on 1 node
Date Mon, 27 Dec 2010 05:04:26 GMT
See also https://github.com/toddlipcon/gremlins



On Sun, Dec 26, 2010 at 11:26 AM, Konstantin Boudnik <cos@apache.org> wrote:

> Hi there.
>
> What are looking at is fault injection.
> I am not sure what version of Hadoop you're looking at, but here's at
> what you take a look in 0.21 and forward:
>  - Herriot system testing framework (which does code instrumentation
> to add special APIs) on a real clusters. Here's some starting
> pointers:
>    - source code is in src/test/system
>    - http://wiki.apache.org/hadoop/HowToUseSystemTestFramework
>  - fault injection framework (should've been ported to 0.20 as well)
>    - Source code is under src/test/aop
>    - http://hadoop.apache.org/hdfs/docs/r0.21.0/faultinject_framework.html
>
> If you are running on simulated infrastructure you don't need to look
> further than fault injection framework. There's a test in HDFS which
> does pretty much what you're looking for but for pipe-lines (look
> under src/test/aop/org/apache/hadoop/hdfs/*).
>
> If you are on a physical cluster then you need to use a combination of
> 1st and 2nd. The implementation of faults in system tests are coming
> into Hadoop at some point of not very distant future, so you might
> want to wait a little bit.
> --
>   Take care,
> Konstantin (Cos) Boudnik
>
> On Sun, Dec 26, 2010 at 04:25, yipeng <yipeng@gmail.com> wrote:
> > Hi everyone,
> >
> > I would like to simulate network delay on 1 node in my cluster, perhaps
> by
> > putting the thread to sleep every time it transfers data non-locally. I'm
> > looking at the source but am not sure where to place the code. Is there a
> > better way to do it... a tool perhaps? Or could someone point me in the
> > right direction?
> >
> > Cheers,
> >
> > Yipeng
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message