hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Fwd: How to simulate network delay on 1 node
Date Sun, 26 Dec 2010 19:26:37 GMT
Hi there.

What are looking at is fault injection.
I am not sure what version of Hadoop you're looking at, but here's at
what you take a look in 0.21 and forward:
  - Herriot system testing framework (which does code instrumentation
to add special APIs) on a real clusters. Here's some starting
    - source code is in src/test/system
    - http://wiki.apache.org/hadoop/HowToUseSystemTestFramework
  - fault injection framework (should've been ported to 0.20 as well)
    - Source code is under src/test/aop
    - http://hadoop.apache.org/hdfs/docs/r0.21.0/faultinject_framework.html

If you are running on simulated infrastructure you don't need to look
further than fault injection framework. There's a test in HDFS which
does pretty much what you're looking for but for pipe-lines (look
under src/test/aop/org/apache/hadoop/hdfs/*).

If you are on a physical cluster then you need to use a combination of
1st and 2nd. The implementation of faults in system tests are coming
into Hadoop at some point of not very distant future, so you might
want to wait a little bit.
  Take care,
Konstantin (Cos) Boudnik

On Sun, Dec 26, 2010 at 04:25, yipeng <yipeng@gmail.com> wrote:
> Hi everyone,
> I would like to simulate network delay on 1 node in my cluster, perhaps by
> putting the thread to sleep every time it transfers data non-locally. I'm
> looking at the source but am not sure where to place the code. Is there a
> better way to do it... a tool perhaps? Or could someone point me in the
> right direction?
> Cheers,
> Yipeng

View raw message