Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of eitan27@gmail.com designates
 209.85.192.170 as permitted sender)
MIME-Version: 1.0
From: Eitan Rosenfeld <eitan27@gmail.com>
Date: Tue, 4 Nov 2014 16:19:35 +0200
Message-ID: 
 <CAFNm6wJ-Aoi3QAX-_itSJjSwTZM+RsD1eWTafhCd9qgpSsp2ig@mail.gmail.com>
Subject: Why do reads take as long as replicated writes?
To: hdfs-dev@hadoop.apache.org
Content-Type: text/plain; charset=UTF-8

I am benchmarking my cluster of 16 nodes (all in one rack) with TestDFSIO on
Hadoop 1.0.4.  For simplicity, I turned off speculative task execution and set
the max map and reduce tasks to 1.

With a replication factor of 2, writing 1 file of 5GB takes twice as long as
reading 1 file. This result seems to make sense since the replication results
in twice the I/O in the cluster versus the read. However, as I scale up the
number of 5GB files from 1 to 64 files, reading ultimately takes as long as
writing. In particular, I see this result when writing and reading 64
such files.

What could cause read performance to degrade faster than write performance
as the number of files increases?

The full results (number of 5GB files, ratio of write time to read
time) are below:
1,  2.02
2,  1.87
4,  1.73
8,  1.54
16,  1.37
32,  1.29
64,  1.01

Thank you,

Eitan