Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 83466117B1 for ; Tue, 19 Aug 2014 08:10:09 +0000 (UTC) Received: (qmail 23978 invoked by uid 500); 19 Aug 2014 08:09:40 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 23843 invoked by uid 500); 19 Aug 2014 08:09:40 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 23833 invoked by uid 99); 19 Aug 2014 08:09:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Aug 2014 08:09:40 +0000 X-ASF-Spam-Status: No, hits=0.7 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [80.67.31.101] (HELO smtprelay06.ispgateway.de) (80.67.31.101) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Aug 2014 08:09:33 +0000 Received: from [80.67.16.214] (helo=[192.168.10.165]) by smtprelay06.ispgateway.de with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1XJeTf-0001cd-Sb for user@hadoop.apache.org; Tue, 19 Aug 2014 10:09:11 +0200 Message-ID: <53F30627.3030909@rocknob.de> Date: Tue, 19 Aug 2014 10:09:11 +0200 From: norbi User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: "user@hadoop.apache.org" Subject: Hadoop HDFS slow after upgrade vom 0.20 -> to 2.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Df-Sender: bm9yYmlAcm9ja25vYi5kZQ== X-Virus-Checked: Checked by ClamAV on apache.org Hi List, we have upgraded Hadoop from our very old version 0.20 to Cloudera 4.7 (hadoop 2.0), we are only using HDFS. After upgrade (no configuration changes), the hdfs seems to bee very slow. It needs more than 2h to copying 40GB(47 files) out of the hdfs, bevor upgrading it was about 1h. We are using 52 Datanodes with 10 discs, all connectet via 1gig. How can we speed up hdfs, or where can be the bottleneck? Norbert