Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7E76B1876E for ; Wed, 14 Oct 2015 00:23:19 +0000 (UTC) Received: (qmail 58612 invoked by uid 500); 14 Oct 2015 00:23:15 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 58504 invoked by uid 500); 14 Oct 2015 00:23:15 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 58494 invoked by uid 99); 14 Oct 2015 00:23:15 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 14 Oct 2015 00:23:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 7CCAD180DD5 for ; Wed, 14 Oct 2015 00:23:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.148 X-Spam-Level: X-Spam-Status: No, score=0.148 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id PbvjDVU0Yy_o for ; Wed, 14 Oct 2015 00:23:13 +0000 (UTC) Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 6A90520F1E for ; Wed, 14 Oct 2015 00:23:13 +0000 (UTC) Received: by igbif5 with SMTP id if5so10933000igb.1 for ; Tue, 13 Oct 2015 17:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=HRlZQv5Cy2pJRd7/sd9HZ9G6BTpDZSGROC9NRpTXfTM=; b=jCsuri4jmn2rbqLXG3YxsJFLJG3BIa2d6xwJ9QG1eYKReTjHhJpfotPs8xopung0Mx 0VJIP8y0YV+Bp2MPrbntYLnfvZE+W6NNnxvBUHIBjtxa+fO7kh6KxxPIX+jG1hG8Hbyl pV2D+CyY+KZkiOvUHmEEDDpWpRU8QWbVsLBqzVZRQ4PCrHrXsGP9JZaehTYB+996Yc0F ub2TWR7d3O9CJ4kMI9blImg6Oc4v+jN+Sz7oRMxq9H+JKOvYCroEnb9yieIscJ/hIfGB 5w8LICTJdjYVonGfhiIDKJycq55qyTFciRH7Y/9tB21yvHctQ0J8D31/iXCBPhIsWaQW 5+zw== MIME-Version: 1.0 X-Received: by 10.50.147.6 with SMTP id tg6mr780496igb.68.1444782192895; Tue, 13 Oct 2015 17:23:12 -0700 (PDT) Received: by 10.107.140.193 with HTTP; Tue, 13 Oct 2015 17:23:12 -0700 (PDT) Date: Wed, 14 Oct 2015 08:23:12 +0800 Message-ID: Subject: Datanode install on cluster Dockers From: =?UTF-8?B?5p6X5a626YqY?= To: user@hadoop.apache.org Content-Type: text/plain; charset=UTF-8 Hi I install three datanodes ( Hadoop 2.7.1) in a multiple Docker cluster which is connected by Flannel. The bridge on each physical node are physical host 1: IP address for flannel0: 172.16.91.0 IP address for docker0: 172.16.91.1 physical host 2: IP address for flannel0: 172.16.46.0 IP address for docker0: 172.16.46.1 physical host 3: IP address for flannel0: 172.16.31.0 IP address for docker0: 172.16.31.1 There are two datanodes are installed on "physical host 1", with IPs 172.16.91.19 and 172.16.91.20 One on "physical host 2" with IP 172.16.46.22 Namenode is installed on "physical host 2" with IP 172.16.46.21 And the problem is when I check the HDFS health status there is only two datanodes appear. [hdfs@amb1-host-dev bin]$ hadoop dfsadmin -report Name: 172.16.46.22:50010(k8s_POD-afaa08c2_amb4-rc-dev-n35n9_default_42432432-71d9-11e5-8d52-000c2900be4e_5fee9eed) Hostname: amb4-host-dev Decommission Status : Normal Configured Capacity: 87571161088 (81.56 GB) DFS Used: 33554432 (32 MB) Non DFS Used: 32210010112 (30.00 GB) DFS Remaining: 55327596544 (51.53 GB) DFS Used%: 0.04% DFS Remaining%: 63.18% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 2 Last contact: Wed Oct 14 00:16:42 UTC 2015 Name: 172.16.91.0:50010 (172.16.91.0) <-----------------------here is the problem Hostname: amb3-host-dev Decommission Status : Normal Configured Capacity: 87571161088 (81.56 GB) DFS Used: 28672 (28 KB) Non DFS Used: 36738072576 (34.21 GB) DFS Remaining: 50833059840 (47.34 GB) DFS Used%: 0.00% DFS Remaining%: 58.05% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 2 Last contact: Wed Oct 14 00:16:41 UTC 2015 Why there is only one datanode on "physical host2", and why HDFS regards the whole physical host as a datanode?