Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 326A517EF9 for ; Mon, 2 Mar 2015 04:29:35 +0000 (UTC) Received: (qmail 11341 invoked by uid 500); 2 Mar 2015 04:29:23 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 11232 invoked by uid 500); 2 Mar 2015 04:29:22 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 11221 invoked by uid 99); 2 Mar 2015 04:29:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2015 04:29:22 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of varun.uid@gmail.com designates 209.85.214.180 as permitted sender) Received: from [209.85.214.180] (HELO mail-ob0-f180.google.com) (209.85.214.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2015 04:28:56 +0000 Received: by mail-ob0-f180.google.com with SMTP id vb8so28853749obc.11 for ; Sun, 01 Mar 2015 20:28:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=AaG3/GqyIAKcoHCxlPvlhhaRybDkbsqy9DnkqDvwLmI=; b=l3uo96OA/iP71vcpcWMH8ewCE5g6/z/LiI8Je34oiyJ3nbbMrUudO0unWwqTCT9b/A qsparSaoeG1krfb4WIolaT9RcU0e3+aTIUiJu1ODtML3Jzkq8LgPkjupZcG4z4hys/9p aCgYXYN2FMjAPa31tu8T8259ay4xyDoTnW0YK2msSVRuLEPwOhYTUb174sz2gHwg7yJV YMuQocAW3PiggnyStuOJGLhAHMI/ujYk73AObJEhBwuk/Zw3r3IgAGDKBc/UIhd1dFFM a0DDWjRngAe1djwUm7MrG5T3ogeL45R9zmn33O1iGMteHoD4VmHR2bj0l45uM3W23vX5 oPhw== MIME-Version: 1.0 X-Received: by 10.60.38.66 with SMTP id e2mr17882655oek.38.1425270489117; Sun, 01 Mar 2015 20:28:09 -0800 (PST) Received: by 10.60.32.194 with HTTP; Sun, 1 Mar 2015 20:28:09 -0800 (PST) In-Reply-To: <00d801d05479$1a1c9080$4e55b180$@web-computing.de> References: <00d801d05479$1a1c9080$4e55b180$@web-computing.de> Date: Sun, 1 Mar 2015 23:28:09 -0500 Message-ID: Subject: Re: Hadoop 2.6.0 - No DataNode to stop From: Varun Kumar To: user Content-Type: multipart/alternative; boundary=089e0122772e367895051046a58c X-Virus-Checked: Checked by ClamAV on apache.org --089e0122772e367895051046a58c Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable 1.Stop the service 2.Change the permissions for log and pid directory once again to hdfs. 3.Start service with hdfs. This will resolve the issue On Sun, Mar 1, 2015 at 6:40 PM, Daniel Klinger wrote: > Thanks for your answer. > > > > I put the FQDN of the DataNodes in the slaves file on each node (one FQDN > per line). Here=E2=80=99s the full DataNode log after the start (the log = of the > other DataNode is exactly the same): > > > > 2015-03-02 00:29:41,841 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal > handlers for [TERM, HUP, INT] > > 2015-03-02 00:29:42,207 INFO > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > > 2015-03-02 00:29:42,312 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot > period at 10 second(s). > > 2015-03-02 00:29:42,313 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics syste= m > started > > 2015-03-02 00:29:42,319 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is > hadoop.klinger.local > > 2015-03-02 00:29:42,327 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with > maxLockedMemory =3D 0 > > 2015-03-02 00:29:42,350 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server = at > /0.0.0.0:50010 > > 2015-03-02 00:29:42,357 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is > 1048576 bytes/s > > 2015-03-02 00:29:42,358 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for > balancing is 5 > > 2015-03-02 00:29:42,458 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > > 2015-03-02 00:29:42,462 INFO org.apache.hadoop.http.HttpRequestLog: Http > request log for http.requests.datanode is not defined > > 2015-03-02 00:29:42,474 INFO org.apache.hadoop.http.HttpServer2: Added > global filter 'safety' > (class=3Dorg.apache.hadoop.http.HttpServer2$QuotingInputFilter) > > 2015-03-02 00:29:42,476 INFO org.apache.hadoop.http.HttpServer2: Added > filter static_user_filter > (class=3Dorg.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter)= to > context datanode > > 2015-03-02 00:29:42,476 INFO org.apache.hadoop.http.HttpServer2: Added > filter static_user_filter > (class=3Dorg.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter)= to > context logs > > 2015-03-02 00:29:42,476 INFO org.apache.hadoop.http.HttpServer2: Added > filter static_user_filter > (class=3Dorg.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter)= to > context static > > 2015-03-02 00:29:42,494 INFO org.apache.hadoop.http.HttpServer2: > addJerseyResourcePackage: > packageName=3Dorg.apache.hadoop.hdfs.server.datanode.web.resources;org.ap= ache.hadoop.hdfs.web.resources, > pathSpec=3D/webhdfs/v1/* > > 2015-03-02 00:29:42,499 INFO org.mortbay.log: jetty-6.1.26 > > 2015-03-02 00:29:42,555 WARN org.mortbay.log: Can't reuse > /tmp/Jetty_0_0_0_0_50075_datanode____hwtdwq, using > /tmp/Jetty_0_0_0_0_50075_datanode____hwtdwq_3168831075162569402 > > 2015-03-02 00:29:43,205 INFO org.mortbay.log: Started HttpServer2$ > SelectChannelConnectorWithSafeStartup@0.0.0.0:50075 > > 2015-03-02 00:29:43,635 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName =3D hdfs > > 2015-03-02 00:29:43,635 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup =3D supergrou= p > > 2015-03-02 00:29:43,802 INFO org.apache.hadoop.ipc.CallQueueManager: Usin= g > callQueue class java.util.concurrent.LinkedBlockingQueue > > 2015-03-02 00:29:43,823 INFO org.apache.hadoop.ipc.Server: Starting Socke= t > Reader #1 for port 50020 > > 2015-03-02 00:29:43,875 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at / > 0.0.0.0:50020 > > 2015-03-02 00:29:43,913 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received > for nameservices: null > > 2015-03-02 00:29:43,953 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices > for nameservices: > > 2015-03-02 00:29:43,973 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool > (Datanode Uuid unassigned) service to hadoop.klinger.local/10.0.1.148:802= 0 > starting to offer service > > 2015-03-02 00:29:43,981 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > > 2015-03-02 00:29:43,982 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 50020: starting > > 2015-03-02 00:29:44,620 INFO org.apache.hadoop.hdfs.server.common.Storage= : > DataNode version: -56 and NameNode layout version: -60 > > 2015-03-02 00:29:44,641 INFO org.apache.hadoop.hdfs.server.common.Storage= : > Lock on /cluster/storage/datanode/in_use.lock acquired by nodename > 1660@hadoop.klinger.local > > 2015-03-02 00:29:44,822 INFO org.apache.hadoop.hdfs.server.common.Storage= : > Analyzing storage directories for bpid BP-158097147-10.0.1.148-1424966425= 688 > > 2015-03-02 00:29:44,822 INFO org.apache.hadoop.hdfs.server.common.Storage= : > Locking is disabled > > 2015-03-02 00:29:44,825 INFO org.apache.hadoop.hdfs.server.common.Storage= : > Restored 0 block files from trash. > > 2015-03-02 00:29:44,829 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: > nsid=3D330980018;bpid=3DBP-158097147-10.0.1.148-1424966425688;lv=3D-56;ns= Info=3Dlv=3D-60;cid=3DCID-a2c81934-b3ce-44aa-b920-436ee2f0d5a7;nsid=3D33098= 0018;c=3D0;bpid=3DBP-158097147-10.0.1.148-1424966425688;dnuuid=3Da3b6c890-4= 1ca-4bde-855c-015c67e6e0df > > 2015-03-02 00:29:44,996 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adde= d > new volume: /cluster/storage/datanode/current > > 2015-03-02 00:29:44,998 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adde= d > volume - /cluster/storage/datanode/current, StorageType: DISK > > 2015-03-02 00:29:45,035 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: > Registered FSDatasetState MBean > > 2015-03-02 00:29:45,057 INFO > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic Directo= ry > Tree Verification scan starting at 1425265856057 with interval 21600000 > > 2015-03-02 00:29:45,064 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Addi= ng > block pool BP-158097147-10.0.1.148-1424966425688 > > 2015-03-02 00:29:45,071 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: > Scanning block pool BP-158097147-10.0.1.148-1424966425688 on volume > /cluster/storage/datanode/current... > > 2015-03-02 00:29:45,128 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time > taken to scan block pool BP-158097147-10.0.1.148-1424966425688 on > /cluster/storage/datanode/current: 56ms > > 2015-03-02 00:29:45,128 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Tota= l > time to scan all replicas for block pool > BP-158097147-10.0.1.148-1424966425688: 64ms > > 2015-03-02 00:29:45,128 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Addi= ng > replicas to map for block pool BP-158097147-10.0.1.148-1424966425688 on > volume /cluster/storage/datanode/current... > > 2015-03-02 00:29:45,129 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time > to add replicas to map for block pool BP-158097147-10.0.1.148-14249664256= 88 > on volume /cluster/storage/datanode/current: 0ms > > 2015-03-02 00:29:45,134 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Tota= l > time to add all replicas to map: 5ms > > 2015-03-02 00:29:45,138 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool > BP-158097147-10.0.1.148-1424966425688 (Datanode Uuid null) service to > hadoop.klinger.local/10.0.1.148:8020 beginning handshake with NN > > 2015-03-02 00:29:45,316 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool Block pool > BP-158097147-10.0.1.148-1424966425688 (Datanode Uuid null) service to > hadoop.klinger.local/10.0.1.148:8020 successfully registered with NN > > 2015-03-02 00:29:45,316 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: For namenode > hadoop.klinger.local/10.0.1.148:8020 using DELETEREPORT_INTERVAL of > 300000 msec BLOCKREPORT_INTERVAL of 21600000msec CACHEREPORT_INTERVAL of > 10000msec Initial delay: 0msec; heartBeatInterval=3D3000 > > 2015-03-02 00:29:45,751 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Namenode Block pool > BP-158097147-10.0.1.148-1424966425688 (Datanode Uuid > a3b6c890-41ca-4bde-855c-015c67e6e0df) service to hadoop.klinger.local/ > 10.0.1.148:8020 trying to claim ACTIVE state with txid=3D24 > > 2015-03-02 00:29:45,751 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Acknowledging ACTIVE > Namenode Block pool BP-158097147-10.0.1.148-1424966425688 (Datanode Uuid > a3b6c890-41ca-4bde-855c-015c67e6e0df) service to hadoop.klinger.local/ > 10.0.1.148:8020 > > 2015-03-02 00:29:45,883 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports 0 > blocks total. Took 4 msec to generate and 126 msecs for RPC and NN > processing. Got back commands > org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@3d528774 > > 2015-03-02 00:29:45,883 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Got finalize command for > block pool BP-158097147-10.0.1.148-1424966425688 > > 2015-03-02 00:29:45,891 INFO org.apache.hadoop.util.GSet: Computing > capacity for map BlockMap > > 2015-03-02 00:29:45,891 INFO org.apache.hadoop.util.GSet: VM type = =3D > 64-bit > > 2015-03-02 00:29:45,893 INFO org.apache.hadoop.util.GSet: 0.5% max memory > 966.7 MB =3D 4.8 MB > > 2015-03-02 00:29:45,893 INFO org.apache.hadoop.util.GSet: capacity = =3D > 2^19 =3D 524288 entries > > 2015-03-02 00:29:45,894 INFO > org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Periodic > Block Verification Scanner initialized with interval 504 hours for block > pool BP-158097147-10.0.1.148-1424966425688 > > 2015-03-02 00:29:45,900 INFO > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Added > bpid=3DBP-158097147-10.0.1.148-1424966425688 to blockPoolScannerMap, new > size=3D1 > > > > > > Dfsadmin =E2=80=93report (called as user hdfs on NameNode) generated foll= owing > output. It looks like both DataNodes are available: > > > > Configured Capacity: 985465716736 (917.79 GB) > > Present Capacity: 929892360192 (866.03 GB) > > DFS Remaining: 929892302848 (866.03 GB) > > DFS Used: 57344 (56 KB) > > DFS Used%: 0.00% > > Under replicated blocks: 0 > > Blocks with corrupt replicas: 0 > > Missing blocks: 0 > > > > ------------------------------------------------- > > Live datanodes (2): > > > > Name: 10.0.1.148:50010 (hadoop.klinger.local) > > Hostname: hadoop.klinger.local > > Decommission Status : Normal > > Configured Capacity: 492732858368 (458.89 GB) > > DFS Used: 28672 (28 KB) > > Non DFS Used: 27942051840 (26.02 GB) > > DFS Remaining: 464790777856 (432.87 GB) > > DFS Used%: 0.00% > > DFS Remaining%: 94.33% > > Configured Cache Capacity: 0 (0 B) > > Cache Used: 0 (0 B) > > Cache Remaining: 0 (0 B) > > Cache Used%: 100.00% > > Cache Remaining%: 0.00% > > Xceivers: 1 > > Last contact: Mon Mar 02 00:38:00 CET 2015 > > > > > > Name: 10.0.1.89:50010 (hadoop-data.klinger.local) > > Hostname: hadoop-data.klinger.local > > Decommission Status : Normal > > Configured Capacity: 492732858368 (458.89 GB) > > DFS Used: 28672 (28 KB) > > Non DFS Used: 27631304704 (25.73 GB) > > DFS Remaining: 465101524992 (433.16 GB) > > DFS Used%: 0.00% > > DFS Remaining%: 94.39% > > Configured Cache Capacity: 0 (0 B) > > Cache Used: 0 (0 B) > > Cache Remaining: 0 (0 B) > > Cache Used%: 100.00% > > Cache Remaining%: 0.00% > > Xceivers: 1 > > Last contact: Mon Mar 02 00:37:59 CET 2015 > > > > Any further thoughts? > > > > Greets > > DK > > *Von:* Ulul [mailto:hadoop@ulul.org] > *Gesendet:* Sonntag, 1. M=C3=A4rz 2015 13:12 > > *An:* user@hadoop.apache.org > *Betreff:* Re: Hadoop 2.6.0 - No DataNode to stop > > > > Hi > > Did you check your slaves file is correct ? > That the datanode process is actually running ? > Did you check its log file ? > That the datanode is available ? (dfsadmin -report, through the WUI) > > We need more detail > > Ulul > > Le 28/02/2015 22:05, Daniel Klinger a =C3=A9crit : > > Thanks but i know how to kill a process in Linux. But this didn=E2=80=99t= answer the question why the command say no Datanode to stop instead of sto= pping the Datanode: > > > > $HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script h= dfs stop datanode > > > > > > > > > > *Von:* Surbhi Gupta [mailto:surbhi.gupta01@gmail.com > ] > *Gesendet:* Samstag, 28. Februar 2015 20:16 > *An:* user@hadoop.apache.org > *Betreff:* Re: Hadoop 2.6.0 - No DataNode to stop > > > > Issue jps and get the process id or > Try to get the process id of datanode. > > Issue ps-fu userid of the user through which datanode is running. > > Then kill the process using kill -9 > > On 28 Feb 2015 09:38, "Daniel Klinger" wrote: > > Hello, > > > > I used a lot of Hadoop-Distributions. Now I=E2=80=99m trying to install a= pure > Hadoop on a little =E2=80=9Ecluster=E2=80=9C for testing (2 CentOS-VMs: 1= Name+DataNode 1 > DataNode). I followed the instructions on the Documentation site: > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/C= lusterSetup.html > . > > > > I=E2=80=99m starting the Cluster like it is described in the Chapter =E2= =80=9EOperating > the Hadoop Cluster=E2=80=9C(with different users). The starting process w= orks > great. The PID-Files are created in /var/run and u can see that Folders a= nd > Files are created in the Data- and NameNode folders. I=E2=80=99m getting = no errors > in the log-files. > > > > When I try to stop the cluster all Services are stopped (NameNode, > ResourceManager etc.). But when I stop the DataNodes I=E2=80=99m getting = the > message: =E2=80=9ENo DataNode to stop=E2=80=9C. The PID-File and the in_u= se.lock-File are > still there and if I try to start the DataNode again I=E2=80=99m getting = the error > that the Process is already running. When I stop the DataNode as hdfs > instead of root the PID and in_use-File are removed but I=E2=80=99m still= getting > the message: =E2=80=9ENo DataNode to stop=E2=80=9C > > > > What I=E2=80=99m doing wrong? > > > > Greets > > dk > > > --=20 Regards, Varun Kumar.P --089e0122772e367895051046a58c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
1.Stop the service=C2=A0

2.Change the pe= rmissions for log and pid directory once again to hdfs.

= 3.Start service with hdfs.

This will resolve the i= ssue

O= n Sun, Mar 1, 2015 at 6:40 PM, Daniel Klinger <dk@web-computing.de&g= t; wrote:

Thanks for your answer.

=C2=A0

I put the FQDN of the DataNodes in the slaves file on ea= ch node (one FQDN per line). Here=E2=80=99s the full DataNode log after the= start (the log of the other DataNode is exactly the same):

=C2=A0<= /u>

2015-03-02 0= 0:29:41,841 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registere= d UNIX signal handlers for [TERM, HUP, INT]

2015-03-02 00:29:42,207 INFO org.= apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-me= trics2.properties

2015-03-02 00:29:42,312 INFO org.apache.hadoop.metrics2.imp= l.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).

2015-03-02 00= :29:42,313 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode= metrics system started

2015-03-02 00:29:42,319 INFO org.apache.hadoop.hdfs.s= erver.datanode.DataNode: Configured hostname is hadoop.klinger.local=

2015-03-= 02 00:29:42,327 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Start= ing DataNode with maxLockedMemory =3D 0

2015-03-02 00:29:42,350 INFO org.apac= he.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:50010

2015-03-02 00:= 29:42,357 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing b= andwith is 1048576 bytes/s

<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:"Arial"= ,sans-serif;color:black">2015-03-02 00:29:42,358 INFO org.apache.hadoop.hdf= s.server.datanode.DataNode: Number threads for balancing is 5=

2015-03-02 00:2= 9:42,458 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter= (org.mortbay.log) via org.mortbay.log.Slf4jLog

2015-03-02 00:29:42,462 INFO o= rg.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.da= tanode is not defined

2015-03-02 00:29:42,474 INFO org.apache.hadoop.http.Htt= pServer2: Added global filter 'safety' (class=3Dorg.apache.hadoop.h= ttp.HttpServer2$QuotingInputFilter)

2015-03-02 00:29:42,476 INFO org.apache.h= adoop.http.HttpServer2: Added filter static_user_filter (class=3Dorg.apache= .hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode<= u>

= 2015-03-02 00:29:42,476 INFO org.apache.hadoop.http.HttpServer2: Added filt= er static_user_filter (class=3Dorg.apache.hadoop.http.lib.StaticUserWebFilt= er$StaticUserFilter) to context logs

2015-03-02 00:29:42,476 INFO org.apache.= hadoop.http.HttpServer2: Added filter static_user_filter (class=3Dorg.apach= e.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static

201= 5-03-02 00:29:42,494 INFO org.apache.hadoop.http.HttpServer2: addJerseyReso= urcePackage: packageName=3Dorg.apache.hadoop.hdfs.server.datanode.web.resou= rces;org.apache.hadoop.hdfs.web.resources, pathSpec=3D/webhdfs/v1/*<= u>

2015-03-0= 2 00:29:42,499 INFO org.mortbay.log: jetty-6.1.26

<= p class=3D"MsoNormal">2015-03-02 00:29:42,555 WAR= N org.mortbay.log: Can't reuse /tmp/Jetty_0_0_0_0_50075_datanode____hwt= dwq, using /tmp/Jetty_0_0_0_0_50075_datanode____hwtdwq_3168831075162569402<= u>

= 2015-03-02 00:29:43,205 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50075

2015-03-02 00:= 29:43,635 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = =3D hdfs

2015-03-02 00:29:43,635 INFO org.apache.hadoop.hdfs.server.datanode.= DataNode: supergroup =3D supergroup

2015-03-02 00:29:43,802 INFO org.apache.h= adoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.Link= edBlockingQueue

2015-03-02 00:29:43,823 INFO org.apache.hadoop.ipc.Server: = Starting Socket Reader #1 for port 50020

2015-03-02 00:29:43,875 INFO org.a= pache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020=

2015-03-02 00:2= 9:43,913 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh requ= est received for nameservices: null

2015-03-02 00:29:43,953 INFO org.apache.h= adoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservi= ces: <default>

2015-03-02 00:29:43,973 INFO org.apache.hadoop.hdfs.serv= er.datanode.DataNode: Block pool <registering> (Datanode Uuid unassig= ned) service to hadoop.klinger.local/10.0.1.148:8020 starting to offer service<= /span>

2015-03-02 00:29= :43,981 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting

201= 5-03-02 00:29:43,982 INFO org.apache.hadoop.ipc.Server: IPC Server listener= on 50020: starting

2015-03-02 00:29:44,620 INFO org.apache.hadoop.hdfs.serve= r.common.Storage: DataNode version: -56 and NameNode layout version: -60=

2015= -03-02 00:29:44,641 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock= on /cluster/storage/datanode/in_use.lock acquired by nodename 1660@hadoop.= klinger.local

2015-03-02 00:29:44,822 INFO org.apache.hadoop.hdfs.server.comm= on.Storage: Analyzing storage directories for bpid BP-158097147-10.0.1.148-= 1424966425688

2015-03-02 00:29:44,822 INFO org.apache.hadoop.hdfs.server.comm= on.Storage: Locking is disabled

2015-03-02 00:29:44,825 INFO org.apache.hadoo= p.hdfs.server.common.Storage: Restored 0 block files from trash.<= /u>

2015-03-02 0= 0:29:44,829 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting u= p storage: nsid=3D330980018;bpid=3DBP-158097147-10.0.1.148-1424966425688;lv= =3D-56;nsInfo=3Dlv=3D-60;cid=3DCID-a2c81934-b3ce-44aa-b920-436ee2f0d5a7;nsi= d=3D330980018;c=3D0;bpid=3DBP-158097147-10.0.1.148-1424966425688;dnuuid=3Da= 3b6c890-41ca-4bde-855c-015c67e6e0df

2015-03-02 00:29:44,996 INFO org.apache.h= adoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added new volume: = /cluster/storage/datanode/current

2015-03-02 00:29:44,998 INFO org.apache.had= oop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added volume - /clus= ter/storage/datanode/current, StorageType: DISK

2015-03-02 00:29:45,035 INFO = org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Regist= ered FSDatasetState MBean

2015-03-02 00:29:45,057 INFO org.apache.hadoop.hdfs= .server.datanode.DirectoryScanner: Periodic Directory Tree Verification sca= n starting at 1425265856057 with interval 21600000

=

2015-03-02 00:29:45,064 IN= FO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Add= ing block pool BP-158097147-10.0.1.148-1424966425688

2015-03-02 00:29:45,071 = INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: S= canning block pool BP-158097147-10.0.1.148-1424966425688 on volume /cluster= /storage/datanode/current...

2015-03-02 00:29:45,128 INFO org.apache.hadoop.h= dfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time taken to scan block = pool BP-158097147-10.0.1.148-1424966425688 on /cluster/storage/datanode/cur= rent: 56ms

2015-03-02 00:29:45,128 INFO org.apache.hadoop.hdfs.server.datanod= e.fsdataset.impl.FsDatasetImpl: Total time to scan all replicas for block p= ool BP-158097147-10.0.1.148-1424966425688: 64ms

2015-03-02 00:29:45,128 INFO = org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding= replicas to map for block pool BP-158097147-10.0.1.148-1424966425688 on vo= lume /cluster/storage/datanode/current...

2015-03-02 00:29:45,129 INFO org.a= pache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time to add= replicas to map for block pool BP-158097147-10.0.1.148-1424966425688 on vo= lume /cluster/storage/datanode/current: 0ms

2015-03-02 00:29:45,134 INFO org.= apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Total time= to add all replicas to map: 5ms

2015-03-02 00:29:45,138 INFO org.apache.hado= op.hdfs.server.datanode.DataNode: Block pool BP-158097147-10.0.1.148-142496= 6425688 (Datanode Uuid null) service to hadoop.klinger.local/10.0.1.148:8020 beginning handsh= ake with NN

2015-03-02 00:29:45,316 INFO org.apache.hadoop.hdfs.server.datano= de.DataNode: Block pool Block pool BP-158097147-10.0.1.148-1424966425688 (D= atanode Uuid null) service to hadoop.klinger.local/10.0.1.148:8020 successfully registered wi= th NN

2015-03-02 00:29:45,316 INFO org.apache.hadoop.hdfs.server.datanode.Dat= aNode: For namenode hadoop.klinger.local/10.0.1.148:8020 using DELETEREPORT_INTERVAL of 30000= 0 msec=C2=A0 BLOCKREPORT_INTERVAL of 21600000msec CACHEREPORT_INTERVAL of 1= 0000msec Initial delay: 0msec; heartBeatInterval=3D3000

2015-03-02 00:29:45,7= 51 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Namenode Block poo= l BP-158097147-10.0.1.148-1424966425688 (Datanode Uuid a3b6c890-41ca-4bde-8= 55c-015c67e6e0df) service to hadoop.klinger.local/10.0.1.148:8020 trying to claim ACTIVE stat= e with txid=3D24

2015-03-02 00:29:45,751 INFO org.apache.hadoop.hdfs.server.= datanode.DataNode: Acknowledging ACTIVE Namenode Block pool BP-158097147-10= .0.1.148-1424966425688 (Datanode Uuid a3b6c890-41ca-4bde-855c-015c67e6e0df)= service to hadoop.klinger.local/10.0.1.148:8020

2015-03-02 00:29:45,883 INFO org.apache.hadoo= p.hdfs.server.datanode.DataNode: Sent 1 blockreports 0 blocks total. Took 4= msec to generate and 126 msecs for RPC and NN processing.=C2=A0 Got back c= ommands org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@3d528774<= /u>

2015-= 03-02 00:29:45,883 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Go= t finalize command for block pool BP-158097147-10.0.1.148-1424966425688<= /u>

2015-= 03-02 00:29:45,891 INFO org.apache.hadoop.util.GSet: Computing capacity for= map BlockMap

2015-03-02 00:29:45,891 INFO org.apache.hadoop.util.GSet: VM ty= pe=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 64-bit

<= p class=3D"MsoNormal">2015-03-02 00:29:45,893 INF= O org.apache.hadoop.util.GSet: 0.5% max memory 966.7 MB =3D 4.8 MB

2015-03-02= 00:29:45,893 INFO org.apache.hadoop.util.GSet: capacity=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 =3D 2^19 =3D 524288 entries

2015-03-02 00:29:45,894 INFO org.a= pache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Periodic Block Ver= ification Scanner initialized with interval 504 hours for block pool BP-158= 097147-10.0.1.148-1424966425688

2015-03-02 00:29:45,900 INFO org.apache.hadoo= p.hdfs.server.datanode.DataBlockScanner: Added bpid=3DBP-158097147-10.0.1.1= 48-1424966425688 to blockPoolScannerMap, new size=3D1<= /p>

=C2=A0

=C2=A0

Dfsadmin =E2= =80=93report (called as user hdfs on NameNode) generated following output. = It looks like both DataNodes are available:

=C2=A0

Configured Capacity: 9854657= 16736 (917.79 GB)

Present Capacity: 929892360192 (866.03 GB)

DFS Remaining: 929= 892302848 (866.03 GB)

DFS Used: 57344 (56 KB)

DFS Used%: 0.00%

Under replicated b= locks: 0

Blocks with corrupt replicas: 0

Missing blocks: 0=

=C2=A0

-----------------= --------------------------------

Live datanodes (2):

=

=C2=A0

Name: 10.0.1.148:50010 (hadoop.klinger= .local)

Hostname: hadoop.klinger.local

Decommission Status : Normal

Configured Ca= pacity: 492732858368 (458.89 GB)

DFS Used: 28672 (28 KB)=

Non DFS Used: 27942051= 840 (26.02 GB)

DFS Remaining: 464790777856 (432.87 GB)

DFS Used%: 0.00%<= u>

DFS Remai= ning%: 94.33%

Configured Cache Capacity: 0 (0 B)

Cache Used: 0 (0 B)<= /u>

Cache Remain= ing: 0 (0 B)

Cache Used%: 100.00%

Cache Remaining%: 0.00%<= /p>

Xceivers: 1

Last contact:= Mon Mar 02 00:38:00 CET 2015

=C2=A0

=C2=A0

Name: 10.0.1.89:50010 (hadoop-data.klinger.local)

Hostnam= e: hadoop-data.klinger.local

Decommission Status : Normal

Configured Capacity: = 492732858368 (458.89 GB)

DFS Used: 28672 (28 KB)

Non DFS Used: 27631304704 (25.= 73 GB)

DFS Remaining: 465101524992 (433.16 GB)

DFS Used%: 0.00%

DFS Remaining%: 9= 4.39%

Configured Cache Capacity: 0 (0 B)

Cache Used: 0 (0 B)

Cache Remaining: 0 (= 0 B)

Cache Used%: 100.00%

Cache Remaining%: 0.00%

Xceivers: 1

Last contact: Mon Mar= 02 00:37:59 CET 2015

=C2=A0

<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:"Arial"= ,sans-serif;color:black">Any further thoughts?

=C2=A0

Greets

DK

Von: Ulul [mailto:hadoop@ulul.org]
Gesendet: Sonntag, 1. M=C3= =A4rz 2015 13:12

= =C2=A0

Hi
Did you check your slaves file is correct ?
That the datanode process i= s actually running ?
Did you check its log file ?
That the datanode i= s available ? (dfsadmin -report, through the WUI)

We need more detai= l

Ulul

Thanks but i know h=
ow to kill a process in Linux. But this didn=E2=80=99t answer the question =
why the command say no Datanode to stop instead of stopping the Datanode:
=C2=A0
$HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR -=
-script hdfs stop datanode
=C2=A0
=C2=A0

=C2=A0<= /u>

=C2=A0<= /u>

Von: Surbhi Gupta [mailto:surbhi.gupta01@g= mail.com]
Gesendet: Samstag, 28. Februar 2015 20:16
An= : user@hado= op.apache.org
Betreff: Re: Hadoop 2.6.0 - No DataNode to stop=

=C2=A0

= Issue jps and get the process id or
Try to get the process id of datano= de.

Issue ps-fu userid of the user through which datano= de is running.

Then kill the process using kill -9

On 28 Feb 2015 09:38, "Daniel= Klinger" <dk@web-computing.de> wrote:

Hello,

=C2=A0

I used a lot of Hadoop-Distributions. Now I= =E2=80=99m trying to install a pure Hadoop on a little =E2=80=9Ecluster=E2= =80=9C for testing (2 CentOS-VMs: 1 Name+DataNode 1 DataNode). I followed t= he instructions on the Documentation site: http://hadoop.apache.org/docs/current/hadoop-project-dist/hado= op-common/ClusterSetup.html.

=C2=A0

I=E2=80=99m starting the Cluster like it is described in the Ch= apter =E2=80=9EOperating the Hadoop Cluster=E2=80=9C(with different users).= The starting process works great. The PID-Files are created in /var/run an= d u can see that Folders and Files are created in the Data- and NameNode fo= lders. I=E2=80=99m getting no errors in the log-files.=

=C2=A0

When I try to stop the cluster all Servic= es are stopped (NameNode, ResourceManager etc.). But when I stop the DataNo= des I=E2=80=99m getting the message: =E2=80=9ENo DataNode to stop=E2=80=9C.= The PID-File and the in_use.lock-File are still there and if I try to star= t the DataNode again I=E2=80=99m getting the error that the Process is alre= ady running. When I stop the DataNode as hdfs instead of root the PID and i= n_use-File are removed but I=E2=80=99m still getting the message: =E2=80=9E= No DataNode to stop=E2=80=9C

=C2=A0

What I=E2=80=99m doing wrong?

=C2=A0

Greets

dk

=C2=A0




--
Regard= s,
Varun Kumar.P=
--089e0122772e367895051046a58c--