hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Vadhera <project.linux.p...@gmail.com>
Subject Re: hadoop filesystem corrupt
Date Fri, 01 Mar 2013 21:01:22 GMT
Hi,

I solved this problem by running command " hadoop fsck / -delete" . As I
written in my email that partial data has been moved and remaining is not
moved. Can i merge the data again .  Below are directories.  How can i sure
the hadoop cluster is running fine . Is there any way to verify ?

# ll  /var/lib/cache/hdfs/hdfs/dfs/data       (running data)
total 8
drwxr-xr-x. 3 hdfs hdfs 4096 Dec 27 13:31 current
-rw-r--r--  1 hdfs hdfs   29 Mar  1 12:50 in_use.lock


Data that is not
# ll  /var/lib/cache/hdfs/hdfs/dfs/data_brk/ old data
total 4
drwx------ 3 hdfs hdfs 4096 Mar  1 12:20 current

size
# sudo du -sh  /var/lib/cache/hdfs/hdfs/dfs/*
171G    /mnt/san1/hdfs/hdfs/dfs/data
30G     /mnt/san1/hdfs/hdfs/dfs/data_brk
411M    /mnt/san1/hdfs/hdfs/dfs/name
8.0K    /mnt/san1/hdfs/hdfs/dfs/namesecondary

Thanks,




On Sat, Mar 2, 2013 at 1:02 AM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Hi Mohit,
>
> Is your replication factor really setup to 1?
>
> "Default replication factor:    1"
>
> Also, can you look into you data directories and ensure you always
> have the right structur and all the related META files?
>
> JM
>
> 2013/3/1 Mohit Vadhera <project.linux.proj@gmail.com>:
> > Hi,
> >
> > While moving the data my data folder didn't move successfully. Now while
> > running the FSCK i get the file system is corrupt. Can anybody help me at
> > the earliest please. I shall be thankful
> >
> > $ sudo - hdfs hadoop fsck /
> >
> > -----------------------------
> > -------------------------
> >
> > ...Status: CORRUPT
> >  Total size:    739698399117 B
> >  Total dirs:    2179
> >  Total files:   9064 (Files currently being written: 1)
> >  Total blocks (validated):      17060 (avg. block size 43358640 B)
> >   ********************************
> >   CORRUPT FILES:        4
> >   CORRUPT BLOCKS:       4
> >   ********************************
> >  Minimally replicated blocks:   17060 (100.0 %)
> >  Over-replicated blocks:        0 (0.0 %)
> >  Under-replicated blocks:       232 (1.3599062 %)
> >  Mis-replicated blocks:         0 (0.0 %)
> >  Default replication factor:    1
> >  Average block replication:     1.0
> >  Corrupt blocks:                4
> >  Missing replicas:              2088 (10.904533 %)
> >  Number of data-nodes:          1
> >  Number of racks:               1
> > FSCK ended at Fri Mar 01 13:56:41 EST 2013 in 509 milliseconds
> >
> >
> > The filesystem under path '/' is CORRUPT
> >
>

Mime
View raw message