hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bejoy Ks <bejoy.had...@gmail.com>
Subject Re: log
Date Fri, 19 Apr 2013 09:21:50 GMT
This basically happens while running a mapreduce job. When a map reduce job
is triggered the job files are put in hdfs with high replication (
replication is controlled by  - 'mapred.submit.replication'  default value
is 10).

The job files are cleaned up after the job is completed and hence that
could be the reason you are seeing the hdfs file system status as healthy
after running the job.


On Fri, Apr 19, 2013 at 1:04 PM, Mohit Vadhera <project.linux.proj@gmail.com
> wrote:

> its one (1). Output is below.
>
> .......Status: HEALTHY
>  Total size:    903709673179 B
>  Total dirs:    2906
>  Total files:   11110
>  Total blocks (validated):      20906 (avg. block size 43227287 B)
>  Minimally replicated blocks:   20906 (100.0 %)
>  Over-replicated blocks:        0 (0.0 %)
>  Under-replicated blocks:       248 (1.1862624 %)
>  Mis-replicated blocks:         0 (0.0 %)
>  Default replication factor:    1
>  Average block replication:     1.0
>  Corrupt blocks:                0
>  Missing replicas:              2232 (9.646469 %)
>  Number of data-nodes:          1
>  Number of racks:               1
> FSCK ended at Fri Apr 19 03:47:04 EDT 2013 in 2224 milliseconds
>
>
> The filesystem under path '/' is HEALTHY
>
>
>
> On Fri, Apr 19, 2013 at 12:28 PM, S, Manoj <manoj.s@intel.com> wrote:
>
>>  It means that some of your data blocks are not replicated as intended.
>> What is the value of “dfs.replication” in your hadoop-site.xml file?****
>>
>> ** **
>>
>> Can you paste the output of ****
>>
>> ** **
>>
>> *bin/hadoop fsck /****
>>
>> ** **
>>
>> --****
>>
>> Manoj****
>>
>> ** **
>>
>> *From:* Mohit Vadhera [mailto:project.linux.proj@gmail.com]
>> *Sent:* Friday, April 19, 2013 12:09 PM
>> *To:* <user@hadoop.apache.org>
>> *Subject:* log****
>>
>> ** **
>>
>> Can anybody let me know the meaning of the below log plz " Target
>> Replicas is 10 but found 1 replica(s)." ?
>>
>> /var/lib/hadoop-hdfs/cache/mapred/mapred/staging/test_user/.staging/job_201302180313_0623/job.split:
>> Under replicated
>> BP-2091347308-172.20.3.119-1356632249303:blk_6297333561560198850_70720.
>> Target
>> Replicas is 10 but found 1 replica(s).
>>
>> ****
>>
>> Thanks,****
>>
>
>

Mime
View raw message