hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akira Ajisaka (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-11841) libc.so.6 detected double free on TestDFSIO write when erasure coding and enabled Intel ISA-L
Date Thu, 25 Jan 2018 08:01:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Akira Ajisaka resolved HDFS-11841.
----------------------------------
    Resolution: Duplicate

> libc.so.6 detected double free on TestDFSIO write when erasure coding and enabled Intel
ISA-L 
> ----------------------------------------------------------------------------------------------
>
>                 Key: HDFS-11841
>                 URL: https://issues.apache.org/jira/browse/HDFS-11841
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding, native
>         Environment: cluster: 3 nodes
> os:(Red Hat 2.6.33.20,  Red Hat 3.10.0-514.6.1.el7.x86_64, Ubuntu4.4.0-31-generic)
> hadoop version: hadoop-3.0.0-alpha3
> erasure coding: XOR-2-1-64k and enabled Intel ISA-L 
> TestDFSIO: hadoop jar hadoop-mapreduce-client-jobclient-3.0.0-alpha3-SNAPSHOT-tests.jar
TestDFSIO -write -nrFiles 10000 -size 1MB
>            Reporter: liaoyuxiangqin
>            Priority: Critical
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
>     when i execute  hadoop jar hadoop-mapreduce-client-jobclient-3.0.0-alpha3-SNAPSHOT-tests.jar
TestDFSIO -write -nrFiles 10000 -size 1MB on above environment,  glibc detected double free
or corruption, detail information as follows:
> {pane}
>     File System Counters
> 		FILE: Number of bytes read=489391103089
> 		FILE: Number of bytes written=42020064724
> 		FILE: Number of read operations=0
> 		FILE: Number of large read operations=0
> 		FILE: Number of write operations=0
> 		HDFS: Number of bytes read=5751218285
> 		HDFS: Number of bytes written=52456018688978
> 		HDFS: Number of read operations=150165017
> 		HDFS: Number of large read operations=0
> 		HDFS: Number of write operations=150065006
> 	Map-Reduce Framework
> 		Map input records=10000
> 		Map output records=50000
> 		Map output bytes=707116
> 		Map output materialized bytes=867116
> 		Input split bytes=1288890
> 		Combine input records=0
> 		Combine output records=0
> 		Reduce input groups=5
> 		Reduce shuffle bytes=867116
> 		Reduce input records=50000
> 		Reduce output records=5
> 		Spilled Records=100000
> 		Shuffled Maps =10000
> 		Failed Shuffles=0
> 		Merged Map outputs=10000
> 		GC time elapsed (ms)=34575
> 		Total committed heap usage (bytes)=46386259689472
> 	Shuffle Errors
> 		BAD_ID=0
> 		CONNECTION=0
> 		IO_ERROR=0
> 		WRONG_LENGTH=0
> 		WRONG_MAP=0
> 		WRONG_REDUCE=0
> 	File Input Format Counters 
> 		Bytes Read=1148890
> 	File Output Format Counters 
> 		Bytes Written=88
>  glibc detected  /home/hadoop/jdk1.8.0_111/bin/java: double free or corruption (out):
0x00007fcd84a6c6c0 
> ======= Backtrace: =========
> /lib64/libc.so.6[0x3d1fe724a6]
> /lib64/libc.so.6(cfree+0x6c)[0x3d1fe771bc]
> [0x7fcdb138e8e6]
> ======= Memory map: ========
> 00400000-00401000 r-xp 00000000 08:01 3720754                            /home/hadoop/jdk1.8.0_111/bin/java
> 00600000-00601000 rw-p 00000000 08:01 3720754                            /home/hadoop/jdk1.8.0_111/bin/java
> 0204f000-04600000 rw-p 00000000 00:00 0                                  [heap]
> 649400000-6f2100000 rw-p 00000000 00:00 0 
> 6f2100000-743180000 ---p 00000000 00:00 0 
> 743180000-7c0000000 rw-p 00000000 00:00 0 
> 7c0000000-7c03c0000 rw-p 00000000 00:00 0 
> 7c03c0000-800000000 ---p 00000000 00:00 0 
> 3d1fa00000-3d1fa21000 r-xp 00000000 08:01 8126508                        /lib64/ld-2.14.1.so
> 3d1fc20000-3d1fc21000 r--p 00020000 08:01 8126508                        /lib64/ld-2.14.1.so
> 3d1fc21000-3d1fc23000 rw-p 00021000 08:01 8126508                        /lib64/ld-2.14.1.so
> 3d1fe00000-3d1ff83000 r-xp 00000000 08:01 8126562                        /lib64/libc-2.14.1.so
> 3d1ff83000-3d20182000 ---p 00183000 08:01 8126562                        /lib64/libc-2.14.1.so
>  /home/hadoop/jdk1.8.0_111/jre/lib/jsse.jar
> 7fcd9260a000-7fcd9260f000 r--s 00044000 08:01 3852117                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.0.0-alpha3-SNAPSHOT.jar
> 7fcd9260f000-7fcd92612000 r--s 0001f000 08:01 3852191                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-registry-3.0.0-alpha3-SNAPSHOT.jarAborted
> {panel}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message