hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "liaoyuxiangqin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11841) libc.so.6 detected double free on TestDFSIO write when erasure coding and enabled Intel ISA-L
Date Wed, 17 May 2017 10:02:04 GMT
liaoyuxiangqin created HDFS-11841:
-------------------------------------

             Summary: libc.so.6 detected double free on TestDFSIO write when erasure coding
and enabled Intel ISA-L 
                 Key: HDFS-11841
                 URL: https://issues.apache.org/jira/browse/HDFS-11841
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: erasure-coding
         Environment: cluster: 3 nodes
os:(Red Hat 2.6.33.20,  Red Hat 3.10.0-514.6.1.el7.x86_64, Ubuntu4.4.0-31-generic)
hadoop version: hadoop-3.0.0-alpha3
erasure coding: XOR-2-1-64k and enabled Intel ISA-L 
TestDFSIO: hadoop jar hadoop-mapreduce-client-jobclient-3.0.0-alpha3-SNAPSHOT-tests.jar TestDFSIO
-write -nrFiles 10000 -size 1MB
            Reporter: liaoyuxiangqin
            Priority: Critical


    when i execute  hadoop jar hadoop-mapreduce-client-jobclient-3.0.0-alpha3-SNAPSHOT-tests.jar
TestDFSIO -write -nrFiles 10000 -size 1MB on above environment,  glibc detected double free
or corruption, detail information as follows:
    File System Counters
		FILE: Number of bytes read=489391103089
		FILE: Number of bytes written=42020064724
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=5751218285
		HDFS: Number of bytes written=52456018688978
		HDFS: Number of read operations=150165017
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=150065006
	Map-Reduce Framework
		Map input records=10000
		Map output records=50000
		Map output bytes=707116
		Map output materialized bytes=867116
		Input split bytes=1288890
		Combine input records=0
		Combine output records=0
		Reduce input groups=5
		Reduce shuffle bytes=867116
		Reduce input records=50000
		Reduce output records=5
		Spilled Records=100000
		Shuffled Maps =10000
		Failed Shuffles=0
		Merged Map outputs=10000
		GC time elapsed (ms)=34575
		Total committed heap usage (bytes)=46386259689472
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=1148890
	File Output Format Counters 
		Bytes Written=88
*** glibc detected *** /home/hadoop/jdk1.8.0_111/bin/java: double free or corruption (out):
0x00007fcd84a6c6c0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3d1fe724a6]
/lib64/libc.so.6(cfree+0x6c)[0x3d1fe771bc]
[0x7fcdb138e8e6]
======= Memory map: ========
00400000-00401000 r-xp 00000000 08:01 3720754                            /home/hadoop/jdk1.8.0_111/bin/java
00600000-00601000 rw-p 00000000 08:01 3720754                            /home/hadoop/jdk1.8.0_111/bin/java
0204f000-04600000 rw-p 00000000 00:00 0                                  [heap]
649400000-6f2100000 rw-p 00000000 00:00 0 
6f2100000-743180000 ---p 00000000 00:00 0 
743180000-7c0000000 rw-p 00000000 00:00 0 
7c0000000-7c03c0000 rw-p 00000000 00:00 0 
7c03c0000-800000000 ---p 00000000 00:00 0 
3d1fa00000-3d1fa21000 r-xp 00000000 08:01 8126508                        /lib64/ld-2.14.1.so
3d1fc20000-3d1fc21000 r--p 00020000 08:01 8126508                        /lib64/ld-2.14.1.so
3d1fc21000-3d1fc23000 rw-p 00021000 08:01 8126508                        /lib64/ld-2.14.1.so
3d1fe00000-3d1ff83000 r-xp 00000000 08:01 8126562                        /lib64/libc-2.14.1.so
3d1ff83000-3d20182000 ---p 00183000 08:01 8126562                        /lib64/libc-2.14.1.so
3d20182000-3d20186000 r--p 00182000 08:01 8126562                        /lib64/libc-2.14.1.so
3d20186000-3d20187000 rw-p 00186000 08:01 8126562                        /lib64/libc-2.14.1.so
3d20187000-3d2018c000 rw-p 00000000 00:00 0 
3d20200000-3d20282000 r-xp 00000000 08:01 8126567                        /lib64/libm-2.14.1.so
3d20282000-3d20481000 ---p 00082000 08:01 8126567                        /lib64/libm-2.14.1.so
3d20481000-3d20482000 r--p 00081000 08:01 8126567                        /lib64/libm-2.14.1.so
3d20482000-3d20483000 rw-p 00082000 08:01 8126567                        /lib64/libm-2.14.1.so
3d20a00000-3d20a02000 r-xp 00000000 08:01 8126764                        /lib64/libdl-2.14.1.so
3d20a02000-3d20c02000 ---p 00002000 08:01 8126764                        /lib64/libdl-2.14.1.so
3d20c02000-3d20c03000 r--p 00002000 08:01 8126764                        /lib64/libdl-2.14.1.so
3d20c03000-3d20c04000 rw-p 00003000 08:01 8126764                        /lib64/libdl-2.14.1.so
3d21200000-3d21218000 r-xp 00000000 08:01 8126576                        /lib64/libpthread-2.14.1.so
3d21218000-3d21417000 ---p 00018000 08:01 8126576                        /lib64/libpthread-2.14.1.so
3d21417000-3d21418000 r--p 00017000 08:01 8126576                        /lib64/libpthread-2.14.1.so
3d21418000-3d21419000 rw-p 00018000 08:01 8126576                        /lib64/libpthread-2.14.1.so
3d21419000-3d2141d000 rw-p 00000000 00:00 0 
3d21a00000-3d21a0a000 r-xp 00000000 08:01 8126578                        /lib64/librt-2.14.1.so
3d21a0a000-3d21c09000 ---p 0000a000 08:01 8126578                        /lib64/librt-2.14.1.so
3d21c09000-3d21c0a000 r--p 00009000 08:01 8126578                        /lib64/librt-2.14.1.so
3d21c0a000-3d21c0b000 rw-p 0000a000 08:01 8126578                        /lib64/librt-2.14.1.so
3d21c0b000-3d21c1b000 rw-p 00000000 00:00 0 
3d2ae00000-3d2ae16000 r-xp 00000000 08:01 8126574                        /lib64/libgcc_s-4.4.5-20110214.so.1
3d2ae16000-3d2b015000 ---p 00016000 08:01 8126574                        /lib64/libgcc_s-4.4.5-20110214.so.1
3d2b015000-3d2b016000 rw-p 00015000 08:01 8126574                        /lib64/libgcc_s-4.4.5-20110214.so.1
7fcd84000000-7fcd86c57000 rw-p 00000000 00:00 0 
7fcd86c57000-7fcd88000000 ---p 00000000 00:00 0 
7fcd8a5ad000-7fcd8a5b0000 ---p 00000000 00:00 0 
7fcd8a5b0000-7fcd8a82e000 rw-p 00000000 00:00 0 
7fcd8a82e000-7fcd8a8b1000 ---p 00000000 00:00 0 
7fcd8a8b1000-7fcd8a9af000 rw-p 00000000 00:00 0 
7fcd8a9af000-7fcd8a9b2000 ---p 00000000 00:00 0 
7fcd8a9b2000-7fcd8aab0000 rw-p 00000000 00:00 0 
7fcd8aab0000-7fcd8aab3000 ---p 00000000 00:00 0 
7fcd8aab3000-7fcd8abb1000 rw-p 00000000 00:00 0 
7fcd8abb1000-7fcd8abb4000 ---p 00000000 00:00 0 
7fcd8abb4000-7fcd8acb2000 rw-p 00000000 00:00 0 
7fcd8acb2000-7fcd8acb5000 ---p 00000000 00:00 0 
7fcd8acb5000-7fcd8adb3000 rw-p 00000000 00:00 0 
7fcd8adb3000-7fcd8adb6000 ---p 00000000 00:00 0 
7fcd8adb6000-7fcd8aeb4000 rw-p 00000000 00:00 0 
7fcd8aeb4000-7fcd8aeb7000 ---p 00000000 00:00 0 
7fcd8aeb7000-7fcd8afb5000 rw-p 00000000 00:00 0 
7fcd8afb5000-7fcd8afb8000 ---p 00000000 00:00 0 
7fcd8afb8000-7fcd8b2b6000 rw-p 00000000 00:00 0 
7fcd8b2b6000-7fcd8b4b6000 rw-p 00000000 00:00 0 
7fcd8b4b6000-7fcd8b4e8000 r-xp 00000000 08:01 3720813                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libsunec.so
7fcd8b4e8000-7fcd8b6e7000 ---p 00032000 08:01 3720813                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libsunec.so
7fcd8b6e7000-7fcd8b6ed000 rw-p 00031000 08:01 3720813                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libsunec.so
7fcd8b6ed000-7fcd8b8fd000 rw-p 00000000 00:00 0 
7fcd8b8fd000-7fcd8bafd000 rw-p 00000000 00:00 0 
7fcd8bafd000-7fcd8bb00000 ---p 00000000 00:00 0 
7fcd8bb00000-7fcd8bbfe000 rw-p 00000000 00:00 0 
7fcd8bbfe000-7fcd8bc01000 ---p 00000000 00:00 0 
7fcd8bc01000-7fcd8beff000 rw-p 00000000 00:00 0 
7fcd8beff000-7fcd8bf02000 ---p 00000000 00:00 0 
7fcd8bf02000-7fcd8c000000 rw-p 00000000 00:00 0 
7fcd8c000000-7fcd8f0ad000 rw-p 00000000 00:00 0 
7fcd8f0ad000-7fcd90000000 ---p 00000000 00:00 0 
7fcd90015000-7fcd90028000 r--s 00345000 08:01 3720986                    /home/hadoop/jdk1.8.0_111/jre/lib/resources.jar
7fcd90028000-7fcd9002b000 ---p 00000000 00:00 0 
7fcd9002b000-7fcd90129000 rw-p 00000000 00:00 0 
7fcd90129000-7fcd9012c000 ---p 00000000 00:00 0 
7fcd9012c000-7fcd9022a000 rw-p 00000000 00:00 0 
7fcd9022a000-7fcd9022d000 ---p 00000000 00:00 0 
7fcd9022d000-7fcd9032b000 rw-p 00000000 00:00 0 
7fcd9032b000-7fcd9032e000 ---p 00000000 00:00 0 
7fcd9032e000-7fcd9042c000 rw-p 00000000 00:00 0 
7fcd9042c000-7fcd9042f000 ---p 00000000 00:00 0 
7fcd9042f000-7fcd9052d000 rw-p 00000000 00:00 0 
7fcd9052d000-7fcd90530000 ---p 00000000 00:00 0 
7fcd90530000-7fcd9062e000 rw-p 00000000 00:00 0 
7fcd9062e000-7fcd90658000 r-xp 00000000 08:01 1619849                    /usr/lib/libisal.so.2.0.18
7fcd90658000-7fcd90857000 ---p 0002a000 08:01 1619849                    /usr/lib/libisal.so.2.0.18
7fcd90857000-7fcd90858000 r--p 00029000 08:01 1619849                    /usr/lib/libisal.so.2.0.18
7fcd90858000-7fcd9085e000 rw-p 0002a000 08:01 1619849                    /usr/lib/libisal.so.2.0.18
7fcd9085f000-7fcd90860000 rw-p 00000000 00:00 0 
7fcd90860000-7fcd90865000 r--s 002f9000 08:01 3720780                    /home/hadoop/jdk1.8.0_111/jre/lib/charsets.jar
7fcd90865000-7fcd9086b000 r--s 0003f000 08:01 3720879                    /home/hadoop/jdk1.8.0_111/jre/lib/ext/sunjce_provider.jar
7fcd9086b000-7fcd9086d000 r--s 00009000 08:01 3720878                    /home/hadoop/jdk1.8.0_111/jre/lib/ext/sunec.jar
7fcd9086d000-7fcd90871000 r--s 0003a000 08:01 3720876                    /home/hadoop/jdk1.8.0_111/jre/lib/ext/sunpkcs11.jar
7fcd90871000-7fcd90874000 r--s 0001a000 08:01 3721014                    /home/hadoop/jdk1.8.0_111/jre/lib/jce.jar
7fcd90874000-7fcd90a74000 rw-p 00000000 00:00 0 
7fcd90a74000-7fcd90a77000 ---p 00000000 00:00 0 
7fcd90a77000-7fcd90b75000 rw-p 00000000 00:00 0 
7fcd90b75000-7fcd90b78000 ---p 00000000 00:00 0 
7fcd90b78000-7fcd90c76000 rw-p 00000000 00:00 0 
7fcd90c76000-7fcd90c87000 r-xp 00000000 08:01 3720830                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libnio.so
7fcd90c87000-7fcd90e86000 ---p 00011000 08:01 3720830                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libnio.so
7fcd90e86000-7fcd90e87000 rw-p 00010000 08:01 3720830                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libnio.so
7fcd90e87000-7fcd90e8a000 ---p 00000000 00:00 0 
7fcd90e8a000-7fcd91188000 rw-p 00000000 00:00 0 
7fcd91188000-7fcd91388000 rw-p 00000000 00:00 0 
7fcd91388000-7fcd91588000 rw-p 00000000 00:00 0 
7fcd91588000-7fcd91788000 rw-p 00000000 00:00 0 
7fcd91788000-7fcd91a88000 rw-p 00000000 00:00 0 
7fcd91a88000-7fcd91a9e000 r-xp 00000000 08:01 3720805                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libnet.so
7fcd91a9e000-7fcd91c9e000 ---p 00016000 08:01 3720805                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libnet.so
7fcd91c9e000-7fcd91c9f000 rw-p 00016000 08:01 3720805                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libnet.so
7fcd91c9f000-7fcd91ca0000 r-xp 00000000 08:01 3720862                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libjaas_unix.so
7fcd91ca0000-7fcd91e9f000 ---p 00001000 08:01 3720862                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libjaas_unix.so
7fcd91e9f000-7fcd91ea0000 rw-p 00000000 08:01 3720862                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libjaas_unix.so
7fcd91ea0000-7fcd91ec2000 r-xp 00000000 08:01 3852081                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/lib/native/libhadoop.so.1.0.0
7fcd91ec2000-7fcd920c1000 ---p 00022000 08:01 3852081                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/lib/native/libhadoop.so.1.0.0
7fcd920c1000-7fcd920c2000 r--p 00021000 08:01 3852081                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/lib/native/libhadoop.so.1.0.0
7fcd920c2000-7fcd920c3000 rw-p 00022000 08:01 3852081                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/lib/native/libhadoop.so.1.0.0
7fcd920c3000-7fcd920cc000 r-xp 00000000 08:01 3720841                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libmanagement.so
7fcd920cc000-7fcd922cb000 ---p 00009000 08:01 3720841                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libmanagement.so
7fcd922cb000-7fcd922cc000 rw-p 00008000 08:01 3720841                    /home/hadoop/jdk1.8.0_111/jre/lib/amd64/libmanagement.so
7fcd922cc000-7fcd924cc000 rw-p 00000000 00:00 0 
7fcd924cc000-7fcd924cf000 ---p 00000000 00:00 0 
7fcd924cf000-7fcd925cd000 rw-p 00000000 00:00 0 
7fcd925cd000-7fcd925d0000 ---p 00000000 00:00 0 
7fcd925d0000-7fcd92606000 rw-p 00000000 00:00 0 
7fcd92606000-7fcd9260a000 r--s 00097000 08:01 3720779                    /home/hadoop/jdk1.8.0_111/jre/lib/jsse.jar
7fcd9260a000-7fcd9260f000 r--s 00044000 08:01 3852117                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.0.0-alpha3-SNAPSHOT.jar
7fcd9260f000-7fcd92612000 r--s 0001f000 08:01 3852191                    /home/gy/hadoop-3.0.0-alpha3-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-registry-3.0.0-alpha3-SNAPSHOT.jarAborted



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message