hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuting Lin <mikeli...@gmail.com>
Subject Error while random accessing Lzo file in Hadoop
Date Mon, 19 Apr 2010 13:33:09 GMT
Hi all

I try to random access the sequence file which is blocked compressed by
"com.hadoop.compression.lzo.LzoCodec.class".  However, when the program
contains more than one seek(offset) followed by next(), it leads unexpected
error in Java (The offset passed to seek() is the beginning position of the
block).

If the sequence file is compressed by
"org.apache.hadoop.io.compress.GzipCodec.class", there is no such error.

My Linux is 32-bit 2.6.27, Lzo version is 0.1.0. OS, Java version is 1.6.0.
The log of error is shown below.

Is there any approach to random access the lzo-file in Hadoop? Thanks.
-
Regards
Yuting

---------------  S Y S T E M  ---------------

OS:lenny/sid

uname:Linux 2.6.27-14-generic #1 SMP Mon Aug 31 13:01:41 UTC 2009 i686
libc:glibc 2.8.90 NPTL 2.8.90
rlimit: STACK 8192k, CORE 0k, NPROC 26607, NOFILE 1024, AS infinity
load average:0.03 0.11 0.14

CPU:total 2 (2 cores per cpu, 1 threads per core) family 6 model 15 stepping
11, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3

Memory: 4k page, physical 3369768k(1217868k free), swap 1140572k(1140572k
free)

vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE
(1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc 3.2.1-7a
(J2SE release)

---------------  T H R E A D  ---------------

Current thread (0x09542000):  JavaThread "main" [_thread_in_native,
id=22346, stack(0xb7e6e000,0xb7ebf000)]

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
si_addr=0x09a82000

Registers:
EAX=0x00000000, EBX=0x7181cff4, ECX=0x01048f83, EDX=0x00074de0
ESP=0xb7ebd6c4, EBP=0xb7ebd6f8, ESI=0x09a0c7b5, EDI=0x09a0d21b
EIP=0x716d644b, CR2=0x09a82000, EFLAGS=0x00210212

Top of Stack: (sp=0xb7ebd6c4)
0xb7ebd6c4:   00fd419e 00000000 01048f82 09a0d21f
0xb7ebd6d4:   09a0c7b9 099fc003 09a0d21b 099fc21e
0xb7ebd6e4:   095c644c 00000022 7181cff4 09542114
0xb7ebd6f4:   09542000 b7ebd7f8 7181a994 099fc000
0xb7ebd704:   00000003 09a0d000 b7ebd764 00000000
0xb7ebd714:   00000000 b4dec439 716d6200 099fc002
0xb7ebd724:   00000000 ae190570 b7ebd72c 72a18ae0
0xb7ebd734:   b7ebd848 00000003 00010000 72a18af0

Instructions: (pc=0x716d644b)
0x716d643b:   26 00 00 00 00 8b 44 16 04 8b 7d e4 83 6d cc 04
0x716d644b:   89 44 17 04 83 c2 04 83 7d cc 03 77 e8 8d 41 fb
...

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message