Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:from:to:in-reply-to:content-type:
	content-transfer-encoding:mime-version:subject:date:references:x-mailer;
	b=HXuTkKt2LJ7eFOKt3syCHoXmYoROX4QXgQ/nEGFaIkYpQoGcMJpdJOwMT2rA/JGC
Message-Id: <8C2892F2-D5B2-4F86-A0E2-F0493CC1BFD6@yahoo-inc.com>
From: Hong Tang <htang@yahoo-inc.com>
To: general@hadoop.apache.org
In-Reply-To: <h2o999b56ef1004190633jea5b15a2hf1924116bae09840@mail.gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v936)
Subject: Re: Error while random accessing Lzo file in Hadoop
Date: Mon, 19 Apr 2010 12:05:50 -0700
References: <h2o999b56ef1004190633jea5b15a2hf1924116bae09840@mail.gmail.com>

Yuting,

Could you please report the bug at http://code.google.com/p/hadoop-gpl-compression/issues/list? 
  Also it would be helpful if you could share your test program and  
sample data.

-Hong

On Apr 19, 2010, at 6:33 AM, Yuting Lin wrote:

> Hi all
>
> I try to random access the sequence file which is blocked compressed  
> by
> "com.hadoop.compression.lzo.LzoCodec.class".  However, when the  
> program
> contains more than one seek(offset) followed by next(), it leads  
> unexpected
> error in Java (The offset passed to seek() is the beginning position  
> of the
> block).
>
> If the sequence file is compressed by
> "org.apache.hadoop.io.compress.GzipCodec.class", there is no such  
> error.
>
> My Linux is 32-bit 2.6.27, Lzo version is 0.1.0. OS, Java version is  
> 1.6.0.
> The log of error is shown below.
>
> Is there any approach to random access the lzo-file in Hadoop? Thanks.
> -
> Regards
> Yuting
>
> ---------------  S Y S T E M  ---------------
>
> OS:lenny/sid
>
> uname:Linux 2.6.27-14-generic #1 SMP Mon Aug 31 13:01:41 UTC 2009 i686
> libc:glibc 2.8.90 NPTL 2.8.90
> rlimit: STACK 8192k, CORE 0k, NPROC 26607, NOFILE 1024, AS infinity
> load average:0.03 0.11 0.14
>
> CPU:total 2 (2 cores per cpu, 1 threads per core) family 6 model 15  
> stepping
> 11, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3
>
> Memory: 4k page, physical 3369768k(1217868k free), swap  
> 1140572k(1140572k
> free)
>
> vm_info: Java HotSpot(TM) Server VM (11.0-b15) for linux-x86 JRE
> (1.6.0_10-b33), built on Sep 26 2008 01:05:20 by "java_re" with gcc  
> 3.2.1-7a
> (J2SE release)
>
> ---------------  T H R E A D  ---------------
>
> Current thread (0x09542000):  JavaThread "main" [_thread_in_native,
> id=22346, stack(0xb7e6e000,0xb7ebf000)]
>
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
> si_addr=0x09a82000
>
> Registers:
> EAX=0x00000000, EBX=0x7181cff4, ECX=0x01048f83, EDX=0x00074de0
> ESP=0xb7ebd6c4, EBP=0xb7ebd6f8, ESI=0x09a0c7b5, EDI=0x09a0d21b
> EIP=0x716d644b, CR2=0x09a82000, EFLAGS=0x00210212
>
> Top of Stack: (sp=0xb7ebd6c4)
> 0xb7ebd6c4:   00fd419e 00000000 01048f82 09a0d21f
> 0xb7ebd6d4:   09a0c7b9 099fc003 09a0d21b 099fc21e
> 0xb7ebd6e4:   095c644c 00000022 7181cff4 09542114
> 0xb7ebd6f4:   09542000 b7ebd7f8 7181a994 099fc000
> 0xb7ebd704:   00000003 09a0d000 b7ebd764 00000000
> 0xb7ebd714:   00000000 b4dec439 716d6200 099fc002
> 0xb7ebd724:   00000000 ae190570 b7ebd72c 72a18ae0
> 0xb7ebd734:   b7ebd848 00000003 00010000 72a18af0
>
> Instructions: (pc=0x716d644b)
> 0x716d643b:   26 00 00 00 00 8b 44 16 04 8b 7d e4 83 6d cc 04
> 0x716d644b:   89 44 17 04 83 c2 04 83 7d cc 03 77 e8 8d 41 fb
> ...