hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Reading from sequence file using java FS api
Date Tue, 13 Nov 2012 12:56:43 GMT
Yes, the codec information is stored in the file's header.

Same goes for Avro where even the deserialization schema logic is
stored in addition, so you can just directly read into usable
primitive/compound objects and not have to do manual transformation
work.

On Tue, Nov 13, 2012 at 6:07 AM, Mohit Anchlia <mohitanchlia@gmail.com> wrote:
> I was simple able to read using below code. Didn't have to decompress. It
> looks like reader automatically knows and decompresses the file before
> returning it to the user.
>
>
> On Mon, Nov 12, 2012 at 3:16 PM, Mohit Anchlia <mohitanchlia@gmail.com>
> wrote:
>>
>> I am looking for an example that read snappy compressed snappy file. Could
>> someone point me to it? What I have so far is this:
>>
>>
>> Configuration conf =
>>
>> new Configuration();
>>
>> FileSystem fs = FileSystem.get(URI.create(uri), conf);
>>
>> Path path =
>>
>> new Path(uri);
>>
>> SequenceFile.Reader reader =
>>
>> null;
>>
>> org.apache.hadoop.io.LongWritable key =
>>
>> new org.apache.hadoop.io.LongWritable();
>>
>> org.apache.hadoop.io.Text value =
>>
>> new org.apache.hadoop.io.Text();
>>
>> try {
>>
>> reader = new SequenceFile.Reader(fs, path, conf);
>
>



-- 
Harsh J

Mime
View raw message