tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinho Kim <jino...@gmail.com>
Subject Re: [FYI] some remarkable features in hadoop 2.3.0
Date Tue, 25 Feb 2014 02:26:56 GMT
Interesting news

I'm looking forward to the De/Compression


https://issues.apache.org/jira/browse/HADOOP-10047?focusedCommentId=13815175&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13815175

--Jinho
Best regards


2014-02-25 11:13 GMT+09:00 Hyunsik Choi <hyunsik@apache.org>:

> Hi Min,
>
> HADOOP-10047 seems to not support other codecs like LZO. Although we
> firstly consider snappy and deflate that Hadoop currently supports, there
> is no reason to not support other codecs if users want :)
>
> In addition, zero-copy tuple looks very interesting. If you create the
> issue, I'll appreciate it..
>
> - hyunsik
>
>
>
> On Tue, Feb 25, 2014 at 10:48 AM, Min Zhou <coderplay@gmail.com> wrote:
>
> > Hi,
> >
> > +1.
> > It's happy to see 2.3.0 get released. Actually, I did something dev on
> > zero-copy processing tuples based on that version. But haven't finished
> > yet, I can create a ticket for that.  Anyway, currently the zero-copy
> > decompressors in hadoop have only two,  default and snappy. Do you guys
> > decide to support lzo or any other type of compressions?
> >
> > Regards,
> > Min
> >
> >
> > On Mon, Feb 24, 2014 at 5:23 PM, JaeHwa Jung <jhjung@gruter.com> wrote:
> >
> > > +1.
> > >
> > > Thanks Hyunsik, I also agree with you.
> > > We need to bump up hadoop to 2.3.0.
> > >
> > > Cheers
> > >
> > >
> > > 2014-02-25 10:19 GMT+09:00 Hyunsik Choi <hyunsik@apache.org>:
> > >
> > > > Hi folks,
> > > >
> > > > As you already know, Hadoop 2.3.0 release. While I'm reading the
> > > changes, I
> > > > noted some new features that Tajo should consider.
> > > >
> > > > Centralized cache management in HDFS
> > > >   - https://issues.apache.org/jira/browse/HDFS-4949
> > > >
> > > > Ealier, Min mentioned cached table. In offline, I discussed HDFS-4949
> > > with
> > > > him. It may be a candidate feature for our goal.
> > > >
> > > > Enable support for heterogeneous storages in HDFS - DN as a
> collection
> > of
> > > > storages
> > > >  - https://issues.apache.org/jira/browse/HDFS-2832
> > > >
> > > > It's for different storage medias like SSD and HDD.
> > > >
> > > > Add a directbuffer Decompressor API to hadoop
> > > >   - https://issues.apache.org/jira/browse/HADOOP-10047
> > > >
> > > > We already use compression/decompression in text file. We also should
> > > adopt
> > > > comp/decomp to other file formats. For that, HDFS-10047 may be a nice
> > > > candidate feature to be used.
> > > >
> > > > - hyunsik
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Jaehwa Jung
> > > Bigdata Platform Team
> > > Gruter
> > >
> >
> >
> >
> > --
> > My research interests are distributed systems, parallel computing and
> > bytecode based virtual machine.
> >
> > My profile:
> > http://www.linkedin.com/in/coderplay
> > My blog:
> > http://coderplay.javaeye.com
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message