tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jihoon Son <ghoon...@gmail.com>
Subject Re: Tajo C++ worker
Date Thu, 23 Jan 2014 01:50:31 GMT
Hi Yu,

I'm very excited for integrating vecterization and compilation techniques
to Tajo!! As Hyunsik said, it would be great to proceed the work in a new
branch after review, because this is a new feature for Tajo.

I'll expect that your work is finished and submitted to the master branch
as soon as possible.

Thanks!
Jihoon


2014/1/23 JaeHwa Jung <jhjung@gruter.com>

> Awesome!
>
> Thanks guys.
> Your contributions will improve tajo more powerful. :)
>
>
>
> 2014/1/23 Hyunsik Choi <hyunsik@apache.org>
>
> > Hi Yu,
> >
> > Thank you for your contribution. I have been looking forward to seeing
> > this work. Could you submit a patch to proper issues or create a jira?
> > We will create a new branch for that after review.
> >
> > Hi folks,
> >
> > Some of us know Yu in offline. After the discussion early this year,
> > he has mostly worked on Tajo C++ worker. I also contributed a small
> > part about JIT code generation via LLVM, and participated in the
> > design. Another contributor, Hyoung Jun, also contributed some module.
> > Its main objective is to build a high performance query engine to
> > maximize hardware utilization by using SIMD and cache conscious
> > algorithm. It is under heavy development. As he mentioned, it would be
> > a long journey for the future.
> >
> > - hyunsik
> >
> > On Wed, Jan 22, 2014 at 8:19 PM, Yu Dongmin <miniway@gmail.com> wrote:
> > > Hello,
> > >
> > > I'm very glad to hear that Tajo is getting stable and production ready.
> > >
> > > Here's another news. Some of us might have have seen some Hyunsik's
> > presentations commenting about an experimental project, c++ tajo worker.
> > >
> > >
> > > Recently, I've been working on the c++ worker, even though it is a
> > little bit far from behind schedule than what I expected,
> > > it is able to communicate with Tajo Master and Query Master
> successfully.
> > >
> > > This is never a replacement of Java worker, but an exchangeable worker.
> > We could use Java workers as is, C++ workers only, or mixed workers.
> > >
> > > It is designed as a vectorized execution engine hoping to process
> > certain types of data structures very efficiently.
> > >
> > > These are the supported features right now.
> > >
> > > - Reading and parsing csv files in the hadoop data node.
> > > - Filtering rows within LLVM code generated evaluation
> > > - Simple scalar functions
> > > - Simple group by aggregation functions
> > >
> > > Now I'm working on 'order by' clause and doing profiling to get
> expected
> > performance.
> > >
> > > Working on these stuffs, I hope community to allow creating a new git
> > branch, say native_worker, cplus_worker or a nicer name, to make this
> > project together.
> > >
> > >
> > > There's still a long way to go on this project, but it could be
> improved
> > with help of the Tajo community.
> > >
> > >
> > > Thanks
> > > Min
> >
>
>
>
> --
> Thanks,
> Jaehwa Jung
> Bigdata Platform Team
> Gruter
>



-- 
Jihoon Son

Database & Information Systems Group,
Prof. Yon Dohn Chung Lab.
Dept. of Computer Science & Engineering,
Korea University
1, 5-ga, Anam-dong, Seongbuk-gu,
Seoul, 136-713, Republic of Korea

Tel : +82-2-3290-3580
E-mail : jihoonson@korea.ac.kr

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message