tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JaeHwa Jung <jhj...@gruter.com>
Subject Re: Tajo C++ worker
Date Thu, 23 Jan 2014 01:16:55 GMT
Awesome!

Thanks guys.
Your contributions will improve tajo more powerful. :)



2014/1/23 Hyunsik Choi <hyunsik@apache.org>

> Hi Yu,
>
> Thank you for your contribution. I have been looking forward to seeing
> this work. Could you submit a patch to proper issues or create a jira?
> We will create a new branch for that after review.
>
> Hi folks,
>
> Some of us know Yu in offline. After the discussion early this year,
> he has mostly worked on Tajo C++ worker. I also contributed a small
> part about JIT code generation via LLVM, and participated in the
> design. Another contributor, Hyoung Jun, also contributed some module.
> Its main objective is to build a high performance query engine to
> maximize hardware utilization by using SIMD and cache conscious
> algorithm. It is under heavy development. As he mentioned, it would be
> a long journey for the future.
>
> - hyunsik
>
> On Wed, Jan 22, 2014 at 8:19 PM, Yu Dongmin <miniway@gmail.com> wrote:
> > Hello,
> >
> > I'm very glad to hear that Tajo is getting stable and production ready.
> >
> > Here's another news. Some of us might have have seen some Hyunsik's
> presentations commenting about an experimental project, c++ tajo worker.
> >
> >
> > Recently, I've been working on the c++ worker, even though it is a
> little bit far from behind schedule than what I expected,
> > it is able to communicate with Tajo Master and Query Master successfully.
> >
> > This is never a replacement of Java worker, but an exchangeable worker.
> We could use Java workers as is, C++ workers only, or mixed workers.
> >
> > It is designed as a vectorized execution engine hoping to process
> certain types of data structures very efficiently.
> >
> > These are the supported features right now.
> >
> > - Reading and parsing csv files in the hadoop data node.
> > - Filtering rows within LLVM code generated evaluation
> > - Simple scalar functions
> > - Simple group by aggregation functions
> >
> > Now I'm working on 'order by' clause and doing profiling to get expected
> performance.
> >
> > Working on these stuffs, I hope community to allow creating a new git
> branch, say native_worker, cplus_worker or a nicer name, to make this
> project together.
> >
> >
> > There's still a long way to go on this project, but it could be improved
> with help of the Tajo community.
> >
> >
> > Thanks
> > Min
>



-- 
Thanks,
Jaehwa Jung
Bigdata Platform Team
Gruter

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message