tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyunsik Choi <hyun...@apache.org>
Subject Re: Tajo C++ worker
Date Wed, 22 Jan 2014 17:01:25 GMT
Hi Yu,

Thank you for your contribution. I have been looking forward to seeing
this work. Could you submit a patch to proper issues or create a jira?
We will create a new branch for that after review.

Hi folks,

Some of us know Yu in offline. After the discussion early this year,
he has mostly worked on Tajo C++ worker. I also contributed a small
part about JIT code generation via LLVM, and participated in the
design. Another contributor, Hyoung Jun, also contributed some module.
Its main objective is to build a high performance query engine to
maximize hardware utilization by using SIMD and cache conscious
algorithm. It is under heavy development. As he mentioned, it would be
a long journey for the future.

- hyunsik

On Wed, Jan 22, 2014 at 8:19 PM, Yu Dongmin <miniway@gmail.com> wrote:
> Hello,
>
> I'm very glad to hear that Tajo is getting stable and production ready.
>
> Here's another news. Some of us might have have seen some Hyunsik's presentations commenting
about an experimental project, c++ tajo worker.
>
>
> Recently, I've been working on the c++ worker, even though it is a little bit far from
behind schedule than what I expected,
> it is able to communicate with Tajo Master and Query Master successfully.
>
> This is never a replacement of Java worker, but an exchangeable worker. We could use
Java workers as is, C++ workers only, or mixed workers.
>
> It is designed as a vectorized execution engine hoping to process certain types of data
structures very efficiently.
>
> These are the supported features right now.
>
> - Reading and parsing csv files in the hadoop data node.
> - Filtering rows within LLVM code generated evaluation
> - Simple scalar functions
> - Simple group by aggregation functions
>
> Now I'm working on 'order by' clause and doing profiling to get expected performance.
>
> Working on these stuffs, I hope community to allow creating a new git branch, say native_worker,
cplus_worker or a nicer name, to make this project together.
>
>
> There's still a long way to go on this project, but it could be improved with help of
the Tajo community.
>
>
> Thanks
> Min

Mime
View raw message