impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: impala query spend too much time on codegen
Date Fri, 25 Mar 2016 05:03:22 GMT
Webkit recently switched to a new JIT:
https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/

But, I think LLVM is still a better bet for the longer-running queries,
especially as Impala evolves to take more advantage of vector instructions,
etc. A more advanced optimizer is necessary for these kinds of things (but
probably less so for Javascript on a browser)

-Todd

On Thu, Mar 24, 2016 at 8:37 PM, Tim Armstrong <tarmstrong@cloudera.com>
wrote:

> From what I can tell Safari uses a four-tier JIT where at one end
> interpreted code has no start-up time but slow runtime and at the other end
> the full LLVM pipeline generates high-quality code but takes a long time to
> start-up. In-between there are two intermediate stages where they use some
> parts of LLVM. Currently in Impala we only really have two tiers
> (interpreted and high-quality LLVM codegen), so we sometimes spend too much
> time optimising code for short-running queries.
>
> This was the page I found:
> https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ but maybe
> you saw something more recent.
>
> On Thu, Mar 24, 2016 at 8:31 PM, Tim Armstrong <tarmstrong@cloudera.com>
> wrote:
>
>> IMO even though LLVM is more heavyweight than a custom JIT, there is
>> plenty of room to improve codegen time for shorter-running Impala queries
>> given some engineering effort.
>>
>> On Thu, Mar 24, 2016 at 8:18 PM, 吴朱华 <ikewu83@gmail.com> wrote:
>>
>>> I can see that, apple safari team have just switch llvm to something
>>> custom made on their new JS jit engine.
>>>
>>> 2016-03-25 10:35 GMT+08:00 Tim Armstrong <tarmstrong@cloudera.com>:
>>>
>>>> Oops, I meant Impala 2.6
>>>>
>>>> On Thu, Mar 24, 2016 at 8:26 AM, Tim Armstrong <tarmstrong@cloudera.com
>>>> > wrote:
>>>>
>>>>> We'll be upgrading to LLVM 3.8 in Impala 2.7. I've been working on the
>>>>> upgrade. It doesn't help with codegen time (actually it hurts) but there
>>>>> are other benefits.
>>>>>
>>>>> Thanks,
>>>>> Tim
>>>>>
>>>>> On Wed, Mar 23, 2016 at 10:51 PM, zhen <yyzzjj0515@gmail.com> wrote:
>>>>>
>>>>>> just now  impala use the  llvm  version is 3.3  that  api not
>>>>>> compatible with  3.5 + .  cur  could not test
>>>>>>
>>>>>>
>>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Impala User" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to impala-user+unsubscribe@cloudera.org.
>>>>>>
>>>>>
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Impala User" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to impala-user+unsubscribe@cloudera.org.
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Impala User" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to impala-user+unsubscribe@cloudera.org.
>>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Impala User" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to impala-user+unsubscribe@cloudera.org.
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message