impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvius Rus <s...@cloudera.com>
Subject Re: Need to upgrade my one year old build for potential join perf improvement?
Date Sun, 09 Apr 2017 22:36:38 GMT
One known issue that matches this condition was fixed on master,
https://issues.apache.org/jira/browse/IMPALA-4765.  It is not yet part of a
release.  The fix doesn't speed up the loading of the largest table, but it
makes it more likely that loading of smaller tables can progress in
parallel.

On Sun, Apr 9, 2017 at 12:30 AM, 吴朱华 <ikewu83@gmail.com> wrote:

> The issue will occur when the new table has been created or impalad has
> been just restarted.
>
> 2017-04-09 15:28 GMT+08:00 吴朱华 <ikewu83@gmail.com>:
>
>> During 10-15sec loading, from top, we only can see 2-3sec more than
>> 100%-200% java usage, other than that there is nothing serious working,
>> and we will attach profile once we have it, and this question has been
>> haunted us for year.
>>
>> 2017-04-09 15:13 GMT+08:00 吴朱华 <ikewu83@gmail.com>:
>>
>>> Plus the schema we using is simple
>>>
>>> 2017-04-09 15:11 GMT+08:00 吴朱华 <ikewu83@gmail.com>:
>>>
>>>> The current version using early 2016, the  metadata loading is
>>>> 10-15sec.
>>>> We haven't try the latest version, we hope the issue has been solved^_^
>>>>
>>>> 2017-04-07 14:08 GMT+08:00 Alex Behm <alex.behm@cloudera.com>:
>>>>
>>>>> What was your Impala version before and what is it now?
>>>>>
>>>>> How fast was it before and how fast is it now?
>>>>>
>>>>> On Thu, Apr 6, 2017 at 9:16 PM, 吴朱华 <ikewu83@gmail.com> wrote:
>>>>>
>>>>>> Hi guys:
>>>>>>
>>>>>> Is there any change on metadata loading? The metadata loading used
to
>>>>>> 10sec.
>>>>>>
>>>>>> 2017-03-23 12:42 GMT+08:00 吴朱华 <ikewu83@gmail.com>:
>>>>>>
>>>>>>> thx^_^
>>>>>>>
>>>>>>> 2017-03-23 11:51 GMT+08:00 Henry Robinson <henry@apache.org>:
>>>>>>>
>>>>>>>> If your build does not have runtime filters (which are a
little
>>>>>>>> more than a year old), then the answer is yes: you should
upgrade.
>>>>>>>>
>>>>>>>> Performance in general has been improving steadily over the
last
>>>>>>>> year.
>>>>>>>>
>>>>>>>> On 22 March 2017 at 20:36, 吴朱华 <ikewu83@gmail.com>
wrote:
>>>>>>>>
>>>>>>>>> The question is there any big join perf related improvement
has
>>>>>>>>> implemented? Thx
>>>>>>>>>
>>>>>>>>> 2017-03-23 11:34 GMT+08:00 吴朱华 <ikewu83@gmail.com>:
>>>>>>>>>
>>>>>>>>> > Hi guys:
>>>>>>>>> >
>>>>>>>>> > I am currently using my one year old build (I guess
2016.3
>>>>>>>>> CDH5-Trunk),
>>>>>>>>> > now I am facing a lot of join query situation(two
big table
>>>>>>>>> joins), do I
>>>>>>>>> > need to upgrade my one year old build for potential
join perf
>>>>>>>>> improvement?
>>>>>>>>> > Thx at advance^_^
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message