hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deepak Jaiswal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time
Date Thu, 19 Apr 2018 22:52:02 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444937#comment-16444937
] 

Deepak Jaiswal commented on HIVE-19077:
---------------------------------------

Hi Adam,

Thanks for taking this up again. I like your proposal. Can you please provide what kind of
tag you are suggesting?

If I may, I suggest that we can inspect the size of the patch and decide to skip it if it
is large. We can define what can be considered large. IMO, any patch larger than 1MB could
be considered one. Feel free to go ahead and make that change if that sounds good.

> Handle duplicate ptests requests standing in queue at the same time
> -------------------------------------------------------------------
>
>                 Key: HIVE-19077
>                 URL: https://issues.apache.org/jira/browse/HIVE-19077
>             Project: Hive
>          Issue Type: Improvement
>          Components: Testing Infrastructure
>            Reporter: Adam Szita
>            Assignee: Adam Szita
>            Priority: Blocker
>             Fix For: 3.1.0
>
>         Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch, HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I noticed that
sometimes huge queues can build up, that contain jira's more than once. (Yesterday I've seen
a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already long queue),
and 3 hours later I will update it, re-upload and re-queue. Now the current ptest infra seems
to be smart enough to always deal with the latest patch, so what will happen is that the same
patch will be tested 2 times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for Jira X,
then it can take a look on the current queue, and see if X is there again. If so, it can skip
for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your original
place in the queue will be gone (like as a penalty for changing it), but overall it saves
resources for the whole community.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message