hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nishant Bangarwa (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-20349) Implement Retry Logic in HiveDruidSplit for Scan Queries
Date Fri, 17 Aug 2018 11:56:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-20349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nishant Bangarwa updated HIVE-20349:
------------------------------------
    Attachment: HIVE-20349.patch

> Implement Retry Logic in HiveDruidSplit for Scan Queries
> --------------------------------------------------------
>
>                 Key: HIVE-20349
>                 URL: https://issues.apache.org/jira/browse/HIVE-20349
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Nishant Bangarwa
>            Assignee: Nishant Bangarwa
>            Priority: Major
>         Attachments: HIVE-20349.patch
>
>
> while distributing druid scan query we check where the segments are loaded and then each
HiveDruidSplit directly queries the historical node. 
> There are few cases when we need to retry and refetch the segments. 
> # The segment is loaded on multiple historical nodes and one of them went down. in this
case when we do not get response from one segment, we query the next replica. 
> # The segment was loaded onto a realtime task and was handed over, when we query the
realtime task has already finished. In this case there is no replica. The Split needs to query
the broker again for the location of the segment and then send the query to correct historical
node. 
> This is also the root cause of failure of druidkafkamini_basic.q test, where the segment
handover happens before the scan query is executed.
> Note: This is not a problem when we are directly querying Druid brokers as the broker
handles the retry logic. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message