hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <>
Subject [jira] [Created] (HIVE-10092) LLAP: improve how buffers are locked for split
Date Wed, 25 Mar 2015 20:41:53 GMT
Sergey Shelukhin created HIVE-10092:

             Summary: LLAP: improve how buffers are locked for split
                 Key: HIVE-10092
             Project: Hive
          Issue Type: Sub-task
            Reporter: Sergey Shelukhin

Right now, for simplicity, entire split of decompressed buffers is locked in cache, in case
some buffers are shared between RGs, to avoid dealing with situations where we uncompress
some data, pass it on to processor for RG N, then processor processes and unlocks it, and
before we can pass it on for RG N+1 it's evicted. 
However, if split is too big, and cache is small, or many splits are processed at the same
time, this can result in a deadlock as entire cache is locked. We need to improve locking
to be more granular and probably also try to avoid deadlocks in general (bypass cache?)

This message was sent by Atlassian JIRA

View raw message