Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 30 Aug 2017 01:00:48 +0000 (UTC)
From: "Sergey Shelukhin (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13098433.1504054557000.154059.1504054848190@Atlassian.JIRA>
In-Reply-To: <JIRA.13098433.1504054557000@Atlassian.JIRA>
References: <JIRA.13098433.1504054557000@Atlassian.JIRA> <JIRA.13098433.1504054557622@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HIVE-17411) LLAP IO may incorrectly release a
 refcount in some rare cases
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 30 Aug 2017 01:00:56 -0000


     [ https://issues.apache.org/jira/browse/HIVE-17411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sergey Shelukhin updated HIVE-17411:
------------------------------------
    Attachment: HIVE-17411.patch

[~prasanth_j] can you take a look? small bugfix

> LLAP IO may incorrectly release a refcount in some rare cases
> -------------------------------------------------------------
>
>                 Key: HIVE-17411
>                 URL: https://issues.apache.org/jira/browse/HIVE-17411
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-17411.patch
>
>
> In a large stream whose buffers are not reused, separated into many buffers (e.g. due to a small ORC compression buffer size), it may happen that some, but not all, buffers that are read together as a unit are evicted from cache.
> If CacheBuffer follows BufferChunk in the buffer list, the latter will be converted to ProcCacheChunk;  it is possible for early refcount release logic from the former to release the refcount (for a dictionary it would always be released cause by definition there's no reuse), and then backtrack to the latter, and try to decref an uninitialized MemoryBuffer in ProcCacheChunk because ProcCacheChunk looks like a CacheChunk. PCC initial refcounts are released separately after the data is uncompressed.
> I'm assuming it would almost never happen with non-stripe-level streams because one would need both very large RG to span 2+ CBs, no overlap with next/previous RGs in 2+ buffers for the early release to kick in, and an unfortunate eviction order. However it's possible with large-ish dictionaries.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)