hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11030) Enhance storage layer to create one delta file per write
Date Thu, 09 Jul 2015 17:16:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620870#comment-14620870
] 

Alan Gates commented on HIVE-11030:
-----------------------------------

bq. OrcRecordUpdate, end of the constructor (line 265 in your patch)...
bq. This does stat the file but if this check were to fail unnoticed it leads to data loss
which seems really bad. I could wrap this in LOG.isInfoEnabled() for the most perf sensitive
cases...
I don't have a feel for how frequent a case this would be.  What would cause this to happen?
 It just feels like this belongs in test mode but not in production.

Other than that, +1

> Enhance storage layer to create one delta file per write
> --------------------------------------------------------
>
>                 Key: HIVE-11030
>                 URL: https://issues.apache.org/jira/browse/HIVE-11030
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Transactions
>    Affects Versions: 1.2.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-11030.2.patch, HIVE-11030.3.patch, HIVE-11030.4.patch, HIVE-11030.5.patch,
HIVE-11030.6.patch, HIVE-11030.7.patch
>
>
> Currently each txn using ACID insert/update/delete will generate a delta directory like
delta_0000100_0000101.  In order to support multi-statement transactions we must generate
one delta per operation within the transaction so the deltas would be named like delta_0000100_0000101_0001,
etc.
> Support for MERGE (HIVE-10924) would need the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message