hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Piotr Findeisen (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-24082) Expose information whether AcidUtils.ParsedDelta contains statementId
Date Mon, 07 Sep 2020 11:30:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-24082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191652#comment-17191652
] 

Piotr Findeisen commented on HIVE-24082:
----------------------------------------

 

We call these methods today:
{code:java}
AcidUtils.isTransactionalTable
AcidUtils.isFullAcidTable
AcidUtils.getAcidState 
AcidUtils.OrcAcidVersion.getAcidVersionFromMetaFile
AcidUtils.deleteDeltaSubdir
AcidUtils.createBucketFile{code}
I expect further usage as we advance Presto's support for ORC ACID / Transactional tables.

I am aware AcidUtils is not public interface, so I am aware a breakage may occur when we upgrade.
We chose to do this  since the ACID handling logic has quite a few nuances that are easy
to overlook. Copying the logic over to Presto codebase would probably be safer from code
compilation perspective, but could cause trouble as ORC ACID evolves.

 
{quote}we were planning to change things around that.
{quote}
 

sure! I am always curious so please CC me whenever you feel like I could be interested.

 

> Expose information whether AcidUtils.ParsedDelta contains statementId
> ---------------------------------------------------------------------
>
>                 Key: HIVE-24082
>                 URL: https://issues.apache.org/jira/browse/HIVE-24082
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Piotr Findeisen
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> In [Presto|https://prestosql.io] we support reading ORC ACID tables by leveraging AcidUtils
rather than duplicate the file name parsing logic in our code.
> To do this fully correctly, we need information whether {{org.apache.hadoop.hive.ql.io.AcidUtils.ParsedDelta}}
contains {{statementId}} information or not. 
> Currently, a getter of that property does not allow us to access this information.
> [https://github.com/apache/hive/blob/468907eab36f78df3e14a24005153c9a23d62555/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L804-L806]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message