Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Mon, 12 Dec 2016 19:35:58 +0000 (UTC)
From: "Owen O'Malley (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.12978446.1465853948000.501600.1481571358618@Atlassian.JIRA>
In-Reply-To: <JIRA.12978446.1465853948000@Atlassian.JIRA>
References: <JIRA.12978446.1465853948000@Atlassian.JIRA> <JIRA.12978446.1465853948167@arcas>
Subject: [jira] [Issue Comment Deleted] (HIVE-14007) Replace ORC module with
 ORC release
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Mon, 12 Dec 2016 19:36:00 -0000


     [ https://issues.apache.org/jira/browse/HIVE-14007?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:all-tabpanel ]

Owen O'Malley updated HIVE-14007:
---------------------------------
    Comment: was deleted

(was: .bq
The other thing I think we need community wide clarity on before you rip ou=
t orc is how we=E2=80=99re going to keep developing hive afterwards. Right =
now there=E2=80=99s a cyclic dependency. Hive -> ORC -> Hive - because of a=
 shared storage api.

There is agreement within Hive to release the storage-api independently fro=
m Hive.  That would break the cycle and allow a non-cyclic release process.=
 I'll file a Hive jira to do that work. Avoiding have two copies of code ma=
kes the whole ecosystem stronger by making sure that fixes get applied ever=
ywhere. I'd suggest leaving storage-api in the Hive source tree rather than=
 making its own git repository.=20

.bq
There are features that touch all three. And it turns out these are more fr=
equent than expected.=20

They come in waves. In the last three months, there have been 2 changes to =
storage-api.
Most of the patches are in either storage-api or ORC.  For example, HIVE-14=
453 only touches ORC.

.bq
How do you propose to handle development and release of these features give=
n the cyclic dependency? How do you work out feature branches/ snapshots?

For changes that touch one or the other, you'd commit the relevant change a=
nd release either storage-api or ORC and have a jira that updates the versi=
on in Hive. In the worst case, where the change spreads among the three art=
ifacts, you would:

* commit to storage-api & ORC
* release them
* upgrade the pom in Hive

.bq
If a successful feature commit requires sequential hive and orc releases, t=
hen that means minimum several months before commit and that's not great. H=
ow will this be done?

No, ORC releases typically take 3 days. Storage API is much simpler and sho=
uld also take 3 days. By being much smaller and more focused, they are much=
 more nimble. Furthermore, the two votes could completely overlap, so the t=
otal time to get the change into Hive would be roughly 3 days.=20

.bq
Looking over the PMC and committer lists in ORC it looks like many people w=
orking on ACID, vectorization or llap will lose the ability to do what they=
 are doing today with this change.

When we set up the ORC project, we were pretty inclusive in the committer l=
ist and we continue to add new committers and PMC members. I'll take at the=
 contributors to the Hive ORC module to look for new committers.)

> Replace ORC module with ORC release
> -----------------------------------
>
>                 Key: HIVE-14007
>                 URL: https://issues.apache.org/jira/browse/HIVE-14007
>             Project: Hive
>          Issue Type: Bug
>          Components: ORC
>    Affects Versions: 2.2.0
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 2.2.0
>
>         Attachments: HIVE-14007.patch, HIVE-14007.patch, HIVE-14007.patch=
, HIVE-14007.patch
>
>
> This completes moving the core ORC reader & writer to the ORC project.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)