Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
Subject: Re: Hive orc use case
From: Alan Gates <alanfgates@gmail.com>
In-Reply-To: <CAOr_EshUqYOKgLBTVEf+7qWYN+PC=PGkHvVRDF1sL0fPPuQUyg@mail.gmail.com>
Date: Mon, 26 Sep 2016 09:41:38 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <31627FE1-03C2-413D-A51F-E00FE105AA3F@gmail.com>
References: <CAOr_EshUqYOKgLBTVEf+7qWYN+PC=PGkHvVRDF1sL0fPPuQUyg@mail.gmail.com>
To: user@hive.apache.org
archived-at: Mon, 26 Sep 2016 16:41:47 -0000

ORC does not store data row by row.  It decomposes the rows into =
columns, and then stores pointer to those columns, as well as a number =
of indices and statistics, in a footer of the file.  Due to the footer, =
in the simple case you cannot read the file before you close it or =
append to it.  We did address both of these issues to support Hive =
streaming, but it=E2=80=99s a low level interface.  If you want to take =
a look at how Hive streaming handles this you could use it as your =
guide.  The starting point for that is HiveEndPoint in =
org.apache.hive.hcatalog.streaming.

Alan.

> On Sep 26, 2016, at 01:18, Amey Barve <ameybarve15@gmail.com> wrote:
>=20
> Hi All,
>=20
> I have an use case where I need to append either 1 or many rows to =
orcFile as well as read 1 or many rows from it.
>=20
> I observed that I cannot read rows from OrcFile unless I close the =
OrcFile's writer, is this correct?
>=20
> Why doesn't write actually flush the rows to the orcFile, is there any =
alternative where I write the rows as well as read them without closing =
the orcFile's writer ?
>=20
> Thanks and Regards,
> Amey=20