orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <owen.omal...@gmail.com>
Subject Re: Sub-classing implementation classes(e.g., ReaderImpl)
Date Wed, 29 May 2019 15:57:34 GMT
Inheritance, especially in C++ is very hard to maintain compatibility across versions.

Based on my experience on the Java side with LLAP, I’d suggest adding an optional cache
manager than can be supplied to the reader. As Deepak says, there is already an interface
for passing in the file tail. I’d suspect that we should have an API to supply a cache (memory
or local file based) for:

* file footer & tail
* stripe footers
* stripe statistics 
* row indexes
* row data

.. Owen

> On May 29, 2019, at 5:12 AM, naveen mahadevuni <nmahadevuni@gmail.com> wrote:
> Hi,
> We are using ORC C++ API. We would like to be able to sub-class the ORC
> implementation classes such as ReaderImpl, StatisticsImpl,
> StripeInformationImpl... The reason being, when ORC files are stored on a
> cloud store such as S3, we would like to have the metadata/statistics
> stored on a file on local disk and create wrapper implementation classes
> which read metadata/statistics from locally stored file and delegates only
> the data stream reads to the cloud store.
> Will there be any problems if these implementation classes are published?
> Thanks,
> Naveen

View raw message