orc-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <owen.omal...@gmail.com>
Subject Re: Creating a Reader from a Java InputStream
Date Thu, 20 Feb 2020 20:02:09 GMT
The biggest problem is that the java ORC reader needs seeks or positioned reads.  Other than
that, it would currently require making a wrapper that could feed it as a Hadoop FileSystem
API, but that would be easy. In terms of where we’d like to go, it is toward an async API
that has async read of a list of file ranges that gives back CompletableFuture<ByteBuffer>.
That would let us overlap the decompression and io. 

.. Owen

> On Feb 20, 2020, at 11:01, Matamoros, Ronald <ronald.matamoros@accenture.com> wrote:
> Hi All,
> Orc version 1.6.2 (Core Java)
> In the Java context is there a way (direct or indirect) to create a Reader from a Java
> Something like:
>    OrcFile.java -> createReader(InputStream is, ReaderOptions options) ...
>    ReaderImpl.java -> public ReaderImpl(InputStream is, OrcFile.ReaderOptions options)
> I noticed that the C++ Core Java does have this signature version (from the documentation
> Checked the source code and internet, but did not find anything similar for Java.
> In my case, I already have the Orc file on a Java InputStream in my environment; do not
need to load it from the file system (no Path).
> Would like to avoid writing to a temporary file to consume the OrcFile.
> Was wondering if the constructor is omitted due to a performance hit or important design
> Maybe I just missed some utility class or combination of calls that would achieve the
> If not, was planning try implement my own constructor, but wanted to check there is a
better/easier way.
> Would you be so kind to point me to the correct direction.
> Thanks in advance.
> Regards
> Ronald Matamoros
> ________________________________
> This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our systems for the purposes
of information security and assessment of internal compliance with Accenture policy. Your
privacy is important to us. Accenture uses your personal data only in compliance with data
protection laws. For further information on how Accenture processes your personal data, please
see our privacy statement at https://www.accenture.com/us-en/privacy-policy.
> ______________________________________________________________________________________
> www.accenture.com

View raw message