orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From omalley <...@git.apache.org>
Subject [GitHub] orc pull request #259: ORC-248. PhysicalFsWriter sometimes passes negative p...
Date Mon, 30 Apr 2018 22:20:00 GMT
GitHub user omalley opened a pull request:

    https://github.com/apache/orc/pull/259

    ORC-248. PhysicalFsWriter sometimes passes negative padding down to shims

    The PhysicalFsWriter sometimes pass down negative sizes for padding, which leads to exceptions.
    
    My fix:
    * Add a new option to enable writing variable length HDFS blocks "orc.write.shortened.blocks",
which is off by default.
    * Modified the shim to shortenBlock so that the PhysicalFsWriter can track the offsets.
    * shortenBlock isn't gated by the padding tolerance, although it is called when the new
stripe would cross a block boundary.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/omalley/orc orc-248

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/orc/pull/259.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #259
    
----
commit 7a8107b9f6d954ce3d4bddd5245642f58b6bb729
Author: Owen O'Malley <omalley@...>
Date:   2018-04-27T23:58:48Z

    ORC-354. Restore the benchmark module.
    
    This reverts commit b86d70aa73289b86e066cc019ea11e0d83c1e40d.

commit 1ee460dd750dace3bee34131ebd2e0eb260bedfc
Author: Owen O'Malley <omalley@...>
Date:   2018-04-30T21:10:07Z

    ORC-248. PhysicalFsWriter will sometimes pass a negative amount of
    requested padding to the shims.

----


---

Mime
View raw message