hawq-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From superbobry <...@git.apache.org>
Subject [GitHub] incubator-hawq pull request #1376: [RFC] Changed libhdfs3 to be ABI compatib...
Date Tue, 12 Jun 2018 18:35:23 GMT
GitHub user superbobry opened a pull request:

    https://github.com/apache/incubator-hawq/pull/1376

    [RFC] Changed libhdfs3 to be ABI compatible with libhdfs

    I have been working on this in the context of tensorflow/tensorflow#16919, and the three
discrepancies I've identified are:
    
    * libhdfs3 does not define `hdfsPread` which nonetheless can be implemented in terms of
`hdfsTell`, `hdfsSeek` and `hdfsRead`;
    * it also uses the name `hdfsSync` for `hdfsHSync`;
    * `hdfsFlush`, which flushes the internal buffer of the InputStream in libhdfs, in libhdfs3
always flushes to HDFS, i.e. it does what `hdfsHFlush` should do. One way to address this
is to make `hdfsFlush` a noop, but I guess it's already too late to change the implementation,
wdyt? 
    
    Do you want me to add a test for `hdfsPread`? It does not do much on its own, but I can
add one nonetheless.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/superbobry/incubator-hawq libhdfs-abi-compat

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-hawq/pull/1376.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1376
    
----
commit c048cd3345235ba41bfeb961cfeb83f62b595b24
Author: Sergei Lebedev <s.lebedev@...>
Date:   2018-06-12T17:55:02Z

    Deprecated hdfsSync in favour of hdfsHSync
    
    It was probably misnamed at some point, because the implementation seems
    to be doing something very close to its JVM counterpart.

commit 9650b1644c7becf6ca4d97ed84a995d18210a189
Author: Sergei Lebedev <s.lebedev@...>
Date:   2018-06-12T18:17:54Z

    Implemented hdfsPread
    
    Closes Pivotal-Data-Attic/attic-c-hdfs-client#51. AFAIK there is no
    corresponding ticket in HAWQ JIRA.

----


---

Mime
View raw message