hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-573) Porting libhdfs to Windows
Date Fri, 01 Aug 2014 23:23:38 GMT

     [ https://issues.apache.org/jira/browse/HDFS-573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth updated HDFS-573:

    Attachment: HDFS-573.2.patch

Thanks for the heads-up, Colin.  Here is patch v2.

* {{TYPE_CHECKED_PRINTF_FORMAT}} is in platform.h.
* hash table locking is reworked.
* minor changes for const-ness and putting the * with the variable name instead of the data
type for pointers.

bq. Can we avoid this typecast by making {{port}} be a variable of type {{tPort}}?

The challenge here is that {{nmdGetNameNodePort}} returns {{int}}, but subsequent code wants
a {{tPort}} (a {{uint16_t}}), so a cast is unavoidable.  I don't want to change the return
type of {{nmdGetNameNodePort}} right now, because fuse-dfs calls it too, and I don't want
to expand the scope of this patch into fuse-dfs code.  I did however change the type of {{port}}
and cast the return value immediately, which I think better documents intent.

bq. I wish all libhdfs patches could be this good.

Thanks for the constructive feedback!

> Porting libhdfs to Windows
> --------------------------
>                 Key: HDFS-573
>                 URL: https://issues.apache.org/jira/browse/HDFS-573
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: libhdfs
>         Environment: Windows, Visual Studio 2008
>            Reporter: Ziliang Guo
>            Assignee: Chris Nauroth
>         Attachments: HDFS-573.1.patch, HDFS-573.2.patch
>   Original Estimate: 336h
>  Remaining Estimate: 336h
> The current C code in libhdfs is written using C99 conventions and also uses a few POSIX
specific functions such as hcreate, hsearch, and pthread mutex locks.  To compile it using
Visual Studio would require a conversion of the code in hdfsJniHelper.c and hdfs.c to C89
and replacement/reimplementation of the POSIX functions.  The code also uses the stdint.h
header, which is not part of the original C89, but there exists what appears to be a BSD licensed
reimplementation written to be compatible with MSVC floating around.  I have already done
the other necessary conversions, as well as created a simplistic hash bucket for use with
hcreate and hsearch and successfully built a DLL of libhdfs.  Further testing is needed to
see if it is usable by other programs to actually access hdfs, which will likely happen in
the next few weeks as the Condor Project continues with its file transfer work.
> In the process, I've removed a few what I believe are extraneous consts and also fixed
an incorrect array initialization where someone was attempting to initialize with something
like this: JavaVMOption options[noArgs]; where noArgs was being incremented in the code above.
 This was in the hdfsJniHelper.c file, in the getJNIEnv function.

This message was sent by Atlassian JIRA

View raw message