hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ilya Maykov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-10358) libhadoop doesn't compile on Mac OS X
Date Fri, 21 Feb 2014 21:51:28 GMT
Ilya Maykov created HADOOP-10358:

             Summary: libhadoop doesn't compile on Mac OS X
                 Key: HADOOP-10358
                 URL: https://issues.apache.org/jira/browse/HADOOP-10358
             Project: Hadoop Common
          Issue Type: Improvement
          Components: native
         Environment: Mac OS X 10.8.5
Oracle JDK 1.7.0_51
            Reporter: Ilya Maykov
            Priority: Minor

The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib on mac) fails
to compile on Mac OS X. The problem is in  hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
at lines 76-78:

[exec] /Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
error: invalid operands to binary expression ('void' and 'int')
[exec]  if(setnetgrent(cgroup) == 1) {
[exec]      ~~~~~~~~~~~~~~~~~~~ ^  ~

There are two problems in the code:
1) The #ifndef guard only checks for __FreeBSD__ but should check for either one of __FreeBSD__
or __APPLE__. This is because Mac OS X inherits its syscalls from FreeBSD rather than Linux,
and thus the setnetgrent() syscall returns void.
2) setnetgrentCalledFlag = 1 is called outside the #ifndef guard, but the syscall is only
called inside the guard. This means that on FreeBSD, endnetgrent() can be called in the cleanup
code without a corresponding setnetgrent() invocation.

I have a patch that fixes both issues (will attach in a bit). With this patch, I'm able to
compile libhadoop.dylib on Mac OS X, which in turn lets me install native snappy, lzo, etc
compressor libraries on my client. That lets me run commands like 'hadoop fs -text somefile.lzo'
from the macbook rather than having to ssh to a linux box, etc.

Note that this patch only fixes the native build of hadoop-common-project. Some other components
of hadoop still fail to build their native components, but libhadoop.dylib is enough for the

This message was sent by Atlassian JIRA

View raw message