zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ZOOKEEPER-2841) ZooKeeper public include files leak porting changes
Date Mon, 17 Jul 2017 19:54:00 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090446#comment-16090446
] 

ASF GitHub Bot commented on ZOOKEEPER-2841:
-------------------------------------------

Github user andschwa commented on the issue:

    https://github.com/apache/zookeeper/pull/306
  
    I managed to get the Mesos unit tests for ZooKeeper ported to our CMake system, which
much more thoroughly exercises the C client. I've integrated this patch and the CMake system
with the Mesos build on Linux, and all our tests passed:
    
    ```
    [----------] 7 tests from ZooKeeperTest
    [ RUN      ] ZooKeeperTest.Auth
    [       OK ] ZooKeeperTest.Auth (6923 ms)
    [ RUN      ] ZooKeeperTest.SessionTimeoutNegotiation
    [       OK ] ZooKeeperTest.SessionTimeoutNegotiation (46 ms)
    [ RUN      ] ZooKeeperTest.Create
    [       OK ] ZooKeeperTest.Create (6770 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetector
    [       OK ] ZooKeeperTest.LeaderDetector (57 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorTimeoutHandling
    2017-07-14 15:23:18,510:29970(0x7f01e4d0a700):ZOO_ERROR@handle_socket_error_msg@2409:
Socket [127.0.0.1:43434] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-14 15:23:18,510:29970(0x7f01e4d0a700):ZOO_ERROR@handle_socket_error_msg@2384:
Socket [127.0.0.1:43434] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    2017-07-14 15:23:18,514:29970(0x7f01dffff700):ZOO_ERROR@handle_socket_error_msg@2384:
Socket [127.0.0.1:43434] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    2017-07-14 15:23:18,514:29970(0x7f01dffff700):ZOO_ERROR@handle_socket_error_msg@2384:
Socket [127.0.0.1:43434] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    [       OK ] ZooKeeperTest.LeaderDetectorTimeoutHandling (50 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorCancellationHandling
    [       OK ] ZooKeeperTest.LeaderDetectorCancellationHandling (54 ms)
    [ RUN      ] ZooKeeperTest.LeaderContender
    2017-07-14 15:23:18,630:29970(0x7f01dffff700):ZOO_ERROR@handle_socket_error_msg@2409:
Socket [127.0.0.1:38085] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-14 15:23:18,632:29970(0x7f01dffff700):ZOO_ERROR@handle_socket_error_msg@2428:
Socket [127.0.0.1:38085] zk retcode=-112, errno=116(Stale file handle): sessionId=0x10000305fd80000
has expired.
    2017-07-14 15:23:18,655:29970(0x7f01e4d0a700):ZOO_ERROR@handle_socket_error_msg@2409:
Socket [127.0.0.1:38085] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-14 15:23:18,657:29970(0x7f01e4d0a700):ZOO_ERROR@handle_socket_error_msg@2428:
Socket [127.0.0.1:38085] zk retcode=-112, errno=116(Stale file handle): sessionId=0x10000305fd80001
has expired.
    2017-07-14 15:23:18,676:29970(0x7f01dffff700):ZOO_ERROR@handle_socket_error_msg@2409:
Socket [127.0.0.1:38085] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-14 15:23:18,677:29970(0x7f01dffff700):ZOO_ERROR@handle_socket_error_msg@2384:
Socket [127.0.0.1:38085] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    2017-07-14 15:23:18,688:29970(0x7f01e4d0a700):ZOO_ERROR@handle_socket_error_msg@2384:
Socket [127.0.0.1:38085] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    2017-07-14 15:23:18,688:29970(0x7f01e4d0a700):ZOO_ERROR@handle_socket_error_msg@2384:
Socket [127.0.0.1:38085] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    [       OK ] ZooKeeperTest.LeaderContender (304 ms)
    [----------] 7 tests from ZooKeeperTest (14204 ms total)
    
    [----------] Global test environment tear-down
    [==========] 7 tests from 1 test case ran. (14376 ms total)
    [  PASSED  ] 7 tests.
    ```
    
    And then I brought these changes over to Windows too, and while it's currently building
with ~some~ a lot of irrelevant-to-this-patch hacks, they pass:
    
    ```
    [----------] 7 tests from ZooKeeperTest
    [ RUN      ] ZooKeeperTest.Auth
    [       OK ] ZooKeeperTest.Auth (7101 ms)
    [ RUN      ] ZooKeeperTest.SessionTimeoutNegotiation
    [       OK ] ZooKeeperTest.SessionTimeoutNegotiation (737 ms)
    [ RUN      ] ZooKeeperTest.Create
    [       OK ] ZooKeeperTest.Create (6752 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetector
    [       OK ] ZooKeeperTest.LeaderDetector (107 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorTimeoutHandling
    2017-07-17 12:52:08,546:15936(0x3334):ZOO_ERROR@handle_socket_error_msg@2409: Socket [127.0.0.1:54312]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    [       OK ] ZooKeeperTest.LeaderDetectorTimeoutHandling (68 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorCancellationHandling
    [       OK ] ZooKeeperTest.LeaderDetectorCancellationHandling (72 ms)
    [ RUN      ] ZooKeeperTest.LeaderContender
    2017-07-17 12:52:08,709:15936(0x47c8):ZOO_ERROR@handle_socket_error_msg@2409: Socket [127.0.0.1:54336]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-17 12:52:08,714:15936(0x47c8):ZOO_ERROR@handle_socket_error_msg@2428: Socket [127.0.0.1:54336]
zk retcode=-112, errno=19(No such device): sessionId=0x1001b9dddba0000 has expired.
    2017-07-17 12:52:08,749:15936(0x39b8):ZOO_ERROR@handle_socket_error_msg@2409: Socket [127.0.0.1:54336]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-17 12:52:08,754:15936(0x39b8):ZOO_ERROR@handle_socket_error_msg@2428: Socket [127.0.0.1:54336]
zk retcode=-112, errno=19(No such device): sessionId=0x1001b9dddba0001 has expired.
    2017-07-17 12:52:08,788:15936(0xb94):ZOO_ERROR@handle_socket_error_msg@2409: Socket [127.0.0.1:54336]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    [       OK ] ZooKeeperTest.LeaderContender (701 ms)
    [----------] 7 tests from ZooKeeperTest (15548 ms total)
    
    [----------] Global test environment tear-down
    [==========] 7 tests from 1 test case ran. (16724 ms total)
    [  PASSED  ] 7 tests.
    ```


> ZooKeeper public include files leak porting changes
> ---------------------------------------------------
>
>                 Key: ZOOKEEPER-2841
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2841
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: c client
>         Environment: Windows 10 with Visual Studio 2017
>            Reporter: Andrew Schwartzmeyer
>            Assignee: Andrew Schwartzmeyer
>              Labels: windows
>
> The fundamental problem is that the port of the C client to Windows is now close to six
years old, with very few updates. This port leaks a lot of changes that should be internal
to ZooKeeper, and many of those changes are simply no longer relevant. The correct thing to
do is attempt to refactor the Windows port for new versions of ZooKeeper, removing dead/unneeded
porting code, and moving dangerous porting code to C files instead of public headers.
> Two primary examples of this problem are [ZOOKEEPER-2491|https://issues.apache.org/jira/browse/ZOOKEEPER-2491]
and [MESOS-7541|https://issues.apache.org/jira/browse/MESOS-7541].
> The first issue stems from this ancient porting code:
> {noformat}
> #define snprintf _snprintf
> {noformat}
>  in [winconfig.h|https://github.com/apache/zookeeper/blob/ddf0364903bf7ac7cd25b2e1927f0d9d3c7203c4/src/c/include/winconfig.h#L179].
Newer versions of Windows C libraries define {{snprintf}} as a function, and so it cannot
be redefined.
> The second issue comes from this undocumented change:
> {noformat}
> #undef AF_INET6
> {noformat}
> again in [winconfig.h|https://github.com/apache/zookeeper/blob/ddf0364903bf7ac7cd25b2e1927f0d9d3c7203c4/src/c/include/winconfig.h#L169]
which breaks any library that uses IPv6 and {{winsock2.h}}.
> Furthermore, the inclusion of the following defines and headers causes terrible problems
for consuming libraries, as they leak into ZooKeeper's public headers:
> {noformat}
> #define _CRT_SECURE_NO_WARNINGS
> #define WIN32_LEAN_AND_MEAN
> #include <Windows.h>
> #include <Winsock2.h>
> #include <winstdint.h>
> #include <process.h>
> #include <ws2tcpip.h>
> {noformat}
> Depending on the order that a project includes or compiles files, this may or may not
cause {{WIN32_LEAN_AND_MEAN}} to become unexpectedly defined, and {{windows.h}} to be unexpectedly
included. This problem is exacberated by the fact that the {{winsock2.h}} and {{windows.h}}
headers are order-dependent (if you read up on this, you'll see that defining {{WIN32_LEAN_AND_MEAN}}
was meant to work-around this).
> Going forward, porting changes should live next to where they are used, preferably in
source files, not header files, so they remain contained.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message