zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ZOOKEEPER-2841) ZooKeeper public include files leak porting changes
Date Wed, 19 Jul 2017 19:27:00 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093648#comment-16093648
] 

ASF GitHub Bot commented on ZOOKEEPER-2841:
-------------------------------------------

Github user andschwa commented on the issue:

    https://github.com/apache/zookeeper/pull/313
  
    @hanm While the Windows version of Mesos had been using ZooKeeper 3.5.2, the Linux version
was using ZooKeeper 3.4.8. I took this patch and cherry-picked back to 3.4.8, and changed
the Windows version of Mesos to use 3.4.8 with this patch. However, I also needed 97e598b6c
(ZOOKEEPER-1643) in order to build, which is not in `branch-3.4`. Would you be willing to
backport ZOOKEEPER-1643 too? If so, then we can switch to the next 3.4 release without any
patches at all.
    
    As for `Zookeeper_simpleSystem::testAsyncWatcherAutoReset`, I observed it failing consistently
even on `branch-3.4` built with Autotools (i.e. none of my changes at all). I'm inferring
that it's a machine issue, and since it didn't fail on Jenkins, I don't think we need to worry
about it.
    
    Furthermore, with this patch _and_ ZOOKEEPER-1643 (currently in this branch), I was able
to integration test with Mesos on Windows and Linux successfully.
    
    CentOS 7:
    ```
    [==========] Running 7 tests from 1 test case.
    [----------] Global test environment set-up.
    [----------] 7 tests from ZooKeeperTest
    [ RUN      ] ZooKeeperTest.Auth
    [       OK ] ZooKeeperTest.Auth (6898 ms)
    [ RUN      ] ZooKeeperTest.SessionTimeoutNegotiation
    [       OK ] ZooKeeperTest.SessionTimeoutNegotiation (46 ms)
    [ RUN      ] ZooKeeperTest.Create
    [       OK ] ZooKeeperTest.Create (6728 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetector
    [       OK ] ZooKeeperTest.LeaderDetector (70 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorTimeoutHandling
    2017-07-19 12:13:30,626:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1781:
Socket [127.0.0.1:45292] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-19 12:13:30,642:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1757:
Socket [127.0.0.1:45292] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    [       OK ] ZooKeeperTest.LeaderDetectorTimeoutHandling (51 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorCancellationHandling
    [       OK ] ZooKeeperTest.LeaderDetectorCancellationHandling (74 ms)
    [ RUN      ] ZooKeeperTest.LeaderContender
    2017-07-19 12:13:30,759:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1781:
Socket [127.0.0.1:44849] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-19 12:13:34,096:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1799:
Socket [127.0.0.1:44849] zk retcode=-112, errno=116(Stale file handle): sessionId=0x15d5c44fa610000
has expired.
    2017-07-19 12:13:34,119:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1781:
Socket [127.0.0.1:44849] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-19 12:13:34,121:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1799:
Socket [127.0.0.1:44849] zk retcode=-112, errno=116(Stale file handle): sessionId=0x15d5c44fa610001
has expired.
    2017-07-19 12:13:34,142:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1781:
Socket [127.0.0.1:44849] zk retcode=-4, errno=112(Host is down): failed while receiving a
server response
    2017-07-19 12:13:34,153:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1757:
Socket [127.0.0.1:44849] zk retcode=-4, errno=111(Connection refused): server refused to accept
the client
    [       OK ] ZooKeeperTest.LeaderContender (6787 ms)
    [----------] 7 tests from ZooKeeperTest (20654 ms total)
    
    [----------] Global test environment tear-down
    [==========] 7 tests from 1 test case ran. (20795 ms total)
    [  PASSED  ] 7 tests.
    ```
    
    Windows 10:
    ```
    [==========] Running 7 tests from 1 test case.
    [----------] Global test environment set-up.
    [----------] 7 tests from ZooKeeperTest
    [ RUN      ] ZooKeeperTest.Auth
    [       OK ] ZooKeeperTest.Auth (7641 ms)
    [ RUN      ] ZooKeeperTest.SessionTimeoutNegotiation
    [       OK ] ZooKeeperTest.SessionTimeoutNegotiation (71 ms)
    [ RUN      ] ZooKeeperTest.Create
    [       OK ] ZooKeeperTest.Create (7018 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetector
    [       OK ] ZooKeeperTest.LeaderDetector (93 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorTimeoutHandling
    2017-07-19 12:13:55,269:18232(0x53d8):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57541]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,271:18232(0x53d8):ZOO_ERROR@handle_socket_error_msg@1519: Socket [127.0.0.1:57541]
zk retcode=-4, errno=140(Unknown error): failed to send a handshake packet: Unknown error
    [       OK ] ZooKeeperTest.LeaderDetectorTimeoutHandling (76 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorCancellationHandling
    [       OK ] ZooKeeperTest.LeaderDetectorCancellationHandling (66 ms)
    [ RUN      ] ZooKeeperTest.LeaderContender
    2017-07-19 12:13:55,470:18232(0xaf4):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57557]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,473:18232(0xaf4):ZOO_ERROR@handle_socket_error_msg@1799: Socket [127.0.0.1:57557]
zk retcode=-112, errno=19(No such device): sessionId=0x15d5c455ad70000 has expired.
    2017-07-19 12:13:55,519:18232(0x4dfc):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57557]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,523:18232(0x4dfc):ZOO_ERROR@handle_socket_error_msg@1799: Socket [127.0.0.1:57557]
zk retcode=-112, errno=19(No such device): sessionId=0x15d5c455ad70001 has expired.
    2017-07-19 12:13:55,562:18232(0xc0c):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57557]
zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,564:18232(0xc0c):ZOO_ERROR@handle_socket_error_msg@1519: Socket [127.0.0.1:57557]
zk retcode=-4, errno=140(Unknown error): failed to send a handshake packet: Unknown error
    [       OK ] ZooKeeperTest.LeaderContender (714 ms)
    [----------] 7 tests from ZooKeeperTest (15727 ms total)
    
    [----------] Global test environment tear-down
    [==========] 7 tests from 1 test case ran. (16802 ms total)
    [  PASSED  ] 7 tests.
    ```


> ZooKeeper public include files leak porting changes
> ---------------------------------------------------
>
>                 Key: ZOOKEEPER-2841
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2841
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: c client
>         Environment: Windows 10 with Visual Studio 2017
>            Reporter: Andrew Schwartzmeyer
>            Assignee: Andrew Schwartzmeyer
>              Labels: windows
>
> The fundamental problem is that the port of the C client to Windows is now close to six
years old, with very few updates. This port leaks a lot of changes that should be internal
to ZooKeeper, and many of those changes are simply no longer relevant. The correct thing to
do is attempt to refactor the Windows port for new versions of ZooKeeper, removing dead/unneeded
porting code, and moving dangerous porting code to C files instead of public headers.
> Two primary examples of this problem are [ZOOKEEPER-2491|https://issues.apache.org/jira/browse/ZOOKEEPER-2491]
and [MESOS-7541|https://issues.apache.org/jira/browse/MESOS-7541].
> The first issue stems from this ancient porting code:
> {noformat}
> #define snprintf _snprintf
> {noformat}
>  in [winconfig.h|https://github.com/apache/zookeeper/blob/ddf0364903bf7ac7cd25b2e1927f0d9d3c7203c4/src/c/include/winconfig.h#L179].
Newer versions of Windows C libraries define {{snprintf}} as a function, and so it cannot
be redefined.
> The second issue comes from this undocumented change:
> {noformat}
> #undef AF_INET6
> {noformat}
> again in [winconfig.h|https://github.com/apache/zookeeper/blob/ddf0364903bf7ac7cd25b2e1927f0d9d3c7203c4/src/c/include/winconfig.h#L169]
which breaks any library that uses IPv6 and {{winsock2.h}}.
> Furthermore, the inclusion of the following defines and headers causes terrible problems
for consuming libraries, as they leak into ZooKeeper's public headers:
> {noformat}
> #define _CRT_SECURE_NO_WARNINGS
> #define WIN32_LEAN_AND_MEAN
> #include <Windows.h>
> #include <Winsock2.h>
> #include <winstdint.h>
> #include <process.h>
> #include <ws2tcpip.h>
> {noformat}
> Depending on the order that a project includes or compiles files, this may or may not
cause {{WIN32_LEAN_AND_MEAN}} to become unexpectedly defined, and {{windows.h}} to be unexpectedly
included. This problem is exacberated by the fact that the {{winsock2.h}} and {{windows.h}}
headers are order-dependent (if you read up on this, you'll see that defining {{WIN32_LEAN_AND_MEAN}}
was meant to work-around this).
> Going forward, porting changes should live next to where they are used, preferably in
source files, not header files, so they remain contained.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message