Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@zookeeper.apache.org
Date: Wed, 19 Jul 2017 19:27:00 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: dev@zookeeper.apache.org
Message-ID: <JIRA.13085619.1499467710000.285226.1500492420128@Atlassian.JIRA>
In-Reply-To: <JIRA.13085619.1499467710000@Atlassian.JIRA>
References: <JIRA.13085619.1499467710000@Atlassian.JIRA> <JIRA.13085619.1499467710989@jira-lw-us.apache.org>
Subject: [jira] [Commented] (ZOOKEEPER-2841) ZooKeeper public include files
 leak porting changes
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 19 Jul 2017 19:27:06 -0000


    [ https://issues.apache.org/jira/browse/ZOOKEEPER-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093648#comment-16093648 ] 

ASF GitHub Bot commented on ZOOKEEPER-2841:
-------------------------------------------

Github user andschwa commented on the issue:

    https://github.com/apache/zookeeper/pull/313
  
    @hanm While the Windows version of Mesos had been using ZooKeeper 3.5.2, the Linux version was using ZooKeeper 3.4.8. I took this patch and cherry-picked back to 3.4.8, and changed the Windows version of Mesos to use 3.4.8 with this patch. However, I also needed 97e598b6c (ZOOKEEPER-1643) in order to build, which is not in `branch-3.4`. Would you be willing to backport ZOOKEEPER-1643 too? If so, then we can switch to the next 3.4 release without any patches at all.
    
    As for `Zookeeper_simpleSystem::testAsyncWatcherAutoReset`, I observed it failing consistently even on `branch-3.4` built with Autotools (i.e. none of my changes at all). I'm inferring that it's a machine issue, and since it didn't fail on Jenkins, I don't think we need to worry about it.
    
    Furthermore, with this patch _and_ ZOOKEEPER-1643 (currently in this branch), I was able to integration test with Mesos on Windows and Linux successfully.
    
    CentOS 7:
    ```
    [==========] Running 7 tests from 1 test case.
    [----------] Global test environment set-up.
    [----------] 7 tests from ZooKeeperTest
    [ RUN      ] ZooKeeperTest.Auth
    [       OK ] ZooKeeperTest.Auth (6898 ms)
    [ RUN      ] ZooKeeperTest.SessionTimeoutNegotiation
    [       OK ] ZooKeeperTest.SessionTimeoutNegotiation (46 ms)
    [ RUN      ] ZooKeeperTest.Create
    [       OK ] ZooKeeperTest.Create (6728 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetector
    [       OK ] ZooKeeperTest.LeaderDetector (70 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorTimeoutHandling
    2017-07-19 12:13:30,626:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:45292] zk retcode=-4, errno=112(Host is down): failed while receiving a server response
    2017-07-19 12:13:30,642:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1757: Socket [127.0.0.1:45292] zk retcode=-4, errno=111(Connection refused): server refused to accept the client
    [       OK ] ZooKeeperTest.LeaderDetectorTimeoutHandling (51 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorCancellationHandling
    [       OK ] ZooKeeperTest.LeaderDetectorCancellationHandling (74 ms)
    [ RUN      ] ZooKeeperTest.LeaderContender
    2017-07-19 12:13:30,759:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:44849] zk retcode=-4, errno=112(Host is down): failed while receiving a server response
    2017-07-19 12:13:34,096:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1799: Socket [127.0.0.1:44849] zk retcode=-112, errno=116(Stale file handle): sessionId=0x15d5c44fa610000 has expired.
    2017-07-19 12:13:34,119:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:44849] zk retcode=-4, errno=112(Host is down): failed while receiving a server response
    2017-07-19 12:13:34,121:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1799: Socket [127.0.0.1:44849] zk retcode=-112, errno=116(Stale file handle): sessionId=0x15d5c44fa610001 has expired.
    2017-07-19 12:13:34,142:200313(0x7fd96cfff700):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:44849] zk retcode=-4, errno=112(Host is down): failed while receiving a server response
    2017-07-19 12:13:34,153:200313(0x7fd99cb08700):ZOO_ERROR@handle_socket_error_msg@1757: Socket [127.0.0.1:44849] zk retcode=-4, errno=111(Connection refused): server refused to accept the client
    [       OK ] ZooKeeperTest.LeaderContender (6787 ms)
    [----------] 7 tests from ZooKeeperTest (20654 ms total)
    
    [----------] Global test environment tear-down
    [==========] 7 tests from 1 test case ran. (20795 ms total)
    [  PASSED  ] 7 tests.
    ```
    
    Windows 10:
    ```
    [==========] Running 7 tests from 1 test case.
    [----------] Global test environment set-up.
    [----------] 7 tests from ZooKeeperTest
    [ RUN      ] ZooKeeperTest.Auth
    [       OK ] ZooKeeperTest.Auth (7641 ms)
    [ RUN      ] ZooKeeperTest.SessionTimeoutNegotiation
    [       OK ] ZooKeeperTest.SessionTimeoutNegotiation (71 ms)
    [ RUN      ] ZooKeeperTest.Create
    [       OK ] ZooKeeperTest.Create (7018 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetector
    [       OK ] ZooKeeperTest.LeaderDetector (93 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorTimeoutHandling
    2017-07-19 12:13:55,269:18232(0x53d8):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57541] zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,271:18232(0x53d8):ZOO_ERROR@handle_socket_error_msg@1519: Socket [127.0.0.1:57541] zk retcode=-4, errno=140(Unknown error): failed to send a handshake packet: Unknown error
    [       OK ] ZooKeeperTest.LeaderDetectorTimeoutHandling (76 ms)
    [ RUN      ] ZooKeeperTest.LeaderDetectorCancellationHandling
    [       OK ] ZooKeeperTest.LeaderDetectorCancellationHandling (66 ms)
    [ RUN      ] ZooKeeperTest.LeaderContender
    2017-07-19 12:13:55,470:18232(0xaf4):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57557] zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,473:18232(0xaf4):ZOO_ERROR@handle_socket_error_msg@1799: Socket [127.0.0.1:57557] zk retcode=-112, errno=19(No such device): sessionId=0x15d5c455ad70000 has expired.
    2017-07-19 12:13:55,519:18232(0x4dfc):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57557] zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,523:18232(0x4dfc):ZOO_ERROR@handle_socket_error_msg@1799: Socket [127.0.0.1:57557] zk retcode=-112, errno=19(No such device): sessionId=0x15d5c455ad70001 has expired.
    2017-07-19 12:13:55,562:18232(0xc0c):ZOO_ERROR@handle_socket_error_msg@1781: Socket [127.0.0.1:57557] zk retcode=-4, errno=32(Broken pipe): failed while receiving a server response
    2017-07-19 12:13:55,564:18232(0xc0c):ZOO_ERROR@handle_socket_error_msg@1519: Socket [127.0.0.1:57557] zk retcode=-4, errno=140(Unknown error): failed to send a handshake packet: Unknown error
    [       OK ] ZooKeeperTest.LeaderContender (714 ms)
    [----------] 7 tests from ZooKeeperTest (15727 ms total)
    
    [----------] Global test environment tear-down
    [==========] 7 tests from 1 test case ran. (16802 ms total)
    [  PASSED  ] 7 tests.
    ```


> ZooKeeper public include files leak porting changes
> ---------------------------------------------------
>
>                 Key: ZOOKEEPER-2841
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2841
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: c client
>         Environment: Windows 10 with Visual Studio 2017
>            Reporter: Andrew Schwartzmeyer
>            Assignee: Andrew Schwartzmeyer
>              Labels: windows
>
> The fundamental problem is that the port of the C client to Windows is now close to six years old, with very few updates. This port leaks a lot of changes that should be internal to ZooKeeper, and many of those changes are simply no longer relevant. The correct thing to do is attempt to refactor the Windows port for new versions of ZooKeeper, removing dead/unneeded porting code, and moving dangerous porting code to C files instead of public headers.
> Two primary examples of this problem are [ZOOKEEPER-2491|https://issues.apache.org/jira/browse/ZOOKEEPER-2491] and [MESOS-7541|https://issues.apache.org/jira/browse/MESOS-7541].
> The first issue stems from this ancient porting code:
> {noformat}
> #define snprintf _snprintf
> {noformat}
>  in [winconfig.h|https://github.com/apache/zookeeper/blob/ddf0364903bf7ac7cd25b2e1927f0d9d3c7203c4/src/c/include/winconfig.h#L179]. Newer versions of Windows C libraries define {{snprintf}} as a function, and so it cannot be redefined.
> The second issue comes from this undocumented change:
> {noformat}
> #undef AF_INET6
> {noformat}
> again in [winconfig.h|https://github.com/apache/zookeeper/blob/ddf0364903bf7ac7cd25b2e1927f0d9d3c7203c4/src/c/include/winconfig.h#L169] which breaks any library that uses IPv6 and {{winsock2.h}}.
> Furthermore, the inclusion of the following defines and headers causes terrible problems for consuming libraries, as they leak into ZooKeeper's public headers:
> {noformat}
> #define _CRT_SECURE_NO_WARNINGS
> #define WIN32_LEAN_AND_MEAN
> #include <Windows.h>
> #include <Winsock2.h>
> #include <winstdint.h>
> #include <process.h>
> #include <ws2tcpip.h>
> {noformat}
> Depending on the order that a project includes or compiles files, this may or may not cause {{WIN32_LEAN_AND_MEAN}} to become unexpectedly defined, and {{windows.h}} to be unexpectedly included. This problem is exacberated by the fact that the {{winsock2.h}} and {{windows.h}} headers are order-dependent (if you read up on this, you'll see that defining {{WIN32_LEAN_AND_MEAN}} was meant to work-around this).
> Going forward, porting changes should live next to where they are used, preferably in source files, not header files, so they remain contained.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)