hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: QA effort to support windows as a production platform
Date Wed, 07 Apr 2010 21:11:33 GMT

On 04/07/2010 01:51 PM, Vishal K wrote:
> Hi Mahadev,
>
> Thanks for your response. Currently I am running ZK without cygwin on
> windows. I will give it a try on cygwin. I am not quite familiar with
> cppunit. Why will cppinit give me more confidence in native windows
> libraries?
>
> I have few more questions relevant to testing:
> 1. How much code coverage do we get with "test-core-java"? I see 68.8%
> coverage on hudson
> http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/clover/.  Does
> this measure the coverage from junit tests run from "test-core-java"?
>

Right, afaik that's test-core-java.

I think if you plan to run in production under windows you would want to 
run in win, not cygwin. If that's the case you'd have to port the c 
client and c client tests to some windows specific compiler.

> 2. What would be a good and reliable set of tests that will help me verify
> that the cluster holding up fine on windows. I tried to run systest and
> generateload. But I am having issues with the tests (and also understanding
> the output of the tests since I am not familiar with the source). The
> systest did exit with output something ilke Test OK (1). I presume this is a
> good sign :-) generateLoad crashed and I will look into it later. Please let
> me know if you have any suggestions. Last few lines are shown below:
>

Well the best thing would be to run one's own tests for their specific 
use cases. Short of that you've done the core set of tests that we have 
available to us. You might try the smoke/latency tests:
http://github.com/phunt/zk-smoketest
in particular you can run the latency tests from a number of clients in 
parallel and see the results.
http://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview

Patrick


> 2010-04-07 14:44:20,055 - WARN  [QuorumPeer:/0.0.0.0:3155:QuorumPeer@662] -
> QuorumPeer main thread exited
> Got rc = -4
> Got rc = -4
> [many such messages as above]
> WatchedEvent state:Disconnected type:None path:null
> java.lang.InterruptedException: sleep interrupted
>          at java.lang.Thread.sleep(Native Method)
>          at
> org.apache.zookeeper.test.system.GenerateLoad$GeneratorInstance$SenderThread.run(GenerateLoad.java:425)
> java.lang.InterruptedException
>          at java.lang.Object.wait(Native Method)
>          at java.lang.Object.wait(Object.java:485)
>          at
> org.apache.zookeeper.test.system.GenerateLoad$GeneratorInstance$ZooKeeperThread.incOutstanding(GenerateLoad.java:305)
>          at
> org.apache.zookeeper.test.system.GenerateLoad$GeneratorInstance$ZooKeeperThread.run(GenerateLoad.java:353)
>
> 2010-04-07 14:44:20,711 - INFO
> [Thread-42-SendThread(vkher-devd:3155):ClientCnxn$SendThread@1000] - Opening
> socket connection to server<host>/<IP>:48214
> Got rc = -4
> 2010-04-07 14:44:22,008 - INFO  [Thread-42:ZooKeeper@538] -
> Session:0x127d97600cc0000 closed
>
> Thanks.
>
> Regards,
> -Vishal
>
>
> On Wed, Apr 7, 2010 at 3:55 PM, Mahadev Konar<mahadev@yahoo-inc.com>  wrote:
>
>> HI Vishal,
>>   I would be a good think to actually get cppunit working on windows (rather
>> than dropping it) since it would make you more confident on being able to
>> use the native libraries for windows.
>>
>> Though there is already an open jira to try and compile librarires without
>> CPPIUNIT being installed on the machines.
>>
>> http://issues.apache.org/jira/browse/ZOOKEEPER-316
>>
>>
>> Would you want to try and take a shot at fixing the cppunit tests?
>>
>> Would be great to have cppunit tests working on cygwin!
>>
>>
>> Thanks
>> mahadev
>>
>>
>> On 4/6/10 4:15 PM, "Vishal K"<vishalmlst@gmail.com>  wrote:
>>
>>> Hi,
>>>
>>> I had few minor problems (
>>> https://issues.apache.org/jira/browse/ZOOKEEPER-734) after which all
>> junits
>>> passed (I ran ant test-java-core).
>>>
>>> But the build failed later in create-cppunit-configure:
>>>
>>> -------------
>>> test-core-java:
>>> call-test-cppunit:
>>> init:
>>> check-cppunit-makefile:
>>> create-cppunit-makefile:
>>> init:
>>> check-cppunit-configure:
>>> create-cppunit-configure:
>>>      [mkdir] Created dir:
>>> C:\zookeeper\zookeeper-3.3.0\build\test\test-cppunit
>>> BUILD FAILED
>>> C:\zookeeper\zookeeper-3.3.0\build.xml:907: The following error occurred
>>> while executing this line:
>>> C:\zookeeper\zookeeper-3.3.0\build.xml:865: The following error occurred
>>> while executing this line:
>>> C:\zookeeper\zookeeper-3.3.0\build.xml:857: Execute failed:
>>> java.io.IOException: Cannot run program
>>> "C:\zookeeper\zookeeper-3.3.0\src\c\configure" (in directory
>>> "C:\zookeeper\zookeeper-3.3.0\build\test\test-cppunit"): CreateProcess
>>> error=193, %1 is not a valid Win32 application
>>>   at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
>>>   at java.lang.Runtime.exec(Runtime.java:593
>>> -----------
>>> cppunit tests should be probably dropped for windows.
>>>
>>> Also, one point to note the dataDir in zoo.cfg should have UNIX path (or
>>> double backslash instead of single backslash). I suppose all Java
>> developers
>>> might be already aware of that.
>>>
>>> Regards,
>>> -Vishal
>>>
>>> On Fri, Apr 2, 2010 at 1:46 PM, Patrick Hunt<phunt@apache.org>  wrote:
>>>
>>>> Pretty seamless, just do a rolling upgrade (see the faq) of the servers.
>>>> Then upgrade your clients. Code APIs on the client are all b/w compat.
>>>>
>>>> Patrick
>>>>
>>>>
>>>> On 04/02/2010 10:41 AM, Vishal K wrote:
>>>>
>>>>> Hi Patrick,
>>>>>
>>>>> We have not upgraded to 3.3.0 yet. We are using 3.2.2. I did notice the
>>>>> windows specific batch files in 3.3.0. How seemless is upgrade from
>> 3.2.2
>>>>> -
>>>>>
>>>>>> 3.3.0?
>>>>>>
>>>>>
>>>>> I will let you know if I run into any windows related problems. Thanks.
>>>>>
>>>>> On Fri, Apr 2, 2010 at 11:40 AM, Patrick Hunt<phunt@apache.org>
>>   wrote:
>>>>>
>>>>> Are you using 3.3.0? 3.3.0 included a number of fixes for cygwin and
>>>>>> includes windows specific batch files. If you are planning to deploy
>> to
>>>>>> production on windows I'd encourage you to develop under windows
>> directly
>>>>>> as
>>>>>> well.
>>>>>>
>>>>>> If you find issues, bugs, etc... be sure to enter JIRAs. Don't worry,
>> you
>>>>>> won't hurt our feelings, on the contrary we'll be happy if you
>> find/fix
>>>>>> issues on windows and make things better for everyone. (just make
sure
>>>>>> you
>>>>>> are using the latest release).
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 04/02/2010 07:05 AM, Vishal K wrote:
>>>>>>
>>>>>> Hi,
>>>>>>>
>>>>>>> I was able to start zookeeper on windows using cygwin. I had
to do
>> minor
>>>>>>> changes to the shell scripts to use cygpath wherever needed.
>>>>>>> I will run a few tests and post the progress. I greped through
the
>>>>>>> zookeeper
>>>>>>> sources just to check if ZK is using any native code.
>>>>>>> I didn't find any, but just to confirm - is ZK using native code.
>>>>>>>
>>>>>>> I have talked to a few guys around and they said it is fair to
assume
>>>>>>> that
>>>>>>> the programs are portable (to Windows) if they don't have native
>> code.
>>>>>>> Just
>>>>>>> wanted to check. Thanks.
>>>>>>> On Thu, Apr 1, 2010 at 10:09 AM, Vishal K<vishalmlst@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Patrick,
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks for your response. I start running ZK on windows and
let you
>>>>>>>> know
>>>>>>>> if
>>>>>>>> I run into issues.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 31, 2010 at 11:32 AM, Patrick Hunt<phunt@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Vishal K wrote:
>>>>>>>>>
>>>>>>>>> We will be using zookeeper quite extensively for clustering.
>> Windows
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>>> one
>>>>>>>>>> of the platforms that we may need to support. Since
Win32 is not
>>>>>>>>>> supported
>>>>>>>>>> as a production platform I was wondering to what
extent is
>> zookeeper
>>>>>>>>>> tested
>>>>>>>>>> on windows. We are also interested on using Zookeeper
on Win64
>>>>>>>>>> platform.
>>>>>>>>>> Is
>>>>>>>>>> Win64 supported? Are there any plans to support Win32/Win64
for
>>>>>>>>>> production?
>>>>>>>>>> If not, what would one need to do support windows
and what would
>> be
>>>>>>>>>> the
>>>>>>>>>> estimated QA effort?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> My use of ZK is exclusively 32/64bit linux, however
I can tell you
>>>>>>>>> that
>>>>>>>>> given that the client/server are implemented in java
it should
>> work.
>>>>>>>>> Problems you might encounter would be things like NIO
issues with
>> the
>>>>>>>>> JVM
>>>>>>>>> implementation on windows.
>>>>>>>>>
>>>>>>>>> Testing on windows? Pretty much 0 afaik. We do support
development
>> on
>>>>>>>>> cygwin, so provides some basic exercising of the codepaths
with the
>>>>>>>>> windows
>>>>>>>>> jvm, however it's not likely production level qa.
>>>>>>>>>
>>>>>>>>> This question (zk on win) has come up once or twice before,
I
>> haven't
>>>>>>>>> seen
>>>>>>>>> any followup from the users who asked about it previously
though.
>>>>>>>>>
>>>>>>>>> 3.3.0 has batch files for running the server in windows,
give those
>> a
>>>>>>>>> try.
>>>>>>>>> Probably what you'd want to do is run "ant test-core-java"
or
>> similar
>>>>>>>>> in
>>>>>>>>> the
>>>>>>>>> top of the ZK release directory. This will run all the
java tests
>> and
>>>>>>>>> give
>>>>>>>>> you some insight into status. I'd be happy to work with
you to land
>>>>>>>>> patches
>>>>>>>>> that address issues with ZK on windows. Depending on
the interest
>>>>>>>>> level
>>>>>>>>> and
>>>>>>>>> support from win users we could support win as a dev/prod
platform
>> at
>>>>>>>>> some
>>>>>>>>> point in the future - having ongoing support for this
would be
>>>>>>>>> important
>>>>>>>>> though (people interested in testing/fixing under win
I mean). Try
>>>>>>>>> exercising under windows and create some JIRAs based
on what you
>> find.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Patrick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>
>>
>

Mime
View raw message