hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vishal K <vishalm...@gmail.com>
Subject Re: QA effort to support windows as a production platform
Date Wed, 07 Apr 2010 21:34:41 GMT
Hi Patrick,

We are not using C clients so we are not worried about porting them to
Windows. Just out of curiosity, is there any way to confirm if the code
coverage results are a result of test-core-java. If not, I will run the
coverage tools locally.

I had tried the smoketests earlier, but I ran into some issues there. I
forgot to note down the problem. I will revisit and upload the results.
Thanks for your help.

On Wed, Apr 7, 2010 at 5:11 PM, Patrick Hunt <phunt@apache.org> wrote:

>
> On 04/07/2010 01:51 PM, Vishal K wrote:
>
>> Hi Mahadev,
>>
>> Thanks for your response. Currently I am running ZK without cygwin on
>> windows. I will give it a try on cygwin. I am not quite familiar with
>> cppunit. Why will cppinit give me more confidence in native windows
>> libraries?
>>
>> I have few more questions relevant to testing:
>> 1. How much code coverage do we get with "test-core-java"? I see 68.8%
>> coverage on hudson
>> http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/clover/.  Does
>> this measure the coverage from junit tests run from "test-core-java"?
>>
>>
> Right, afaik that's test-core-java.
>
> I think if you plan to run in production under windows you would want to
> run in win, not cygwin. If that's the case you'd have to port the c client
> and c client tests to some windows specific compiler.
>
>
> 2. What would be a good and reliable set of tests that will help me verify
>> that the cluster holding up fine on windows. I tried to run systest and
>> generateload. But I am having issues with the tests (and also
>> understanding
>> the output of the tests since I am not familiar with the source). The
>> systest did exit with output something ilke Test OK (1). I presume this is
>> a
>> good sign :-) generateLoad crashed and I will look into it later. Please
>> let
>> me know if you have any suggestions. Last few lines are shown below:
>>
>>
> Well the best thing would be to run one's own tests for their specific use
> cases. Short of that you've done the core set of tests that we have
> available to us. You might try the smoke/latency tests:
> http://github.com/phunt/zk-smoketest
> in particular you can run the latency tests from a number of clients in
> parallel and see the results.
> http://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview
>
> Patrick
>
>
>
> 2010-04-07 14:44:20,055 - WARN  [QuorumPeer:/0.0.0.0:3155:QuorumPeer@662]
>> -
>> QuorumPeer main thread exited
>> Got rc = -4
>> Got rc = -4
>> [many such messages as above]
>> WatchedEvent state:Disconnected type:None path:null
>> java.lang.InterruptedException: sleep interrupted
>>         at java.lang.Thread.sleep(Native Method)
>>         at
>>
>> org.apache.zookeeper.test.system.GenerateLoad$GeneratorInstance$SenderThread.run(GenerateLoad.java:425)
>> java.lang.InterruptedException
>>         at java.lang.Object.wait(Native Method)
>>         at java.lang.Object.wait(Object.java:485)
>>         at
>>
>> org.apache.zookeeper.test.system.GenerateLoad$GeneratorInstance$ZooKeeperThread.incOutstanding(GenerateLoad.java:305)
>>         at
>>
>> org.apache.zookeeper.test.system.GenerateLoad$GeneratorInstance$ZooKeeperThread.run(GenerateLoad.java:353)
>>
>> 2010-04-07 14:44:20,711 - INFO
>> [Thread-42-SendThread(vkher-devd:3155):ClientCnxn$SendThread@1000] -
>> Opening
>> socket connection to server<host>/<IP>:48214
>> Got rc = -4
>> 2010-04-07 14:44:22,008 - INFO  [Thread-42:ZooKeeper@538] -
>> Session:0x127d97600cc0000 closed
>>
>> Thanks.
>>
>> Regards,
>> -Vishal
>>
>>
>> On Wed, Apr 7, 2010 at 3:55 PM, Mahadev Konar<mahadev@yahoo-inc.com>
>>  wrote:
>>
>> HI Vishal,
>>>  I would be a good think to actually get cppunit working on windows
>>> (rather
>>> than dropping it) since it would make you more confident on being able to
>>> use the native libraries for windows.
>>>
>>> Though there is already an open jira to try and compile librarires
>>> without
>>> CPPIUNIT being installed on the machines.
>>>
>>> http://issues.apache.org/jira/browse/ZOOKEEPER-316
>>>
>>>
>>> Would you want to try and take a shot at fixing the cppunit tests?
>>>
>>> Would be great to have cppunit tests working on cygwin!
>>>
>>>
>>> Thanks
>>> mahadev
>>>
>>>
>>> On 4/6/10 4:15 PM, "Vishal K"<vishalmlst@gmail.com>  wrote:
>>>
>>> Hi,
>>>>
>>>> I had few minor problems (
>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-734) after which all
>>>>
>>> junits
>>>
>>>> passed (I ran ant test-java-core).
>>>>
>>>> But the build failed later in create-cppunit-configure:
>>>>
>>>> -------------
>>>> test-core-java:
>>>> call-test-cppunit:
>>>> init:
>>>> check-cppunit-makefile:
>>>> create-cppunit-makefile:
>>>> init:
>>>> check-cppunit-configure:
>>>> create-cppunit-configure:
>>>>     [mkdir] Created dir:
>>>> C:\zookeeper\zookeeper-3.3.0\build\test\test-cppunit
>>>> BUILD FAILED
>>>> C:\zookeeper\zookeeper-3.3.0\build.xml:907: The following error occurred
>>>> while executing this line:
>>>> C:\zookeeper\zookeeper-3.3.0\build.xml:865: The following error occurred
>>>> while executing this line:
>>>> C:\zookeeper\zookeeper-3.3.0\build.xml:857: Execute failed:
>>>> java.io.IOException: Cannot run program
>>>> "C:\zookeeper\zookeeper-3.3.0\src\c\configure" (in directory
>>>> "C:\zookeeper\zookeeper-3.3.0\build\test\test-cppunit"): CreateProcess
>>>> error=193, %1 is not a valid Win32 application
>>>>  at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
>>>>  at java.lang.Runtime.exec(Runtime.java:593
>>>> -----------
>>>> cppunit tests should be probably dropped for windows.
>>>>
>>>> Also, one point to note the dataDir in zoo.cfg should have UNIX path (or
>>>> double backslash instead of single backslash). I suppose all Java
>>>>
>>> developers
>>>
>>>> might be already aware of that.
>>>>
>>>> Regards,
>>>> -Vishal
>>>>
>>>> On Fri, Apr 2, 2010 at 1:46 PM, Patrick Hunt<phunt@apache.org>  wrote:
>>>>
>>>> Pretty seamless, just do a rolling upgrade (see the faq) of the servers.
>>>>> Then upgrade your clients. Code APIs on the client are all b/w compat.
>>>>>
>>>>> Patrick
>>>>>
>>>>>
>>>>> On 04/02/2010 10:41 AM, Vishal K wrote:
>>>>>
>>>>> Hi Patrick,
>>>>>>
>>>>>> We have not upgraded to 3.3.0 yet. We are using 3.2.2. I did notice
>>>>>> the
>>>>>> windows specific batch files in 3.3.0. How seemless is upgrade from
>>>>>>
>>>>> 3.2.2
>>>
>>>>  -
>>>>>>
>>>>>> 3.3.0?
>>>>>>>
>>>>>>>
>>>>>> I will let you know if I run into any windows related problems.
>>>>>> Thanks.
>>>>>>
>>>>>> On Fri, Apr 2, 2010 at 11:40 AM, Patrick Hunt<phunt@apache.org>
>>>>>>
>>>>>  wrote:
>>>
>>>>
>>>>>> Are you using 3.3.0? 3.3.0 included a number of fixes for cygwin
and
>>>>>>
>>>>>>> includes windows specific batch files. If you are planning to
deploy
>>>>>>>
>>>>>> to
>>>
>>>>  production on windows I'd encourage you to develop under windows
>>>>>>>
>>>>>> directly
>>>
>>>>  as
>>>>>>> well.
>>>>>>>
>>>>>>> If you find issues, bugs, etc... be sure to enter JIRAs. Don't
worry,
>>>>>>>
>>>>>> you
>>>
>>>>  won't hurt our feelings, on the contrary we'll be happy if you
>>>>>>>
>>>>>> find/fix
>>>
>>>>  issues on windows and make things better for everyone. (just make sure
>>>>>>> you
>>>>>>> are using the latest release).
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Patrick
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 04/02/2010 07:05 AM, Vishal K wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>>
>>>>>>>> I was able to start zookeeper on windows using cygwin. I
had to do
>>>>>>>>
>>>>>>> minor
>>>
>>>>   changes to the shell scripts to use cygpath wherever needed.
>>>>>>>> I will run a few tests and post the progress. I greped through
the
>>>>>>>> zookeeper
>>>>>>>> sources just to check if ZK is using any native code.
>>>>>>>> I didn't find any, but just to confirm - is ZK using native
code.
>>>>>>>>
>>>>>>>> I have talked to a few guys around and they said it is fair
to
>>>>>>>> assume
>>>>>>>> that
>>>>>>>> the programs are portable (to Windows) if they don't have
native
>>>>>>>>
>>>>>>> code.
>>>
>>>>   Just
>>>>>>>> wanted to check. Thanks.
>>>>>>>> On Thu, Apr 1, 2010 at 10:09 AM, Vishal K<vishalmlst@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Patrick,
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks for your response. I start running ZK on windows
and let you
>>>>>>>>> know
>>>>>>>>> if
>>>>>>>>> I run into issues.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Mar 31, 2010 at 11:32 AM, Patrick Hunt<phunt@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Vishal K wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> We will be using zookeeper quite extensively for
clustering.
>>>>>>>>>>
>>>>>>>>> Windows
>>>
>>>>    is
>>>>>>>>>>
>>>>>>>>>> one
>>>>>>>>>>> of the platforms that we may need to support.
Since Win32 is not
>>>>>>>>>>> supported
>>>>>>>>>>> as a production platform I was wondering to what
extent is
>>>>>>>>>>>
>>>>>>>>>> zookeeper
>>>
>>>>    tested
>>>>>>>>>>> on windows. We are also interested on using Zookeeper
on Win64
>>>>>>>>>>> platform.
>>>>>>>>>>> Is
>>>>>>>>>>> Win64 supported? Are there any plans to support
Win32/Win64 for
>>>>>>>>>>> production?
>>>>>>>>>>> If not, what would one need to do support windows
and what would
>>>>>>>>>>>
>>>>>>>>>> be
>>>
>>>>    the
>>>>>>>>>>> estimated QA effort?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> My use of ZK is exclusively 32/64bit linux, however
I can tell
>>>>>>>>>>> you
>>>>>>>>>>>
>>>>>>>>>> that
>>>>>>>>>> given that the client/server are implemented in java
it should
>>>>>>>>>>
>>>>>>>>> work.
>>>
>>>>    Problems you might encounter would be things like NIO issues with
>>>>>>>>>>
>>>>>>>>> the
>>>
>>>>    JVM
>>>>>>>>>> implementation on windows.
>>>>>>>>>>
>>>>>>>>>> Testing on windows? Pretty much 0 afaik. We do support
development
>>>>>>>>>>
>>>>>>>>> on
>>>
>>>>    cygwin, so provides some basic exercising of the codepaths with the
>>>>>>>>>> windows
>>>>>>>>>> jvm, however it's not likely production level qa.
>>>>>>>>>>
>>>>>>>>>> This question (zk on win) has come up once or twice
before, I
>>>>>>>>>>
>>>>>>>>> haven't
>>>
>>>>    seen
>>>>>>>>>> any followup from the users who asked about it previously
though.
>>>>>>>>>>
>>>>>>>>>> 3.3.0 has batch files for running the server in windows,
give
>>>>>>>>>> those
>>>>>>>>>>
>>>>>>>>> a
>>>
>>>>    try.
>>>>>>>>>> Probably what you'd want to do is run "ant test-core-java"
or
>>>>>>>>>>
>>>>>>>>> similar
>>>
>>>>    in
>>>>>>>>>> the
>>>>>>>>>> top of the ZK release directory. This will run all
the java tests
>>>>>>>>>>
>>>>>>>>> and
>>>
>>>>    give
>>>>>>>>>> you some insight into status. I'd be happy to work
with you to
>>>>>>>>>> land
>>>>>>>>>> patches
>>>>>>>>>> that address issues with ZK on windows. Depending
on the interest
>>>>>>>>>> level
>>>>>>>>>> and
>>>>>>>>>> support from win users we could support win as a
dev/prod platform
>>>>>>>>>>
>>>>>>>>> at
>>>
>>>>    some
>>>>>>>>>> point in the future - having ongoing support for
this would be
>>>>>>>>>> important
>>>>>>>>>> though (people interested in testing/fixing under
win I mean). Try
>>>>>>>>>> exercising under windows and create some JIRAs based
on what you
>>>>>>>>>>
>>>>>>>>> find.
>>>
>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Patrick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>
>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message