Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Tue, 17 Nov 2015 14:55:11 +0000 (UTC)
From: "Paulo Motta (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12905451.1444986370000.93445.1447772111028@Atlassian.JIRA>
In-Reply-To: <JIRA.12905451.1444986370000@Atlassian.JIRA>
References: <JIRA.12905451.1444986370000@Atlassian.JIRA>
 <JIRA.12905451.1444986370404@arcas>
Subject: [jira] [Commented] (CASSANDRA-10541) cqlshlib tests cannot run on
 Windows
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008770#comment-15008770 ] 

Paulo Motta commented on CASSANDRA-10541:
-----------------------------------------

Most of the tests (except the auto-completion suite) were fixed on Windows with {{subprocess}}. Luckily the code was well modularized and allowed to replace the posix-only pty terminal-access module with a custom WinPty emulation based on {{subprocess}}. 

The problem of accessing a subprocess interactively with {{subprocess.PIPE}} is that {{subproc.stdout.read}} blocks until new data is present, so I used a trick (described in [this blog post|http://eyalarubas.com/python-subproc-nonblock.html]) that uses another thread to read characters in the background from stdout and put them in a queue which is consumed by the testing thread without blocking. This trick is not necessary in dtests because all input is piped to the process, which processes it and exits, and then all the output is verified at once. The cqlshlib tests executes each command interactively and checks the output after each command.

Some other fixes:
* Replace the symlink trick to load cqlsh as a module with creating an empty {{__init__.py}} file during tests on {{cassandra/bin}}, so python will allow loading {{cqlsh.py}} as a module on both windows and Linux.
* Replaced the signal-based timeout handling on Windows with the eventlet library, which is now necessary to run cqlshlib tests on Windows.

Although this was sufficient to fix the {{test_cqlsh_output.py}} and {{test_cql_parsing.py}} suites, it did not fix the command completion tests of {{test_cqlsh_completion.py}}, probably because readline disables auto-completing from stdin piped input. For now, I disabled these tests on Windows and created CASSANDRA-10720 to create platform-independent auto-completion tests.

[~mambocab] [~philipthompson] any volunteer to review? :-) Also, could we set up a Windows/Linux cqlshlib cassci run with this branch?

Tests will appear shortly below:
||2.2||3.0||trunk||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-10541]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-10541]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-10541]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10541-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10541-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10541-testall/lastCompletedBuild/testReport/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10541-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10541-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10541-dtest/lastCompletedBuild/testReport/]|

> cqlshlib tests cannot run on Windows
> ------------------------------------
>
>                 Key: CASSANDRA-10541
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10541
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>            Reporter: Benjamin Lerer
>            Assignee: Paulo Motta
>            Priority: Minor
>              Labels: cqlsh, windows
>
> If I try to run the {{cqlshlib}} tests on Windows, I got the following error:
> {quote}
> ======================================================================
> ERROR: Failure: AttributeError ('module' object has no attribute 'symlink')
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "C:\Python27\lib\site-packages\nose\loader.py", line 414, in loadTestsFromName
>     addr.filename, addr.module)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 47, in importFromPath
>     return self.importFromDir(dir_path, fqname)
>   File "C:\Python27\lib\site-packages\nose\importer.py", line 94, in importFromDir
>     mod = load_module(part_fqname, fh, filename, desc)
>   File "[...]\pylib\cqlshlib\test\__init__.py", line 17, in <module>
>     from .cassconnect import create_test_db, remove_test_db
>   File "[...]\pylib\cqlshlib\test\cassconnect.py", line 22, in <module>
>     from .basecase import cql, cqlsh, cqlshlog, TEST_HOST, TEST_PORT, rundir
>   File "[...]\pylib\cqlshlib\test\basecase.py", line 43, in <module>
>     os.symlink(path_to_cqlsh, modulepath)
> AttributeError: 'module' object has no attribute 'symlink'
> ----------------------------------------------------------------------
> Ran 1 test in 0.002s
> FAILED (errors=1)
> {quote}
> The problem comes from the fact tha Windows has no support for symlinks.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)