Return-Path: 
 <commits-return-147945-apmail-cassandra-commits-archive=cassandra.apache.org@cassandra.apache.org>
X-Original-To: apmail-cassandra-commits-archive@www.apache.org
Delivered-To: apmail-cassandra-commits-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 1E4B71806B
	for <apmail-cassandra-commits-archive@www.apache.org>;
 Thu,  8 Oct 2015 12:21:27 +0000 (UTC)
Received: (qmail 7435 invoked by uid 500); 8 Oct 2015 12:21:27 -0000
Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org
Received: (qmail 7409 invoked by uid 500); 8 Oct 2015 12:21:26 -0000
Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:commits-help@cassandra.apache.org>
List-Unsubscribe: <mailto:commits-unsubscribe@cassandra.apache.org>
List-Post: <mailto:commits@cassandra.apache.org>
List-Id: <commits.cassandra.apache.org>
Reply-To: dev@cassandra.apache.org
Delivered-To: mailing list commits@cassandra.apache.org
Received: (qmail 7391 invoked by uid 99); 8 Oct 2015 12:21:26 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Oct 2015 12:21:26 +0000
Date: Thu, 8 Oct 2015 12:21:26 +0000 (UTC)
From: "Stefania (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12901440.1443563357000.56045.1444306886846@Atlassian.JIRA>
In-Reply-To: <JIRA.12901440.1443563357000@Atlassian.JIRA>
References: <JIRA.12901440.1443563357000@Atlassian.JIRA>
 <JIRA.12901440.1443563357506@arcas>
Subject: [jira] [Commented] (CASSANDRA-10415) Fix cqlsh bugs
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/CASSANDRA-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14948574#comment-14948574 ] 

Stefania commented on CASSANDRA-10415:
--------------------------------------

[~thobbs] I would like to see if you have any ideas (without spending too much time) about this problem. Basically since we removed the {{K_XXX}} rules, we no longer complete correctly {{IF NOT EXIST}}. For example if we type {{CREATE TABLE IF}} followed by a TAB, with the {{K_XXX}} rules the valid completion was {{NOT EXISTS}} whereas now it's {{(    NOT}}

Here is the rule:

{code}
<createColumnFamilyStatement> ::= "CREATE" wat=( "COLUMNFAMILY" | "TABLE" ) ("IF" "NOT" "EXISTS")?
                                    ( ks=<nonSystemKeyspaceName> dot="." )? cf=<cfOrKsName>
                                    "(" ( <singleKeyCfSpec> | <compositeKeyCfSpec> ) ")"
                                   ( "WITH" <cfamProperty> ( "AND" <cfamProperty> )* )?
                                ;
{code}

And here is the debug information:

{code}
item 'NOT' added by:
  - word_match('NOT').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:294)
    - <ParseContext matched=(('identifier', 'CREATE', (0, 6)), ('identifier', 'TABLE', (7, 12)), ('identifier', 'IF', (13, 15))) remainder=() prodname='createColumnFamilyStatement' bindings={'wat': 'TABLE', 'partial': '', 'cassandra_conn': <__main__.Shell instance at 0x7f0273b63ef0>, '*LASTTYPE*': 'identifier', '*DEBUG*': True, '*SRC*': 'CREATE TABLE
 IF '}>
  - rule_series([word_match('IF'), word_match('NOT'), word_match('EXISTS')]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:216)
  - one_or_none(rule_series([word_match('IF'), word_match('NOT'), word_match('EXISTS')])).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:179)
  - rule_series([word_match('CREATE'), named_symbol('wat', choice([word_match('COLUMNFAMILY'), word_match('TABLE')])), one_or_none(rule_series([word_match('IF'), word_match('NOT'), word_match('EXISTS')])), one_or_none(rule_series([named_symbol('ks', rule_reference('nonSystemKeyspaceName')), named_symbol('dot', text_match('.'))])), named_symbol('cf', rule_reference('cfOrKsName')), text_match('('), choice([rule_reference('singleKeyCfSpec'), rule_reference('compositeKeyCfSpec')]), text_match(')'), one_or_none(rule_series([word_match('WITH'), rule_reference('cfamProperty'), repeat(rule_series([word_match('AND'), rule_reference('cfamProperty')]))]))]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:216)
  - rule_reference('createColumnFamilyStatement').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:205)
  - choice([rule_reference('createKeyspaceStatement'), rule_reference('createColumnFamilyStatement'), rule_reference('createIndexStatement'), rule_reference('createUserTypeStatement'), rule_reference('createFunctionStatement'), rule_reference('createAggregateStatement'), rule_reference('createTriggerStatement'), rule_reference('dropKeyspaceStatement'), rule_reference('dropColumnFamilyStatement'), rule_reference('dropIndexStatement'), rule_reference('dropUserTypeStatement'), rule_reference('dropFunctionStatement'), rule_reference('dropAggregateStatement'), rule_reference('dropTriggerStatement'), rule_reference('alterTableStatement'), rule_reference('alterKeyspaceStatement'), rule_reference('alterUserTypeStatement')]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:171)
  - rule_reference('schemaChangeStatement').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:205)
  - choice([rule_reference('useStatement'), rule_reference('selectStatement'), rule_reference('dataChangeStatement'), rule_reference('schemaChangeStatement'), rule_reference('authenticationStatement'), rule_reference('authorizationStatement')]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:171)
  - rule_reference('statementBody').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:205)
  - rule_reference('statementBody').match_with_results() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:136)
Trying completer <function completerwrapper_on_create_cf_ks_dot_completer at 0x7f0273b41488> with <ParseContext matched=(('identifier', 'CREATE', (0, 6)), ('identifier', 'TABLE', (7, 12)), ('identifier', 'IF', (13, 15))) remainder=() prodname='createColumnFamilyStatement' bindings={'ks': 'IF', 'ksname': 'IF', '*LASTTYPE*': 'identifier', 'wat': 'TABLE', 'partial': '', 'cassandra_conn': <__main__.Shell instance at 0x7f0273b63ef0>, '*DEBUG*': True, '*SRC*': 'CREATE TABLE IF '}>
got []

item '(' added by:
  - text_match('(').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:294)
    - <ParseContext matched=(('identifier', 'CREATE', (0, 6)), ('identifier', 'TABLE', (7, 12)), ('identifier', 'IF', (13, 15))) remainder=() prodname='createColumnFamilyStatement' bindings={'*LASTTYPE*': 'identifier', '*SRC*': 'CREATE TABLE IF ', 'partial': '', 'cassandra_conn': <__main__.Shell instance at 0x7f0273b63ef0>, '*DEBUG*': True, 'wat': 'TABLE', 'cf': 'IF'}>
  - rule_series([word_match('CREATE'), named_symbol('wat', choice([word_match('COLUMNFAMILY'), word_match('TABLE')])), one_or_none(rule_series([word_match('IF'), word_match('NOT'), word_match('EXISTS')])), one_or_none(rule_series([named_symbol('ks', rule_reference('nonSystemKeyspaceName')), named_symbol('dot', text_match('.'))])), named_symbol('cf', rule_reference('cfOrKsName')), text_match('('), choice([rule_reference('singleKeyCfSpec'), rule_reference('compositeKeyCfSpec')]), text_match(')'), one_or_none(rule_series([word_match('WITH'), rule_reference('cfamProperty'), repeat(rule_series([word_match('AND'), rule_reference('cfamProperty')]))]))]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:216)
  - rule_reference('createColumnFamilyStatement').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:205)
  - choice([rule_reference('createKeyspaceStatement'), rule_reference('createColumnFamilyStatement'), rule_reference('createIndexStatement'), rule_reference('createUserTypeStatement'), rule_reference('createFunctionStatement'), rule_reference('createAggregateStatement'), rule_reference('createTriggerStatement'), rule_reference('dropKeyspaceStatement'), rule_reference('dropColumnFamilyStatement'), rule_reference('dropIndexStatement'), rule_reference('dropUserTypeStatement'), rule_reference('dropFunctionStatement'), rule_reference('dropAggregateStatement'), rule_reference('dropTriggerStatement'), rule_reference('alterTableStatement'), rule_reference('alterKeyspaceStatement'), rule_reference('alterUserTypeStatement')]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:171)
  - rule_reference('schemaChangeStatement').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:205)
  - choice([rule_reference('useStatement'), rule_reference('selectStatement'), rule_reference('dataChangeStatement'), rule_reference('schemaChangeStatement'), rule_reference('authenticationStatement'), rule_reference('authorizationStatement')]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:171)
  - rule_reference('statementBody').match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:205)
  - rule_reference('statementBody').match_with_results() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:136)
  - named_collector('statements', rule_reference('statementBody')).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:249)
  - rule_series([named_collector('statements', rule_reference('statementBody')), text_match(';')]).match() (/home/stefi/git/cstar/cassandra/bin/../pylib/cqlshlib/pylexotron.py:216)
{code}

It looks like {{IF}} is treated as a valid table name because it doesn't know it is a reserved keyword. We need to somehow treat reserved keywords in a special way, the {{identifier}} rule seems too generic. Do you have any ideas? Would adding all reserved keywords to a very long {{(aaa|bbb|........)}} rule work?

> Fix cqlsh bugs
> --------------
>
>                 Key: CASSANDRA-10415
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10415
>             Project: Cassandra
>          Issue Type: Sub-task
>            Reporter: Jim Witschey
>            Assignee: Stefania
>              Labels: cqlsh
>             Fix For: 3.0.0 rc2
>
>
> This is followup to CASSANDRA-10289
> The tests currently failing should be:
> * {{cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_create_columnfamily}}
> ** uses {{create_columnfamily_table_template}}. Stefania says "the {{(}} after {{CREATE ... IF}} does not look valid to me."
> * {{cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_create_table}}
> ** uses {{create_columnfamily_table_template}}, see above.
> * {{cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_delete}}
> ** Stefania says: "I don't think keyspaces are a valid completion after {{DELETE a [}} and after {{DELETE FROM twenty_rows_composite_table USING TIMESTAMP 0 WHERE TOKEN(a) >=}}. From a quick analysis of {{cqlhandling.py}} I think it comes from {{<term>}}, which picks up {{<functionName>}}, which was changed to include {{ks.}} by CASSANDRA-7556.
> * {{cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_drop_keyspace}}
> ** Stefania says: "the {{;}} after {{DROP KEYSPACE IF}} is not valid.
> * {{cqlshlib.test.test_cqlsh_output.TestCqlshOutput.test_timestamp_output}}
> ** already documented with CASSANDRA-10313 and CASSANDRA-10397
> I'm happy to break these out into separate tickets if necessary. 
> To run the tests locally, I cd to {{cassandra/pylib/cqlshlib}} and run the following:
> {code}
> ccm create -n 1 --install-dir=../.. test
> ccm start --wait-for-binary-proto
> nosetests test 2>&1
> ccm remove
> {code}
> This requires nose and ccm. Until CASSANDRA-10289 is resolved, you'll have to use my branch here: https://github.com/mambocab/cassandra/tree/fix-cqlsh-tests
> Tests for this branch are run (non-continuously) here:
> http://cassci.datastax.com/job/scratch_mambocab-fix_cqlsh/
> Assigning [~Stefania] for now, since she's already looked at 10289, but feel free to reassign.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)