db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dag H. Wanvik (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DERBY-5125) Derby 10.7.1.1 : Build fails with javacc 5 (for Debian packaging)
Date Tue, 22 May 2012 23:24:41 GMT

    [ https://issues.apache.org/jira/browse/DERBY-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281306#comment-13281306
] 

Dag H. Wanvik commented on DERBY-5125:
--------------------------------------

The problem with the empty "encountered" token lies in the generated code; SQLParser#jj_add_error_token

In version 4.0, the code looks like this:

  private void jj_add_error_token(int kind, int pos) {
    if (pos >= 100) return;
    if (pos == jj_endpos + 1) {
      jj_lasttokens[jj_endpos++] = kind;
    } else if (jj_endpos != 0) {
      jj_expentry = new int[jj_endpos];
      for (int i = 0; i < jj_endpos; i++) {
        jj_expentry[i] = jj_lasttokens[i];
      }
      boolean exists = false;
      for (java.util.Enumeration e = jj_expentries.elements(); e.hasMoreElements();) {
        int[] oldentry = (int[])(e.nextElement());
        if (oldentry.length == jj_expentry.length) {
          exists = true;
          for (int i = 0; i < jj_expentry.length; i++) {
            if (oldentry[i] != jj_expentry[i]) {
              exists = false;
              break;
            }
          }
          if (exists) break;
        }
      }
      if (!exists) jj_expentries.addElement(jj_expentry);  // <------- Missing in 5.0
      if (pos != 0) jj_lasttokens[(jj_endpos = pos) - 1] = kind;
    }


In JavaCC 5.0, this has been changed. The representation of jj_expentries ha sbeen changed
from Vector to List, and the logic for initializing it has also been changed, viz.

  private void jj_add_error_token(int kind, int pos) {
    if (pos >= 100) return;
    if (pos == jj_endpos + 1) {
      jj_lasttokens[jj_endpos++] = kind;
    } else if (jj_endpos != 0) {
      jj_expentry = new int[jj_endpos];
      for (int i = 0; i < jj_endpos; i++) {
        jj_expentry[i] = jj_lasttokens[i];
      }
      jj_entries_loop: for (java.util.Iterator it = jj_expentries.iterator(); it.hasNext();)
{
        int[] oldentry = (int[])(it.next());
        if (oldentry.length == jj_expentry.length) {
          for (int i = 0; i < jj_expentry.length; i++) {
            if (oldentry[i] != jj_expentry[i]) {
              continue jj_entries_loop;
            }
          }
          jj_expentries.add(jj_expentry);
          break jj_entries_loop;
        }
      }
      if (pos != 0) jj_lasttokens[(jj_endpos = pos) - 1] = kind;
    }
  }

Now, the line "if (!exists) jj_expentries.addElement(jj_expentry);" from 4.0 has gone missing
due to the new logic which removes the use of the exists variable. Unfortunately this is broken,
because the "jj_entries_loop" would never get executed since jj_expentries is empty initially.
So, it remains uninitialized (empty), hence the problem of the empty encountered token.

We could fix this by building our own JavaCC, or by patching up the generated code on each
build. 

I tried to use the new generated version of ParseException, but had to adjust that code a
bit too:

This stanza gave a bit more output than we want:

    String retval = "Encountered \"";
    Token tok = currentToken.next;
    for (int i = 0; i < maxSize; i++) {
      if (i != 0) retval += " ";
      if (tok.kind == 0) {
        retval += tokenImage[0];
        break;
      }
      retval += " " + tokenImage[tok.kind];
      retval += " \"";
      retval += add_escapes(tok.image);
      retval += " \"";
      tok = tok.next;
    }

which would show:

ij> values CHAR(DEC(67.21,4,2)); 
ERROR 42X01: Syntax error: Encountered " "dec" "DEC "" at line 1, column 13.

I found reverting that code to:

    String retval = "Encountered \"";
    Token tok = currentToken.next;
    
    for (int i = 0; i < maxSize; i++) {
        if (i != 0) retval += " ";
        if (tok.kind == 0) {
            retval += tokenImage[0];
            break;
        }
        retval += add_escapes(tok.image);
        tok = tok.next;
    }

did the trick. I also had to comment out the code to show the possible expected tokens, as
done in our old checked in version of ParseException. With that change and a fix to SQLParser,
the output would look right for the cases I attempted. I presume the "encountered" bug would
apply for ij as well.





                
> Derby 10.7.1.1 : Build fails with javacc 5 (for Debian packaging)
> -----------------------------------------------------------------
>
>                 Key: DERBY-5125
>                 URL: https://issues.apache.org/jira/browse/DERBY-5125
>             Project: Derby
>          Issue Type: Improvement
>          Components: Build tools
>    Affects Versions: 10.7.1.1
>         Environment: Debian Linux Linux inf-7590.int-evry.fr 2.6.32-5-amd64 #1 SMP Wed
Jan 12 03:40:32 UTC 2011 x86_64 GNU/Linux
> java -version   (also tested with sun jvm)
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.7) (6b18-1.8.7-2)
> OpenJDK 64-Bit Server VM (build 16.0-b13, mixed mode)
> ant -version
> Apache Ant version 1.8.0 compiled on March 11 2010
>            Reporter: Zied ABID
>              Labels: build, libtool
>             Fix For: 10.7.1.1, 10.7.1.4, 10.8.2.2, 11.0.0.0
>
>         Attachments: d5125-javacc5-and-tests.diff, javacc5.diff
>
>
> I try to and Apache derby into Debian, so I begin to compile derby
> sources [1], derby sources provide each own libs and javacc is one of
> them. In Debian, javacc version is 5.0 [2].
> After some tries, I'm in a this situation : compilation success with the
> official [3] javacc 4.0  and fail with the 5.0 official one.
> I think that is related with javacc options passed in .jj files 
> (ex.  java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj lines 2-14)
> I don't know much about javacc, so I can't resolve unless modifying 
> options blindly and I'm afraid that isn't enough ...
> Best regards,
> Zied
> [1] http://svn.apache.org/repos/asf/db/derby/code/tags/10.7.1.1/
> [2] http://packages.debian.org/squeeze/javacc
> [3] http://java.net/projects/javacc/downloads

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message