flex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christofer Dutz <christofer.d...@c-ware.de>
Subject AW: [Falcon] ASDocTokenizer and RawASDocTokenizer JFlex deps
Date Tue, 19 May 2015 07:23:43 GMT
Well I did manage to convert the ANTLR3 and ANTLR2 parts to an ANTLR4 grammar and I seem to
have been able to parse AS and CSS files with that. The problem was that because the Falcon
Compiler was not compiling my test-projects correcty then, sort of dropped the ball. It's
something different to refactor something that's working cause you can always compare the
new with the old output, but if the initial thing isn't working, that's pretty tricky. I also
worked together with the JBurg guy to get Jburg 2 closer to being finished and wrote a maven
plugin for jburg that actually allowed me to build Falcon with Maven (But that's a personal
crusade of mine ;-) )

The coolest thing is that Antlr4 seems to generate an interface with callbacks that are called
whenever a rule is entered or left, this makes it easier to separate the actual parser code
from the falcon code. And it seems some of the compilcated constructs can be written a lot
easier. Cant't say anything about the performance though ... from my gut feeling it seemed
to be faster than Antlr2 and 3 ... but I would like to confirm that first.

Just have a look at the falcon "falcon-antlr4" branch (https://github.com/apache/flex-falcon/tree/falcon-antlr4)
My stuff is in there ... I was always planning on continuing the work as soon as I could have
a working test-suite.


Von: Michael Schmalle <teotigraphixllc@gmail.com>
Gesendet: Sonntag, 17. Mai 2015 20:29
An: dev@flex.apache.org
Betreff: Re: [Falcon] ASDocTokenizer and RawASDocTokenizer JFlex deps

I actually have a grammar I used for ASDocs years ago, check it out. I was
actually looking at that last night when I hit the JFlex problem.

I think the grammar I created used Antlr 3 but we could get it working with
4 pretty easy I am sure. I am rusty with Antlr right now, been about 2-3
years since I did it. But I did write a pretty decent AS3 grammar for my
asblocks projects.

So you are saying you took the EXISTING as3 grammar from falcon and made it
compatible with antlr4 correct?



On Sun, May 17, 2015 at 1:56 PM, Christofer Dutz <christofer.dutz@c-ware.de>

> Eventually it could be a good Idea to sort of streamline all the parsing
> done in falcon ... I think I created the Antlr4 branch as a first test to
> build falcon with only one parser-generator. Currently we are using 3
> incompatible types of parsers JFlex, Antlr2 and Antl3 ... I was
> successfully able to create an Antlr4 grammar for parsing ActionScript and
> CSS ... so eventually simply adding the ASDoc stuff to the ActionScript
> parser would be the cleanest solution ... what do you think?
> Chris
> -----Ursprüngliche Nachricht-----
> Von: Michael Schmalle [mailto:teotigraphixllc@gmail.com]
> Gesendet: Sonntag, 17. Mai 2015 00:20
> An: dev@flex.apache.org
> Betreff: Re: [Falcon] ASDocTokenizer and RawASDocTokenizer JFlex deps
> No, this has nothing to do with your code. I am using the ASDocTokenizer
> which uses the RawASDocTokenizer that is generated by JFlex.
> I really doubt you have touched that class, all it does is call
> nextToken() on the JFlex scanner. It's the JFlex scanner that is hitting a
> loop while trying to end the scan of the doc comment.
> Mike
> On Sat, May 16, 2015 at 6:01 PM, Alex Harui <aharui@adobe.com> wrote:
> > I don’t remember which version is which, but IIRC, more recent Jflex
> > was licensed under Apache or a more permissive license where older
> > ones weren’t, so we opted to simply the bundling by going to a more
> > recent version.
> >
> > If you are getting a loop it is more likely a bug in our source
> > exposed by some new scenario that wasn’t around two years ago.
> >
> > -Alex
> >
> > On 5/16/15, 2:26 PM, "Michael Schmalle" <teotigraphixllc@gmail.com>
> wrote:
> >
> > >Hi,
> > >
> > >I am going through a lot of my code, one in particular is a new
> > >version of the ASDoc tool that uses velocity templates.
> > >
> > >I know 2 years ago I was able to document the whole Flex SDK with the
> > >current code base.
> > >
> > >I noticed in the compiler downloads.xml the JFlex version was changed
> > >from
> > >1.5.1 to 1.6.0 by Erik.
> > >
> > >Was there a reason to update the version? I know dealing with
> > >parser/lexer/scanner generators it's not a good idea to update
> > >versions unless it's tested.
> > >
> > >Obviously the ASTokenizer that is used with the ASParser still seems
> > >to function correctly but the ASDocTokenizer is running into an
> > >infinite loop when it reaches the end of tokenizing the asdoc IE */.
> > >
> > >I changed the download version back to 1.5.1 and regenerated the
> > >scanner but I am still getting the loop where I had never had this
> problem before.
> > >
> > >I doubt anybody has answers but I was wondering if Alex might have
> > >any ideas to other things that changed in the dependencies.
> > >
> > >As it stands, maybe there is a bug in the grammar, I might check this
> > >out when I have time but it worked before.
> > >
> > >It seems that when Falcon was donated it used 1.5.1, so I am totally
> > >clueless as to what changed and why the infinite loop in ALL scanning.
> > >
> > >Mike
> >
> >
View raw message