incubator-flex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Smith <gosm...@adobe.com>
Subject RE: Falcon compiler source code / Falcon architecture
Date Fri, 20 Jan 2012 18:34:23 GMT
Here is some other possibly-interesting info about Falcon:

It was designed from the beginning for incremental compilation of a complete workspace, so
it tracks dependencies between compilation units in various projects. (I.e., if you edit a
file, it knows which other files it should recompile.)

It can compile an app against a library project without creating the SWC for that library
on disk.

It tracks where in the source every token/tree node/definition came from, to support operations
like go-to-definition in Flash Builder.

It has the general concept of a "problem", where a problem can end up being considered an
error, a warning, or ignored.

It provides live problem highlighting to Flash Builder. When you pause during editing a file,
the file is compiled and the problems show up as red squiggly underlines.

It tries to continue after almost all problems, so that you don't necessarily have to fix
every problem before you can get a SWF to run.

And these are probably obvious:

It includes code for reading and writing SWFs and SWCs. 

It includes code for reading and writing ActionScript byte code, at the level of individual
instructions.

It includes transcoders for the various types of embedded assets, which read, for example,
a PNG file and convert it to the form required inside a SWF.

- Gordon

-----Original Message-----
From: Gordon Smith 
Sent: Friday, January 20, 2012 10:17 AM
To: flex-dev@incubator.apache.org
Subject: RE: Falcon compiler source code / Falcon architecture

> Does the ANTLR parser use the standard token stream if it's not lexing the source?

For AS, where we use JFlex to do the lexing (for better performance), we implement our own
token stream.

For CSS, where ANTLR is doing the lexing, we use its standard token stream.

> Would there be a way to create a linked list token stream with what the lexer creates?

We don't buffer visited tokens for AS, but it would be straightforward to implement your own
linked list token stream.

> Is there any plan on adding FXG grammar and DOM?

I forgot to mention that Falcon does support compiling FXG. But we didn't innovate much in
this area... we mainly just brought over the FXG code from the old compiler, which you can
take a look at. I believe it uses a SAX parser to parse the XML of the FXG file, and it builds
a custom DOM with classes like RectNode and TextNode which it then uses for semantic analysis
and code generation. One improvement is that the FXG compilation takes place in the same multithreaded
fashion as other Falcon compilation.

- Gordon

-----Original Message-----
From: Michael Schmalle [mailto:mike@teotigraphix.com]
Sent: Thursday, January 19, 2012 4:48 PM
To: flex-dev@incubator.apache.org
Subject: RE: Falcon compiler source code / Falcon architecture

Gordon, when you have time;

Does the ANTLR parser use the standard token stream if it's not lexing the source?

Would there be a way to create a linked list token stream with what the lexer creates?

Is there any plan on adding FXG grammar and DOM (this would be awesome but maybe something
the community could implement)?

Thanks,
Mike


Quoting Gordon Smith <gosmith@adobe.com>:

> ANTLR is used to parse (but not lex) AS, and to both lex and parse CSS.
>
> - Gordon.
>
> -----Original Message-----
> From: Michael Schmalle [mailto:mike@teotigraphix.com]
> Sent: Thursday, January 19, 2012 11:16 AM
> To: flex-dev@incubator.apache.org
> Subject: RE: Falcon compiler source code / Falcon architecture
>
> Gordon this is about the best news we could get.
>
> It's great to see this structure! I can't wait.
>
> What parts are using ANTLR grammar?
>
> Thanks,
> Mike
>
> Quoting Gordon Smith <gosmith@adobe.com>:
>
>> Until we release specs, Javadoc, or code for Falcon, here is a very 
>> brief overview of Falcon's architecture:
>>
>> Falcon is designed from the beginning to support compiling multiple 
>> targets in multiple projects in a workspace (as you have in an IDE 
>> like Flash Builder). By contrast, asc was designed to compile a 
>> single AS file, and then mxmlc was built on top of that.
>>
>> Falcon uses multiple threads to compile multiple files at the same 
>> time. The more cores you have, the faster it goes.
>>
>> Critical data structures such as the symbol table (which stores 
>> information about which classes are known, what methods they have,
>> etc.) are shared across the entire workspace, to minimize memory 
>> usage.
>>
>> Critical data structures are maintained in memory to support both 
>> compilation and IDS code intelligence in an efficient and consistent 
>> way. For example: If you open a file in the IDE, Falcon builds a 
>> syntax tree and symbol table for it to support intelligent editing.
>> Compiling the file requires just one additional code generation step.
>> By contrast, in the current Flash Builder which uses the compiler in 
>> the SDK, Flash Builder builds its own parse trees and symbol tables 
>> to support editing, and then when you compile the compiler in the SDK 
>> builds another set of parse trees and symbol tables. This is slow and 
>> a waste of memory.
>>
>> Falcon understands .as, .mxml, .css, and .properties files.
>>
>> The parse trees for AS consist of nodes from about 100 classes, such 
>> as LiteralNode, BinaryOperatorNode, FunctionNode, ClassNode. The 
>> parse trees for MXML consist of nodes from about 50 classes, such as 
>> MXMLDocumentNode, MXMLInstanceNode, MXMLScriptNode, etc. The 
>> ActionScript-y parts of MXML are represented by AS nodes inside of 
>> MXML nodes.
>>
>> The symbol table consists of objects representing the things your 
>> code defines, such as ClassDefiniition, FunctionDefinition, and 
>> VariableDefinition, arranged into a hierarchy of scope objects.
>>
>> Falcon uses 3rd-party grammars like JFlex, ANTLR, and JBurg to 
>> generate various lexers, parsers, and code generators.
>>
>> MXML is compiled directly to ABC, not to ActionScript source code or 
>> an ActionScript parse tree.
>>
>> - Gordon Smith, Adobe
>>
>> P.S. After working on the Flex framework since its inception for 
>> about
>> 8 years, I joined the Falcon team 18 months ago.
>>
>>
>>
>> -----Original Message-----
>> From: Raju Bitter [mailto:rajubitter@googlemail.com]
>> Sent: Thursday, January 19, 2012 2:10 AM
>> To: flex-dev@incubator.apache.org
>> Subject: Re: Falcon compiler source code / Falcon architecture
>>
>> 2012/1/19 Alex Harui <aharui@adobe.com>:
>>> I was in the Falcon code in November.  It was way easier to figure 
>>> out how to make changes than with MXMLC.  Even if it is not 
>>> documented, I think it will be easier to make progress that with the 
>>> Falcon code base.
>> I agree, that sounds like it would be much better to use the Falcon 
>> code base. There are normally fewer community members with much 
>> experience in language/compiler design, and the cleaner the 
>> architecture is, the better for the community.
>>
>
>
>


Mime
View raw message