incubator-lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: Charmonizer
Date Sat, 08 Jul 2006 19:04:31 GMT

On Jul 7, 2006, at 7:16 PM, David Balmain wrote:

> This is just an idea, but it might be worth doing away with the DSL
> idea and just doing it all in C

I'd thought about that.  And thanks for your implementation, which is  
nicely done.  It has the advantage of a simple main program, to be  
sure.  The downsides are A) it's hard to get your head around writing  
C code which writes C code which writes C code (argh, *when* does  
that macro get expanded?), and B) it's kind of noisy, because  
embedding C source code in printf's and C quoted strings involves a  
lot of escaping.  Streamlining the main app is a Good Thing, but I  
think using pure C will make module-writing harder and more error-prone.

One way we can limit ourselves to a single layer of indirection is to  
make the .charm files be data files, with delimited snippets of C  
code.  The Charmonizer application would then process the .charm file  
record by record.  In fact, that was the original idea (proposed as  

However, some of the routines in these modules are going to be very  
repetitive -- if you were to spelunk Perl's Configure script, you'd  
see what I mean.  We'll want to abstract out some functions so that  
we obey the principle of DRY.  OK, then, how do we indicate what kind  
of record we have, and thus what function should be used to process  
it?  We add another field which holds the name of the function.

At this point, our data file is starting to look like a program.  But  
one with awkward syntax.  Sorta like programming in CSV:

    "printf", "Hello, %s!\n", "\"world\""

What's more, we still need a parser for that data file.

 From this point, it's a short, nimble leap to your excellent  
suggestion of a DSL.  We expend some energy up-front writing and  
testing a slightly more sophisticated parser, and then programming,  
debugging, and maintaining the modules gets easier.

My current goal with Charmonizer is to define the language API in as  
forward-compatible a way as possible.  I'm happy with everything  
except the quoting mechanism.

It occurred to me that it wouldn't be back-compatible later to add in  
support for escapes in a string surrounded by single quotes, because  
this would be treated differently:

   'a backslash: \'

So, I think I'll add interpolation of backslashes and single quotes,  
so that Charmonizer's singly-quoted strings behave like those in Perl  
and Ruby.  That's not that hard.

The other thing I'm not perfectly happy with is the CH_q operator.   
Here's my idea for a replacement:

    /* "\nMarvin's literal text\n" */
Marvin's literal text

Sort of a combination of here docs and the q operator in Perl (%q  
operator in Ruby).  We could also add support for here docs, but here  
docs are fragile and quirky.  The one advantage here docs have over  
this construct, other than familiarity, is that they automatically  
begin on the next line, which makes things slightly cleaner if you  
don't want to begin with a newline:

/* Charmonizer string equivalent to "\nSalutations, earthlings!\n" */
Salutations, earthlings!

# Perl here doc equivalent to "Salutations, earthlings!\n"
Salutations, earthlings!

/* Charmonizer string equivalent to "Salutations, earthlings!\n" */
q'DELIMITER'Salutations, earthlings!

However, 1) I prefer the conceptual consistency and robustness of the  
modified q operator, 2) we certainly won't care about the extra  
newline for the C fragments we're writing, 3) I think that in  
general, the extra newline would rarely turn out to be a problem, 4)  
when it does turn out to be a problem, the solution looks a little  
funny but is easy to understand, and 5) here docs and this operator  
are equally lousy for indenting (but again, it doesn't matter for us).

So, off to finish the parser...

Marvin Humphrey
Rectangular Research

View raw message