just curious

[Stephen Weeks]

> Lex tables makes sense.  I'm surprised you are thinking of using your regexp
> library for the lexer. 

I am confused.  I thought that lex table = DFA table (i.e. the same kind of
table I am using in my regexp library).  I thought I was just proposing using my
regexp's tables instead of the mllex ones.

> When ever I have worked on things like this I always
> wrote the lexer by hand.  It was faster, smaller and just as easy.  The point
> being that the lexical structure of languages was rather simple and ad-hoc.
> Even Mathematica, which was pretty goofy.  The C lexxer I wrote was no larger
> than the lex lexer it replaced, way faster and more readable.
> I don't think that regular expressions, although sufficient, are really needed
> for this kind of stuff.  I don't know about ML's, lexical structure, but I
> wouldn't think that it would be any different.

I agree it might be better to write by hand.  If you're interested to try, take
a look at src/mlton/front-end/ml.lex.  I'd be interested to see how you'd do
it.  I have found that code tends to get quite a bit larger than the regexps,
but maybe I haven't found the right abstractions.  I'd be interested to see some
of your code.