It's nice being able to do this now.
This starts to pave the path toward ultimately sharing code with Ulambda.
* build-aux/bootstrap/rebirth.scm: Extract steps 0--2 into separate source
files.
* build-aux/bootstrap/rebirth/es.scm: New file containing step 2.
* build-aux/bootstrap/rebirth/macro.scm: New file containing step 1.
* build-aux/bootstrap/rebirth/relibprebirth.scm: New file contaiing step 0.
This is technically not a correct implementation---it doesn't wrap in
`begin'. But that's just because we don't have a `begin' yet, and it's not
yet necessary.
This will allow us to begin breaking the mess that is rebirth.scm into
separate files, and does so cheaply (not much development effort, that is).
* build-aux/bootstrap/rebirth.scm (expand-cond-expand): Recognize "include".
(fnmap): Add "include".
(rebirth->ecmascript): Do not wrap compiled ECMAScript in self-executing
function (see below).
(rebirth->ecmascript-prog): Wrap compiled ECMAScript in self-executing
function, as `rebirth->ecmascript' was. Use it at end of script.
Note that this doesn't yet provide any useful abstractions for creating
discrete environments---it merely provides support for them moving
forward. Hopefully.
This is the next big step toward rewriting the compiler as a series of macro
passes. I'll write more on this later; it has profound consequences (well,
as a jump from Rebirth into Ulambda, which is a name that I haven't
mentioned until now).
It was yesterday that I also got word that I'll be speaking at
LibrePlanet 2018 in March. This is great news, but unfortunate news for
this project---it has had very little of time time to begin with, and now
it's going to have even less until after the conference.
* build-aux/bootstrap/rebirth.scm: Add toplevel note about Ulambda.
Add notes to "Step 2" regarding environments.
(es:envf, %es:env): Add macros.
(lambda, let*, let, set!): Macro forms use environments.
(%es:has-own-prop, %es:proto-of, %es:envobj-for, %es:setenv): New
procedures for `set!' macro.
(es:null?): Add procedure.
(_macros): Remove ES global (now using `_env.macros').
(cdfn-macro, macro?): Use `_env.macros' in place of `macros'.
(tparam->es, tname-verbatim?, env-ref, env-params): Add procedures.
(tname->id): Use `tname-verbatim?', which extracts the digit check.
(cdfn-var, cdfn-proc): Also assign to current environment.
(apply-proc-or-macro): Use `_env.macros'. Apply function from
environment.
(sexp->es)[symbol]: Generate environment reference.
(rebirth->ecmascript): Generate toplevel environment.
This removes the need to use the ES `arguments' reference. Not only is this
what we want to do, but it's _necessary_---the next commit will introduce
environments, and wrapping procedures in lambdas breaks the `arguments'
reference in each of these cases.
Note that fold and map can now be written in Rebirth Lisp, but I'm just
leaving them alone for now.
* build-aux/bootstrap/rebirth.scm
(append, string-append, +, -, map): Use define cdr in place of ES
`arguments'.
This is a significant step toward abandoning the Rebirth compiler (which is
a slight, mostly unaltered version of the Birth compiler, at this point);
anything written for these macros from this point on can be re-used moving
forward, regardless of what compiler we have underneath it. I'll continue
writing more Scheme-like abstractions moving forward to begin to normalize
the syntax as I get closer to a point where it's worth starting to create
proper R⁷RS implementations.
Things are moving along slowly, but they're moving. I don't have a whole
lot of free time between kids and other obligations.
* build-aux/bootstrap/birth.scm (fnmap): Doc corrections.
* build-aux/bootstrap/rebirth.scm: Add more documentation.
Add numerous macros to replace built-in forms; I'm not listing them
here.
(%list-item): Whitespace fix.
(fnmap): Split into `fnmap-premacro'.
(fnmap-premacro): New procedure.
There were some instances where fnmap forms defined themselves in terms of
themselves (e.g. `if' using `if'), or defined themselves in terms of one of
their dependents (`let*' used `let' which uses `let*').
They work just fine because of how we're just transpiling directly into JS,
but we're converting them into macros, and we're going to run into issues
once we do that.
This was never intentional---I didn't realize that I was doing it.
* build-aux/bootstrap/rebirth.scm (fnmap)[if, let*]: Fix definitions.
I've been torn on this for a bit, but we're using "es" (ECMAScript) in a lot
of important places. Thinks like "js:console" I was considering leaving
because "console" is not an ECMAScript thing---it is JavaScript. But now we
will take "es:" to mean "outputting in ECMAScript".
* build-aux/bootstrap/birth.scm, build-aux/bootstrap/rebirth.scm,
build-aux/bootstrap/prebirth.js:
s/js:/es:/g.
* build-aux/bootstrap/libprebirth.js: s/\$\$js\$/\$\$es\$/g.
This provides preliminary support for traditional Lisp macros, _not_ Scheme
macros as we know them today; this implementation is easy to implement, and
gives us a great foundation for moving forward.
There are caveats to this simple and naive implementation, documented
ad nauseam. It's probably worth a read if you're studying Rebirth in any
level of detail for whatever reason.
This is a pretty exciting change---it liberates us from rigid compiler
changes and will allow us to rewrite fnmap into macros almost as-is. The
change is fairly elegant, all things considered---the amount of code is
minimal; most of the change consists of comments describing it and its
caveats. This is a defining step in Rebirth, and brings it a step closer to
being an actual Lisp rather than a fragment of one. (Though I still don't
know if Rebirth Lisp will ever actually be a full Lisp. Please hold the
arguments about Scheme not being a Lisp or I'll respond very immaturely with
"your mom is not a Lisp", and we'll both be very confused and somewhat
offended by one-another, with a net loss overall. Oh, wait, you're actually
reading this?)
Anyway, changes:
* build-aux/bootstrap/rebirth.scm: Some rephrasing of toplevel comments,
and addition of macro comments.
(_macros): New ECMAScript variable.
(cdfn-macro, macro-compile-result, list->ast): New procedures. Little
full of comments. Lots of sap.
(parse-lisp): Fix typo. Add third argument to `cdfn-proc' (#f).
(cdfn-proc): Add third argument `id-override'. Use it in place of token
value, if set. Remove semicolon from generated ES function (it was
unneeded to begin with) so that it can be used in macro ES expressions.
(macro?): New procedure, conditional based on availability of
`string->es'.
(apply-proc-or-macro): New procedure. Conditionally apply macro during
compiler runtime or compile output for a procedure application.
(fnmap)[define-macro]: Apply `cdfn-macro'. This will try to apply it even
if the procedure isn't defined yet (e.g. first Rebirth pass), so don't
call it until then!
[else]: Use `apply-proc-or-macro'.
This will crudely detect "features" by seeing if the requested feature is a
definition. Ideally that definition would be a procedure, but we don't
check for that. This does what we need it to do.
Obviously this is a poor implementation and will not persist past Rebirth.
* build-aux/bootstrap/rebirth.scm (es:defined?): New procedure.
(expand-cond-expand): New procedure. If first Rebirth pass, only support
`string->es' and `else'; otherwise detect features using `es:defined?'.
(fnmap)[cond-expand]: Use it.
Since we don't have reader support for ",@", this will have to do for now.
* build-aux/bootstrap/rebirth.scm (%quote-maybe): Add `unquote@'.
Rename from `-quote-maybe'.
(%sexp-maybe-type): Rename from `-sexp-maybe-type'.
Birth is pretty much at a feature freeze at this point, but `cond-expand'
really was difficult to work with in certain circumstances when it didn't
even support `else'.
* build-aux/bootstrap/birth.scm (fnmap)[cond-expand]: Always expand `else'
while dropping all others.
They were just out of date, having been copied from Prebirth.
* build-aux/bootstrap/birth.scm
(prebirth->ecmascript, birth>ecmascript): Former renamed to latter.
* build-aux/bootstrap/rebirth.scm
(prebirth->ecmascript, rebirth>ecmascript): Former renamed to latter.
Many of the procedures were encapsulated within `prebirth->ecmascript';
these have been moved out so that they can be accessed by other procedures,
allowing me to organize the code how I please.
Which also makes me realize that the procedure name is incorrect.
* build-aux/bootstrap/rebirth.scm: Move all procedures out of
`prebirth->ecmascript'.
This was pretty much a prerequisite for introducing macros (in commits to
follow)---the code would otherwise be far too verbose.
Note that this does _not_ introduce the shorthand forms ("'", "`", ",",
",@"), as implementing those in a reasonable manner would require
preprocesing the AST, which is not what I want to focus on right now. So
macros will still be a bit verbose, using the full
"(quasiquote ... (unquote-splicing ...))" and such instead of simply
"`(... ,@(...))"
* build-aux/bootstrap/rebirth.scm
(quote-sexp, quasiquote-sexp): Add procedures.
(fnmap)[quote]: Use `quote-sexp'.
[quasiquote]: Add special form.
This re-implements libprebirth in Rebirth Lisp, finally cutting the
cord. (Are these birth puns getting out of control?) We are finally purely
in Lisp land!
* build-aux/bootstrap/rebirth.scm: Define libprebith primitives when
`string->es' is available (using `cond-expand').
(prebirth->ecmascript): Do not include `libprebirth.js' in output.
This modifies each of the three lexers just to avoid confusion, even
though prebirth has no need for it. Birth does have a need, however, since
rebirth will contain newlines---despite the relevant rebirth code (next
commit) not being expanded, it still has to go through the lexer, which
otherwise errors out.
* build-aux/bootstrap/birth.scm (lex): Permit newlines in string regex.
* build-aux/bootstrap/prebirth.js (lex): Permit newlines in string regex.
* build-aux/bootstrap/rebirth.scm (lex): Permit newlines in string regex.
`cond-expand' allows moving forward with implementing features based on the
current bootstrapped state of the system---rebirth will be able to
recursively compile itself and introduce new features along the way.
`string->es' allows outputting raw ECMAScript, which gives us more control
over the code that is generated without having to hard-code it in the
compiler itself.
* build-aux/bootstrap/birth.scm
(fnmap)[cond-expand]: Always yield the empty string (do nothing).
* build-aux/bootstrap/rebirth.scm
(fnmap)[cond-expand]: Expand `string->es' only.
(fnmap)[string->es]: Add macro.
`define' can now be used to define values in addition to the procedure
short-hand.
* build-aux/bootstrap/rebirth.scm
(cdfn): New procedure. Renamed original `cdfn' to `cdfn-proc'.
(cdfn-var): New procedure.
(cdfn-proc): Renamed from `cdfn'.
This allows bootstrapping in either a development environment (Node.js) or
simply using the intended runtime environment: the user's browser.
* bootstrap.html: Add file (browser).
* bootstrap.js: Add file (command-line).
* bootstrap/Bootstrap.js: Add class. Formalize bootstrap process.
* bootstrap/libprebirth.js
(fsdata): Add variable to serve as filesystem stub.
(fs): Always throw error when `fs' module is unavailable.
($$js$file$_$$g$string): Consider `fsdata'.
* bootstrap/prebirth.js: Export as CommonJS module if in proper
environment. Abort automatic processing via stdin if root CommonJS
module.
This provides the complete compiler output, which is consistent with the
output of Birth. This will be useful for browser-based bootstrapping.
* build-aux/bootstrap/prebirth.js: Extract code into `Prebirth' class and
use the class.
(Prebirth): Add class.
This needs to run in the browser too, where we have no control over stack
limits.
* build-aux/bootstrap/birth.scm
(lex): Non-recursive strategy (loop with mutable list).
(make-token): Update doc. Produce list of token, new string, and
position. Don't recurse.
(body->es): Add `ret' param. Only produce `return' statement if new param
is set.
(cdfn): Use it.
(fnmap)
[js:while, js:break]: Add forms.
[lambda, let, case]: Use new `body->es' `ret' param.
[let*]: Define JS variables in output using `let' instead of `const' to
permit mutating with new `set!' form. Use new `body->es' `ret' param.
[set!]: Add form.
(prebirth->ecmascript): Adjust libprebirth path to be relative to self.
* build-aux/bootstrap/libprebirth.js
($$append$b$): Add `append!' procedure.
($$js$regexp, $$js$match, $$js$replace): Move a few lines up.
(fs): Provide stub if `require' is not defined.
* build-aux/bootstrap/prebirth.js
(_lex): Non-recursive strategy (loop with array-appending).
(_token): No more mutual recursion with `#_lex'. Return new string
and position.
(_bodyToEs): Add `ret' param. Only produce `return' statement if new
param is set.
(fnmap) [js:while, js:break]: Add forms.
[let*]: Define JS variables in output using `let' instead of `const' to
permit mutating with new `set!' form. Use new `body->es' `ret' param.
[set!]: Add form.
This completes bootstrapping for Prebirth Lisp. The next step will be
Rebirth, which will replace libprebirth.js, removing hand-written JavaScript
entirely.
* build-aux/bootstrap/prebirth.js
(_cdfn): Remove second argument to `#_idFromName'.
(_idFromName): Remove second parameter `global'. Identify and echo
integers. Remove distinction between global and non-global
identifiers---process everything.
Just about ready for that sloppy code generation!
* build-aux/bootstrap/birth.scm: Update file header documentation.
Add some whitespace between existing procedures.
Invoke `parse-lisp' as the program in place of `lex', producing an AST as
output to the console.
(cadddr): Add procedure.
(token-{type,lexeme,value,pos}): Add procedures.
(parse-lisp): Add procedure (contains other procedures).
* build-aux/bootstrap/libprebirth.js
($$append): Add function (append).
($$$_$): Correct implementation (-).
($$zero$7$): Add predicate (zero?).
($$fold): Add function (fold).
* build-aux/bootstrap/prebirth.js (parseLisp): Lowercase some errors.
(Compiler): Update class docblock.
(fnmap)[labmda]: Add `lambda' form.
This just reorganizes things slightly to allow for nested `define's. This
is a common means of encapsulation, and will Just Work™ because JS has
pretty much identical block scoping rules in this regard.
* build-aux/bootstrap/prebirth.js (compile): Invoke `#_sexpsToEs' directly
instead of `#_cdfn' on tree.
(_cdfn): Remove non-`define' check to proxy to `#_sexpsToEs'.
(_sexpsToEs): Treat `define' as a special case, invoking `#_cdfn'.
Exciting first step! Though it required a much more complicated Prebirth
Lisp than I was hoping to create. And I never intended to go into a full
Scheme implementation, but that's the route this is headed in. I just can't
stomach creating this full system in a block language.
With that said, the block language will still be able to work with all Lisp
code; you'll see.
* build-aux/bootstrap/birth.scm: Add beginning of Birth, capable of parsing
itself! Baby steps!
Already implemented by `case', `or', `and'.
* build-aux/bootstrap/prebirth.js (fnmap)[if]: Use proper truth check, as
expected by RnRS Scheme. For JS, this means only `false' is non-true.
So much for simple. But it's not worth my suffering to not add them.
* build-aux/bootstrap/prebirth.js (fnmap): Add more information to docblock.
(and, or, case): Add special forms.
Empty string was not matching, which was causing its value to be '""', which
expanded (in JS) into the compiled string '""""'. Oops.
* build-aux/bootstrap/prebirth.js (Parser#_token): Distinguish empty values
from non-matches.
The Prebirth JS compiler is getting much more sophisticated than I had
hoped, but I need something to work with here. This starts to get us
in a good spot.
There are other conveniences I will want. More to come.
* build-aux/bootstrap/prebirth.js
(Compiler#constructor): Add method, accept fnmap.
(Compiler#_sexpToEs): Use fnmap as needed.
(fnmap): Add initial map.
Instantiate `Compiler' with fnmap.
This was implicit in the `define-block' implementation because of the
conversion from `<foo>' to `$foo$', but that's no longer the case.
This is just a simple, temporary implementation, so don't be alarmed.
* build-aux/bootstrap/libprebirth.js: Updated header documentation to
mention `$$' prefix.
($$js$console): Renamed from `js$console'.
* build-aux/bootstrap/prebirth.js
(_idFromName): Add `global' parameter to add `$$' prefix to generated
identifiers.
(_cdfn, _sexpToEs): Use it.
So we can invoke the main function for the program.
* build-aux/bootstrap/prebirth.js
(Compiler#_cdfn): Handle non-`define' applications.
(Compiler#assertApply): Remove function.
Turns out, I'll kill myself before writing a Prebirth compiler in a
block-define-based Prebirth Lisp. So, let's degrade even further into a
primitive Scheme. This is going down a dangerous path to simply
implementing Scheme...
Nonetheless, here I remove `define-block' in favor of a simple shorthand
function definition `define', as is custom in Scheme. We will worry
about block definitions later as metadata mapping to normal functions.
Mike Gerwitz