These changes have been sitting here uncommitted for a almost a couple of
years now. It's become clear that, while this project deeply interests me,
I do not have time for it. Implementing a full-fledged Scheme is simply not
something I have time to do.
The original goal of this project is also something I'm still interested in,
but will not have time to do any time soon. But it has been folded into
broader, more grand ideas.
I hope that this will, at the very least, be interested to someone else. It
is terribly lacking in certain cases, e.g. it does not even implement pairs
properly, because I never needed to.
See also https://github.com/google/schism, which compiles to WASM.
Happy hacking!
This is consistent with Guile and can be parsed with `(ice-9 documentation)'.
* bootstrap/birth.scm: Update header.
* bootstrap/rebirth.scm: Likewise.
* bootstrap/rebirth/es.scm: Likewise.
* bootstrap/rebirth/macro.scm: Likewise.
* bootstrap/rebirth/relibprebirth.scm: Likewise.
This is an initial test suite that is by no means comprehensive, but is a
pretty good starting point. At this point, it's important that we be
confident in certain aspects of the system to preempt difficulty in
debugging problems when using 3rd-party code (we'll be using psyntax at some
point, for example).
We also need to be able to have confidence in refactoring. We do gain
limited confidence when Rebirth is able to successfully compile itself, but
finer details are missed, as has been demonstrated by recent commits (which
fix bugs found by this test suite).
This is also the first test of writing another program in Rebirth that isn't
Rebirth itself. It's exciting, and helps make obvious certain pain points,
like the need to include core packages. Further, we can't include important
definitions in `rebith.scm' itself. Consequently, I think it may be in my
best interest to just add support to Birth for `include' so that the first
pass of Rebirth can benefit from it. I didn't want to do so because I
wanted Birth to be a faithful re-implementation of Prebirth, but it's not
worth it. This would be a minor addition and would save so much trouble.
I saw a couple mentions of JS Lisps today on HN (one a tiny Lisp that
compiles to JS, one an interpreter)---neither of them come close to what
Ulambda will be (and in many ways with Rebirth already is), so I'm excited
to keep development going. There is a niche to be filled. And I intend to
keep this project secret until it can actually be called "Scheme" (and maybe
beyond that too, since I have Gibble to work on as well).
Anyway, enjoy the first non-Rebirth Rebirth program! Hello, world!
* bootstrap/Bootstrap.js (bootstrap): Include `#_testViable' call after
Rebirth compilation.
(_birth): Throw any error received during Birth compilation.
(_makeCompiler): Accumulate output for each `console.log' in compiler,
rather than overwriting with each call (that latter behavior was never
intended in the first place).
(_compileRebirth): Throw error on compilation failure.
(_testViable): New method.
(_strmap)[rebirthTestCompiling, rebirthTestCompiled, rebirthTestFailed]
[rebirthTestDone]: New functions.
* bootstrap/rebirth.scm: Add comment clarifying that Rebirth should _not_ be
used as a general-purpose little Lisp.
* bootstrap/rebirth/.dir-locals.el: New file.
* bootstrap/rebirth/es.scm: Add comment comparing environment implementation
to a spaghetti stack (a term which I was unfamiliar with until recently
while researching some Lisp history).
(es:json-stringify): New procedure.
(es:try): New macro.
* bootstrap/rebirth/relibprebirth.scm (es:null): New procedure for use by
test suite.
(fold): Fix argument naming order.
(zero?): Correctly use `=' instead of `eq?'.
* bootstrap/rebirth/test.scm: New viability test suite.
Some of them snuck through.
Gibble is still a goal. It really is fascinating how projects like these
evolve...as a complete distraction from my original goals.
* bootstrap/birth.scm: s/GIBBLE/REBIRTH/.
* bootstrap/libprebirth.js: Likewise. Also s/Gibble/Ulambda/.
This provides a very basic Automake+Autoconf configuration. At the moment
it handles only bootstrapping, but at least allows for the user to enter
`make' and have something useful happen.
* .gitignore: New file.
* Makefile.am: New file.
* autogen.sh: New file.
* bootstrap/Bootstrap.js (_error): Re-throw error.
* bootstrap/bootstrap.js: Exit with non-zero status code on failure.
* configure.ac: New file.
NOTE: To see history for files in this directory, run:
$ git log --follow PATH
The previous home for the bootstrapping process was build-aux, but this has
evolved into quite the project in its own that such a directory strucutre is
no longer honest. Further, I want the bootstrap process to get the
visibility in the source tree that it deserves, considering how vitally
important it is.
I preserved the original history rather than rewriting the commits because I
did not want the hashes to change---I have recorded some hashes elsewhere
as proof that early Ulambda existed when it did, since it is yet unpublished.
* bootstrap/Bootstrap.js: Renamed from build-aux/bootstrap/Bootstrap.js.
* bootstrap/birth.scm: Renamed from build-aux/bootstrap/birth.scm.
* bootstrap/bootstrap.html: Renamed from build-aux/bootstrap.html.
* bootstrap/bootstrap.js: Renamed from build-aux/bootstrap.js.
* bootstrap/libprebirth.js: Renamed from build-aux/bootstrap/libprebirth.js.
* bootstrap/prebirth.js: Renamed from build-aux/bootstrap/prebirth.js.
* bootstrap/rebirth.scm: Renamed from build-aux/bootstrap/rebirth.scm.
* bootstrap/rebirth/es.scm: Renamed from build-aux/bootstrap/rebirth/es.scm.
* bootstrap/rebirth/macro.scm: Renamed from build-aux/bootstrap/rebirth/macro.scm.
* bootstrap/rebirth/relibprebirth.scm: Renamed from
build-aux/bootstrap/rebirth/relibprebirth.scm.
* build-aux/bootstrap/rebirth/relibprebirth.scm (<, >): Reverse
comparisons. This is quite the embarassing mess-up. It appears that it
was only used in one place, and so I did not notice the problem until I
began writing test cases.
* build-aux/bootstrap/rebirth/es.scm (cdfn-proc): Add missing semicolon.
This was causing serious problems when the folloiwng line introduced
lexical scope, since that encloses the scope in a self-executing function,
but the opening parenthesis were interpreted as invoking the code
preceding it.
This bootstraps up to the final Rebirth generation. I had been doing this
manually from the command line for far too long.
This also includes an ASCII Ulambda logo.
* README.md: Add Ulabmda logo
* build-aux/bootstrap/Bootstrap.js:
(bootstrap): Extract majority of existing code into separate methods.
Add invocation of `_rebirth'.
(_birth): Extracted from `bootstrap'. Extracted compiler function
creation into separate method `_makeCompiler'. Extract self-
verification into `_verifyBirthOutput'.
(_verifyBirthOutput, _makeCompiler): Extracted from `_birth'/`bootstrap'.
(_rebirth, _compileRebirth): New methods.
(_loadPath)[desc]: Default param value to empty string.
(_strmap)[header]: Change logo.
[loadingf]: Account for optional empty `desc'.
[prebirthDesc, birthCompiled, birthVerifyOk]: Prefix with "+".
[prebirthComplete]: Remove prefix "[prebirth]".
[birthSelfCompiling, birthVerify]: Remove prefix "[birth]".
[rebirthCompiling, rebirthCompiled, rebirthDone]: New keys.
[ok]: Update to indicate that bootstrapping isn't complete yet.
(_supmap): New field.
See code comments. This is _not_ a long-term solution; we just need
something working in the meantime to get out of this repass cycle we're in.
* build-aux/bootstrap/rebirth.scm (cdfn-proc): Assign proc to env root.
(es:empty-env): Add `root' property to reference self. This allows
easy access to the root of the prototype chain without traversing.
Gibble is still the ultimate goal, but since I seem to have gone down
precisely the path that I did _not_ want to go down (implementing a full
Scheme), this deserves to be its own project.
Oh, and Gibble is the "{General=>Graphical} Block-Based Learning
Environment".
* build-aux/bootstrap/rebirth.scm: Provide env to `rebirth->ecmascript-prog'.
(rebirth->ecmascript-prog)[env-es]: New param. Use for `_env' in
generated self-executing ES function.
(es:empty-env): New procedure.
* build-aux/bootstrap/rebirth/es.scm (es:raw): Remove unused macro.
(es:envf)[env]: New parameter. Use in place of hard-coded inherited env.
(es:inherit-env): New procedure, extracted from proceeding.
(lambda, let): Use it.
(null-environment, eval): New procedures.
It's nice being able to do this now.
This starts to pave the path toward ultimately sharing code with Ulambda.
* build-aux/bootstrap/rebirth.scm: Extract steps 0--2 into separate source
files.
* build-aux/bootstrap/rebirth/es.scm: New file containing step 2.
* build-aux/bootstrap/rebirth/macro.scm: New file containing step 1.
* build-aux/bootstrap/rebirth/relibprebirth.scm: New file contaiing step 0.
This is technically not a correct implementation---it doesn't wrap in
`begin'. But that's just because we don't have a `begin' yet, and it's not
yet necessary.
This will allow us to begin breaking the mess that is rebirth.scm into
separate files, and does so cheaply (not much development effort, that is).
* build-aux/bootstrap/rebirth.scm (expand-cond-expand): Recognize "include".
(fnmap): Add "include".
(rebirth->ecmascript): Do not wrap compiled ECMAScript in self-executing
function (see below).
(rebirth->ecmascript-prog): Wrap compiled ECMAScript in self-executing
function, as `rebirth->ecmascript' was. Use it at end of script.
Note that this doesn't yet provide any useful abstractions for creating
discrete environments---it merely provides support for them moving
forward. Hopefully.
This is the next big step toward rewriting the compiler as a series of macro
passes. I'll write more on this later; it has profound consequences (well,
as a jump from Rebirth into Ulambda, which is a name that I haven't
mentioned until now).
It was yesterday that I also got word that I'll be speaking at
LibrePlanet 2018 in March. This is great news, but unfortunate news for
this project---it has had very little of time time to begin with, and now
it's going to have even less until after the conference.
* build-aux/bootstrap/rebirth.scm: Add toplevel note about Ulambda.
Add notes to "Step 2" regarding environments.
(es:envf, %es:env): Add macros.
(lambda, let*, let, set!): Macro forms use environments.
(%es:has-own-prop, %es:proto-of, %es:envobj-for, %es:setenv): New
procedures for `set!' macro.
(es:null?): Add procedure.
(_macros): Remove ES global (now using `_env.macros').
(cdfn-macro, macro?): Use `_env.macros' in place of `macros'.
(tparam->es, tname-verbatim?, env-ref, env-params): Add procedures.
(tname->id): Use `tname-verbatim?', which extracts the digit check.
(cdfn-var, cdfn-proc): Also assign to current environment.
(apply-proc-or-macro): Use `_env.macros'. Apply function from
environment.
(sexp->es)[symbol]: Generate environment reference.
(rebirth->ecmascript): Generate toplevel environment.
This removes the need to use the ES `arguments' reference. Not only is this
what we want to do, but it's _necessary_---the next commit will introduce
environments, and wrapping procedures in lambdas breaks the `arguments'
reference in each of these cases.
Note that fold and map can now be written in Rebirth Lisp, but I'm just
leaving them alone for now.
* build-aux/bootstrap/rebirth.scm
(append, string-append, +, -, map): Use define cdr in place of ES
`arguments'.
This is a significant step toward abandoning the Rebirth compiler (which is
a slight, mostly unaltered version of the Birth compiler, at this point);
anything written for these macros from this point on can be re-used moving
forward, regardless of what compiler we have underneath it. I'll continue
writing more Scheme-like abstractions moving forward to begin to normalize
the syntax as I get closer to a point where it's worth starting to create
proper R⁷RS implementations.
Things are moving along slowly, but they're moving. I don't have a whole
lot of free time between kids and other obligations.
* build-aux/bootstrap/birth.scm (fnmap): Doc corrections.
* build-aux/bootstrap/rebirth.scm: Add more documentation.
Add numerous macros to replace built-in forms; I'm not listing them
here.
(%list-item): Whitespace fix.
(fnmap): Split into `fnmap-premacro'.
(fnmap-premacro): New procedure.
There were some instances where fnmap forms defined themselves in terms of
themselves (e.g. `if' using `if'), or defined themselves in terms of one of
their dependents (`let*' used `let' which uses `let*').
They work just fine because of how we're just transpiling directly into JS,
but we're converting them into macros, and we're going to run into issues
once we do that.
This was never intentional---I didn't realize that I was doing it.
* build-aux/bootstrap/rebirth.scm (fnmap)[if, let*]: Fix definitions.
I've been torn on this for a bit, but we're using "es" (ECMAScript) in a lot
of important places. Thinks like "js:console" I was considering leaving
because "console" is not an ECMAScript thing---it is JavaScript. But now we
will take "es:" to mean "outputting in ECMAScript".
* build-aux/bootstrap/birth.scm, build-aux/bootstrap/rebirth.scm,
build-aux/bootstrap/prebirth.js:
s/js:/es:/g.
* build-aux/bootstrap/libprebirth.js: s/\$\$js\$/\$\$es\$/g.
This provides preliminary support for traditional Lisp macros, _not_ Scheme
macros as we know them today; this implementation is easy to implement, and
gives us a great foundation for moving forward.
There are caveats to this simple and naive implementation, documented
ad nauseam. It's probably worth a read if you're studying Rebirth in any
level of detail for whatever reason.
This is a pretty exciting change---it liberates us from rigid compiler
changes and will allow us to rewrite fnmap into macros almost as-is. The
change is fairly elegant, all things considered---the amount of code is
minimal; most of the change consists of comments describing it and its
caveats. This is a defining step in Rebirth, and brings it a step closer to
being an actual Lisp rather than a fragment of one. (Though I still don't
know if Rebirth Lisp will ever actually be a full Lisp. Please hold the
arguments about Scheme not being a Lisp or I'll respond very immaturely with
"your mom is not a Lisp", and we'll both be very confused and somewhat
offended by one-another, with a net loss overall. Oh, wait, you're actually
reading this?)
Anyway, changes:
* build-aux/bootstrap/rebirth.scm: Some rephrasing of toplevel comments,
and addition of macro comments.
(_macros): New ECMAScript variable.
(cdfn-macro, macro-compile-result, list->ast): New procedures. Little
full of comments. Lots of sap.
(parse-lisp): Fix typo. Add third argument to `cdfn-proc' (#f).
(cdfn-proc): Add third argument `id-override'. Use it in place of token
value, if set. Remove semicolon from generated ES function (it was
unneeded to begin with) so that it can be used in macro ES expressions.
(macro?): New procedure, conditional based on availability of
`string->es'.
(apply-proc-or-macro): New procedure. Conditionally apply macro during
compiler runtime or compile output for a procedure application.
(fnmap)[define-macro]: Apply `cdfn-macro'. This will try to apply it even
if the procedure isn't defined yet (e.g. first Rebirth pass), so don't
call it until then!
[else]: Use `apply-proc-or-macro'.
This will crudely detect "features" by seeing if the requested feature is a
definition. Ideally that definition would be a procedure, but we don't
check for that. This does what we need it to do.
Obviously this is a poor implementation and will not persist past Rebirth.
* build-aux/bootstrap/rebirth.scm (es:defined?): New procedure.
(expand-cond-expand): New procedure. If first Rebirth pass, only support
`string->es' and `else'; otherwise detect features using `es:defined?'.
(fnmap)[cond-expand]: Use it.
Since we don't have reader support for ",@", this will have to do for now.
* build-aux/bootstrap/rebirth.scm (%quote-maybe): Add `unquote@'.
Rename from `-quote-maybe'.
(%sexp-maybe-type): Rename from `-sexp-maybe-type'.
Birth is pretty much at a feature freeze at this point, but `cond-expand'
really was difficult to work with in certain circumstances when it didn't
even support `else'.
* build-aux/bootstrap/birth.scm (fnmap)[cond-expand]: Always expand `else'
while dropping all others.
They were just out of date, having been copied from Prebirth.
* build-aux/bootstrap/birth.scm
(prebirth->ecmascript, birth>ecmascript): Former renamed to latter.
* build-aux/bootstrap/rebirth.scm
(prebirth->ecmascript, rebirth>ecmascript): Former renamed to latter.
Many of the procedures were encapsulated within `prebirth->ecmascript';
these have been moved out so that they can be accessed by other procedures,
allowing me to organize the code how I please.
Which also makes me realize that the procedure name is incorrect.
* build-aux/bootstrap/rebirth.scm: Move all procedures out of
`prebirth->ecmascript'.
This was pretty much a prerequisite for introducing macros (in commits to
follow)---the code would otherwise be far too verbose.
Note that this does _not_ introduce the shorthand forms ("'", "`", ",",
",@"), as implementing those in a reasonable manner would require
preprocesing the AST, which is not what I want to focus on right now. So
macros will still be a bit verbose, using the full
"(quasiquote ... (unquote-splicing ...))" and such instead of simply
"`(... ,@(...))"
* build-aux/bootstrap/rebirth.scm
(quote-sexp, quasiquote-sexp): Add procedures.
(fnmap)[quote]: Use `quote-sexp'.
[quasiquote]: Add special form.
This re-implements libprebirth in Rebirth Lisp, finally cutting the
cord. (Are these birth puns getting out of control?) We are finally purely
in Lisp land!
* build-aux/bootstrap/rebirth.scm: Define libprebith primitives when
`string->es' is available (using `cond-expand').
(prebirth->ecmascript): Do not include `libprebirth.js' in output.
This modifies each of the three lexers just to avoid confusion, even
though prebirth has no need for it. Birth does have a need, however, since
rebirth will contain newlines---despite the relevant rebirth code (next
commit) not being expanded, it still has to go through the lexer, which
otherwise errors out.
* build-aux/bootstrap/birth.scm (lex): Permit newlines in string regex.
* build-aux/bootstrap/prebirth.js (lex): Permit newlines in string regex.
* build-aux/bootstrap/rebirth.scm (lex): Permit newlines in string regex.
`cond-expand' allows moving forward with implementing features based on the
current bootstrapped state of the system---rebirth will be able to
recursively compile itself and introduce new features along the way.
`string->es' allows outputting raw ECMAScript, which gives us more control
over the code that is generated without having to hard-code it in the
compiler itself.
* build-aux/bootstrap/birth.scm
(fnmap)[cond-expand]: Always yield the empty string (do nothing).
* build-aux/bootstrap/rebirth.scm
(fnmap)[cond-expand]: Expand `string->es' only.
(fnmap)[string->es]: Add macro.
`define' can now be used to define values in addition to the procedure
short-hand.
* build-aux/bootstrap/rebirth.scm
(cdfn): New procedure. Renamed original `cdfn' to `cdfn-proc'.
(cdfn-var): New procedure.
(cdfn-proc): Renamed from `cdfn'.
This allows bootstrapping in either a development environment (Node.js) or
simply using the intended runtime environment: the user's browser.
* bootstrap.html: Add file (browser).
* bootstrap.js: Add file (command-line).
* bootstrap/Bootstrap.js: Add class. Formalize bootstrap process.
* bootstrap/libprebirth.js
(fsdata): Add variable to serve as filesystem stub.
(fs): Always throw error when `fs' module is unavailable.
($$js$file$_$$g$string): Consider `fsdata'.
* bootstrap/prebirth.js: Export as CommonJS module if in proper
environment. Abort automatic processing via stdin if root CommonJS
module.
This provides the complete compiler output, which is consistent with the
output of Birth. This will be useful for browser-based bootstrapping.
* build-aux/bootstrap/prebirth.js: Extract code into `Prebirth' class and
use the class.
(Prebirth): Add class.
This needs to run in the browser too, where we have no control over stack
limits.
* build-aux/bootstrap/birth.scm
(lex): Non-recursive strategy (loop with mutable list).
(make-token): Update doc. Produce list of token, new string, and
position. Don't recurse.
(body->es): Add `ret' param. Only produce `return' statement if new param
is set.
(cdfn): Use it.
(fnmap)
[js:while, js:break]: Add forms.
[lambda, let, case]: Use new `body->es' `ret' param.
[let*]: Define JS variables in output using `let' instead of `const' to
permit mutating with new `set!' form. Use new `body->es' `ret' param.
[set!]: Add form.
(prebirth->ecmascript): Adjust libprebirth path to be relative to self.
* build-aux/bootstrap/libprebirth.js
($$append$b$): Add `append!' procedure.
($$js$regexp, $$js$match, $$js$replace): Move a few lines up.
(fs): Provide stub if `require' is not defined.
* build-aux/bootstrap/prebirth.js
(_lex): Non-recursive strategy (loop with array-appending).
(_token): No more mutual recursion with `#_lex'. Return new string
and position.
(_bodyToEs): Add `ret' param. Only produce `return' statement if new
param is set.
(fnmap) [js:while, js:break]: Add forms.
[let*]: Define JS variables in output using `let' instead of `const' to
permit mutating with new `set!' form. Use new `body->es' `ret' param.
[set!]: Add form.
This completes bootstrapping for Prebirth Lisp. The next step will be
Rebirth, which will replace libprebirth.js, removing hand-written JavaScript
entirely.
* build-aux/bootstrap/prebirth.js
(_cdfn): Remove second argument to `#_idFromName'.
(_idFromName): Remove second parameter `global'. Identify and echo
integers. Remove distinction between global and non-global
identifiers---process everything.
Just about ready for that sloppy code generation!
* build-aux/bootstrap/birth.scm: Update file header documentation.
Add some whitespace between existing procedures.
Invoke `parse-lisp' as the program in place of `lex', producing an AST as
output to the console.
(cadddr): Add procedure.
(token-{type,lexeme,value,pos}): Add procedures.
(parse-lisp): Add procedure (contains other procedures).
* build-aux/bootstrap/libprebirth.js
($$append): Add function (append).
($$$_$): Correct implementation (-).
($$zero$7$): Add predicate (zero?).
($$fold): Add function (fold).
* build-aux/bootstrap/prebirth.js (parseLisp): Lowercase some errors.
(Compiler): Update class docblock.
(fnmap)[labmda]: Add `lambda' form.
This just reorganizes things slightly to allow for nested `define's. This
is a common means of encapsulation, and will Just Work™ because JS has
pretty much identical block scoping rules in this regard.
* build-aux/bootstrap/prebirth.js (compile): Invoke `#_sexpsToEs' directly
instead of `#_cdfn' on tree.
(_cdfn): Remove non-`define' check to proxy to `#_sexpsToEs'.
(_sexpsToEs): Treat `define' as a special case, invoking `#_cdfn'.
Mike Gerwitz