Some notes on this: The "Calc DSL" is the name of the DSL before it
became "TAME". This takes the entire core library and squashes its 91
commits into a single one; the reason for this is because those
commits often contain LoVullo-specific details that are either
irrelevant or should not be included.
This library has limited value to the public at the time of this
commit, since TAME has not yet been released (it requires some
additional cleanup and filtering before then). It is also in need of
heavy refactoring and reorganization, since it has accumulated a lot
of cruft, especially since the project in which the Calc DSL was
introduced was rushed (to put it lightly). Forgive the mess.
[LoVullo employees: the commit was extracted from dsl.git 4a3aea9;
full history can be found there. This commit contains some additional
minor tweaks in addition to squashing. It filters on the :/core/
directory.]
This has a significant performance impact: processing time is cut in about
half and memory usage is reduced by more than 50%. For example, a package
that previously took 30s and 2.1GiB of memory to link now takes 14s and less
than 900MiB of memory.
This was throwing a warning in non-ancient versions of Saxon. It does not
need to be there, nor should it be, nor do I know why it was put there.
* src/current/include/preproc/template.xsl (eseq:is-expandable): Remove
@override.
This has a significant performance impact: processing time is cut in about
half and memory usage is reduced by more than 50%. For example, a
package that previously took 30s and 2.1GiB of memory to link now takes
14s and less than 900MiB of memory.
I had tried to perform this optimization a couple years ago but was
thwarted (I think) by the classifier markers. The previous commit did away
with those. I'm encouraged by the gains from the low-hanging fruit.
* src/current/compiler/linker.xsl
(l:process-empty, l:stack-empty): Convert from l:pstack and
l:sym-stack (respectively) to empty preproc:sym sequences.
(l:depgen-process-sym)[preproc:sym]: Append to sequence rather than
outputting new l:sym-stack tree.
Update all annotations and uses accordingly.
This is something that I thought would be useful back in the day when TAME
was in its infancy, but it is not important. Rather than having the linker
spend time trying to figure out what symbols belong in the classifier---and
rather than keeping that complexity around---this simplifies things by
making the existing `classify' method simply perform _all_ calculations, and
then yield only the classification portion of the result.
This isn't a problem in practice because, if we only desire the use of a
classifier, then we create a "supplier" that only uses classifications and
has no other dependencies. The end result is, as far as we care, the same.
* src/current/compiler/js.xsl (compiler:entry-rater)[lv:package]: Initialize
`classes' rather than invoking classifier
(compiler:entry-classifier)[lv:package]: Invoke all calculations and
return only classes to provide equivalent behavior.
(compiler:exit-classifier): Post-process classifications from calculation
results, iterating through classmap.
(compiler:classifier-yields-map)[lv:package]: Output all classifications
that are not generated. This differs slightly from the original
implementation in that it includes all non-generated classes rather than
just classes that have a non-generated `@yields'; this distinction is
important since `compiler:exit-classifier' is now using it to produce a
classification result set that doesn't contain all the generated
stuff (since it didn't before, and shouldn't now).
* src/current/compiler/linker.xsl: Update copyright year.
(l:resolv-deps)[preproc:sym[@l:mark-inclass]]: Remove template.
(l:resolv-deps)[preproc:sym...@l:mark-inclass...]: Remove template.
(l:depgen-sym): Set type of result to `element(preproc:sym)', since
`l:mark-inclass' is no longer produced.
[inclass, needs-class-mark]: Remove variables and all instances where
they are used.
(l:dep-aug)[inclass]: Remove param. Stop producing `@inclass' attribute.
(l:link-classifier)[lv:package]: Do not process any dependencies. This
can be removed entirely in the future since it now only produces static
code, which we can perhaps combine with a different block.
(l:link-rater)[lv:package]: Remove mention of `inclass' for dependencies;
all dependencies will now be compiled into this block.
This solves the dependency-related problems with rebuilding c1map files.
* build-aux/gen-c1make: New file.
* build-aux/Makefile.2.in (suppliers.mk): Use it.
These were forgotten during the initial liberation.
* build-aux/Makefile.2.in: New file.
* build-aux/Makefile.am: New file.
* build-aux/m4/calcdsl.m4: New file.
* build-aux/progtest-runner: New file.
* build-aux/suppmk-gen: New file.
This includes a SHA256 implementation which is _not_ intended for secure
cryptographic operations; see src/js/sha256.js header for more information.
* src/current/compiler/js.xsl (compiler:static): Echo src/js/sha256.js.
[map_method_uppercase, map_method_hash]: New functions.
* src/current/link.xsl: Include dslc-base.xsl.
* src/js/sha256.js: New file.
* src/current/compiler/map.xsl
(lvmc:get-method-func, lvmc:value-ref, lvmc:transformation-wrap): New
functions, partyl extracted from below.
(lvmc:compile)[lvm:map//lvm:from]: Use `lvmc:value-ref'.
[lvm:map//lvm:from/lvm:translate]: Add `[@key]' to match.
[lvm:map//lvm:transform]: New match. Ignore node entirely.
(lvmc:concat-compile): Propagate symtable to `lvmc:compile'.
* src/current/include/dslc-base.xsl (__path-root): New param.
* src/current/src/com/lovullo/dslc/DslCompiler.java
(DslCompiler)[compile]: Resolve TAME root path.
[_transform]: Set it.
DEV-3115
Oops---we do not want a newline as part of the preimage, which is what
`<<<' does.
* build-aux/list2typedef (output-item)[value]: Remove newline from preimage.
* build-aux/test/test-list2typedef (test-typedef-gen): Update hashes.
(test-collision-value-check): Add new collisions.
This needs to be generalized, but will do for now.
* build-aux/gen-make: Recognize typelists as dependencies. Always include
source XML file as a dependency.
Before this change, it was impossible for the initial paths provided to the
script to be more than one level deep (relative to the cwd).
* build-aux/gen-make: Handle multi-level children.
Not a very useful test runner if it doesn't ever fail, now is it?
* Makefile.am (check): Invoke new test/runner-test. Depend on modindex.
* bin/runner.js: Exit with non-zero status on assertion failure.
* test/_stub: Add stub program with good and bad test cases to test
exit code.
* test/runner-test: Add system test.
This is all really confusing because this doesn't use the same import
specification as packages; maps got stuck in a partial transition. So,
let's provide some helpful errors rather than silently failing.
* src/current/compiler/map.xsl (preproc:symtable)[lvm:import]:
Error if missing `@path'. Provide more information if `@package' was
provided to help clarify.
We need to cut down on symbol imports as much as possible; the whole system
starts dragging if we are importing thousands of symbols into a single
package.
* src/current/include/preproc/symtable.xsl (preproc:symtable)[lv:rate,c:*]: Mark
as local if `@preproc:generated`.
* src/current/include/preproc/template.xsl (preproc:macros)[lv:inline-template]:
Mark generated templates as such.
* src/symtable/symbols.xsl (preproc:symtable)[lv:template]: Mark as local if
`@preproc:generated'.
* src/symtable.xsl (symtable:find-duplicates):
Update description. Find duplicates in same package.
* test/symtable.xspec: Update test case accordingly.
* src/current/compiler/js.xsl (compiler:js-number): New function to
remove leading zeroes.
(compile)[lv:const]: Use it.
* src/current/compiler/js-calc.xsl (compile-calc)[c:const]: Use it.
* src/current/compiler/js.xsl (compile-class-condtion)[lv:rate]: Do not
consider @no's in predicate generation when `@preproc:gentle-no' is set.
* src/current/include/preproc/macros.xsl (preproc:macros)[lv:rate-each]: Set
`@preproc:gentle-no' on generated `lv:rate', since the generator handles
`@no' itself.
Mike Gerwitz
Mike Gerwitz