This was a bit of a nasty one. Fortunately, this was only used as a
validation, so the code that the compiler produced was still correct.
The problem was that a version of Saxon sometime between 9.5 and 9.8 added
an optimization to eliminate conditionals with no body. Consequently, the
kluge to force the variable to be evaluated was optimized away,
`lvmc:get-symbol' was never called, and no error was ever produced.
This would be best refactored, but that's not something I have time to take
up at the moment priority-wise. This should be future-proof since this
would never be a noop.
* src/current/compiler/map.xsl (lvmc:compile)[lvm:map//lvm:from[*]]: Force
evaluation of `$sym' by ensuring that the condition will not be a noop.
This will ensure that tamed does not stall while e.g. make is still
running. This makes TAMED_STALL_SECONDS almost useless; maybe it'll be
removed in future versions.
* bin/tame (TAMED_SPAWNER_PID): Export variable.
* bin/tamed (TAMED_SPAWNER_PID): New variable, default to PPID.
(spawner-dead): New function.
(stall-monitor): Use it.
(usage): Update documentation.
* build-aux/Makefile.am: Set TAMED_SPAWNER_PID to own id and export.
* src/current/c1map.xsl (lvm:c1map): Copy `@namespace' to generated
`lvmp:root'.
* src/current/c1map/render.xsl (lvmp:render)[lvmp:root]: Output
`@namespace' rather than using hardcoded string and dynamic program.
This maintains BC for existing raters that have not yet been migrated to use
the new c1-import service.
* build-aux/Makefile.am (path_c1root): New variable.
(.PHONY): Add c1root target dependency.
(program-data-copy): Copy to `@C1_IMPORT_MAPDEST@'.
(c1root): New target.
* build-aux/m4/calcdsl.m4 (C1_IMPORT_MAPDEST): Configure depending on the
existence of the `c1-import' directory.
This is a long-standing bug, apparently. The location of this code makes it
difficult to test directly (that is in dire need of correcting), but
fortunately we have a number of tests in systems that use TAME that
indirectly test this.
The problem manifested when a matrix was already in the store, but then a
scalar or vector predicate was considered. Without making the branch that
was modified here, it modified store such that it would always yield a
vector.
* src/current/compiler/js.xsl (anyValue): Consider store dimension when
recursing.
This is the start of a working build for core.
* .gitignore: Ignore generated files from configuration and build.
* build.xml: Copy from rater repo. This is the last remaining ant-based
dependency and can be gotten rid of; see comments.
* configure.ac: New file.
* rater/build-aux, rater/src: New symlinks.
This begins to decouple the rater directory conventions using an incremental
approach, defaulting to the existing structure. Not all things were
modified (for example, cleaning will not work properly with a custom
SRCPATHS if those directories do not exist); WIP.
* build-aux/Makefile.am (path_dsl): Use `CALCROOT'.
(suppliers.mk): Test for existence of program.dep and c1map directory
before acting on them.
* build-aux/m4/calcdsl.m4: Default SRCPATHS. Output it during configure.
Expose CALCROOT and SRCPATHS using AC_SUBST.
Invoke suppmk-gen using SRCPATHS.
* build-aux/suppmk-gen: Use arguments (SRCPATHS) in place of hard-coded paths.
This frees us from requiring a rater/ directory in the working
directory. However, it is important that we continue using it if it
exists, since there are additional things that haven't yet been moved
into the tame repo.
* bin/dslc.in: Provide path to rater/ directory.
* src/current/src/com/lovullo/dslc/DslCompiler.java: Use provided rater/ path.
This was broken by a previous commit, but was not noticed because
the test cases aren't being compiled as part of the build yet!
Now that we have tamed, that is an option.
* test/core/insurance.xml: Add missing @desc@.
* bin/tame (TAME_CMD_WAITTIME): Renamed from `RUNNER_CMD_WAITTIME'.
Inherit from environment, default 3.
(command-runner): Sleep for an additional TAME_CMD_WAITTIME seconds after
requesting runner reload to give more time in case of high load.
(verify-runner-ack): Rename variable.
(usage): Document env var.
* build-aux/Makefile.am: Export TAME_CMD_WAITTIME.
* build-aux/gen-make: Do not add ".xmlo" suffix for deps with a
trailing `$'.
* src/current/pkg-dep.xsl (lvm:program|lvm:return-map): Append ".xml$" to
dep for map/@src (new dep).
This is the one we always want in the UI. Rather than stripping with an
outside build process, just use this.
* build-aux/Makefile.am (program-data-copy, lvroot): Copy ui/program{=>.strip}.js.
This significantly improves speed and reduces memory usage when dealing with
hundreds of test cases.
* build-aux/Makefile.am (dest_standalone_strip): New variable.
(strip, %.strip.js: New targets.
(.PHONY): Add strip target.
(check-am): Depend on strip.
* build-aux/progtest-runner: Use stripped executables.
Try to re-post message, since the previous message will have already been
read (otherwise the previous echo would have hung).
* bin/tame (EX_STALLED): New exit code.
(command-runner): Re-post message after stall. If unrecoverable, provide
a more clear error and exit with EX_STALLED.
This tries to be a bit more resilient in case a runner becomes unresponsive,
rather than waiting for tamed to kill itself.
* bin/tame (RUNNER_CMD_WAITTIME): New variable.
(command-runner): Tell runner to reload if it does not respond in
RUNNER_CMD_WAITTIME seconds.
(verify-runner-ack): New function.
* bin/tamed (mkfifos): Only keep stdin open. stdout isn't necessary, and
may have actually been causing subtle issues.
(spawn-runner): Support restarting dslc on SIGHUP.
This fixes a lot of the problems with the build by using a normal Makefile
as it is intended to be used. To do this, tamed was created. See the
manual and commit messages for more information. bin/tame{,d} also have
more information. More information will follow in the manual in the future.
There is also more cleanup to follow; I just want to get this committed so
that people can take advantage of it and stop some of the suffering.
This does not include a great deal of information, but it is a start.
* README.md: Modernize.
* doc/Makefile.am (tame_TEXINFOS): Add `about.texi'.
* doc/about.texi: New file.
* doc/tame.texi: Include it.
This will keep the intermediate files around but will still delete them on
build failure.
* build-aux/Makefile.am (.SECONDARY): Renamed from `.PRECIOUS'.
Please excuse the mess. This was taken from an existing bootstrap script in
a private repository; it can be cleaned up in the future.
* bootstrap: New file.
* README.md (Getting Started): New section.
This is a major step toward normalcy---removing the kluge of a build process
that was causing so many issues. Rather than echoing all operations to a
queue file before passing it off to dslc, the new build scripts in `bin/'
are used to invoke tame normally, as needed. This solves all of the current
issues with things not rebuilding when they should. And, as a bonus, tab
completion on targets works.
Sorry this took so long. There wasn't much motivation until we hired so
many people that are suffering from this.
This does a few major things, along with some miscellaneous others:
- Invoke bin/tame directly;
- Merge Makefile.2.in into Makefile.am; and
- Fix up some targets.
* build-aux/Makefile.2.in: Delete file. Mostly merged with Makefile.am.
* build-aux/Makefile.am: Add a bunch of new targets and definitions from
Makefile.2.in. Modify all that previously used .cqueue to now invoke
`$(TAME)' directly. Remove miscellaneous targets for trying to proxy
targets to Makefile.2.
(saneout, _go): Remove definitions.
(.NOTPARALLEL): Add to prevent parallel builds.
(ui/program.expanded.xml)[.version.xml]: Remove dependency for now.
(clean): Also clean generated PHP files. Follow symlinks to clean core.
This is still incomplete (does not clean all rate table stuff).
(suppliers.mk)[xmlo_cmd]: Remove. See `gen-make' and `gen-c1make'.
(lvroot)[summary-html]: New dependency.
(kill-tamed, tamed-die): New targets (former alias of latter) to kill
tamed.
* build-aux/gen-c1make: Generate `$(TAME)' invocation.
* build-aux/gen-make: Likewise. Remove `xmlo_cmd' output. Ignore recursive
`tame' symlink (this can be removed once we clean `rater/' up.
* build-aux/m4/calcdsl.m4 (TAME): Update description to reflect that it
should now be the path to `bin/tame'. Adjust `AC_CHECK_FILE' lines
accordingly.
(tame_needed_ver): Remove. We have been in the same repo as TAME itself
for quite some time. Remove associated code.
(AC_CONFIG_FILES): Remove `Makefile.2'.
* src/current/src/com/lovullo/dslc/DslCompiler.java (_DslCompiler)[compile]:
Perform validation prefore `compile' command rather than a separate
`validate' step. Remove `rm'.
[compileSrc]: Stop echoing command. This was only necessary because of
the previous Makefile klugery; now Make echoes on its own correctly.
These scripts allow the TAME compiler stack to be invoked naturally, rather
than requiring the use of a Makefile today. This will not only allow users
to more easily invoke the compiler, but will also allow us to invoke TAME
naturally from Makefile and remove the klugery that has existed for so
long.
This users a server/client architecture in order to mitigate the startup
cost of the JVM. More documentation will follow.
Note that there are a bunch of symlinks in rater/---this is a transition
step to allow the build to continue working as it did before, which relies
on a directory structure that exists outside of this repository. This will
be cleaned up in the future.
* .gitignore (bin/dslc): Add ignore for generated file.
* bin/dslc.in: New script to encapsulate Java invocation.
* bin/tame: New script (client).
* bin/tamed: New script (server).
* configure.ac (JAVA_OPTS, DSLC_CLASSPATH, AUTOGENERATED): New variables for
dslc.in. Output bin/dslc.
* rater/README.md: Note that this symlink mess is temporary.
* rater/c1map: New symlink for dslc assumptions.
* rater/c1map.xsl: Likewise.
* rater/calc.xsd: Likewise.
* rater/compile.xsl: Likewise.
* rater/compiler: Likewise.
* rater/dot.xsl: Likewise.
* rater/include: Likewise.
* rater/link.xsl: Likewise.
* rater/standalone.xsl: Likewise.
* rater/summary.xsl: Likewise.
* rater/tame: Likewise (warning: circular symlink).
* src/current/src/com/lovullo/dslc/DslCompiler.java (_DslCompiler)[compile]:
Output `DONE' lines.
This will now automatically build on recursive target `all'.
* Makefile.am (SUBDIRS): Add `src/current/src'.
* src/current/src/Makefile: (.PHONY): Add `all'.
(all): New target. Alias to `dslc'.
This will allow the variable abstractions to fully encapsulate values while
still permitting binary searches on sorted rows.
* csvm-expand: Renamed from `csvm2csv'. Add directive support.
* csvm2csv: New script to perform sorting. Invokes aforementioned.
* test/test-csvm2csv: Update for sorting.
* build-aux/csvm2csv: Fail on invalid var definition.
(expand_vars): Fail in invalid reference.
(parseline): Fail on non-numeric range.
* build-aux/test/test-csvm2csv
(test-fail-unknown-var-ref, test-fail-non-numeric-range)
(test-fail-invalid-var-dfn): New tests.
This allows variables to contain any type of expansion, treating them
as (recursively) string replacements.
* csvm2csv: Add Copyright year.
(expand_vars): New function.
(parseline): Use it. Expand before all other replacements so that
variable expansions may support all types of expressions. Expand
variables in range expressions.
* test/test-csvm2csv: New file.
Mike Gerwitz