Note that, because of the way this is implemented, the timestamps may become
mangled (multiple per line) for parallel builds.
Output can be prettied up in the future.
tamed was originally designed with support for parallel builds in mind, but
I hadn't completed that work because we didn't have enough hardware that
we'd benefit strongly from it. That has since changed.
tamed will now spawn additional runners as needed to fulfill requests, which
works around the issue of not knowing how many jobs GNU Make is going to try
to do at once.
There were a couple minor dependency fixes/workarounds for now in the
Makefile, but otherwise everything appears to be working great.
This now uses year ranges, which I'll update annually.
This also renames "R-T Specialty" to "Ryan Specialty Group". The latter is
the parent company of the former. I was originally employed under the
former when LoVullo Associates was purchased, by I now work for the parent
company.
This allows customizing from the command-line what suppliers should be
checked. This motivation for this is both to run as part of a distributed
pipeline (where each supplier may be built individually), and for during
development of a single supplier.
BC BREAK: Note that this will now check for `package' in the test path for
UI tests. To keep the old directory around, a symlink of `packages' to `ui'
would suffice.
* build-aux/Makefile (SUPPLIERS, suppliers_strip): New variables.
(check-am): BC-BREAK: Build and check only requested suppliers.
* build-aux/progtest-runner: BC-BREAK: First argument is now test directory
and all remaining arguments specify the supplier XML files to check.
We want to be able to build the UI independently of the
suppliers. Historically, this did not provide much of a benefit, but this
change allows us to build independently as a job in a distributed pipeline,
and allows testing out the UI when rating is unneeded.
* build-aux/Makefile.am (program-ui): Remove `standalones'.
This target has not been used for years.
* build-aux/Makefile.am (program-ui-immediate): Remove target.
(program-ui): Use dependency of old `program-ui-immediate'.
(.PHONY): Remove `program-ui-immediate'.
Note that such files may not actually exist, which is why `nullglob' is set
and the `for' loop is used.
* build-aux/Makefile.am (SHELL): Set `nullglob'.
(program-data-copy, lvroot): Copy srv/!(rater).js to destination JS paths.
This will ensure that tamed does not stall while e.g. make is still
running. This makes TAMED_STALL_SECONDS almost useless; maybe it'll be
removed in future versions.
* bin/tame (TAMED_SPAWNER_PID): Export variable.
* bin/tamed (TAMED_SPAWNER_PID): New variable, default to PPID.
(spawner-dead): New function.
(stall-monitor): Use it.
(usage): Update documentation.
* build-aux/Makefile.am: Set TAMED_SPAWNER_PID to own id and export.
This maintains BC for existing raters that have not yet been migrated to use
the new c1-import service.
* build-aux/Makefile.am (path_c1root): New variable.
(.PHONY): Add c1root target dependency.
(program-data-copy): Copy to `@C1_IMPORT_MAPDEST@'.
(c1root): New target.
* build-aux/m4/calcdsl.m4 (C1_IMPORT_MAPDEST): Configure depending on the
existence of the `c1-import' directory.
This begins to decouple the rater directory conventions using an incremental
approach, defaulting to the existing structure. Not all things were
modified (for example, cleaning will not work properly with a custom
SRCPATHS if those directories do not exist); WIP.
* build-aux/Makefile.am (path_dsl): Use `CALCROOT'.
(suppliers.mk): Test for existence of program.dep and c1map directory
before acting on them.
* build-aux/m4/calcdsl.m4: Default SRCPATHS. Output it during configure.
Expose CALCROOT and SRCPATHS using AC_SUBST.
Invoke suppmk-gen using SRCPATHS.
* build-aux/suppmk-gen: Use arguments (SRCPATHS) in place of hard-coded paths.
* bin/tame (TAME_CMD_WAITTIME): Renamed from `RUNNER_CMD_WAITTIME'.
Inherit from environment, default 3.
(command-runner): Sleep for an additional TAME_CMD_WAITTIME seconds after
requesting runner reload to give more time in case of high load.
(verify-runner-ack): Rename variable.
(usage): Document env var.
* build-aux/Makefile.am: Export TAME_CMD_WAITTIME.
This is the one we always want in the UI. Rather than stripping with an
outside build process, just use this.
* build-aux/Makefile.am (program-data-copy, lvroot): Copy ui/program{=>.strip}.js.
This significantly improves speed and reduces memory usage when dealing with
hundreds of test cases.
* build-aux/Makefile.am (dest_standalone_strip): New variable.
(strip, %.strip.js: New targets.
(.PHONY): Add strip target.
(check-am): Depend on strip.
* build-aux/progtest-runner: Use stripped executables.
This will keep the intermediate files around but will still delete them on
build failure.
* build-aux/Makefile.am (.SECONDARY): Renamed from `.PRECIOUS'.
This is a major step toward normalcy---removing the kluge of a build process
that was causing so many issues. Rather than echoing all operations to a
queue file before passing it off to dslc, the new build scripts in `bin/'
are used to invoke tame normally, as needed. This solves all of the current
issues with things not rebuilding when they should. And, as a bonus, tab
completion on targets works.
Sorry this took so long. There wasn't much motivation until we hired so
many people that are suffering from this.
This does a few major things, along with some miscellaneous others:
- Invoke bin/tame directly;
- Merge Makefile.2.in into Makefile.am; and
- Fix up some targets.
* build-aux/Makefile.2.in: Delete file. Mostly merged with Makefile.am.
* build-aux/Makefile.am: Add a bunch of new targets and definitions from
Makefile.2.in. Modify all that previously used .cqueue to now invoke
`$(TAME)' directly. Remove miscellaneous targets for trying to proxy
targets to Makefile.2.
(saneout, _go): Remove definitions.
(.NOTPARALLEL): Add to prevent parallel builds.
(ui/program.expanded.xml)[.version.xml]: Remove dependency for now.
(clean): Also clean generated PHP files. Follow symlinks to clean core.
This is still incomplete (does not clean all rate table stuff).
(suppliers.mk)[xmlo_cmd]: Remove. See `gen-make' and `gen-c1make'.
(lvroot)[summary-html]: New dependency.
(kill-tamed, tamed-die): New targets (former alias of latter) to kill
tamed.
* build-aux/gen-c1make: Generate `$(TAME)' invocation.
* build-aux/gen-make: Likewise. Remove `xmlo_cmd' output. Ignore recursive
`tame' symlink (this can be removed once we clean `rater/' up.
* build-aux/m4/calcdsl.m4 (TAME): Update description to reflect that it
should now be the path to `bin/tame'. Adjust `AC_CHECK_FILE' lines
accordingly.
(tame_needed_ver): Remove. We have been in the same repo as TAME itself
for quite some time. Remove associated code.
(AC_CONFIG_FILES): Remove `Makefile.2'.
* src/current/src/com/lovullo/dslc/DslCompiler.java (_DslCompiler)[compile]:
Perform validation prefore `compile' command rather than a separate
`validate' step. Remove `rm'.
[compileSrc]: Stop echoing command. This was only necessary because of
the previous Makefile klugery; now Make echoes on its own correctly.
These were forgotten during the initial liberation.
* build-aux/Makefile.2.in: New file.
* build-aux/Makefile.am: New file.
* build-aux/m4/calcdsl.m4: New file.
* build-aux/progtest-runner: New file.
* build-aux/suppmk-gen: New file.
Mike Gerwitz
Mike Gerwitz