%.xml{=>o}: %csvo rater/core/vector/table.xmlo
That is: we'll only build an object file when we try to build another object
file. This was causing problems with dependency generation, because it will
triggering compilation early.
This should have been done many years ago. This will determine if any of
the dependencies have changed for the included suppliers.mk and regenerate
it as needed, without the developer having to do so manually when imports
change.
* build-aux/Makefile.am (suppliers.mk): Invoke ant with `-q` to eliminate
"processing" messages for each and every file. This also speeds up
operation slightly.
* build-aux/gen-make: Remove information echos for each file.
These changes will allow for suppliers.mk to be regenerated automatically
without being so invasive.
Note: this really belongs in liza-proguic, and should be moved in the near
future.
liza-proguic is being modified to generate step-level packages, which are
significantly faster to build than larger ones (XSLT TAME scales
terribly). These changes handle those new dependencies.
One important thing to note with this change is that suppliers.mk now
requires proguic to have run before generation so that those generated
dependencies can be properly examined. This is a quick operation, so that
is not problematic.
This also depends on the .version.xml change that was previously made: when
the timestamp changed every time, we got into an infinite build loop.
This will be generated automatically by the Makefile. It's not appropriate
to generate in the configure script, and I do not recall why I did
so---possibly to work around the issue of delayed tab completion when it
needs regeneration?
This removes suppmk-gen in favor of more generic Makefile targets---in this
case, having `%.tdat` depend upon `rater/core/tdat.xml`, even though that's
not quite true (the %.xml file generated from it needs it). But these files
are going away soon; a pending TAME optimization branch removes support for
the underlying pattern primitive entirely; CSVMs should be used instead.
The timestamp of the file will now only be updated if the hash (version)
_actually_ changes. This allows this to be used as a target dependency
without forcing a rebuild each and every time.
This handles moving to another repository structure (our gigarepo) where
this relative path is no longer true. The absolute path generated by this
is okay since it's ephemeral and only used for this build invocation.
This begins providing release notes for changes and provides scripts to
facilitate this:
- tools/mkrelease will update RELEASES.md and run some checks.
- build-aux/release-check is intended for use in pipelines (e.g. see
.gitlab-ci.yml) to verify that releases were done properly.
Add a new step to the build process that copies the `xml` file to an
`xmli` file. Eventually, the new compiler will create the `xmli` file
and the old compiler will convert it to an `amle` file during the
transition.
We want to add an option to set the output file to the linker so we do
not need to redirect output to awk any longer.
This also adds integration tests for tameld.
Note that, because of the way this is implemented, the timestamps may become
mangled (multiple per line) for parallel builds.
Output can be prettied up in the future.
csvm2csv was not failing when csvm-expand exited with a non-zero
status. Further, the tests were written incorrectly to account for this.
* build-aux/csvm2csv: Set `pipefail' option.
* build-aux/test/test-csvm2csv: Fix tests.
While tabs aren't desirable, users that are not developers will be modifying
these files, and so we need to be permissive in what we want to
accept. That doesn't mean that we need to forego occasional formatting, though.
tamed was originally designed with support for parallel builds in mind, but
I hadn't completed that work because we didn't have enough hardware that
we'd benefit strongly from it. That has since changed.
tamed will now spawn additional runners as needed to fulfill requests, which
works around the issue of not knowing how many jobs GNU Make is going to try
to do at once.
There were a couple minor dependency fixes/workarounds for now in the
Makefile, but otherwise everything appears to be working great.
A table with a couple hundred thousand rows was taking minutes to
generate. This gets it down to a few seconds.
* build-aux/csvm-expand (parse_date): New function.
(parseline): use it.
This now uses year ranges, which I'll update annually.
This also renames "R-T Specialty" to "Ryan Specialty Group". The latter is
the parent company of the former. I was originally employed under the
former when LoVullo Associates was purchased, by I now work for the parent
company.
lsimports will be able to be used to replace the last remaining Ant script
that generates depfiles.
* build-aux/check-coupling:
* build-aux/lsimports: New files.
This allows customizing from the command-line what suppliers should be
checked. This motivation for this is both to run as part of a distributed
pipeline (where each supplier may be built individually), and for during
development of a single supplier.
BC BREAK: Note that this will now check for `package' in the test path for
UI tests. To keep the old directory around, a symlink of `packages' to `ui'
would suffice.
* build-aux/Makefile (SUPPLIERS, suppliers_strip): New variables.
(check-am): BC-BREAK: Build and check only requested suppliers.
* build-aux/progtest-runner: BC-BREAK: First argument is now test directory
and all remaining arguments specify the supplier XML files to check.
We want to be able to build the UI independently of the
suppliers. Historically, this did not provide much of a benefit, but this
change allows us to build independently as a job in a distributed pipeline,
and allows testing out the UI when rating is unneeded.
* build-aux/Makefile.am (program-ui): Remove `standalones'.
This target has not been used for years.
* build-aux/Makefile.am (program-ui-immediate): Remove target.
(program-ui): Use dependency of old `program-ui-immediate'.
(.PHONY): Remove `program-ui-immediate'.
Note that such files may not actually exist, which is why `nullglob' is set
and the `for' loop is used.
* build-aux/Makefile.am (SHELL): Set `nullglob'.
(program-data-copy, lvroot): Copy srv/!(rater).js to destination JS paths.
This will ensure that tamed does not stall while e.g. make is still
running. This makes TAMED_STALL_SECONDS almost useless; maybe it'll be
removed in future versions.
* bin/tame (TAMED_SPAWNER_PID): Export variable.
* bin/tamed (TAMED_SPAWNER_PID): New variable, default to PPID.
(spawner-dead): New function.
(stall-monitor): Use it.
(usage): Update documentation.
* build-aux/Makefile.am: Set TAMED_SPAWNER_PID to own id and export.
This maintains BC for existing raters that have not yet been migrated to use
the new c1-import service.
* build-aux/Makefile.am (path_c1root): New variable.
(.PHONY): Add c1root target dependency.
(program-data-copy): Copy to `@C1_IMPORT_MAPDEST@'.
(c1root): New target.
* build-aux/m4/calcdsl.m4 (C1_IMPORT_MAPDEST): Configure depending on the
existence of the `c1-import' directory.
This begins to decouple the rater directory conventions using an incremental
approach, defaulting to the existing structure. Not all things were
modified (for example, cleaning will not work properly with a custom
SRCPATHS if those directories do not exist); WIP.
* build-aux/Makefile.am (path_dsl): Use `CALCROOT'.
(suppliers.mk): Test for existence of program.dep and c1map directory
before acting on them.
* build-aux/m4/calcdsl.m4: Default SRCPATHS. Output it during configure.
Expose CALCROOT and SRCPATHS using AC_SUBST.
Invoke suppmk-gen using SRCPATHS.
* build-aux/suppmk-gen: Use arguments (SRCPATHS) in place of hard-coded paths.
* bin/tame (TAME_CMD_WAITTIME): Renamed from `RUNNER_CMD_WAITTIME'.
Inherit from environment, default 3.
(command-runner): Sleep for an additional TAME_CMD_WAITTIME seconds after
requesting runner reload to give more time in case of high load.
(verify-runner-ack): Rename variable.
(usage): Document env var.
* build-aux/Makefile.am: Export TAME_CMD_WAITTIME.
* build-aux/gen-make: Do not add ".xmlo" suffix for deps with a
trailing `$'.
* src/current/pkg-dep.xsl (lvm:program|lvm:return-map): Append ".xml$" to
dep for map/@src (new dep).
This is the one we always want in the UI. Rather than stripping with an
outside build process, just use this.
* build-aux/Makefile.am (program-data-copy, lvroot): Copy ui/program{=>.strip}.js.
This significantly improves speed and reduces memory usage when dealing with
hundreds of test cases.
* build-aux/Makefile.am (dest_standalone_strip): New variable.
(strip, %.strip.js: New targets.
(.PHONY): Add strip target.
(check-am): Depend on strip.
* build-aux/progtest-runner: Use stripped executables.
Mike Gerwitz
Joseph Frazer