Commit Graph

687 Commits (da7a2c71c75d72c7e9f21f9d73a7aa439520e4ec)

Author SHA1 Message Date
Mike Gerwitz da7a2c71c7 tamed: TAMED_JAVA_OPTS: New environment variable
This will be passed to dslc and then to the JVM.  The intent is to permit
fine-grained heap ratio tuning.
2020-08-19 10:19:04 -04:00
Mike Gerwitz 680691c4cf bootstrap: Permit directory for hoxsl
Now I recall the reason I had -e: we clone hoxsl in the pipeline.
2020-07-27 12:51:10 -04:00
Mike Gerwitz 2627b8eef5 bootstrap: Check explicitly for hoxsl symbolic link
Using `-e` resulted in a situation where a broken symbolic link would cause
`ln` to be executed in error.
2020-07-24 15:33:17 -04:00
Mike Gerwitz 13cd372eb7 RELEASES.md: Include mention of lsimports fix 2020-07-23 14:33:06 -04:00
Mike Gerwitz 59b2b32756 build-aux/lsimports: Fix awk gensub warning
Third argument must be numeric indicating which match to replace.
This error did not exist in previous versions.
2020-07-23 14:31:54 -04:00
Mike Gerwitz 4610f5d4a4 RELEASES.md (17.4.3): Fix {product=>produce} typo 2020-07-15 14:38:07 -04:00
Mike Gerwitz d4d412f20a RELEASES.md: Update for v17.5.0 2020-07-15 11:16:14 -04:00
Mike Gerwitz 6baa88136a RELEASES.md: Remove heading underling artifact 2020-07-15 11:15:40 -04:00
Mike Gerwitz 6784090cf0 Use experimental TCO for heavily recursive portion of table lookup
This was urgently needed for a project using TAME.  Somehow, we've gone
all of these years without a table in which the first predicate is unable to
sufficiently filter out enough results that we do not hit stack limits.

Each recursive step of mrange before inlining and TCO, at the time of
writing, was adding eight stack frames.  This is because each let (and many
other things) compile into self-applying functions.  Since mrange is invoked
once for every single row for a given value, we quickly run out of stack
space.

For example, consider this table:

  1, $a, $b
  2, $a, $b
  2, $b, $c
  2, $c, $d
  3, $a, $b

If we were to filter the first column on the value 2, it would first bisect
to find the middle row, backtrack to the first, and then move forward to the
last, producing:

  2, $a, $b
  2, $b, $c
  2, $c, $d

This is at least three mrange calls, for a potential total of 8*3=24 stack
frames, depending on implementation details I don't quite recall at the
moment about the how the query system works.

We had over 1000 rows after applying the first predicate; the stack was
exhausted before it could even reach the last row.

Tail call optimization (TCO) is the process of turning recursive calls in
tail position into jumps.  So, rather than the stack growing on a recursive
call, it stays constant.  A common way to accomplish this in stack-based
languages is using a trampoline.

In our case, we enclose the entirety of the function in a `do` loop, and
clear a flag indicating that a tail call took place.  When we reach a
recursive tail call, we set that flag.  Then, instead of invoking the
function again, we _overwrite the original arguments_ with their new
values, and simply return 0.  When the function hits the end of the loop, it
will see that the flag is set, and jump back to the beginning of the
function, starting all over with the new values.

Compiling in this functionality is not difficult.  Tracking whether a given
call is in tail position, however, is a bit of a pain given how the XSLT
code is currently written.  Given that this is all being replaced with
TAMER, it's difficult to stomach making too many changes to the compiler,
when we can do it properly in the future with TAMER.  But we need the
feature now.

As a compromise, I call this implementation "guided" TCO---we rely on a
human to indicate that a call is in tail position by setting an experimental
flag manually.  That frees us from having to have the compiler do it, but
does create some nasty problems if the human is wrong.  Consequently, this
should only be used in core, and people should not use it unless they know
what they're doing.

Using this feature currently outputs a warning---that way, if there are
problems, people have some idea of where they maybe can look.  The warning
will be removed in the future after this has been in production for some
time (granted, our test suite passes).

Once again: TAMER will implement proper tail calls automatically, without
the need for a human to intervene.

For more information on tail calls:

  - https://en.wikipedia.org/wiki/Tail_call
2020-07-15 10:55:38 -04:00
Mike Gerwitz 61aec5f714 [DEV-8130] core: mrange: Use experimental guided TCO
This alleviates stack exhaustion issues with large rate tables where the
predicates fail to reduce the search space to a reasonable size.
2020-07-15 10:33:05 -04:00
Mike Gerwitz 3418023269 core: mrange: Inline _mrange_cmp
This will permit the use of TCO in the following commit.
2020-07-15 10:33:05 -04:00
Mike Gerwitz 26b1bdacec Experimental guided TCO
This implements TCO in the XSLT compiler by requiring a human to manually
indicate when a recursive call is in tail position.  This was somewhat
urgently needed to resolve stack exhaustion on large rate tables.

TAMER will do this properly by determining itself whether a call is in tail
position.  Until then, this will serve as a test for this type of feature.
2020-07-15 10:33:04 -04:00
Mike Gerwitz 79c4116190 src/current/Makefile (all, html): Phony targets
Standardization for recursive make.  This Makefile will
go away at some point anyway.
2020-07-14 10:56:24 -04:00
Mike Gerwitz 1fbcededa8 build-aux/Makefile.am (ui/Program.js): include-path with arbitrary parent
This handles moving to another repository structure (our gigarepo) where
this relative path is no longer true.  The absolute path generated by this
is okay since it's ephemeral and only used for this build invocation.
2020-07-07 16:41:35 -04:00
Mike Gerwitz b5cfc11c34 RELEASES.md: Update for v17.4.3 2020-07-02 12:48:16 -04:00
Mike Gerwitz af222e2777 RELEASES.md: Summary for NEXT release 2020-07-02 12:47:12 -04:00
Mike Gerwitz e0356324bd TAMER: Linker: Human-readable unresolved object errors
Example output:

  error: unresolved extern `taxHomeState` of type `rate[float; 0]`, declared
  in `common/tax/state/dc`
2020-07-02 12:43:17 -04:00
Mike Gerwitz 96ffd5f6e5 [DEV-8000] ir::asg: Error types for unresolved identifiers during sorting
This checks explicitly for unresolved objects while sorting and provides an
explicit error for them.  For example, this will catch externs that have no
concrete resolution.

This previously fell all the way through to the unreachable! block.  The old
POC implementation was catching unresolved objects, albeit with a debug
error.
2020-07-02 01:38:32 -04:00
Mike Gerwitz a2415c8c6f [DEV-8000] ir::asg::base: Replace Symbol::new_dummy
Use symbol_dummy!.
2020-07-01 15:53:56 -04:00
Mike Gerwitz 0d4bbe5e4e [DEV-8000] ir::asg: Introduce SortableAsgError
This will be used for the next commit, but this change has been isolated
both because it distracts from the implementation change in the next commit,
and because it cleans up the code by removing the need for a type parameter
on `AsgError`.

Note that the sort test cases now use `unwrap` instead of having
`{,Sortable}AsgError` support one or the other---this is because that does
not currently happen in practice, and there is not supposed to be a
hierarchy; they are siblings (though perhaps their name may imply otherwise).
2020-07-01 13:42:14 -04:00
Mike Gerwitz f832feb3fa [DEV-8000] ir::asg::base::BaseAsg::check_cycles: Extract into function
The only reason this function was a method of `BaseAsg` was because of
`self.graph`, which is accessible within the scope of this
module.  `check_cycles` is logically associated with `SortableAsg`, and so
should exist alongside it (though it can't exist as an associated function
of that trait).
2020-07-01 11:02:20 -04:00
Joseph Frazer a5b3730410 RELEASES.md: Update for v17.4.2 2020-05-13 08:09:48 -04:00
Joseph Frazer 45b7adbf54 [DEV-7504] Add GraphML generation
Merge branch 'jira-7504'

* jira-7504:
  [DEV-7504] Update RELEASES.md to make it less technical
  [DEV-7504] Add cypher script for post-graph import
  [DEV-7504] Add make target for "graphml"
  [DEV-7504] Add GraphML generation
2020-05-13 08:05:57 -04:00
Joseph Frazer c72409d617 [DEV-7504] Update RELEASES.md to make it less technical 2020-05-13 08:04:48 -04:00
Joseph Frazer 71ba6a95eb [DEV-7504] Add cypher script for post-graph import
After we import the GraphML output into Neo4J, we need to make a few
modifications to make the data easier to work with.
2020-05-13 08:04:48 -04:00
Joseph Frazer 09350d0ada [DEV-7504] Add make target for "graphml" 2020-05-13 08:04:48 -04:00
Joseph Frazer 43d00a8268 [DEV-7504] Add GraphML generation
We want to be able to build a representation of the dependency graph so
we can easily inspect it.

We do not want to make GraphML by default. It is better to use a tool.
We use "petgraph-graphml".
2020-05-13 08:04:48 -04:00
Mike Gerwitz 18d87a6b00 Merge old Neo4j graph generation POC
This was never completed and will be able to be deleted entirely, but I
didn't want to lose this history by having it sit out in a branch.  Joe is
working on something better.
2020-05-13 00:45:01 -04:00
Mike Gerwitz b8d9128f18 POC for full graph 2020-05-13 00:44:13 -04:00
Mike Gerwitz be61d41ca5 RELEASES.md: Clarification and formatting fix of intro 2020-04-30 13:06:43 -04:00
Mike Gerwitz d53ee68e6d RELEASES.md: Fix tame-core heading 2020-04-29 15:49:57 -04:00
Mike Gerwitz 6f61ae7788 RELEASES.md: Add missing semver link 2020-04-29 15:48:21 -04:00
Mike Gerwitz 35010f6882 build-aux/release-check: Fix readonly variable issue 2020-04-29 15:38:22 -04:00
Mike Gerwitz baaecc7181 RELEASES.md: Update for v17.4.1 2020-04-29 15:34:29 -04:00
Mike Gerwitz 99f2d5054e Release notes and associated scripts
This begins providing release notes for changes and provides scripts to
facilitate this:

 - tools/mkrelease will update RELEASES.md and run some checks.
 - build-aux/release-check is intended for use in pipelines (e.g. see
   .gitlab-ci.yml) to verify that releases were done properly.
2020-04-29 15:33:46 -04:00
Mike Gerwitz 0127d4b698 TAMER: sym::Interner::index_lookup
This was originally omitted because there wasn't a use case for it.  Now
that we're adding context to errors, however, an owned value is highly
desirable.

This adds almost no measurable overhead to the internment system in
benchmarks (largely within the margin of error).
2020-04-29 11:33:41 -04:00
Mike Gerwitz 4b643385c8 TAMER: Update Cargo dependencies 2020-04-29 11:33:38 -04:00
Mike Gerwitz 1ddd9cceea TAMER: Finalize dependency graph construction
There is more refactoring that can be done, but this extracts it out of POC.
2020-04-29 11:14:51 -04:00
Mike Gerwitz bcca5f7c49 [DEV-7084] TAMER: AsgBuilder and IR lowering docs 2020-04-28 13:39:55 -04:00
Mike Gerwitz 0f4b2d75f8 [DEV-7084] TAMER: obj::xmlo: Private inner modules 2020-04-28 11:08:05 -04:00
Mike Gerwitz 549e9ca23b [DEV-7084] TAMER: AsgBuilderState:🆕 New constructor 2020-04-28 09:06:25 -04:00
Mike Gerwitz 9893d56775 [DEV-7084] TAMER: Finalize AsgBuilder 2020-04-28 09:06:25 -04:00
Mike Gerwitz 32abc7dce2 [DEV-7084] TAMER: impl PartialEq for XmloError
This cannot be dervied because XmlError does not implement PartialEq,
which is quite the annoyance in tests.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 21a0bdcce1 [DEV-7084] TAMER: AsgBuilderError: Introduce proper error variants
This is a union (sum type) of three other errors types, plus errors specific
to this builder.

This commit does a good job demonstrating the boilerplate, as well as a need
for additional context (in the case of `IdentKindError`), that we'll want to
work on abstracting away.
2020-04-28 09:06:25 -04:00
Mike Gerwitz ef79a763ac [DEV-7084] TAMER: Correct Ix trait bound for AsgError
The `Debug` bound is inconvenient and requires propagation to any types that
use it.  Further, it's really awkward having `Display` depend on `Debug`; if
we want to render a useful display here, we can write one.

To be clear: IndexType implements Debug.

For now, this is pretty-printed by another part of the code, which we don't
want to implement in `Display` because it requires looking things up from
the graph.
2020-04-28 09:06:25 -04:00
Mike Gerwitz cfc13f9016 [DEV-7084] TAMER: ir::asg::IdentKindError: Replace string with enum 2020-04-28 09:06:25 -04:00
Mike Gerwitz 0a9a3214b7 [DEV-7084] TAMER: ir::asg::BaseAsg:🆕 New associated function
Profiling showed that creating an initial capacity of 0 did not have a
notable affect on performance.
2020-04-28 09:06:25 -04:00
Mike Gerwitz ecc2e33ba7 [DEV-7084] TAMER: xmlo::AsgBuilder: Accept XmloResult iterator
This flips the API from using XmloWriter as the context to using Asg and
consuming anything that can produce XmloResults.  This not only makes more
sense, but avoids having to create a trait for XmloReader, and simplifies
the trait bounds we have to concern ourselves with.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 323ea79bf8 [DEV-7084] TAMER: Basic AsgBuilder cleanup
This just tidies things up a little bit before I get into some further
refactoring.  I wrote the original code when I was just learning Rust not
too long ago, so it's interesting to see how my understanding has changed
over that relatively short period of time.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 9220de4769 [DEV-7084] TAMER: Finish encapsulating petgraph
This will allow us to migrate away from Petgraph in the future should we
choose to do so.
2020-04-28 09:06:25 -04:00