Commit Graph

135 Commits (01722c9c3bd191179e4dcbac9841f743fb6e2166)

Author SHA1 Message Date
Mike Gerwitz 01722c9c3b tamer: Symbol{Index=>Id}
The former was a misnomer (it represents an index _entry_).  This name is
also shorter, which is nice, considering how often it'll be used.
2021-07-30 13:32:32 -04:00
Mike Gerwitz 0fc8a1a4df tamer: Remove default SymbolIndex (et al) index type
Oh boy.  What a mess of a change.

This demonstrates some significant issues we have with Symbol.  I had
originally modelled the system a bit after Rustc's, but deviated in certain
regards:

  1. This has a confurable base type to enable better packing without bit
     twiddling and potentially unsafe tricks I'd rather avoid unless
     necessary; and
  2. The lifetime is not static, and there is no global, singleton interner;
     and
  3. I pass around references to a Symbol rather than passing around an
     index into an interner.

For #3---this is done because there's no singleton interner and therefore
resolving a symbol requires a direct reference to an available interner.  It
also wasn't clear to me (and still isn't, in fact) whether more than one
interner may be used for different contexts.

But, that doesn't preclude removing lifetimes and just passing around
indexes; in fact, I plan to do this in the frontend where the parser and
such will have direct interner access and can therefore just look up based
on a symbol index.  We could reserve references for situations where
exposing an interner would be undesirable.

Anyway, more to come...
2021-07-29 14:26:40 -04:00
Mike Gerwitz e6ad2be5b9 tamer: sym: Primitive-based SupportedSymbolIndex
As mentioned in the previous commit, this flips the types such that the base
type if the primitive and the associated type is the `NonZero*` type; this
is much more natural, concise, and allows Rust to infer the proper type in
most every situation.

The next step will be to stop defaulting the index type for SymbolIndex and
related, since we are about to care very much what size it is (compiler
vs. linker).
2021-07-28 15:21:24 -04:00
Mike Gerwitz e562d7fcc8 tamer: sym: Begin SymbolIndex base data generalization
This was previously a NonZeroU32, but it was intended to support NonZeroU16
as well for packages, so that we can fit symbols into smaller spaces.  In
particular, the upcoming Span wants to fit within 8 bytes, and so requires a
smaller SymbolIndex type.

I'm unhappy with this current implementation, and so comments are unfinished
and there are a couple ignores for dead code warnings.  I want to flip the
`SupportedSymbolIndex` trait so that users can specify the primitive rather
than the NonZero* type, which is really awkward-looking and verbose,
especially if you have to do `SymbolIndex::<NonZeroU32>::from_int` or
something.  It also prevents (at least in the cases I've observed) Rust from
inferring the proper type for you based on the argument you provide.

So, the goal will be `SymbolIndex::<u32>::from_int(n)`, for example.
2021-07-28 15:21:15 -04:00
Mike Gerwitz ca6ef3ed36 tamer: frontend: Begin basic XML parsing
The first step in the process is to emit the raw XML events that can then be
immediately output again to echo the results into another file.  This will
then allow us to begin parsing the input incrementally, and begin to morph
the output into a real `xmlo` file.
2021-07-27 00:37:13 -04:00
Mike Gerwitz d9dcfe8777 tamer: Introduce tpwrap module to contain quick_xml::Error adapter
This adapter exists to implement PartialEq so that it can be derived on
Error objects.  This is used primarily (well, exclusively atm) for tests.
2021-07-23 23:23:55 -04:00
Mike Gerwitz fb8422d670 tamer: Initial frontend concept
This introduces the beginnings of frontends for TAMER, gated behind a
`wip-features` flag.

This will be introduced in stages:

  1. Replace the existing copy with a parser-based copy (echo back out the
     tokens), when the flag is on.
  2. Begin to parse portions of the source, augmenting the output xmlo (xmli
     at the moment).  The XSLT-based compiler will be modified to skip
     compilation steps as necessary.

As portions of the compilation are implemented in TAMER, they'll be placed
behind their own feature flags and stabalized, which will incrementally
remove the compilation steps from the XSLT-based system.  The result should
be substantial incremental performance improvements.

Short-term, the priorities are for loading identifiers into an IR
are (though the order may change):

  1. Echo
  2. Imports
  3. Extern declarations.
  4. Simple identifiers (e.g. param, const, template, etc).
  5. Classifications.
  6. Documentation expressions.
  7. Calculation expressions.
  8. Template applications.
  9. Template definitions.
  10. Inline templates.

After each of those are done, the resulting xmlo (xmli) will have fully
reconstructed the source document from the IR produced during parsing.
2021-07-23 22:24:08 -04:00
Mike Gerwitz 60372d2960 tamer: Makefile.am (all): Binaries and doc
`all` was previously the target for binaries only.
2021-07-23 22:23:10 -04:00
Mike Gerwitz 6ec1a49506 tamer: Makefile.am: Include feature flags for doc generation and tests
This was forgotten in the previous commit.
2021-07-23 15:56:33 -04:00
Mike Gerwitz f1a3273ee3 tamer: configure.ac: Configure-time feature flags (via Cargo) 2021-07-23 10:16:44 -04:00
Mike Gerwitz 5aaa1106cb tamer: obj::xmlo::reader::mock: Extract into crate::test::quick_xml
Other mocks exist here, and here it can be re-used for the upcoming XML
frontend.
2021-07-22 15:32:30 -04:00
Mike Gerwitz 2e50af1220 Copyright year update 2021 2021-07-22 15:00:15 -04:00
Mike Gerwitz e5bbd49166 tamer: obj::xmlo::reader: Extract tests separate file
The file's getting a bit large and the tests are rather complex.  Further,
LSP does better on smaller, less complex files.
2021-07-22 14:39:06 -04:00
Mike Gerwitz 1f24cfdf25 Remove :map: sym-dep generation
This was incorrect to begin with---it does not make sense that an input
mapping should depend upon the identifier that it maps to, in the sense that
we make use of these dependencies.  If we add weak symbol references in the
future, then this can be reintroduced.

By removing this, we free tameld from having to perform the check itself.

.rev-xmlo bumped to force rebuilding of object files since the linker now
expects that no such dependencies will exist within them.
2021-07-22 14:27:15 -04:00
Mike Gerwitz 90c6b51fd5 tamer: tameld: Place constants into static section in executable
This is something that changed when the TAMER POC was initially created, as
I was learning Rust.  I don't recall the original reason why this was moved,
but it could have been moved back long ago.

In our systems, constants can hold tables (as matrices) with tens or
hundreds of thousands of rows, and there are a number of them in certain
projects.  As an example, the YAML-based test cases for one of our systems
went from ~2m30s to ~45s after this change was made.  Much of the cost
savings comes from saving GC.
2021-07-21 14:53:15 -04:00
Mike Gerwitz 93fb1f1bdd tamer: Rust v1.{48=>53}.0 for rustdoc tool lints
A previous commit used a rustdoc tool lint, but that support wasn't added
until 1.52.0 (2021-05-06).

Note that this represents the minimum _required_ version to build TAMER; you
can use a later version.
2021-06-22 09:07:53 -04:00
Mike Gerwitz 716556c39f tamer: Rust 1.{42=>48}.0 for stable intra-doc links without nightly 2021-06-21 13:10:00 -04:00
Mike Gerwitz 96ea0302cc tamer: Cargo.lock: Dependency updates
This project has been on pause for over a year.
2021-06-21 12:46:38 -04:00
Mike Gerwitz 96ffd5f6e5 [DEV-8000] ir::asg: Error types for unresolved identifiers during sorting
This checks explicitly for unresolved objects while sorting and provides an
explicit error for them.  For example, this will catch externs that have no
concrete resolution.

This previously fell all the way through to the unreachable! block.  The old
POC implementation was catching unresolved objects, albeit with a debug
error.
2020-07-02 01:38:32 -04:00
Mike Gerwitz a2415c8c6f [DEV-8000] ir::asg::base: Replace Symbol::new_dummy
Use symbol_dummy!.
2020-07-01 15:53:56 -04:00
Mike Gerwitz 0d4bbe5e4e [DEV-8000] ir::asg: Introduce SortableAsgError
This will be used for the next commit, but this change has been isolated
both because it distracts from the implementation change in the next commit,
and because it cleans up the code by removing the need for a type parameter
on `AsgError`.

Note that the sort test cases now use `unwrap` instead of having
`{,Sortable}AsgError` support one or the other---this is because that does
not currently happen in practice, and there is not supposed to be a
hierarchy; they are siblings (though perhaps their name may imply otherwise).
2020-07-01 13:42:14 -04:00
Mike Gerwitz f832feb3fa [DEV-8000] ir::asg::base::BaseAsg::check_cycles: Extract into function
The only reason this function was a method of `BaseAsg` was because of
`self.graph`, which is accessible within the scope of this
module.  `check_cycles` is logically associated with `SortableAsg`, and so
should exist alongside it (though it can't exist as an associated function
of that trait).
2020-07-01 11:02:20 -04:00
Joseph Frazer 43d00a8268 [DEV-7504] Add GraphML generation
We want to be able to build a representation of the dependency graph so
we can easily inspect it.

We do not want to make GraphML by default. It is better to use a tool.
We use "petgraph-graphml".
2020-05-13 08:04:48 -04:00
Mike Gerwitz 0127d4b698 TAMER: sym::Interner::index_lookup
This was originally omitted because there wasn't a use case for it.  Now
that we're adding context to errors, however, an owned value is highly
desirable.

This adds almost no measurable overhead to the internment system in
benchmarks (largely within the margin of error).
2020-04-29 11:33:41 -04:00
Mike Gerwitz 4b643385c8 TAMER: Update Cargo dependencies 2020-04-29 11:33:38 -04:00
Mike Gerwitz bcca5f7c49 [DEV-7084] TAMER: AsgBuilder and IR lowering docs 2020-04-28 13:39:55 -04:00
Mike Gerwitz 0f4b2d75f8 [DEV-7084] TAMER: obj::xmlo: Private inner modules 2020-04-28 11:08:05 -04:00
Mike Gerwitz 549e9ca23b [DEV-7084] TAMER: AsgBuilderState:🆕 New constructor 2020-04-28 09:06:25 -04:00
Mike Gerwitz 9893d56775 [DEV-7084] TAMER: Finalize AsgBuilder 2020-04-28 09:06:25 -04:00
Mike Gerwitz 32abc7dce2 [DEV-7084] TAMER: impl PartialEq for XmloError
This cannot be dervied because XmlError does not implement PartialEq,
which is quite the annoyance in tests.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 21a0bdcce1 [DEV-7084] TAMER: AsgBuilderError: Introduce proper error variants
This is a union (sum type) of three other errors types, plus errors specific
to this builder.

This commit does a good job demonstrating the boilerplate, as well as a need
for additional context (in the case of `IdentKindError`), that we'll want to
work on abstracting away.
2020-04-28 09:06:25 -04:00
Mike Gerwitz ef79a763ac [DEV-7084] TAMER: Correct Ix trait bound for AsgError
The `Debug` bound is inconvenient and requires propagation to any types that
use it.  Further, it's really awkward having `Display` depend on `Debug`; if
we want to render a useful display here, we can write one.

To be clear: IndexType implements Debug.

For now, this is pretty-printed by another part of the code, which we don't
want to implement in `Display` because it requires looking things up from
the graph.
2020-04-28 09:06:25 -04:00
Mike Gerwitz cfc13f9016 [DEV-7084] TAMER: ir::asg::IdentKindError: Replace string with enum 2020-04-28 09:06:25 -04:00
Mike Gerwitz 0a9a3214b7 [DEV-7084] TAMER: ir::asg::BaseAsg:🆕 New associated function
Profiling showed that creating an initial capacity of 0 did not have a
notable affect on performance.
2020-04-28 09:06:25 -04:00
Mike Gerwitz ecc2e33ba7 [DEV-7084] TAMER: xmlo::AsgBuilder: Accept XmloResult iterator
This flips the API from using XmloWriter as the context to using Asg and
consuming anything that can produce XmloResults.  This not only makes more
sense, but avoids having to create a trait for XmloReader, and simplifies
the trait bounds we have to concern ourselves with.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 323ea79bf8 [DEV-7084] TAMER: Basic AsgBuilder cleanup
This just tidies things up a little bit before I get into some further
refactoring.  I wrote the original code when I was just learning Rust not
too long ago, so it's interesting to see how my understanding has changed
over that relatively short period of time.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 9220de4769 [DEV-7084] TAMER: Finish encapsulating petgraph
This will allow us to migrate away from Petgraph in the future should we
choose to do so.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 0f423f3b24 [DEV-7084] TAMER: Simplify path canonicalization
This abstracts away the canonicalizer and solves the problem whereby
canonicalization was not being performed prior to recording whether a path
has been visited.  This ensures that multiple relative paths to the same
file will be properly recognized as visited.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 4a7e00c404 [DEV-7084] TAMER: ld::poc: Remove unused fragments arg 2020-04-28 09:06:25 -04:00
Mike Gerwitz c94120335f [DEV-7084] TAMER: ld::poc: Remove unnecessary initial path canonicalization
Less to refactor and test.
2020-04-28 09:06:25 -04:00
Mike Gerwitz da69118592 [DEV-7084] TAMER: AsgBuilderState
This completes the POC extraction for AsgBuilder, but is still POC
code.  The commits that follow will clean it up and provide tests.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 3f46917da9 [DEV-7084] TAMER: AsgBuilder extracted from POC
This extracts the changes nearly verbatim before doing refactoring so that
it's easier to observe what changes have been made.
2020-04-28 09:06:25 -04:00
Mike Gerwitz 7ed0691c45 [DEV-7084] TAMER: fs: impl File for BufReader
This further simplifies the POC linker.
2020-04-28 09:06:25 -04:00
Mike Gerwitz fbfb3c4ba2 [DEV-7084] TAMER: CanonicalFile
This will be entirely replaced in an upcoming commit.  See that for
details.  I don't feel like dealing with the conflicts for rearranging and
squashing these commits.
2020-04-28 09:06:25 -04:00
Mike Gerwitz d97e53a835 [DEV-7084] TAMER: fs: Basic filesystem abstraction
This also includes an implementation to visit paths only once.  Note that it
does not yet canonicalize the path before visiting, so relative paths to the
same file can slip through, and relative paths to _different_ files could be
erroneously considered to have been visited.

This will be fixed in an upcoming commit.
2020-04-28 09:06:19 -04:00
Mike Gerwitz 90ed4e9bd6 [DEV-7084] TAMER: From<B, &I> for XmloReader
This serves as a constructor for the time being, decoupling from POC.  We
may do something better once we have a better idea of how the various
abstractions around this will evolve.
2020-04-20 10:53:51 -04:00
Joseph Frazer 2c587e2d9d [DEV-7147] Add "tamec" executable
Add a stub executable that will eventually become a full-featured TAME
compiler. The first implementation will only copy the source file to an
intermediary file that will be compiled by the XSLT compiler.
2020-04-09 09:46:46 -04:00
Mike Gerwitz 8385b64e1d [DEV-7086] TAMER: Remove WIP linker warning
While it is true that this is still being finalized, the warnings originally
existed because tameld was not feature complete.  It is now.
2020-04-06 10:04:19 -04:00
Mike Gerwitz 68c7636be8 [DEV-7086] TAMER: ir::asg::base::test Add missing set_fragment failure test
Results the last remaining BaseAsg test TODO.
2020-04-06 09:56:13 -04:00
Mike Gerwitz b870480944 [DEV-7086] TAMER: ir::asg::TransitionError::BadFragmentDest tuple=>struct
Consistency.
2020-04-06 09:56:13 -04:00