tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
// ASG IR
|
|
|
|
//
|
2023-01-17 23:09:25 -05:00
|
|
|
// Copyright (C) 2014-2023 Ryan Specialty, LLC.
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
//
|
|
|
|
// This file is part of TAME.
|
|
|
|
//
|
|
|
|
// This program is free software: you can redistribute it and/or modify
|
|
|
|
// it under the terms of the GNU General Public License as published by
|
|
|
|
// the Free Software Foundation, either version 3 of the License, or
|
|
|
|
// (at your option) any later version.
|
|
|
|
//
|
|
|
|
// This program is distributed in the hope that it will be useful,
|
|
|
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
// GNU General Public License for more details.
|
|
|
|
//
|
|
|
|
// You should have received a copy of the GNU General Public License
|
|
|
|
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
|
tamer: asg::air: AIR as a sum IR
This introduces a new macro `sum_ir!` to help with a long-standing problem
of not being able to easily narrow types in Rust without a whole lot of
boilerplate. This patch includes a bit of documentation, so see that for
more information.
This was not a welcome change---I jumped down this rabbit hole trying to
decompose `AirAggregate` so that I can share portions of parsing with the
current parser and a template parser. I can now proceed with that.
This is not the only implementation that I had tried. I previously inverted
the approach, as I've been doing manually for some time: manually create
types to hold the sets of variants, and then create a sum type to hold those
types. That works, but it resulted in a mess for systems that have to use
the IR, since now you have two enums to contend with. I didn't find that to
be appropriate, because we shouldn't complicate the external API for
implementation details.
The enum for IRs is supposed to be like a bytecode---a list of operations
that can be performed with the IR. They can be grouped if it makes sense
for a public API, but in my case, I only wanted subsets for the sake of
delegating responsibilities to smaller subsystems, while retaining the
context that `match` provides via its exhaustiveness checking but does not
expose as something concrete (which is deeply frustrating!).
Anyway, here we are; this'll be refined over time, hopefully, and
portions of it can be generalized for removing boilerplate from other IRs.
Another thing to note is that this syntax is really a compromise---I had to
move on, and I was spending too much time trying to get creative with
`macro_rules!`. It isn't the best, and it doesn't seem very Rust-like in
some places and is therefore not necessarily all that intuitive. This can
be refined further in the future. But the end result, all things
considered, isn't too bad.
DEV-13708
2023-03-02 15:15:28 -05:00
|
|
|
//! Intermediate representation for construction of the
|
|
|
|
//! [abstract semantic graph (ASG)](super) (AIR).
|
|
|
|
//!
|
|
|
|
//! AIR serves as an abstraction layer between higher-level parsers and the
|
|
|
|
//! aggregate ASG.
|
|
|
|
//! It allows parsers to operate as a raw stream of data without having to
|
|
|
|
//! worry about ownership of or references to the ASG,
|
|
|
|
//! and allows for multiple such parsers to be joined.
|
|
|
|
//!
|
|
|
|
//! AIR is _not_ intended to replace the API of the ASG---it
|
|
|
|
//! is intended as a termination point for the parsing pipeline,
|
|
|
|
//! and as such implements a subset of the ASG's API that is suitable
|
|
|
|
//! for aggregating raw data from source and object files.
|
|
|
|
//! Given that it does so little and is so close to the [`Asg`] API,
|
|
|
|
//! one might say that the abstraction is as light as air,
|
|
|
|
//! but that would surely result in face-palming and so we're not going
|
|
|
|
//! air such cringeworthy dad jokes here.
|
|
|
|
|
2023-03-08 11:18:51 -05:00
|
|
|
use self::expr::AirExprAggregateReachable;
|
|
|
|
|
|
|
|
use super::{graph::object::Pkg, Asg, AsgError, ObjectIndex};
|
2023-03-08 14:47:31 -05:00
|
|
|
use crate::{parse::prelude::*, sym::SymbolId};
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
use std::fmt::{Debug, Display};
|
|
|
|
|
tamer: asg::air: AIR as a sum IR
This introduces a new macro `sum_ir!` to help with a long-standing problem
of not being able to easily narrow types in Rust without a whole lot of
boilerplate. This patch includes a bit of documentation, so see that for
more information.
This was not a welcome change---I jumped down this rabbit hole trying to
decompose `AirAggregate` so that I can share portions of parsing with the
current parser and a template parser. I can now proceed with that.
This is not the only implementation that I had tried. I previously inverted
the approach, as I've been doing manually for some time: manually create
types to hold the sets of variants, and then create a sum type to hold those
types. That works, but it resulted in a mess for systems that have to use
the IR, since now you have two enums to contend with. I didn't find that to
be appropriate, because we shouldn't complicate the external API for
implementation details.
The enum for IRs is supposed to be like a bytecode---a list of operations
that can be performed with the IR. They can be grouped if it makes sense
for a public API, but in my case, I only wanted subsets for the sake of
delegating responsibilities to smaller subsystems, while retaining the
context that `match` provides via its exhaustiveness checking but does not
expose as something concrete (which is deeply frustrating!).
Anyway, here we are; this'll be refined over time, hopefully, and
portions of it can be generalized for removing boilerplate from other IRs.
Another thing to note is that this syntax is really a compromise---I had to
move on, and I was spending too much time trying to get creative with
`macro_rules!`. It isn't the best, and it doesn't seem very Rust-like in
some places and is therefore not necessarily all that intuitive. This can
be refined further in the future. But the end result, all things
considered, isn't too bad.
DEV-13708
2023-03-02 15:15:28 -05:00
|
|
|
#[macro_use]
|
|
|
|
mod ir;
|
|
|
|
pub use ir::Air;
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
|
2023-03-07 13:35:01 -05:00
|
|
|
mod expr;
|
2023-03-07 14:00:46 -05:00
|
|
|
mod tpl;
|
2023-03-07 13:35:01 -05:00
|
|
|
use expr::AirExprAggregate;
|
2023-03-07 14:00:46 -05:00
|
|
|
use tpl::AirTplAggregate;
|
2023-03-07 13:35:01 -05:00
|
|
|
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
pub type IdentSym = SymbolId;
|
|
|
|
pub type DepSym = SymbolId;
|
|
|
|
|
tamer: asg::air::AirAggregate: Initial impl of nested exprs
This introduces a number of concepts together, again to demonstrate that
they were derived.
This introduces support for nested expressions, extending the previous
work. It also supports error recovery for dangling expressions.
The parser states are a mess; there is a lot of duplicate code here that
needs refactoring, but I wanted to commit this first at a known-good state
so that the diff will demonstrate the need for the change that will
follow; the opportunities for abstraction are plainly visible.
The immutable stack introduced here could be generalized, if needed, in the
future.
Another important note is that Rust optimizes away the `memcpy`s for the
stack that was introduced here. The initial Parser Context was introduced
because of `ArrayVec` inhibiting that elision, but Vec never had that
problem. In the future, I may choose to go back and remove ArrayVec, but I
had wanted to keep memory allocation out of the picture as much as possible
to make the disassembly and call graph easier to reason about and to have
confidence that optimizations were being performed as intended.
With that said---it _should_ be eliding in tamec, since we're not doing
anything meaningful yet with the graph. It does also elide in tameld, but
it's possible that Rust recognizes that those code paths are never taken
because tameld does nothing with expressions. So I'll have to monitor this
as I progress and adjust accordingly; it's possible a future commit will
call BS on everything I just said.
Of course, the counter-point to that is that Rust is optimizing them away
anyway, but Vec _does_ still require allocation; I was hoping to keep such
allocation at the fringes. But another counter-point is that it _still_ is
allocated at the fringe, when the context is initialized for the parser as
part of the lowering pipeline. But I didn't know how that would all come
together back then.
...alright, enough rambling.
DEV-13160
2023-01-05 15:57:06 -05:00
|
|
|
/// AIR parser state.
|
2023-03-07 16:28:32 -05:00
|
|
|
#[derive(Debug, PartialEq, Default)]
|
2022-06-02 13:26:46 -04:00
|
|
|
pub enum AirAggregate {
|
tamer: asg::air::AirAggregate: Initial impl of nested exprs
This introduces a number of concepts together, again to demonstrate that
they were derived.
This introduces support for nested expressions, extending the previous
work. It also supports error recovery for dangling expressions.
The parser states are a mess; there is a lot of duplicate code here that
needs refactoring, but I wanted to commit this first at a known-good state
so that the diff will demonstrate the need for the change that will
follow; the opportunities for abstraction are plainly visible.
The immutable stack introduced here could be generalized, if needed, in the
future.
Another important note is that Rust optimizes away the `memcpy`s for the
stack that was introduced here. The initial Parser Context was introduced
because of `ArrayVec` inhibiting that elision, but Vec never had that
problem. In the future, I may choose to go back and remove ArrayVec, but I
had wanted to keep memory allocation out of the picture as much as possible
to make the disassembly and call graph easier to reason about and to have
confidence that optimizations were being performed as intended.
With that said---it _should_ be eliding in tamec, since we're not doing
anything meaningful yet with the graph. It does also elide in tameld, but
it's possible that Rust recognizes that those code paths are never taken
because tameld does nothing with expressions. So I'll have to monitor this
as I progress and adjust accordingly; it's possible a future commit will
call BS on everything I just said.
Of course, the counter-point to that is that Rust is optimizing them away
anyway, but Vec _does_ still require allocation; I was hoping to keep such
allocation at the fringes. But another counter-point is that it _still_ is
allocated at the fringe, when the context is initialized for the parser as
part of the lowering pipeline. But I didn't know how that would all come
together back then.
...alright, enough rambling.
DEV-13160
2023-01-05 15:57:06 -05:00
|
|
|
/// Parser is not currently performing any work.
|
2023-03-06 12:03:55 -05:00
|
|
|
#[default]
|
|
|
|
Empty,
|
tamer: Initial concept for AIR/ASG Expr
This begins to place expressions on the graph---something that I've been
thinking about for a couple of years now, so it's interesting to finally be
doing it.
This is going to evolve; I want to get some things committed so that it's
clear how I'm moving forward. The ASG makes things a bit awkward for a
number of reasons:
1. I'm dealing with older code where I had a different model of doing
things;
2. It's mutable, rather than the mostly-functional lowering pipeline;
3. We're dealing with an aggregate ever-evolving blob of data (the graph)
rather than a stream of tokens; and
4. We don't have as many type guarantees.
I've shown with the lowering pipeline that I'm able to take a mutable
reference and convert it into something that's both functional and
performant, where I remove it from its container (an `Option`), create a new
version of it, and place it back. Rust is able to optimize away the memcpys
and such and just directly manipulate the underlying value, which is often a
register with all of the inlining.
_But_ this is a different scenario now. The lowering pipeline has a narrow
context. The graph has to keep hitting memory. So we'll see how this
goes. But it's most important to get this working and measure how it
performs; I'm not trying to prematurely optimize. My attempts right now are
for the way that I wish to develop.
Speaking to #4 above, it also sucks that I'm not able to type the
relationships between nodes on the graph. Rather, it's not that I _can't_,
but a project to created a typed graph library is beyond the scope of this
work and would take far too much time. I'll leave that to a personal,
non-work project. Instead, I'm going to have to narrow the type any time
the graph is accessed. And while that sucks, I'm going to do my best to
encapsulate those details to make it as seamless as possible API-wise. The
performance hit of performing the narrowing I'm hoping will be very small
relative to all the business logic going on (a single cache miss is bound to
be far more expensive than many narrowings which are just integer
comparisons and branching)...but we'll see. Introducing branching sucks,
but branch prediction is pretty damn good in modern CPUs.
DEV-13160
2022-12-21 16:47:04 -05:00
|
|
|
|
2023-03-06 12:03:55 -05:00
|
|
|
/// Expecting a package-level token.
|
2023-03-08 13:21:31 -05:00
|
|
|
Toplevel(ObjectIndex<Pkg>, AirExprAggregateReachable<Pkg>),
|
2023-01-30 16:51:24 -05:00
|
|
|
|
2023-03-06 12:03:55 -05:00
|
|
|
/// Parsing an expression.
|
tamer: Initial concept for AIR/ASG Expr
This begins to place expressions on the graph---something that I've been
thinking about for a couple of years now, so it's interesting to finally be
doing it.
This is going to evolve; I want to get some things committed so that it's
clear how I'm moving forward. The ASG makes things a bit awkward for a
number of reasons:
1. I'm dealing with older code where I had a different model of doing
things;
2. It's mutable, rather than the mostly-functional lowering pipeline;
3. We're dealing with an aggregate ever-evolving blob of data (the graph)
rather than a stream of tokens; and
4. We don't have as many type guarantees.
I've shown with the lowering pipeline that I'm able to take a mutable
reference and convert it into something that's both functional and
performant, where I remove it from its container (an `Option`), create a new
version of it, and place it back. Rust is able to optimize away the memcpys
and such and just directly manipulate the underlying value, which is often a
register with all of the inlining.
_But_ this is a different scenario now. The lowering pipeline has a narrow
context. The graph has to keep hitting memory. So we'll see how this
goes. But it's most important to get this working and measure how it
performs; I'm not trying to prematurely optimize. My attempts right now are
for the way that I wish to develop.
Speaking to #4 above, it also sucks that I'm not able to type the
relationships between nodes on the graph. Rather, it's not that I _can't_,
but a project to created a typed graph library is beyond the scope of this
work and would take far too much time. I'll leave that to a personal,
non-work project. Instead, I'm going to have to narrow the type any time
the graph is accessed. And while that sucks, I'm going to do my best to
encapsulate those details to make it as seamless as possible API-wise. The
performance hit of performing the narrowing I'm hoping will be very small
relative to all the business logic going on (a single cache miss is bound to
be far more expensive than many narrowings which are just integer
comparisons and branching)...but we'll see. Introducing branching sucks,
but branch prediction is pretty damn good in modern CPUs.
DEV-13160
2022-12-21 16:47:04 -05:00
|
|
|
///
|
2023-03-06 12:03:55 -05:00
|
|
|
/// This expects to inherit an [`AirExprAggregate`] from the prior state
|
|
|
|
/// so that we are not continuously re-allocating its stack for each
|
|
|
|
/// new expression root.
|
2023-03-08 11:18:51 -05:00
|
|
|
PkgExpr(ObjectIndex<Pkg>, AirExprAggregateReachable<Pkg>),
|
2023-02-28 15:31:49 -05:00
|
|
|
|
|
|
|
/// Parser is in template parsing mode.
|
|
|
|
///
|
|
|
|
/// All objects encountered until the closing [`Air::TplClose`] will be
|
|
|
|
/// parented to this template rather than the parent [`Pkg`].
|
|
|
|
/// See [`Air::TplOpen`] for more information.
|
2023-03-08 11:18:51 -05:00
|
|
|
PkgTpl(
|
|
|
|
ObjectIndex<Pkg>,
|
|
|
|
AirExprAggregateReachable<Pkg>,
|
|
|
|
AirTplAggregate,
|
|
|
|
),
|
tamer: asg::air::AirAggregate: Initial impl of nested exprs
This introduces a number of concepts together, again to demonstrate that
they were derived.
This introduces support for nested expressions, extending the previous
work. It also supports error recovery for dangling expressions.
The parser states are a mess; there is a lot of duplicate code here that
needs refactoring, but I wanted to commit this first at a known-good state
so that the diff will demonstrate the need for the change that will
follow; the opportunities for abstraction are plainly visible.
The immutable stack introduced here could be generalized, if needed, in the
future.
Another important note is that Rust optimizes away the `memcpy`s for the
stack that was introduced here. The initial Parser Context was introduced
because of `ArrayVec` inhibiting that elision, but Vec never had that
problem. In the future, I may choose to go back and remove ArrayVec, but I
had wanted to keep memory allocation out of the picture as much as possible
to make the disassembly and call graph easier to reason about and to have
confidence that optimizations were being performed as intended.
With that said---it _should_ be eliding in tamec, since we're not doing
anything meaningful yet with the graph. It does also elide in tameld, but
it's possible that Rust recognizes that those code paths are never taken
because tameld does nothing with expressions. So I'll have to monitor this
as I progress and adjust accordingly; it's possible a future commit will
call BS on everything I just said.
Of course, the counter-point to that is that Rust is optimizing them away
anyway, but Vec _does_ still require allocation; I was hoping to keep such
allocation at the fringes. But another counter-point is that it _still_ is
allocated at the fringe, when the context is initialized for the parser as
part of the lowering pipeline. But I didn't know how that would all come
together back then.
...alright, enough rambling.
DEV-13160
2023-01-05 15:57:06 -05:00
|
|
|
}
|
|
|
|
|
tamer: Initial concept for AIR/ASG Expr
This begins to place expressions on the graph---something that I've been
thinking about for a couple of years now, so it's interesting to finally be
doing it.
This is going to evolve; I want to get some things committed so that it's
clear how I'm moving forward. The ASG makes things a bit awkward for a
number of reasons:
1. I'm dealing with older code where I had a different model of doing
things;
2. It's mutable, rather than the mostly-functional lowering pipeline;
3. We're dealing with an aggregate ever-evolving blob of data (the graph)
rather than a stream of tokens; and
4. We don't have as many type guarantees.
I've shown with the lowering pipeline that I'm able to take a mutable
reference and convert it into something that's both functional and
performant, where I remove it from its container (an `Option`), create a new
version of it, and place it back. Rust is able to optimize away the memcpys
and such and just directly manipulate the underlying value, which is often a
register with all of the inlining.
_But_ this is a different scenario now. The lowering pipeline has a narrow
context. The graph has to keep hitting memory. So we'll see how this
goes. But it's most important to get this working and measure how it
performs; I'm not trying to prematurely optimize. My attempts right now are
for the way that I wish to develop.
Speaking to #4 above, it also sucks that I'm not able to type the
relationships between nodes on the graph. Rather, it's not that I _can't_,
but a project to created a typed graph library is beyond the scope of this
work and would take far too much time. I'll leave that to a personal,
non-work project. Instead, I'm going to have to narrow the type any time
the graph is accessed. And while that sucks, I'm going to do my best to
encapsulate those details to make it as seamless as possible API-wise. The
performance hit of performing the narrowing I'm hoping will be very small
relative to all the business logic going on (a single cache miss is bound to
be far more expensive than many narrowings which are just integer
comparisons and branching)...but we'll see. Introducing branching sucks,
but branch prediction is pretty damn good in modern CPUs.
DEV-13160
2022-12-21 16:47:04 -05:00
|
|
|
impl Display for AirAggregate {
|
|
|
|
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
|
|
|
|
use AirAggregate::*;
|
|
|
|
|
|
|
|
match self {
|
2023-03-06 12:03:55 -05:00
|
|
|
Empty => write!(f, "awaiting AIR input for ASG"),
|
2023-03-08 13:21:31 -05:00
|
|
|
Toplevel(_, _) => {
|
2023-03-06 12:03:55 -05:00
|
|
|
write!(f, "expecting package header or an expression")
|
|
|
|
}
|
|
|
|
PkgExpr(_, expr) => {
|
|
|
|
write!(f, "defining a package expression: {expr}")
|
tamer: asg::air::AirAggregate: Initial impl of nested exprs
This introduces a number of concepts together, again to demonstrate that
they were derived.
This introduces support for nested expressions, extending the previous
work. It also supports error recovery for dangling expressions.
The parser states are a mess; there is a lot of duplicate code here that
needs refactoring, but I wanted to commit this first at a known-good state
so that the diff will demonstrate the need for the change that will
follow; the opportunities for abstraction are plainly visible.
The immutable stack introduced here could be generalized, if needed, in the
future.
Another important note is that Rust optimizes away the `memcpy`s for the
stack that was introduced here. The initial Parser Context was introduced
because of `ArrayVec` inhibiting that elision, but Vec never had that
problem. In the future, I may choose to go back and remove ArrayVec, but I
had wanted to keep memory allocation out of the picture as much as possible
to make the disassembly and call graph easier to reason about and to have
confidence that optimizations were being performed as intended.
With that said---it _should_ be eliding in tamec, since we're not doing
anything meaningful yet with the graph. It does also elide in tameld, but
it's possible that Rust recognizes that those code paths are never taken
because tameld does nothing with expressions. So I'll have to monitor this
as I progress and adjust accordingly; it's possible a future commit will
call BS on everything I just said.
Of course, the counter-point to that is that Rust is optimizing them away
anyway, but Vec _does_ still require allocation; I was hoping to keep such
allocation at the fringes. But another counter-point is that it _still_ is
allocated at the fringe, when the context is initialized for the parser as
part of the lowering pipeline. But I didn't know how that would all come
together back then.
...alright, enough rambling.
DEV-13160
2023-01-05 15:57:06 -05:00
|
|
|
}
|
2023-03-07 12:41:47 -05:00
|
|
|
PkgTpl(_, _, tpl) => {
|
|
|
|
write!(f, "building a template: {tpl}",)
|
2023-02-28 15:31:49 -05:00
|
|
|
}
|
tamer: Initial concept for AIR/ASG Expr
This begins to place expressions on the graph---something that I've been
thinking about for a couple of years now, so it's interesting to finally be
doing it.
This is going to evolve; I want to get some things committed so that it's
clear how I'm moving forward. The ASG makes things a bit awkward for a
number of reasons:
1. I'm dealing with older code where I had a different model of doing
things;
2. It's mutable, rather than the mostly-functional lowering pipeline;
3. We're dealing with an aggregate ever-evolving blob of data (the graph)
rather than a stream of tokens; and
4. We don't have as many type guarantees.
I've shown with the lowering pipeline that I'm able to take a mutable
reference and convert it into something that's both functional and
performant, where I remove it from its container (an `Option`), create a new
version of it, and place it back. Rust is able to optimize away the memcpys
and such and just directly manipulate the underlying value, which is often a
register with all of the inlining.
_But_ this is a different scenario now. The lowering pipeline has a narrow
context. The graph has to keep hitting memory. So we'll see how this
goes. But it's most important to get this working and measure how it
performs; I'm not trying to prematurely optimize. My attempts right now are
for the way that I wish to develop.
Speaking to #4 above, it also sucks that I'm not able to type the
relationships between nodes on the graph. Rather, it's not that I _can't_,
but a project to created a typed graph library is beyond the scope of this
work and would take far too much time. I'll leave that to a personal,
non-work project. Instead, I'm going to have to narrow the type any time
the graph is accessed. And while that sucks, I'm going to do my best to
encapsulate those details to make it as seamless as possible API-wise. The
performance hit of performing the narrowing I'm hoping will be very small
relative to all the business logic going on (a single cache miss is bound to
be far more expensive than many narrowings which are just integer
comparisons and branching)...but we'll see. Introducing branching sucks,
but branch prediction is pretty damn good in modern CPUs.
DEV-13160
2022-12-21 16:47:04 -05:00
|
|
|
}
|
|
|
|
}
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
}
|
|
|
|
|
2022-06-02 13:26:46 -04:00
|
|
|
impl ParseState for AirAggregate {
|
2022-12-13 14:36:38 -05:00
|
|
|
type Token = Air;
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
type Object = ();
|
|
|
|
type Error = AsgError;
|
|
|
|
|
|
|
|
/// Destination [`Asg`] that this parser lowers into.
|
|
|
|
///
|
tamer: asg::air: AIR as a sum IR
This introduces a new macro `sum_ir!` to help with a long-standing problem
of not being able to easily narrow types in Rust without a whole lot of
boilerplate. This patch includes a bit of documentation, so see that for
more information.
This was not a welcome change---I jumped down this rabbit hole trying to
decompose `AirAggregate` so that I can share portions of parsing with the
current parser and a template parser. I can now proceed with that.
This is not the only implementation that I had tried. I previously inverted
the approach, as I've been doing manually for some time: manually create
types to hold the sets of variants, and then create a sum type to hold those
types. That works, but it resulted in a mess for systems that have to use
the IR, since now you have two enums to contend with. I didn't find that to
be appropriate, because we shouldn't complicate the external API for
implementation details.
The enum for IRs is supposed to be like a bytecode---a list of operations
that can be performed with the IR. They can be grouped if it makes sense
for a public API, but in my case, I only wanted subsets for the sake of
delegating responsibilities to smaller subsystems, while retaining the
context that `match` provides via its exhaustiveness checking but does not
expose as something concrete (which is deeply frustrating!).
Anyway, here we are; this'll be refined over time, hopefully, and
portions of it can be generalized for removing boilerplate from other IRs.
Another thing to note is that this syntax is really a compromise---I had to
move on, and I was spending too much time trying to get creative with
`macro_rules!`. It isn't the best, and it doesn't seem very Rust-like in
some places and is therefore not necessarily all that intuitive. This can
be refined further in the future. But the end result, all things
considered, isn't too bad.
DEV-13708
2023-03-02 15:15:28 -05:00
|
|
|
/// This ASG will be yielded by [`crate::parse::Parser::finalize`].
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
type Context = Asg;
|
|
|
|
|
|
|
|
fn parse_token(
|
|
|
|
self,
|
|
|
|
tok: Self::Token,
|
|
|
|
asg: &mut Self::Context,
|
|
|
|
) -> crate::parse::TransitionResult<Self> {
|
tamer: asg::air: AIR as a sum IR
This introduces a new macro `sum_ir!` to help with a long-standing problem
of not being able to easily narrow types in Rust without a whole lot of
boilerplate. This patch includes a bit of documentation, so see that for
more information.
This was not a welcome change---I jumped down this rabbit hole trying to
decompose `AirAggregate` so that I can share portions of parsing with the
current parser and a template parser. I can now proceed with that.
This is not the only implementation that I had tried. I previously inverted
the approach, as I've been doing manually for some time: manually create
types to hold the sets of variants, and then create a sum type to hold those
types. That works, but it resulted in a mess for systems that have to use
the IR, since now you have two enums to contend with. I didn't find that to
be appropriate, because we shouldn't complicate the external API for
implementation details.
The enum for IRs is supposed to be like a bytecode---a list of operations
that can be performed with the IR. They can be grouped if it makes sense
for a public API, but in my case, I only wanted subsets for the sake of
delegating responsibilities to smaller subsystems, while retaining the
context that `match` provides via its exhaustiveness checking but does not
expose as something concrete (which is deeply frustrating!).
Anyway, here we are; this'll be refined over time, hopefully, and
portions of it can be generalized for removing boilerplate from other IRs.
Another thing to note is that this syntax is really a compromise---I had to
move on, and I was spending too much time trying to get creative with
`macro_rules!`. It isn't the best, and it doesn't seem very Rust-like in
some places and is therefore not necessarily all that intuitive. This can
be refined further in the future. But the end result, all things
considered, isn't too bad.
DEV-13708
2023-03-02 15:15:28 -05:00
|
|
|
use ir::{
|
2023-03-08 13:21:31 -05:00
|
|
|
AirBind::*, AirIdent::*, AirPkg::*, AirSubsets::*, AirTodo::*,
|
|
|
|
AirTpl::*,
|
tamer: asg::air: AIR as a sum IR
This introduces a new macro `sum_ir!` to help with a long-standing problem
of not being able to easily narrow types in Rust without a whole lot of
boilerplate. This patch includes a bit of documentation, so see that for
more information.
This was not a welcome change---I jumped down this rabbit hole trying to
decompose `AirAggregate` so that I can share portions of parsing with the
current parser and a template parser. I can now proceed with that.
This is not the only implementation that I had tried. I previously inverted
the approach, as I've been doing manually for some time: manually create
types to hold the sets of variants, and then create a sum type to hold those
types. That works, but it resulted in a mess for systems that have to use
the IR, since now you have two enums to contend with. I didn't find that to
be appropriate, because we shouldn't complicate the external API for
implementation details.
The enum for IRs is supposed to be like a bytecode---a list of operations
that can be performed with the IR. They can be grouped if it makes sense
for a public API, but in my case, I only wanted subsets for the sake of
delegating responsibilities to smaller subsystems, while retaining the
context that `match` provides via its exhaustiveness checking but does not
expose as something concrete (which is deeply frustrating!).
Anyway, here we are; this'll be refined over time, hopefully, and
portions of it can be generalized for removing boilerplate from other IRs.
Another thing to note is that this syntax is really a compromise---I had to
move on, and I was spending too much time trying to get creative with
`macro_rules!`. It isn't the best, and it doesn't seem very Rust-like in
some places and is therefore not necessarily all that intuitive. This can
be refined further in the future. But the end result, all things
considered, isn't too bad.
DEV-13708
2023-03-02 15:15:28 -05:00
|
|
|
};
|
2022-06-02 13:26:46 -04:00
|
|
|
use AirAggregate::*;
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
|
2023-02-28 15:31:49 -05:00
|
|
|
// TODO: Seems to be about time for refactoring this...
|
tamer: asg::air: AIR as a sum IR
This introduces a new macro `sum_ir!` to help with a long-standing problem
of not being able to easily narrow types in Rust without a whole lot of
boilerplate. This patch includes a bit of documentation, so see that for
more information.
This was not a welcome change---I jumped down this rabbit hole trying to
decompose `AirAggregate` so that I can share portions of parsing with the
current parser and a template parser. I can now proceed with that.
This is not the only implementation that I had tried. I previously inverted
the approach, as I've been doing manually for some time: manually create
types to hold the sets of variants, and then create a sum type to hold those
types. That works, but it resulted in a mess for systems that have to use
the IR, since now you have two enums to contend with. I didn't find that to
be appropriate, because we shouldn't complicate the external API for
implementation details.
The enum for IRs is supposed to be like a bytecode---a list of operations
that can be performed with the IR. They can be grouped if it makes sense
for a public API, but in my case, I only wanted subsets for the sake of
delegating responsibilities to smaller subsystems, while retaining the
context that `match` provides via its exhaustiveness checking but does not
expose as something concrete (which is deeply frustrating!).
Anyway, here we are; this'll be refined over time, hopefully, and
portions of it can be generalized for removing boilerplate from other IRs.
Another thing to note is that this syntax is really a compromise---I had to
move on, and I was spending too much time trying to get creative with
`macro_rules!`. It isn't the best, and it doesn't seem very Rust-like in
some places and is therefore not necessarily all that intuitive. This can
be refined further in the future. But the end result, all things
considered, isn't too bad.
DEV-13708
2023-03-02 15:15:28 -05:00
|
|
|
match (self, tok.into()) {
|
|
|
|
(st, AirTodo(Todo(_))) => Transition(st).incomplete(),
|
tamer: asg::air::AirAggregate: Initial impl of nested exprs
This introduces a number of concepts together, again to demonstrate that
they were derived.
This introduces support for nested expressions, extending the previous
work. It also supports error recovery for dangling expressions.
The parser states are a mess; there is a lot of duplicate code here that
needs refactoring, but I wanted to commit this first at a known-good state
so that the diff will demonstrate the need for the change that will
follow; the opportunities for abstraction are plainly visible.
The immutable stack introduced here could be generalized, if needed, in the
future.
Another important note is that Rust optimizes away the `memcpy`s for the
stack that was introduced here. The initial Parser Context was introduced
because of `ArrayVec` inhibiting that elision, but Vec never had that
problem. In the future, I may choose to go back and remove ArrayVec, but I
had wanted to keep memory allocation out of the picture as much as possible
to make the disassembly and call graph easier to reason about and to have
confidence that optimizations were being performed as intended.
With that said---it _should_ be eliding in tamec, since we're not doing
anything meaningful yet with the graph. It does also elide in tameld, but
it's possible that Rust recognizes that those code paths are never taken
because tameld does nothing with expressions. So I'll have to monitor this
as I progress and adjust accordingly; it's possible a future commit will
call BS on everything I just said.
Of course, the counter-point to that is that Rust is optimizing them away
anyway, but Vec _does_ still require allocation; I was hoping to keep such
allocation at the fringes. But another counter-point is that it _still_ is
allocated at the fringe, when the context is initialized for the parser as
part of the lowering pipeline. But I didn't know how that would all come
together back then.
...alright, enough rambling.
DEV-13160
2023-01-05 15:57:06 -05:00
|
|
|
|
2023-03-06 12:03:55 -05:00
|
|
|
(Empty, AirPkg(PkgOpen(span))) => {
|
2023-01-31 22:00:51 -05:00
|
|
|
let oi_pkg = asg.create(Pkg::new(span)).root(asg);
|
2023-03-08 13:21:31 -05:00
|
|
|
Transition(Toplevel(oi_pkg, AirExprAggregate::new_in(oi_pkg)))
|
2023-03-07 16:28:32 -05:00
|
|
|
.incomplete()
|
2023-01-30 16:51:24 -05:00
|
|
|
}
|
|
|
|
|
2023-03-08 13:21:31 -05:00
|
|
|
(Toplevel(oi_pkg, expr), AirPkg(PkgOpen(span))) => {
|
|
|
|
Transition(Toplevel(oi_pkg, expr))
|
2023-01-30 16:51:24 -05:00
|
|
|
.err(AsgError::NestedPkgOpen(span, oi_pkg.span()))
|
|
|
|
}
|
2023-03-06 12:03:55 -05:00
|
|
|
|
|
|
|
(PkgExpr(oi_pkg, expr), AirPkg(PkgOpen(span))) => {
|
|
|
|
Transition(PkgExpr(oi_pkg, expr))
|
2023-01-30 16:51:24 -05:00
|
|
|
.err(AsgError::NestedPkgOpen(span, oi_pkg.span()))
|
|
|
|
}
|
|
|
|
|
2023-03-06 12:03:55 -05:00
|
|
|
// No expression was started.
|
2023-03-08 13:21:31 -05:00
|
|
|
(Toplevel(oi_pkg, _expr), AirPkg(PkgClose(span))) => {
|
2023-02-07 12:19:27 -05:00
|
|
|
oi_pkg.close(asg, span);
|
2023-03-06 12:03:55 -05:00
|
|
|
Transition(Empty).incomplete()
|
|
|
|
}
|
|
|
|
|
2023-03-08 13:21:31 -05:00
|
|
|
// TODO: We don't support package ids yet
|
|
|
|
(st @ Toplevel(..), AirBind(BindIdent(id))) => {
|
|
|
|
Transition(st).err(AsgError::InvalidExprBindContext(id))
|
|
|
|
}
|
|
|
|
(st @ Toplevel(..), AirBind(RefIdent(id))) => {
|
|
|
|
Transition(st).err(AsgError::InvalidExprRefContext(id))
|
2023-03-06 12:03:55 -05:00
|
|
|
}
|
|
|
|
|
2023-03-08 13:21:31 -05:00
|
|
|
(Toplevel(oi_pkg, expr), tok @ AirExpr(..)) => {
|
2023-03-06 12:03:55 -05:00
|
|
|
Transition(PkgExpr(oi_pkg, expr))
|
|
|
|
.incomplete()
|
|
|
|
.with_lookahead(tok)
|
|
|
|
}
|
|
|
|
|
2023-03-07 12:41:47 -05:00
|
|
|
// Note that templates may preempt expressions at any point,
|
|
|
|
// unlike in NIR at the time of writing.
|
|
|
|
(
|
2023-03-08 13:21:31 -05:00
|
|
|
Toplevel(oi_pkg, expr) | PkgExpr(oi_pkg, expr),
|
2023-03-07 12:41:47 -05:00
|
|
|
tok @ AirTpl(..),
|
|
|
|
) => Transition(PkgTpl(
|
|
|
|
oi_pkg,
|
|
|
|
expr,
|
2023-03-07 14:00:46 -05:00
|
|
|
AirTplAggregate::new_in_pkg(oi_pkg),
|
2023-03-07 12:41:47 -05:00
|
|
|
))
|
|
|
|
.incomplete()
|
|
|
|
.with_lookahead(tok),
|
|
|
|
|
2023-03-06 12:03:55 -05:00
|
|
|
// Note: We unfortunately can't match on `AirExpr | AirBind`
|
|
|
|
// and delegate in the same block
|
|
|
|
// (without having to duplicate type checks and then handle
|
|
|
|
// unreachable paths)
|
|
|
|
// because of the different inner types.
|
|
|
|
(PkgExpr(oi_pkg, expr), AirExpr(etok)) => {
|
|
|
|
Self::delegate_expr(asg, oi_pkg, expr, etok)
|
|
|
|
}
|
|
|
|
(PkgExpr(oi_pkg, expr), AirBind(etok)) => {
|
|
|
|
Self::delegate_expr(asg, oi_pkg, expr, etok)
|
|
|
|
}
|
|
|
|
|
2023-03-07 12:41:47 -05:00
|
|
|
// Templates can contain just about anything,
|
|
|
|
// so completely hand over parsing when we're in template mode.
|
|
|
|
(PkgTpl(oi_pkg, stored_expr, tplst), tok) => {
|
|
|
|
Self::delegate_tpl(asg, oi_pkg, stored_expr, tplst, tok.into())
|
|
|
|
}
|
|
|
|
|
|
|
|
(Empty, AirTpl(TplClose(..))) => {
|
|
|
|
todo!("Empty AirTpl::TplClose")
|
2023-03-06 12:03:55 -05:00
|
|
|
}
|
|
|
|
|
|
|
|
(Empty, AirPkg(PkgClose(span))) => {
|
|
|
|
Transition(Empty).err(AsgError::InvalidPkgCloseContext(span))
|
|
|
|
}
|
|
|
|
|
|
|
|
(PkgExpr(oi_pkg, expr), AirPkg(PkgClose(span))) => {
|
|
|
|
match expr.is_accepting(asg) {
|
|
|
|
true => {
|
|
|
|
// TODO: this is duplicated with the above
|
|
|
|
oi_pkg.close(asg, span);
|
|
|
|
Transition(Empty).incomplete()
|
|
|
|
}
|
|
|
|
false => Transition(PkgExpr(oi_pkg, expr))
|
|
|
|
.err(AsgError::InvalidPkgCloseContext(span)),
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2023-03-06 15:17:02 -05:00
|
|
|
(Empty, tok @ (AirExpr(..) | AirBind(..) | AirTpl(TplOpen(_)))) => {
|
|
|
|
Transition(Empty).err(AsgError::PkgExpected(tok.span()))
|
|
|
|
}
|
2023-03-06 12:03:55 -05:00
|
|
|
|
|
|
|
(Empty, AirIdent(IdentDecl(name, kind, src))) => {
|
|
|
|
asg.declare(name, kind, src).map(|_| ()).transition(Empty)
|
|
|
|
}
|
|
|
|
|
|
|
|
(Empty, AirIdent(IdentExternDecl(name, kind, src))) => asg
|
|
|
|
.declare_extern(name, kind, src)
|
|
|
|
.map(|_| ())
|
|
|
|
.transition(Empty),
|
|
|
|
|
|
|
|
(Empty, AirIdent(IdentDep(sym, dep))) => {
|
|
|
|
asg.add_dep_lookup(sym, dep);
|
|
|
|
Transition(Empty).incomplete()
|
|
|
|
}
|
|
|
|
|
|
|
|
(Empty, AirIdent(IdentFragment(sym, text))) => {
|
|
|
|
asg.set_fragment(sym, text).map(|_| ()).transition(Empty)
|
|
|
|
}
|
|
|
|
|
|
|
|
(Empty, AirIdent(IdentRoot(sym))) => {
|
|
|
|
let obj = asg.lookup_or_missing(sym);
|
|
|
|
asg.add_root(obj);
|
|
|
|
|
|
|
|
Transition(Empty).incomplete()
|
|
|
|
}
|
|
|
|
|
|
|
|
(st, tok @ AirIdent(..)) => todo!("{st:?}, {tok:?}"),
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
fn is_accepting(&self, _: &Self::Context) -> bool {
|
|
|
|
matches!(self, Self::Empty)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
impl AirAggregate {
|
|
|
|
/// Delegate to the expression parser [`AirExprAggregate`].
|
|
|
|
///
|
|
|
|
/// TODO: This ought to be further reduced into primitives in the core
|
|
|
|
/// [`crate::parse`] framework.
|
|
|
|
fn delegate_expr(
|
|
|
|
asg: &mut <Self as ParseState>::Context,
|
|
|
|
oi_pkg: ObjectIndex<Pkg>,
|
2023-03-08 11:18:51 -05:00
|
|
|
expr: AirExprAggregateReachable<Pkg>,
|
|
|
|
etok: impl Into<<AirExprAggregateReachable<Pkg> as ParseState>::Token>,
|
2023-03-06 12:03:55 -05:00
|
|
|
) -> TransitionResult<Self> {
|
|
|
|
let tok = etok.into();
|
|
|
|
|
2023-03-07 10:44:28 -05:00
|
|
|
expr.parse_token(tok, asg).branch_dead::<Self, _>(
|
2023-03-08 13:21:31 -05:00
|
|
|
|expr, ()| Transition(Self::Toplevel(oi_pkg, expr)).incomplete(),
|
2023-03-07 10:44:28 -05:00
|
|
|
|expr, result, ()| {
|
2023-03-06 12:03:55 -05:00
|
|
|
result
|
|
|
|
.map(ParseStatus::reflexivity)
|
|
|
|
.transition(Self::PkgExpr(oi_pkg, expr))
|
|
|
|
},
|
2023-03-07 10:44:28 -05:00
|
|
|
(),
|
2023-03-06 12:03:55 -05:00
|
|
|
)
|
|
|
|
}
|
2023-03-07 12:41:47 -05:00
|
|
|
|
|
|
|
/// Delegate to the expression parser [`AirTplAggregate`].
|
|
|
|
///
|
|
|
|
/// After template parsing is complete
|
|
|
|
/// (when reaching a dead state),
|
|
|
|
/// the stored expression [`AirExprAggregate`] is reinstated,
|
|
|
|
/// allowing parsing to continue where it left off before being
|
|
|
|
/// preempted by template parsing.
|
|
|
|
fn delegate_tpl(
|
|
|
|
asg: &mut <Self as ParseState>::Context,
|
|
|
|
oi_pkg: ObjectIndex<Pkg>,
|
2023-03-08 11:18:51 -05:00
|
|
|
stored_expr: AirExprAggregateReachable<Pkg>,
|
2023-03-07 12:41:47 -05:00
|
|
|
tplst: AirTplAggregate,
|
|
|
|
tok: Air,
|
|
|
|
) -> TransitionResult<Self> {
|
2023-03-07 16:28:32 -05:00
|
|
|
tplst.parse_token(tok, asg).branch_dead::<Self, _>(
|
|
|
|
|_, stored_expr| {
|
|
|
|
Transition(Self::PkgExpr(oi_pkg, stored_expr)).incomplete()
|
|
|
|
},
|
|
|
|
|tplst, result, stored_expr| {
|
|
|
|
result
|
|
|
|
.map(ParseStatus::reflexivity)
|
|
|
|
.transition(Self::PkgTpl(oi_pkg, stored_expr, tplst))
|
|
|
|
},
|
|
|
|
stored_expr,
|
|
|
|
)
|
2023-03-07 12:41:47 -05:00
|
|
|
}
|
2023-03-06 12:03:55 -05:00
|
|
|
}
|
|
|
|
|
tamer: Refactor asg_builder into obj::xmlo::lower and asg::air
This finally uses `parse` all the way up to aggregation into the ASG, as can
be seen by the mess in `poc`. This will be further simplified---I just need
to get this committed so that I can mentally get it off my plate. I've been
separating this commit into smaller commits, but there's a point where it's
just not worth the effort anymore. I don't like making large changes such
as this one.
There is still work to do here. First, it's worth re-mentioning that
`poc` means "proof-of-concept", and represents things that still need a
proper home/abstraction.
Secondly, `poc` is retrieving the context of two parsers---`LowerContext`
and `Asg`. The latter is desirable, since it's the final aggregation point,
but the former needs to be eliminated; in particular, packages need to be
worked into the ASG so that `found` can be removed.
Recursively loading `xmlo` files still happens in `poc`, but the compiler
will need this as well. Once packages are on the ASG, along with their
state, that responsibility can be generalized as well.
That will then simplify lowering even further, to the point where hopefully
everything has the same shape (once final aggregation has an abstraction),
after which we can then create a final abstraction to concisely stitch
everything together. Right now, Rust isn't able to infer `S` for
`Lower<S, LS>`, which is unfortunate, but we'll be able to help it along
with a more explicit abstraction.
DEV-11864
2022-05-27 13:51:29 -04:00
|
|
|
#[cfg(test)]
|
2022-12-13 13:57:04 -05:00
|
|
|
mod test;
|