employer
/
tame
1
0
Fork 0
Code Packages Releases Activity

tamer: pipeline::macro: Partially applied pipeline 

This configures the pipeline and returns a closure that can then be provided
with the source and sink.

The next obvious step would be to curry the source and sink.

But I wanted to commit this before I take a different (but equivalent)
approach that makes the pipeline operations more explicit and helps to guide
the user (developer) in developing and composing them.  The FP approach is
less boilerplate, but is also more general and provides less
guidance.  Given that composition at the topmost levels of the system,
especially with all the types involved, is one of the most confusing aspects
of the system---and one of the most important to get right and make clear,
since it's intended to elucidate the entire system at a high level, and
guide the reader.  Well, it does a poor job at that now, but that's the
ultimate goal.

In essence---brutally general abstractions make sense at lower levels, but
the complexity at higher levels benefits from rigid guardrails, even though
it does not necessitate it.

DEV-13162
main
Mike Gerwitz 2023-06-08 11:48:33 -04:00
parent 26c4076579
commit 31f6a102eb
3 changed files with 34 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
tamer/src/bin/tamec.rs
View File

@ -150,10 +150,8 @@ fn compile<R: Reporter>(
// TODO: Determine a good default capacity once we have this populated
// and can come up with some heuristics.
let (air_ctx,) = parse_package_xml(
src,
DefaultAsg::with_capacity(1024, 2048),
report_err,
let (air_ctx,) = parse_package_xml(DefaultAsg::with_capacity(1024, 2048))(
src, report_err,
)?;
match reporter.has_errors() {
@ -203,7 +201,7 @@ fn derive_xmli(
// TODO: Remove bad file?
// Let make do it?
let mut st = WriterState::default();
let (_asg,) = pipeline::lower_xmli(src, &asg, |result| {
let (_asg,) = pipeline::lower_xmli(&asg)(src, |result| {
// Write failures should immediately bail out;
// we can't skip writing portions of the file and
// just keep going!

7
tamer/src/ld/poc.rs
View File

@ -106,9 +106,10 @@ fn load_xmlo<P: AsRef<Path>, S: Escaper>(
let src = &mut lowerable(XmlXirReader::new(file, escaper, ctx));
let (mut state, mut air_ctx) = pipeline::load_xmlo::<_, TameldError, _>(
src, state, air_ctx, identity,
)?;
let (mut state, mut air_ctx) =
pipeline::load_xmlo::<_, TameldError, _, _, _>(state, air_ctx)(
src, identity,
)?;
let mut dir = path;
dir.pop();

49
tamer/src/pipeline/macro.rs
View File

@ -72,19 +72,20 @@ macro_rules! lower_pipeline {
/// of the types of all parsers in the pipeline.
/// It can be understood as:
///
/// 1. A function accepting three classes of arguments:
/// 1. The _source_ token stream,
/// which consists of tokens expected by the first parser
/// in the pipeline;
/// 2. _Context_ for certain parsers that request it,
/// allowing for state to persist between separate
/// pipelines; and
/// 3. A _sink_ that serves as the final destination for the
/// token stream.
/// 2. A [`Result`] consisting of the updated context that was
/// 1. A function accepting _context_ for whatever parsers request
/// it,
/// allowing both for configuration and for state to
/// persist between separate pipelines.
/// This returns a closure representing a configured pipeline.
/// 2. The _source_ token stream is accepted by the closure,
/// which consists of tokens expected by the first parser
/// in the pipeline;
/// 4. A _sink_ serves as the final destination for the token
/// stream.
/// 5. A [`Result`] consisting of the updated context that was
/// originally passed into the function,
/// so that it may be utilized in future pipelines.
/// 3. A _recoverable error_ type `ER` that may be utilized when
/// 6. A _recoverable error_ type `ER` that may be utilized when
/// compilation should continue despite an error.
/// All parsers are expected to perform their own error
/// recovery in an attempt to continue parsing to discover
@ -94,25 +95,17 @@ macro_rules! lower_pipeline {
/// errors of any parser in the pipeline,
/// which is the reason for the large block of
/// [`From`]s in this function's `where` clause.
/// 4. An _unrecoverable error_ type `EU` that may be yielded by
/// 7. An _unrecoverable error_ type `EU` that may be yielded by
/// the sink to terminate compilation immediately.
/// This is a component of the [`Result`] type that is
/// ultimately yielded as the result of this function.
$vis fn $fn<$($l,)? ES: Diagnostic, ER: Diagnostic, EU: Diagnostic>(
src: impl LowerSource<
UnknownToken,
lower_pipeline!(@first_tok_ty $($lower),*),
ES
>,
$vis fn $fn<$($l,)? ES: Diagnostic, ER: Diagnostic, EU: Diagnostic, SA, SB>(
$(
// Each parser may optionally receive context from an
// earlier run.
$($ctx: impl Into<<$lower as ParseState>::PubContext>,)?
)*
sink: impl FnMut(
Result<lower_pipeline!(@last_obj_ty $($lower),*), ER>
) -> Result<(), EU>,
) -> Result<
) -> impl FnOnce(SA, SB) -> Result<
(
$(
// Any context that is passed in is also returned so
@ -147,7 +140,18 @@ macro_rules! lower_pipeline {
// which is _not_ an error that parsers are expected to
// recover from.
EU: From<FinalizeError>,
SA: LowerSource<
UnknownToken,
lower_pipeline!(@first_tok_ty $($lower),*),
ES
>,
SB: FnMut(
Result<lower_pipeline!(@last_obj_ty $($lower),*), ER>
) -> Result<(), EU>
{
move |src, sink| {
let lower_pipeline!(@ret_pat $($($ctx)?)*) = lower_pipeline!(
@body_head(src, sink)
$((|> $lower $([$ctx])? $(, until ($until))?))*
@ -156,6 +160,7 @@ macro_rules! lower_pipeline {
Ok(($(
$($ctx,)?
)*))
}
}
)*};