Fork 0

555 lines
16 KiB
Raw Normal View History

@c This document is part of the Liza Data Collection Framework manual.
@c Copyright (C) 2017 LoVullo Associates, Inc.
@c Permission is granted to copy, distribute and/or modify this document
@c under the terms of the GNU Free Documentation License, Version 1.3
@c or any later version published by the Free Software Foundation;
@c with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
@c Texts. A copy of the license is included in the section entitled ``GNU
@c Free Documentation License''.
@node Design
@chapter Design & Architecture
N.B. This document may reference features that are not yet
present in Liza; they will be moved into this repository in time.}
Liza is fundamentally a data collection framework@mdash{
}a fancy form for collecting, validating, and lightly processing
user data.
The main components of the system are:
@table @strong
2017-03-16 13:23:19 -04:00
@cindex Assertions
@item Assertions
Basic validations against bucket data,
producing errors and manipulating control flow.
Invokes triggers to manipulate the UI and document.
Assertions are compiled from Program sources.
@cindex Bucket
@item Bucket
The key/value store into which all document data are stored.
Supports staging and rollback of data,
processing deltas,
and provides hooks that drive the rest of the system.
@cindex Calculated Value
@item Calculated Values
A small sub-system for calculating bucket values from other values.
@cindex Client
@item Client
Basic logic for navigating between steps,
prompting for user actions,
display help text and basic document data,
communicate with server,
@cindex Developer Dialog
@item Developer Dialog
Renders information about the system for debugging the client.
Can monitor the bucket, assertions, classification results, and
provides other useful features.
@cindex Predicate
@item Predicate System
Processes classification data from external classifiers to determine
applicability of specific questions.
These data are used to determine what assertions are performed,
what questions and groups display,
and more.
@xref{Predicate System}.
@cindex Program
@item Program
2017-03-16 13:23:19 -04:00
Internal representation of the Program with delegation of events to
the assertion system.
Contains compiled representation of all steps, groups, questions,
assertions, metadata, and others.
@cindex Program, User Interface
@cindex User Interface, Program
@item Program UI
2017-03-16 13:23:19 -04:00
Rendering of elements specific to Programs,
such as steps, groups, and questions.
This is the equivalent of an HTML form.
Directly monitors the bucket to perform UI updates.
@xref{Program UI}.
@cindex Program, XML
@item Program XML
2017-03-16 13:23:19 -04:00
The source code for a Program, in XML format.
@xref{Program XML}.
@cindex Server
@item Server
2017-03-16 13:23:19 -04:00
Provides REST API for serving Programs; saving data;
revalidating, filtering, and recalculating data;
and other types of processing.
Code is shared with the client,
ensuring identical behavior for appropriate behaviors.
@cindex Type Validation
@cindex Validation, Type
@item Type Validation
Validates and formats bucket values for specific field (question)
For example, a date field must be in a recognized date format,
and will be normalized for display.
@end table
2017-03-16 13:23:19 -04:00
More information about each can be found in their respective section.
* Assertions::
* Bucket::
* Client::
* Predicate System::
2017-03-16 13:23:19 -04:00
* Program::
* Program UI::
* Program XML::
* Validation::
@end menu
@node Assertions
@section Assertions
Assertions are compiled from the Program XML (@pxref{Program XML}).
Rather than using a library,
it compiles a mess of largely duplicate code inline.
This system needs to be @emph{replaced},
not modified.
A replacement can either be in the form of a library (removing most
if not all code generation from the Program XML compiler),
or possibly compile into classifications and use the classification
@emph{The latter option is preferred,
and would be more powerful with less maintenance.}}
2017-03-16 13:23:19 -04:00
@node Bucket
@section Bucket
* Value Assignment:Bucket Assignment. Writing data to the Bucket.
@end menu
@node Bucket Assignment
@subsection Bucket Value Assignment
@node Client
@section Client
The client is largely managed by a single class,
@srcref{src/client/Client.js, Client},
which has grown out of control.
@code{Client} mediates essentially the entire system.
Code is to be extracted out of this class as it is touched.
The other system mammoth is @code{Ui} (@pxref{Program UI}).}
2017-03-16 13:23:19 -04:00
* Error Handling::
@end menu
@node Error Handling
@subsection Error Handling
The complexity of this system and integration into legacy
layers has caused maintenance trouble in the past.
Each of the error checking layers need to be integrated
to reduce complexity.}
@cindex Error
@cindex Failure
2017-03-16 13:23:19 -04:00
There are three layers of error checking:@footnote{
Primarily for legacy reasons.
They are being consolodated as the system is touched.}
@item Required field checking@mdash{
}whether all required questions have been answered.
@item Type Validation@mdash{
2017-03-16 13:23:19 -04:00
}verify that questions contain valid data according to their
declared type. @ref{Validation}.
@item Assertions@mdash{
2017-03-16 13:23:19 -04:00
}arbitrary checks on data.
2017-03-16 13:23:19 -04:00
@end enumerate
@cindex Required Field
@cindex Field, Required
@cindex Field, Fixed
@cindex Error, Fixed
@cindex Error, Required
@cindex Fixed, Error
2017-03-16 13:23:19 -04:00
Required fields fail serially@mdash{
}the system will notify the user of the required field,
and direct him/her to it (usually through scrolling).
A field is marked as @dfn{fixed} according to the rules
in @ref{Managing Error State}.
* Managing Error State:: Determining when failures should be marked as ``fixed''
@end menu
@node Managing Error State
@subsubsection Managing Error State
@cindex Failure Stack
@cindex Error Stack
Each failure caused by assertions is associated with a
@dfn{failure stack}.
The stack represents the trail of assertions that have run,
containing the ids of all values asserted against.
When any field or classification changes that is represented on the
failure stack,
the failure for the failed field associated with that failure stack
is cleared.
If an assertion for some question @var{foo} first checked the value
of bucket field @var{bar},
and within its failure checked the value @var{c:predicate},
the failure stack would contain both of those ids.
If either @var{bar} or the @var{predicate} classification changed,
the question @var{foo} would have its error cleared.}
Error state is managed by
@srcref{src/validate/ValidStateMonitor.js, ValidStateMonitor}.
2017-03-16 13:23:19 -04:00
@node Predicate System
@section Predicate System
New programs (using the old incarnation of TAME) use the classifier
embedded into the rater by TAME.
Old ones, however, still use the @dfn{Global Classifier}.
This system isn't as well tested as TAME's@mdash{
}which needs to work properly for the sake of calculating premium@mdash{
}and has suffered from a number of bugs in the past.
The solution is to migrate all programs to TAME and remove that old
For a practical application of these concepts, see its use in the
Program@tie{}XML (@pxref{Specifying Predicates}).}
@cindex Predicate
@cindex Classifier
@cindex Applicability
@cindex Domain of discourse, Predicate
The @dfn{predicate system} determines the @dfn{applicability} of
certain objects (like questions and assertions) by associating them
with predicates.
The domain of discourse (variables which may be quantified) is listed
in @ref{t:predicate-dod}.
What it means for some object to be applicable depends on the context.
@float Table, t:predicate-dod
@multitable @columnfractions 0.25 0.10 0.65
@headitem Type @tab Prefix @tab Description
@item Classifications
@tab @emph{None}
@tab Results of applying an external @dfn{classifier} to the bucket
@item Bucket Truth Predicate
@tab @code{q:}
Whether the given name in the bucket (@pxref{Bucket}) is
non-empty and non-zero.
The prefix @samp{q:} refers to its most common use
case---questions (@pxref{Program UI,,Program@tie{}UI}).
@end multitable
@caption{Predicate system domain of discourse}
@end float
This system is limited to universal quantification over the domain of
For other quantifiers and higher-order logic,
defer to one of the systems that contributes to the domain of
like the classifier.@footnote{
This is usually TAME.
The Program XML also supports inline classifications
with TAME's syntax (@pxref{Specifying Predicates}).}
Predicates are usually specified in the Program XML
(@pxref{Specifying Predicates}) and compiled into the program
2017-03-16 13:23:19 -04:00
@node Program
@section Program
The @code{Program} class was one of the first prototypes created,
and has evolved poorly with the rest of the system.
It is the base class for all compiled programs,
and it glues together too many other systems with a terrible
API and little to no encapsulation.
With that said, it is one of the least touched classes (thus its
state); developers rarely have the need to touch @code{Program}.}
The core @code{Program} class that causes maintenance concerns needs
to be added to Liza.
It is currently mocked as necessary in test cases.}
2017-03-16 13:23:19 -04:00
@cindex Program
The @dfn{Program} is a declarative representation of an entire system.
It is the highest level of abstraction from a data perspective.
The user observes and interacts with a Program using the
@ref{Program UI,,Program@tie{}UI}.
@cindex Program, XML
Programs contain a lot of metadata that is not in a convenience
human-readable (or modifiable) format,
some of which are redundant.
Programs are ideally compiled from a @ref{Program XML,,Program@tie{}XML}
@node Program UI
@section Program UI
The @code{Ui} class,
in addition to @srcref{src/client/Client,Client} (@pxref{Client}),
represent the two monoliths of the system.
This mediates all UI-related tasks,
and still has far too many concerns with far too many
Code is to be extracted out of this class as it is touched.
All @file{program/ui/} contents still need to be moved into Liza.}
2017-03-16 13:23:19 -04:00
@cindex Program, User Interface
@cindex User Interface, Program
The @dfn{Program UI} renders a @ref{Program} as a form.
@cindex Step
@cindex Group
At the highest level,
steps are rendered in a tab-like manner,
above the main form content.
A step contains groups,
which in turn contain elements such as questions.
Groups are delimited in some manner defined by their style
(@pxref{Group Styles}).
@cindex Question
@cindex Question, Value Formatting
@cindex Bucket, Updating
Questions are rendered as form fields.
Any time the respective @ref{Bucket} field is changed,
the form field is updated to reflect those changes,
after having first been formatted with the appropriate validator
(@pxref{Formatting Values}).
When a question is changed by the user,
the value is expected to be propagated to the Bucket
(@pxref{Bucket Assignment}).
@cindex Navigation Bar
@cindex User Interface, Navigation Bar
@cindex User Interface, Button Navigation
Navigation between steps can be done via the
@dfn{Navigation Bar} above the step@tie{}content,
or using ``Go@tie{}Back'' and ``Continue'' buttons at the foot of the
@cindex Sidebar
A @dfn{Sidebar} is rendered adjacent to the step content.
It displays the name of the Program,
as well as configurable metadata (usually through the @samp{sidebar}
node of the @ref{Program XML,,Program@tie{}XML}).
It also displays question help text (also configured through the XML)
and any error messages (@pxref{Error Handling}).
* Group Styles:: Different ways of displaying groups of questions to
the user.
@end menu
@node Group Styles
@subsection Group Styles
Some group styles still use jQuery;
they should be modified to use modern formatters and Liza DOM
abstractions (see @srcrefraw{src/ui/field}
and @srcrefraw{src/ui/styler}).}
2017-03-16 13:23:19 -04:00
@cindex Group, Styling
Groups support a number of @dfn{group styles} that determine how
they are delimited from other groups;
how the elements they contain are rendered and laid out;
and how multiple indexes are displayed, added, and removed.
A list of available styles is detailed in @ref{t:group-styles}.
2017-03-16 13:23:19 -04:00
@float Table, t:group-styles
2017-03-16 13:23:19 -04:00
@multitable @columnfractions 0.15 0.65 0.10 0.10
@headitem Name @tab Description @tab Multi-Index? @tab Add/Remove Index?
@item @samp{default}
Groups are unstyled by default@mdash{
}they render elements as flat fields like a traditional form.
Only the first index of elements is rendered.
@tab@center N
@tab@center N
@item @samp{collapsetable}
Renders element label in the leftmost column like @samp{sidetable}.
Indexes are groups of rows delimited by headings,
which collapse the respective group of rows when clicked.
@tab@center Y
@tab@center Add
@item @samp{sidetable}
Renders elements as rows with label in the leftmost column rather
than the top row.
Each index is rendered as a column.
@tab@center Y
@tab@center Add
@item @samp{tabbedblock}
Each group is rendered as a block,
with each index rendered as a tab to the right of it.
Clicking a tab toggles the body content to the associated index.
Elements are rendered within the box.
@tab@center Y
@tab@center N
@item @samp{tabbed}
Like @samp{default},
but each index has a tab at the top of the group.
Clicking a tab toggles the body content to the associated index.
@tab@center Y
@tab@center Y
@item @samp{table}
A vanilla table with elements as columns,
their labels across the top row.
Each index is rendered in its own row.
@tab@center Y
@tab@center Y
@end multitable
@caption{Group styles and index support}
@end float
2017-03-16 13:23:19 -04:00
@node Program XML
@section Program XML
* Specifying Predicates::
@end menu
@node Specifying Predicates
@subsection Specifying Predicates
Object predicates (@pxref{Predicate System}) are specified using the
@xmlattr{when} attribute of certain nodes.
It must contain a string of references understood by the system
(see domain of discourse, @ref{Predicate System}),
all of which much match for the predicate to be true.
@float Figure, f:pred-when
<question id="describe" type="noyes"
label="Any special notes for this location?" />
<question id="vacant_desc" type="textarea"
when="q:describe vacant property"
label="Show only when a vacant property with the
question 'describe' non-empty and non-zero" />
@end example
@caption{Using the @xmlattr{when} attribute}
@end float
In @ref{f:pred-when} above,
question @samp{vacant_desc} will be applicable when @emph{all} of
the values of @samp{vacant}, @samp{property},
and@tie{}@samp{q:describe} are true.@footnote{
@xref{Predicate System} for what ``true'' means for a particular
variable in the domain of discourse.}
Within the context of the @progxml,
this concretely means that the classifications
@samp{vacant} and@tie{}@samp{property} are true,
and that the question @samp{describe} is answered ``yes''.
It reads as a sentence:
``@samp{vacant_desc}'' is applicable when we should @tie{}``describe
a vacant property''.
2017-03-16 13:23:19 -04:00
@node Validation
@section Validation
* Formatting Values::
@end menu
@node Formatting Values
@subsection Formatting Values
@cindex Question
@cindex Question, Value Formatting