liza/doc/design.texi

@c  This document is part of the Liza Data Collection Framework manual.
@c  Copyright (C) 2017 LoVullo Associates, Inc.
@c
@c    Permission is granted to copy, distribute and/or modify this document
@c    under the terms of the GNU Free Documentation License, Version 1.3
@c    or any later version published by the Free Software Foundation;
@c    with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
@c    Texts.  A copy of the license is included in the section entitled ``GNU
@c    Free Documentation License''.

@node Design
@chapter Design & Architecture

@emph{N.B. This document may reference features that are not yet
  present in Liza; they will be moved into this repository in time.}

Liza is fundamentally a data collection framework@mdash{
  }a fancy form for collecting, validating, and lightly processing
  user data.

The main components of the system are:

@table @strong
  @cindex Assertions
  @item Assertions
  Basic validations against bucket data,
    producing errors and manipulating control flow.
  Invokes triggers to manipulate the UI and document.
  Assertions are compiled from Program sources.
  @xref{Assertions}.

  @cindex Bucket
  @item Bucket
  The key/value store into which all document data are stored.
  Supports staging and rollback of data,
    processing deltas,
    and provides hooks that drive the rest of the system.
  @xref{Bucket}.

  @cindex Calculated Value
  @item Calculated Values
  A small sub-system for calculating bucket values from other values.

  @cindex Client
  @item Client
  Basic logic for navigating between steps,
    prompting for user actions,
    display help text and basic document data,
    communicate with server,
    etc.
  @xref{Client}.

  @cindex Developer Dialog
  @item Developer Dialog
  Renders information about the system for debugging the client.
  Can monitor the bucket, assertions, classification results, and
    provides other useful features.

  @cindex Predicate
  @item Predicate System
  Processes classification data from external classifiers to determine
    applicability of specific questions.
  These data are used to determine what assertions are performed,
    what questions and groups display,
    and more.

  @cindex Program
  @item Program
  Internal representation of the Program with delegation of events to
    the assertion system.
  Contains compiled representation of all steps, groups, questions,
    assertions, metadata, and others.
  @xref{Program}.

  @cindex Program, User Interface
  @cindex User Interface, Program
  @item Program UI
  Rendering of elements specific to Programs,
    such as steps, groups, and questions.
  This is the equivalent of an HTML form.
  Directly monitors the bucket to perform UI updates.
  @xref{Program UI}.

  @cindex Program, XML
  @item Program XML
  The source code for a Program, in XML format.
  @xref{Program XML}.

  @cindex Server
  @item Server
  Provides REST API for serving Programs; saving data;
    revalidating, filtering, and recalculating data;
    and other types of processing.
  Code is shared with the client,
    ensuring identical behavior for appropriate behaviors.

  @cindex Type Validation
  @cindex Validation, Type
  @item Type Validation
  Validates and formats bucket values for specific field (question)
    types.
  For example, a date field must be in a recognized date format,
    and will be normalized for display.
  @xref{Validation}.
@end table

More information about each can be found in their respective section.


@menu
* Assertions::
* Bucket::
* Client::
* Program::
* Program UI::
* Program XML::
* Validation::
@end menu



@node Assertions
@section Assertions
@helpwanted



@node Bucket
@section Bucket
@helpwanted

@menu
* Value Assignment:Bucket Assignment.  Writing data to the Bucket.
@end menu


@c TODO
@node Bucket Assignment
@subsection Bucket Value Assignment
@helpwanted



@node Client
@section Client
@helpwanted

@menu
* Error Handling::
@end menu


@node Error Handling
@subsection Error Handling

@maintenance{
  The complexity of this system and integration into legacy
    layers has caused maintenance trouble in the past.
  Each of the error checking layers need to be integrated
    to reduce complexity.}

@cindex Error
@cindex Failure
There are three layers of error checking:@footnote{
  Primarily for legacy reasons.
  They are being consolodated as the system is touched.}

@enumerate
  @item Required field checking@mdash{
    }whether all required questions have been answered.
  @item Type Validation@mdash{
    }verify that questions contain valid data according to their
    declared type. @ref{Validation}.
  @item Assertions@mdash{
    }arbitrary checks on data.
    @ref{Assertions}.
@end enumerate

@cindex Required Field
@cindex Field, Required
@cindex Field, Fixed
@cindex Error, Fixed
@cindex Error, Required
@cindex Fixed, Error
Required fields fail serially@mdash{
  }the system will notify the user of the required field,
    and direct him/her to it (usually through scrolling).
A field is marked as @dfn{fixed} according to the rules
  in @ref{Managing Error State}.


@menu
* Managing Error State::  Determining when failures should be marked as ``fixed''
@end menu


@node Managing Error State
@subsubsection Managing Error State

@cindex Failure Stack
@cindex Error Stack
Each failure caused by assertions is associated with a
  @dfn{failure stack}.
The stack represents the trail of assertions that have run,
  containing the ids of all values asserted against.
When any field or classification changes that is represented on the
  failure stack,
    the failure for the failed field associated with that failure stack
    is cleared.

@exnotice{
  If an assertion for some question @var{foo} first checked the value
    of bucket field @var{bar},
    and within its failure checked the value @var{c:predicate},
    the failure stack would contain both of those ids.
  If either @var{bar} or the @var{predicate} classification changed,
   the question @var{foo} would have its error cleared.}

Error state is managed by
  @srcref{src/validate/ValidStateMonitor.js, ValidStateMonitor}.


@node Program
@section Program

@cindex Program
The @dfn{Program} is a declarative representation of an entire system.
It is the highest level of abstraction from a data perspective.
The user observes and interacts with a Program using the
  @ref{Program UI,,Program@tie{}UI}.

@cindex Program, XML
Programs contain a lot of metadata that is not in a convenience
  human-readable (or modifiable) format,
    some of which are redundant.
Programs are ideally compiled from a @ref{Program XML,,Program@tie{}XML}
  document.



@node Program UI
@section Program UI

@cindex Program, User Interface
@cindex User Interface, Program
The @dfn{Program UI} renders a @ref{Program} as a form.

@cindex Step
@cindex Group
At the highest level,
  steps are rendered in a tab-like manner,
  above the main form content.
A step contains groups,
  which in turn contain elements such as questions.
Groups are delimited in some manner defined by their style
  (@pxref{Group Styles}).

@cindex Question
@cindex Question, Value Formatting
@cindex Bucket, Updating
Questions are rendered as form fields.
Any time the respective @ref{Bucket} field is changed,
  the form field is updated to reflect those changes,
    after having first been formatted with the appropriate validator
    (@pxref{Formatting Values}).
When a question is changed by the user,
  the value is expected to be propagated to the Bucket
  (@pxref{Bucket Assignment}).

@cindex Navigation Bar
@cindex User Interface, Navigation Bar
@cindex User Interface, Button Navigation
Navigation between steps can be done via the
  @dfn{Navigation Bar} above the step@tie{}content,
  or using ``Go@tie{}Back'' and ``Continue'' buttons at the foot of the
  step@tie{}content.

@cindex Sidebar
A @dfn{Sidebar} is rendered adjacent to the step content.
It displays the name of the Program,
  as well as configurable metadata (usually through the @samp{sidebar}
  node of the @ref{Program XML,,Program@tie{}XML}).
It also displays question help text (also configured through the XML)
  and any error messages (@pxref{Error Handling}).

@menu
* Group Styles:: Different ways of displaying groups of questions to
                 the user.
@end menu


@node Group Styles
@subsection Group Styles

@cindex Group, Styling
Groups support a number of @dfn{group styles} that determine how
  they are delimited from other groups;
  how the elements they contain are rendered and laid out;
  and how multiple indexes are displayed, added, and removed.

@multitable @columnfractions 0.15 0.65 0.10 0.10
  @headitem Name @tab Description @tab Multi-Index? @tab Add/Remove Index?

  @item @samp{default}
  @tab
  Groups are unstyled by default@mdash{
    }they render elements as flat fields like a traditional form.
  Only the first index of elements is rendered.
  @tab@center N
  @tab@center N

  @item @samp{collapsetable}
  @tab
  Renders element label in the leftmost column like @samp{sidetable}.
  Indexes are groups of rows delimited by headings,
    which collapse the respective group of rows when clicked.
  @tab@center Y
  @tab@center Add

  @item @samp{sidetable}
  @tab
  Renders elements as rows with label in the leftmost column rather
    than the top row.
  Each index is rendered as a column.
  @tab@center Y
  @tab@center Add

  @item @samp{tabbedblock}
  @tab
  Each group is rendered as a block,
    with each index rendered as a tab to the right of it.
  Clicking a tab toggles the body content to the associated index.
  Elements are rendered within the box.
  @tab@center Y
  @tab@center N

  @item @samp{tabbed}
  @tab
  Like @samp{default},
    but each index has a tab at the top of the group.
  Clicking a tab toggles the body content to the associated index.
  @tab@center Y
  @tab@center Y

  @item @samp{table}
  @tab
  A vanilla table with elements as columns,
    their labels across the top row.
  Each index is rendered in its own row.
  @tab@center Y
  @tab@center Y
@end multitable



@node Program XML
@section Program XML
@helpwanted


@node Validation
@section Validation
@helpwanted

@menu
* Formatting Values::
@end menu


@node Formatting Values
@subsection Formatting Values

@cindex Question
@cindex Question, Value Formatting
@helpwanted