This is unfortunately not production-ready code, but we need to get
something out there in the meantime. The RatingServicePublish's docblock
mentions some of the shortcomings, which will be addressed in the near
future.
There is also more documentation to come once we settle on an implementation.
DEV-4400
Forward validation failures were never updated to use the new Failure
object, so they ended up showing the default message. The problem was that
`ui.invalidateForm' was replacing the Failure object with a new one with an
empty message.
This change just does the same thing as the change and save events.
* src/client/Client.js (_forwardValidate): Invoke `#_genValidationMessages'
and `_dataValidator.updateFailures'. Stop calling `ui.invalidateForm'.
It looks like the metabucket is never initialized, so saving the quote is
right now the only thing that sets default values. That should be fixed in
the future.
This also begins adding tests for the terrible MongoServerDao, that could
use some refactoring.
* src/server/db/MongoServerDao.js: Make `meta' mutable. I had forgotten to
remove the code that mutates it (since our version of v8 right now does
not blow up for const assignments), so this is all that's needed.
* test/server/db/MongoServerDaoTest.js: New file to test this situation.
The metabucket expects vectors, not scalars. This causes the value
to be inaccessible when loaded by the ProcessManager for rating,
because it uses the Bucket implementation expecting vectors.
This wasn't noticed at first because it works through TAME's map.
* doc/bucket.texi (Metabucket): Clarify expected data format.
* src/server/db/MongoServerDao.js (saveQuote): Store
`meta.liza_timestamp_initial_rated' as a vector.
This value already existed on the document, but was inaccessible to external
systems. This is now accessible to e.g. raters.
* doc/bucket.texi (Metabucket): New section.
* src/server/db/MongoServerDao.js (saveQuote): Set initial quoted date as
liza_timestamp_initial_rated. Update metabucket keys individually so as
not to inadvertently overwrite the entire metabucket.
DEV-3715
Default data was converted to an empty array if the data evaluated to
false. We only want to convert it if it is undefined so values that are
false remain false.
Note that this permits animation if you just add:
transition: transform 0.25s;
for example to `dt.stack-header::before'. I didn't add that because nothing
else is animated atm, and so it seems out-of-place.
* src/css/base.css: Add styling for Accordion group.
DEV-3989
* src/client/ClientDependencyFactory.js: Remove old AccordionGroupUi
require, add new one.
* src/ui/group/AccordionGroupUi.js: New class.
* src/ui/group/GroupUi.js (postAddRow): Make virtual.
The `*_label' fields were not being properly set because they were being
considered just as every other expanded field, which is subject to other
considerations to ensure that we do not overwrite user data. Label fields,
on the other hand, _must_ always be set whenever the value changes, and are
_not_ modifiable by the user.
This codifies that `_label' assumption that I would like removed in the
future, but time does not permit that sort of refactoring right
now. Ideally, labels would be treated generically like any other expanded
field and have an attribute set of them that would change their expansion
behavior.
* src/client/dapi/DataApiMediator.js (_updateFieldData): Pass label field id
to `_populateWithMap'.
(_populateWithMap): Use it to always populate the label field when the key
matches, ignoring the other expansion criteria that would inhibit it.
* test/client/dapi/DataApiMediatorTest.js: Update test accordingly.
* src/dapi/DataApiManager.js (triggerFieldUpdate): Provide id to
`_genUiFieldData'.
(_genUiFieldData)[label_id]: New field. Return it as part of the return
object's `label_id' field.
DEV-4077
The postProcessResults method occasionally failed to be overridden in
local environments due to issues with how npm handled its dependencies.
For some reason, changing this to a public method corrected the issue. I
am changing this to, hopefully, prevent future frustrations.
* src/program/ProgramInit.js (_isKnownType): Account for ancient qtype
representation (as a string).
* src/server/quote/ProgramQuoteCleaner.js (_isKnownType): Likewise.
* test/program/ProgramInitTest.js: New test case for this situation.
* test/server/quote/ProgramQuoteCleanerTest.js: Modify existing test case
for this situation.
If only one supplier returns this datum and that supplier is ineligible,
then the value may not exist if this is not defined as an external
value. (Really, that's the bigger problem, but let's be resilient.)
* src/ui/group/TabbedBlockGroupUi.js (visit): Ignore missing index value.
These denote fields that are generated but do not actually have any data
associated with them. For example, select options with predicates have a
field generated so that they contribute to the group field count (so that
the group will automatically show/hide appropriately), but those should
never have values associated with them in the bucket.
This was manifesting as a nasty bug: The bucket contained a key for
generated options. When the quote is loaded, the client "empties" the
bucket. In doing so, it set the option value to the empty string, which had
the effect of rendering the dropdown useless---every value was the empty
string!
* src/program/ProgramInit.js (_isKnownType): New method.
(init): Use it and ignore fields with unknown types.
* src/server/Server.js: Add note that we shouldn't have this logic
duplicated between ProgramInit and ProgramQuoteCleaner.
* src/server/quote/ProgramQuoteCleaner.js (_fixGroup): Ignore fields with
unknown types.
(_isKnownType): New method.
* test/program/ProgramInitTest.js: Update existing tests. Add new.
* test/server/quote/ProgramQuoteCleanerTest.js: Test this case.
The new checkbox question type is setup with an empty validator. When
checked, it was not unchecking because it was trying to behave like the
"noyes" questions. I updated the logic to not behave like that if there
is only one checkbox in a group.
* src/server/quote/ProgramQuoteCleaner.js (clean): Add docblock.
Replace previous linked group cleaning with call to `_fixGroup'.
(_fixGroup): New method. Similar logic to previous linked group cleaning,
except that fields are never truncated.
(_fixLinkedGroups, _getLinkedIndexLength): Remove methods.
(_getGroupLength): New method determining group size from leader length,
which also accounts for linked groups.
* test/server/quote/ProgramQuoteCleanerTest.js: New test case.
* src/server/daemon/Daemon.js (getProgramController): Do not initialize
`no_results_url' to an empty string if not provided; instead, keep any
existing value. Otherwise, later calls with no arguments will clear
previously-set values.
DEV-3254
This further highlights that we probably should not be handling this
here. We want to ensure that we check the state of the bucket right before
we're about to do option setting, otherwise we could be in an inconsistent
state (looking at old data).
* src/client/dapi/DataApiMediator.js (_updateFieldData): Check bucket after
stack clears (along with all other operations).
Allowing the stack to clear ensures that (in practice) DelayedStagingBucket
is given a chance to do necessary processing before data are requested from
it by bucket hooks as a result of _this_ invocation, which in turn
results (in some cases) in infinite recursion.
* src/client/dapi/DataApiMediator.js (_updateFieldData): Allow stack to
clear before invoking `quote.setData'.
* test/client/dapi/DataApiMediatorTest.js: Test respectively.
DEV-3257
Without this, we have a BC break.
* src/client/Client.js (_init): Default program.dapimap to DataApiMediator
instance to an empty object to maintain BC.
* src/client/dapi/DataApiMediator.js (_populateWithMap): Update
docblock. Ignore field during expansion if it would overwrite an
existing value.
* test/client/dapi/DataApiMediatorTest.js: Update tests data to include
values for all bucket fields, not just `name'. Add test for new
condition.
DEV-3257
[*] You should not use this commit directly since this may wipe out data in
fields the user has changed. See future commit where this situation is
properly handled.
* src/client/Client (_init): Provide dapimap to DataApiMediator instance.
* src/client/dapi/DataApiMediator.js
(_dapi_map): New field.
(constructor): Accept dapimap. BC break (which is okay, since this is
still part of a topic branch). Assing to _dapi_map. Update docblock.
(monitor): Bind `dapi_manager' to first argument of handlers.
(_updateFieldData): Accept `dapi_manager' as first argument. Use
`_populateWithMap' to generate additional update data.
(_populateWithMap): New method.
(_clearFieldFailures): Accept `dapi_manager' as first argument.
* src/dapi/DataApiManager.js: Update copyright year.
(getDataExpansion): Return empty object (consistent with interface) rather
than `undefined' when field value is undefined. Use
{Error=>MissingDataError} when field data are missing. Throw instead of
emit. Fix missing comma in var declarations.
* src/dapi/MissingDataError.js: New class.
* test/client/dapi/DataApiMediatorTest.js: Update test data to test field
expansion. New test against ignoring field expansion when data are not
available. Update Sut constructors of other tests for new dapimap
parameter.
DEV-3257
This extracts existing code from Client and adds tests. The glue code is
far from ideal and highlights the amount of work needed to decouple Client
from so many parts of the system.
* src/client/Client.js (_dapiManager): New field.
(_init): Use DataApiMediator.
(_createProgram): Assign `_dapiManager' (this is not at all
ideal). Remove hooks from it: fieldLoading, updateFieldData,
clearFieldData.
* src/client/ClientDependencyFactory.js (createDataApiMediator): New alias
to DataApiMediator constructor.
* src/client/dapi/DataApiMediator.js: New class.
* test/client/dapi/DataApiMediatorTest.js: New test case.
DEV-3257
This does not go all the way, but helps improve the readability of the
algorithm a little bit and modernizes the code.
* src/field/FieldClassMatcher.js (constructor): Renamed from
`__constructor'.
(__constructor): Remove method.
(match): Extract most code into `#_reduceFieldMatches'.
(_reduceFieldMatches): New method, simplifying the algorithm slightly.
(_reduceMatch): Simplify.
* test/field/FieldClassMatcherTest.js: Update accordingly.
If the submission failed, we probably want to try again next time around.
* src/server/service/RatingServiceSubmitNotify.js
(_maybeNotify): Extract logic from `#postProcessRaterData'. Only set
notification flag in absence of dapi error.
(postProcessRaterData): Use it.
* test/server/service/RatingServiceSubmitNotifyTest.js: Update tests
accordingly.
Previously, it was just trying to serialize an object, which was useless.
* src/server/Server.js (_formatValidationFailures): New method.
(_doQuoteSave): Use it.
Requests that came in between (for example) a post request and the actual
validation of the posted data would potentially reset the flag, causing
internal assertions to fail.
* src/server/Server.js
(sendStep): Provide session to #_forwardValidate.
(_forwardValidate): Set `program.isInternal' immediately before invoking
event. Provide session to #quoteFill.
[session]: New argument.
(quoteFill): Add `session' argument. Provide session to hooks.
(loadProgram): Receive session via #quoteFill hook. Set
`program.isInternal' before submit hook.
* src/server/daemon/controller.js (doRoute): Remove `program.isInternal'
set.
This is not the ideal implementation. Instantiation of RatingService and
such should be moved out of the controller entirely, but we don't have the
time for that right now.
* src/server/db/MongoServerDao.js
(getDocumentField,setDocumentField): New methods.
* src/server/service/RatingServiceSubmitNotify.js
(postProcessRaterData): Only notify when notification flag is not set.
(_getNotifyState, _setNotified): New methods.
* test/server/service/RatingServiceSubmitNotifyTest.js: Modify accordingly.
This wasn't intended to make its way into a public repo. :)
The existing key was a long-forgotten kluge that was supposed to be
temporary, allowing internal services to create quotes without
authentication. The chances of this being practically exploited are minimal
in our environment, and it's auditable using webserver logs.
This moves the skey into a configuration file, which allows it to vary by
server and be rotated until a better solution is made available. skey is
disabled by default (empty string), and when used by us internally, the keys
are now generated using a CSPRNG rather than a brute-forcable 5-byte key
that was hard-coded.
The fact that this appears in webserver logs is a big issue as well. I
added a task to address that.
* conf/vanilla-server.json (skey): New key. Default empty.
* src/server/daemon/Daemon.js (start): Provide skey to `#getRouters'.
(getRouters): Provide skey to `#getProgramController'.
(getProgramController): Set skey on `controller'.
* src/server/daemon/controller.js (skey): New mutable export (unideal; quick
change).
(has_skey): Use it.
quicksave data wouldn't always be cleared on save, meaning it might not be
cleared until the next _empty_ quicksave from the client.
* src/server/db/MongoServerDao.js (saveQuote): Always clear quicksave.
Before this change, since `undefined' is encoded as `null' when serialized,
there was no way for the server to disambiguate between unmodified values
and a truncation point. For example:
[ undefined, undefined, null, null, null ]
The above array represents two unmodified and three removed indexes. But
this is serialzed into JSON as:
[ null, null, null, null, null ]
It isn't possible for the server to determine what the truncation point is
from that diff. The solution is to therefore truncate the array _before_
sending it to the server, providing a trailing null to indicate that a
truncation has occurred:
[ null, null, null ]
The above means that the first two indexes are unmodified, and that index 2
and later should all be truncated.
* doc/client.texi (Saving to Server): New section.
* src/client/transport/XhttpQuoteTransport.js (_truncateDiff): New method to
perform truncation.
(getBucketDataJson): Use it.
* test/client/transport/XhttpQuoteTransportTest.js: New file with respective
test case.
* src/client/Client.js (_defaultId): New property.
(__construct)[default_id]: New parameter. Set `_defaultId'.
(_getProgramId): Use `_defaultId' if not found in URL.
* src/ui/nav/HashNav.js (_initUrl): Do not require leading program id.
This technically worked fine before with our URL scheme, but that is not
guaranteed to be the case generally.
This is something that managed to slip by (but not unnoticed) for almost
exactly one year to this day (028606242a). It
can only be reproduced by changing classes that result in visibility changes
differing on the same field by index. The issue hides itself on first
load (because all fields are shown by default) and on refresh.
The problem is that, when one index shows a field but another hides it, the
hide overrode the show indexes, so only the hide took place.
* src/client/Cmatch.js (markShowHide): Make virtual. New implementation to
support concurrent show/hide.
(_handleClassMatch): Use it.
* test/client/CmatchTest.js: New test.
* npm-shrinkwrap.json: ease.js v0.2.{8=>9}.
This is to give us a fairly easy means of testing this logic for a bugfix.
This refactoring also obviates a pretty nasty bug; see docblock.
* src/client/Cmatch.js (_handleClassMatch): Extract show/hide marking.
(markShowHide): New method.
This makes the minimal number of changes necessary to ensure that all object
references remain available. It is a mess.
And despite moving all of this, Client is still a massive clusterfuck.
* src/client/Client.js (system): Add import.
(_cmatch): Now references new Cmatch class instance.
(_cmatchHidden, _classMatcher): Remove fields.
(_forceCmatchAction): Rename to `forceCmatchAction'.
(_hookClassifier, _postProcessCmatch, _cmatchVisFromUi, _handleClassMatch,
_mergeCmatchHidden, _clearCmatchFields): Extract methods. Update
references as necessary.
(getCmatchData): Remove unused method.
(_handleError): Rename to `handleError' to make accessible to
Cmatch. Update references.
* src/client/ClientDependencyFactory.js (FieldClassMatcher): Remove import.
(createFieldClassMatcher): Remove method. See `system/client'.
* src/client/Cmatch.js: New class.
* src/system/client.js (Cmatch, field): Add imports.
(cmatch): Add export.
All of the old events have been removed!
* src/client/Client.js (handleEvent): Remove remainder of old system (after
last commit, all that was left was error handling). Correct docblock.
The `set' event already existed---this merely extracts it into its own
handler.
* src/client/Client.js (handleEvent): Extract `set' handler.
* src/client/ClientDependencyFactory.js (createClientEventHandler): Add
`set'.
* src/client/event/ValueSetEventHandler.js: New class.
* test/event/ValueSetEventHandlerTest.js: Associated test case.
Mark Goldsmith