Fork 0
Commit Graph

765 Commits (master)

Author SHA1 Message Date
Mike Gerwitz 1aa51775f3 [DEV-6968] Client: Hook classifier after initQuote
This saves a lot of time for large classifiers by waiting to invoke it until
after some initial bucket setup has been done.  In particular, initQuote.

DelayedStagingBucket exists to try to limit the number of events that are
kicked off.  In this case, initQuote typically uses setCommittedValues,
which DelayedStagingBucket does not override.  However, overriding it did
not provide a whole lot of benefit, since initQuote also calls
getDataByName, which forces DelayedStagingBucket to flush so that hooks can
run validations and such.

The last step in Client#_changeQuote is to force the classifier to run, so
this should be safe, unless calculated values happen to use classification
results, in which case they'll be momentarily wrong and then immediately
updated after the classifier runs.
2020-02-03 15:11:25 -05:00
Austin Schaffer 671f7cc6cb Pass canTerm to individual raters 2020-02-03 09:41:38 -05:00
Austin Schaffer e81516ca80 [DEV-6721] Add optional code coverage 2020-01-31 15:57:33 -05:00
Austin Schaffer 3c5cffc34f [DEV-6721] Pass canterm to individual raters 2020-01-31 14:15:22 -05:00
Shelly Shaver 3b76a9829e [DEV-6325] Return previous rating results on rate call
Merge branch 'jira-6325'
2020-01-27 08:21:02 -05:00
Shelly Shaver 8967583349 [DEV-6325] Return previous rating results on rate call 2020-01-24 09:08:47 -05:00
Austin Schaffer 09a84e7caf Pass along information about whether a save step was the last one 2020-01-15 10:26:40 -05:00
Austin Schaffer 61b66c24a3 [DEV-6730] Store concluding_save with the reverse delta 2020-01-15 10:26:07 -05:00
Austin Schaffer 103c06da39 [DEV-6730] Post whether or not a save step will conclude the process 2020-01-15 10:26:07 -05:00
Austin Schaffer d918d56a7e Update docker image for gitlab ci 2020-01-15 10:24:16 -05:00
Austin Schaffer 97c226690b Update docker image for gitlab ci 2020-01-15 09:27:32 -05:00
Schaffer, Austin a7cbb37b62 [DEV-5314] Do not publish directly to postrate exchange after rating
See merge request floss/liza!77
2020-01-02 16:42:25 -05:00
Schaffer, Austin aea913e767 [DEV-5314] Do not publish directly to postrate exchange after rating 2020-01-02 16:42:25 -05:00
Shelly Shaver 1508b66cb7 [DEV-6763] Include ratedata in data deltas 2019-12-27 10:27:15 -05:00
Austin Schaffer 3fbe9b31ff [DEV-6763] Include ratedata in data deltas 2019-12-26 15:28:46 -05:00
Joseph Frazer 8ae6e236e9 [DEV-6756] Prevent special chars from "local-part" of email address
Merge branch 'jira-6756'

* jira-6756:
  [DEV-6756] Prevent special chars from "local-part" of email address
2019-12-23 08:42:48 -05:00
Joseph Frazer aa2ba885af [DEV-6756] Prevent special chars from "local-part" of email address
The `"(),:;<>@[\]`` are allowed in the "local-part" of an email address
under certain circumstances, but our regex is not checking those
circumstances. It is better if we do not allow them until we improve
2019-12-23 08:26:54 -05:00
Austin Schaffer b48a9f0de2 Do not use Object.create when applying delta 2019-12-20 11:20:27 -05:00
Austin Schaffer 9f22db72fd [DEV-5312] Use milliseconds for timestamp 2019-12-19 14:42:10 -05:00
Austin Schaffer e2a9d17dc6 [DEV-5312] Do not use Object.create when applying delta 2019-12-19 14:42:10 -05:00
Schaffer, Austin caf2c776b6 [DEV-6710] Introduce Concept of Environment to Quotes 2019-12-19 14:19:26 -05:00
Austin Schaffer 1cb47d4acf [DEV-6710] [WIP] Save environment on quote save 2019-12-19 14:17:06 -05:00
Austin Schaffer e819edc53b [DEV-5312] Revert schema change; filter by double underscore; add programid; add step field on rate delta 2019-12-18 16:03:16 -05:00
Austin Schaffer f4b4a9d5d9 [DEV-5312] Correct lowercase environment variable 2019-12-18 11:37:20 -05:00
Austin Schaffer 5e5932fe9d [DEV-5312] Get delta processor program field from mongodb 2019-12-18 11:36:40 -05:00
Austin Schaffer d7f01e12c8 [DEV-5312] Remove recursive avro schema and filter bucket keys starting with two underscores 2019-12-18 09:48:26 -05:00
Austin Schaffer b9f05b174f [DEV-5312] Update avro schema error context so invalid path is not overwritten 2019-12-18 09:43:14 -05:00
Austin Schaffer c64ecbc816 [DEV-5312] Update the avro schema to accept arbitrary array depths 2019-12-17 13:20:20 -05:00
Austin Schaffer 699ee29a95 [DEV-5312] Use formatted time when logging events 2019-12-17 11:59:06 -05:00
Austin Schaffer 542745da8a [DEV-5312] Log quote_id along with doc_id 2019-12-17 11:59:06 -05:00
Austin Schaffer ccfd40c64f [DEV-5312] Allow for arbitrary depth in bucket schema 2019-12-17 11:59:01 -05:00
Austin Schaffer 33f39d5760 Use milliseconds for timestamps and mongo index creation 2019-12-16 09:41:44 -05:00
Austin Schaffer b6f72b4b09 [DEV-5312] Create additional indexes and process less frequently 2019-12-16 09:27:29 -05:00
Austin Schaffer abf76beac6 Update avro and prometheus dependencies 2019-12-13 09:50:50 -05:00
Austin Schaffer b93188c578 Add bin/delta-processor to EXTRA_DIST in Makefile 2019-12-12 16:03:50 -05:00
Schaffer, Austin 6c38f7d164 Merge branch 'jira-5312' into 'master'
[DEV-5312] Process to publish quote modifications to new exchange

See merge request floss/liza!68
2019-12-12 15:21:04 -05:00
Austin Schaffer cf0ca51cef [DEV-5312] Include created and modified fields in avro schema 2019-12-12 14:31:12 -05:00
Austin Schaffer ad9e66c83e [DEV-5312] Add delta-processor to bin 2019-12-12 12:38:17 -05:00
Austin Schaffer e3dded760d [DEV-5312] Define document meta data and restructure message writer.
Add better tests for message writer
2019-12-12 10:27:09 -05:00
Mike Gerwitz 9eb1f3afca DeltaProcessor: Correct delta application
Deltas must be applied in reverse to reproduce the state of the bucket or
rate data at that point in time, which is then paired with the delta that
will transform it into the previous state.
2019-12-12 10:27:09 -05:00
Mike Gerwitz 819701eca3 DeltaProcessor: Encapsulate getDeltas and remove redundant tests
getDeltas should be encapsulated.  It looks like it was public for the sake
of the tests, and its behavior can be inferred by looking at the result of
2019-12-12 10:27:09 -05:00
Mike Gerwitz d0a3ba573f DeltaProcessor: Remove redundant tests
These are covered by other tests and are testing too closely to the
implementation.  This should be encapsulated within the processor.
2019-12-12 10:27:09 -05:00
Mike Gerwitz 8e0c0e22a8 DeltaPublisher#_sendMessage: Promise.reject=>throw
Purely stylistic.
2019-12-12 10:27:09 -05:00
Mike Gerwitz fb80d2a8b4 .env: Use uppercase for env vars 2019-12-12 10:27:09 -05:00
Mike Gerwitz e13a062f2b bin/delta-processor: Log DeltaProcessor#process errors 2019-12-12 10:27:09 -05:00
Mike Gerwitz 2fb47cc993 bin/delta-processor: Provide error to EventMediator
This will log in the same format as the rest of the system.
2019-12-12 10:27:09 -05:00
Mike Gerwitz 84c2601f1c EventMediator: Log error stack under context 2019-12-12 10:27:09 -05:00
Mike Gerwitz e885026e0a DeltaPublisher: Remove parsing from constructor 2019-12-12 10:27:09 -05:00
Mike Gerwitz 4383d15c5a AvroFactory: Extract avro-js type definitions into src/types
The declarations that were intended for the library were moved, but
additional things that were outside of the library were kept in place.
2019-12-12 10:27:09 -05:00
Mike Gerwitz 9d6cb23e16 AmqpConnection: Propagate assertExchange promise
Failures were being ignored.
2019-12-12 10:27:09 -05:00