1
0
Fork 0
Commit Graph

38 Commits (master)

Author SHA1 Message Date
Austin Schaffer 61b66c24a3 [DEV-6730] Store concluding_save with the reverse delta 2020-01-15 10:26:07 -05:00
Austin Schaffer 3fbe9b31ff [DEV-6763] Include ratedata in data deltas 2019-12-26 15:28:46 -05:00
Austin Schaffer 9f22db72fd [DEV-5312] Use milliseconds for timestamp 2019-12-19 14:42:10 -05:00
Austin Schaffer e2a9d17dc6 [DEV-5312] Do not use Object.create when applying delta 2019-12-19 14:42:10 -05:00
Austin Schaffer 1cb47d4acf [DEV-6710] [WIP] Save environment on quote save 2019-12-19 14:17:06 -05:00
Austin Schaffer 5e5932fe9d [DEV-5312] Get delta processor program field from mongodb 2019-12-18 11:36:40 -05:00
Austin Schaffer d7f01e12c8 [DEV-5312] Remove recursive avro schema and filter bucket keys starting with two underscores 2019-12-18 09:48:26 -05:00
Austin Schaffer b9f05b174f [DEV-5312] Update avro schema error context so invalid path is not overwritten 2019-12-18 09:43:14 -05:00
Austin Schaffer 699ee29a95 [DEV-5312] Use formatted time when logging events 2019-12-17 11:59:06 -05:00
Austin Schaffer 542745da8a [DEV-5312] Log quote_id along with doc_id 2019-12-17 11:59:06 -05:00
Austin Schaffer ccfd40c64f [DEV-5312] Allow for arbitrary depth in bucket schema 2019-12-17 11:59:01 -05:00
Austin Schaffer b6f72b4b09 [DEV-5312] Create additional indexes and process less frequently 2019-12-16 09:27:29 -05:00
Austin Schaffer cf0ca51cef [DEV-5312] Include created and modified fields in avro schema 2019-12-12 14:31:12 -05:00
Austin Schaffer e3dded760d [DEV-5312] Define document meta data and restructure message writer.
Add better tests for message writer
2019-12-12 10:27:09 -05:00
Mike Gerwitz 9eb1f3afca DeltaProcessor: Correct delta application
Deltas must be applied in reverse to reproduce the state of the bucket or
rate data at that point in time, which is then paired with the delta that
will transform it into the previous state.
2019-12-12 10:27:09 -05:00
Mike Gerwitz 819701eca3 DeltaProcessor: Encapsulate getDeltas and remove redundant tests
getDeltas should be encapsulated.  It looks like it was public for the sake
of the tests, and its behavior can be inferred by looking at the result of
processing.
2019-12-12 10:27:09 -05:00
Mike Gerwitz d0a3ba573f DeltaProcessor: Remove redundant tests
These are covered by other tests and are testing too closely to the
implementation.  This should be encapsulated within the processor.
2019-12-12 10:27:09 -05:00
Mike Gerwitz 8e0c0e22a8 DeltaPublisher#_sendMessage: Promise.reject=>throw
Purely stylistic.
2019-12-12 10:27:09 -05:00
Mike Gerwitz fb80d2a8b4 .env: Use uppercase for env vars 2019-12-12 10:27:09 -05:00
Mike Gerwitz 84c2601f1c EventMediator: Log error stack under context 2019-12-12 10:27:09 -05:00
Mike Gerwitz e885026e0a DeltaPublisher: Remove parsing from constructor 2019-12-12 10:27:09 -05:00
Mike Gerwitz 4383d15c5a AvroFactory: Extract avro-js type definitions into src/types
The declarations that were intended for the library were moved, but
additional things that were outside of the library were kept in place.
2019-12-12 10:27:09 -05:00
Mike Gerwitz 9d6cb23e16 AmqpConnection: Propagate assertExchange promise
Failures were being ignored.
2019-12-12 10:27:09 -05:00
Austin Schaffer d9ee999adb [DEV-5312] Refactor class dependencies
Move create config functions out of the main delta-processor.ts file. Change some any types to more specific types. Define document structure better. Move prometheus logic out of processor class. Do not call logger directly, listen for events and log them. Change logger to more PSR-3 compliant style. Extract amqp connection into its own class. Create avro and mongo objects in factory.
2019-12-12 10:14:54 -05:00
Austin Schaffer 1ddb6f29eb [DEV-5312] Add logic to apply delta to bucket then publish modified bucket 2019-12-12 10:13:27 -05:00
Austin Schaffer 1b96cd9147 [DEV-5312] Refactor references from 'self' to 'this', pass console into event logger, and add factory for prometheus 2019-12-12 10:10:45 -05:00
Austin Schaffer 5ee9a5d340 [DEV-5312] Generalize event subscribers and dispatchers 2019-11-25 12:42:03 -05:00
Austin Schaffer e781a841b1 [DEV-5312] Reconnect AMQP when connection drops 2019-11-25 12:26:39 -05:00
Austin Schaffer faa7e15760 [DEV-5312] Add signal handling and prometheus monitoring 2019-11-22 16:37:57 -05:00
Austin Schaffer 9b5cd4e89f [DEV-5312] Call data-processor and instantiate classes 2019-11-20 09:48:28 -05:00
Austin Schaffer 309585cf6e [DEV-5312] Set processed flag 2019-11-13 12:04:27 -05:00
Austin Schaffer c5733d1dff [DEV-5312] Add interface for amqp publisher and implement a delta publisher 2019-11-12 17:04:54 -05:00
Austin Schaffer 950ae8818b [DEV-5312] Add preliminary processor 2019-11-12 16:21:01 -05:00
Mike Gerwitz e058c8b509 Copyright range update
All files now have the same range, beginning from the conception of this
project.
2019-08-30 09:41:35 -04:00
Mike Gerwitz 4b114d0137 Client: Extract cmatch methods
This makes the minimal number of changes necessary to ensure that all object
references remain available.  It is a mess.

And despite moving all of this, Client is still a massive clusterfuck.

* src/client/Client.js (system): Add import.
  (_cmatch): Now references new Cmatch class instance.
  (_cmatchHidden, _classMatcher): Remove fields.
  (_forceCmatchAction): Rename to `forceCmatchAction'.
  (_hookClassifier, _postProcessCmatch, _cmatchVisFromUi, _handleClassMatch,
    _mergeCmatchHidden, _clearCmatchFields): Extract methods.  Update
    references as necessary.
  (getCmatchData): Remove unused method.
  (_handleError): Rename to `handleError' to make accessible to
    Cmatch.  Update references.
* src/client/ClientDependencyFactory.js (FieldClassMatcher): Remove import.
  (createFieldClassMatcher): Remove method.  See `system/client'.
* src/client/Cmatch.js: New class.
* src/system/client.js (Cmatch, field): Add imports.
  (cmatch): Add export.
2018-02-09 11:20:45 -05:00
Mike Gerwitz 1bb5191e3e LoVullo Associates => R-T Specialty
Copyright notices updated.  More casual references to "LoVullo
Associates" replaced with "RT Specialty / Lovullo", which will be "RT
Specialty Buffalo" in the future.  Or "RT Specialty", depending on how
this is rolled out.  Or "Ryan Specialty Group".  Who knows.

"R-T Specialty, LLC." is the legal name, which includes the dash.  Not
to be confused with a certain television network.
2017-06-08 14:48:43 -04:00
Mike Gerwitz 61a59db4e0 system.client: working data.diffStore
* src/system/client.js (data.diffStore): Compose all stores.
* test/system/clientTest.js: Update test case.

DEV-2296
2017-01-30 00:43:13 -05:00
Mike Gerwitz 38b4a58dde Began system/ with client
This will contain various factories (compounded in some cases) to
instantiate various parts of a coherent system.  It aims to replace
(as one of many pieces) the ClientDependencyFactory that's referenced
in the code, which is still part of rating-fw (internal LoVullo repo
from which liza is being liberated).

* src/system/client.js: Add module.
* test/system/clientTest.js: Add functional test case.

DEV-2296
2017-01-29 22:44:34 -05:00