1
0
Fork 0
Commit Graph

765 Commits (master)

Author SHA1 Message Date
Austin Schaffer d9ee999adb [DEV-5312] Refactor class dependencies
Move create config functions out of the main delta-processor.ts file. Change some any types to more specific types. Define document structure better. Move prometheus logic out of processor class. Do not call logger directly, listen for events and log them. Change logger to more PSR-3 compliant style. Extract amqp connection into its own class. Create avro and mongo objects in factory.
2019-12-12 10:14:54 -05:00
Austin Schaffer 1ddb6f29eb [DEV-5312] Add logic to apply delta to bucket then publish modified bucket 2019-12-12 10:13:27 -05:00
Austin Schaffer 1b96cd9147 [DEV-5312] Refactor references from 'self' to 'this', pass console into event logger, and add factory for prometheus 2019-12-12 10:10:45 -05:00
Austin Schaffer 5ee9a5d340 [DEV-5312] Generalize event subscribers and dispatchers 2019-11-25 12:42:03 -05:00
Austin Schaffer e781a841b1 [DEV-5312] Reconnect AMQP when connection drops 2019-11-25 12:26:39 -05:00
Austin Schaffer faa7e15760 [DEV-5312] Add signal handling and prometheus monitoring 2019-11-22 16:37:57 -05:00
Austin Schaffer 9b5cd4e89f [DEV-5312] Call data-processor and instantiate classes 2019-11-20 09:48:28 -05:00
Austin Schaffer 309585cf6e [DEV-5312] Set processed flag 2019-11-13 12:04:27 -05:00
Austin Schaffer e737850efd [DEV-5312] Set processed flag 2019-11-12 17:10:19 -05:00
Austin Schaffer c5733d1dff [DEV-5312] Add interface for amqp publisher and implement a delta publisher 2019-11-12 17:04:54 -05:00
Mike Gerwitz de94f69e8f bin/server.ts: fs.unlink requires two arguments on Node v12 2019-11-12 16:21:01 -05:00
Burzlaff, Herbert 341aeb6a08 [DEVOPS-143] Node 12, needs a callback for asynchronous file writes, so switching to synchronous file writes instead 2019-11-12 16:21:01 -05:00
Austin Schaffer 950ae8818b [DEV-5312] Add preliminary processor 2019-11-12 16:21:01 -05:00
Austin Schaffer 91a7cf94b2 [DEV-5312] Add dao for deltas 2019-11-12 16:21:01 -05:00
Mike Gerwitz d0b2a4ce73 bin/server: Convert to TypeScript 2019-11-12 16:21:00 -05:00
Mike Gerwitz e450bbb3d6 bin/delta-processor: Placeholder script 2019-11-11 16:53:58 -05:00
Austin Schaffer bdd77f8de4 [DEV-5312] Workaround multiple easejs instances 2019-11-11 15:15:28 -05:00
Austin Schaffer 015a3f5d47 [DEV-5312] Convert MongoServerDao to TS 2019-11-11 15:14:49 -05:00
Austin Schaffer c2458dee78 [DEV-5312] Convert most parts of RatingService Publish to TS 2019-11-11 09:29:07 -05:00
Mike Gerwitz 31a820a34f Mitigate problems caused by changes in EventEmitter
This kicks the can down the road a bit since we do not have time atm the
investigate what changes in EventEmitter are causing the underlying
problem.  With that said, the hope is to get rid of EventEmitter entirely.

DEV-6488
2019-11-11 09:16:29 -05:00
Mike Gerwitz 1e5a0fb701 events: Use EventEmitter from old version of Node
This was copied from a _very_ early version of node.  The problem is that,
for reasons not yet investigated, Node v12's EventEmitter (at least when
extended by easejs) is causing data to be shared across instances.

This kicks the can down the road a little bit.  The goal is to eventually
get rid of EventEmitter entirely.

Here's the script used to do the conversion in this commit:

git grep -l "require( 'events' )" -- src \
  | while read f; do
      path=$( sed 's|src/||;s|[^/]\+|..|g' <<< "$(dirname "$f")" )
      sed -i "s|require( 'events' )|require( '$path/events' )|" "$f"
    done
2019-11-08 13:06:18 -05:00
Mike Gerwitz 11c2bcb9b1 configure (TS_NODE_VERSION): Make configurable
Apparently @types/node is not always available for the current node version...
2019-11-08 13:05:34 -05:00
Mike Gerwitz c8c7dfda5d Makefile.am (check-ts-out): New check for TS->JS compilation
tsc handles its own incremental builds, and if a file is removed, it isn't
always regenerated.  This resulted in a bad distribution being generated and
published to npm.
2019-11-08 11:26:35 -05:00
Schaffer, Austin 6cc72d9d11 [DEV-6353] Generate and store reverse delta on step save
Co-Authored-By: Mike Gerwitz <mike.gerwitz@ryansg.com>

See merge request floss/liza!66
2019-11-08 08:41:37 -05:00
Austin Schaffer cda048dd8c [DEV-6353] Generate and save reverse deltas for ratingdata and step save 2019-11-07 16:25:33 -05:00
Austin Schaffer a4d151ef79 [DEV-6353] Add function to create delta from one data set to another
The type of the delta must accomodate undefined and null values for array
data, as well as null for a key value to represent removing a key
2019-11-07 16:25:33 -05:00
Austin Schaffer 26c4367ead [DEV-6353] Accept push data in MongoDao 2019-11-07 16:25:33 -05:00
Austin Schaffer 48aa315c2a [DEV-6353] Get and set past rate data 2019-11-07 16:25:33 -05:00
Austin Schaffer 9f8fb90d0d [DEV-6353] Add dapi definitions 2019-11-07 16:25:33 -05:00
Austin Schaffer 4f39c754a2 [DEV-6353] Add bucket definitions 2019-11-07 16:25:33 -05:00
Austin Schaffer 6fcf72f1b6 [DEV-6353] Convert DataProcessor to TS 2019-11-07 16:25:33 -05:00
Mike Gerwitz 2bdcacd11f package.json.in: @types/node: Use installed node version
This will ensure that the right API version is installed.
2019-11-07 16:24:19 -05:00
Mike Gerwitz 365fe702cf typescript v3.{6=>7} upgrade 2019-11-07 16:24:18 -05:00
Mike Gerwitz 64718d5513 RatingService: Fix initialRatedDate and lastRatedDate reply
Recent changes caused the quote to be read before it was mutated by
_saveRatingData.
2019-10-31 14:26:12 -04:00
Mike Gerwitz f0e246deef RatingService#postProcessRaterData: Fix accidental indvRate change
I accidentally changed id=>after when prototyping data for the _new_
deferred rating (not yet written).  There were no tests for this code, which
was originally written in July of 2012 (see private lovullo.git repo), so it
was not caught until QA.
2019-10-31 11:21:01 -04:00
Mike Gerwitz d2f9f5f18f Save rating data to another database field
This maintains the old behavior while also writing rating result to another
field in the database.
2019-10-29 15:25:38 -04:00
Mike Gerwitz cefd6e95cb RatingService: Remove unnecessary program param
It was kept for postProcessRaterData for now since that would involve
modifying a subtype as well.
2019-10-29 13:36:55 -04:00
Mike Gerwitz 65e7880c81 RatingService: Improved error handling
This does only a slightly better job than before.
2019-10-29 13:36:55 -04:00
Mike Gerwitz a23f2040dd RatingService: Remove unreachable error check 2019-10-29 13:36:55 -04:00
Mike Gerwitz 1ed8ad1cd7 RatingService: Return promise
This only replaces the callbacks at the highest level and starts to move
toward proper error handling.  The private methods do not yet properly
propagate errors.
2019-10-29 13:36:55 -04:00
Mike Gerwitz 1aa69c2a56 RatingService: Save rating data to separate field (ratedata)
This adds ratedata to the quote collection.  We use the existing saveQuote
method so that this operation is atomic.  We're also continuing to save to
the bucket for now so as not to break existing code, but the intent in the
future will be to remove all but necessary data that should be exposed to
the client.
2019-10-29 13:36:55 -04:00
Mike Gerwitz cffd9ddeb0 RatingServiceTest: Add type information for stubs
This also gets rid of the RatingServiceStub module, which is not used by
anything else.  I suspect that I originally added it to be shared by traits,
but that's no longer going to be the case (and the only remaining trait is
unfortunately untested atm, and will be going away).
2019-10-29 13:36:55 -04:00
Mike Gerwitz 312142b3e8 RatingService: Use ServerDao
I had forgotten an interface exists, and this will be needed for the next
commit.  Unfortunately it requires a bit of duplication with
MongoServerDao.d.ts, for now.
2019-10-29 13:36:55 -04:00
Mike Gerwitz 767a248e44 RatingService: Convert to TypeScript
This was an adventure, and was also used as a peer programming exercise to
introduce TypeScript to other programmers in the office.

This class has far too many dependencies, which made this difficult.  The
approach was to create .d.ts files for dependencies and wait on moving those
over for now, otherwise the task will never get done.

The RatingServicePublic trait was left as such for the time being; I was
able to work around a bug that was making it difficult to mix it into a
prototype.

There were no logic changes; this was just type refactoring.
2019-10-29 13:36:41 -04:00
Mike Gerwitz 8f7afd22e5 doc/hacking.texi (TypeScript Migration): .d.ts reference
The original suggestion for interfaces (see diff) is much more
confusing.  We've found .d.ts files to work well for incremental migration.
2019-10-24 10:30:14 -04:00
Mike Gerwitz fa80c79650 doc/hacking.texi (Nominal Typing)[isPositiveInteger]: Fix arg name 2019-10-24 10:30:14 -04:00
Mike Gerwitz 2771cf2a71 src/types/misc: Add UnixTimestampMillis, Seconds, and Milliseconds
These should be moved into their own module at some point and provide
functions to convert between and to yield these types.
2019-10-24 10:30:14 -04:00
Mike Gerwitz 5552de93d5 Rater: Remove interface
It's used only by HttpRater and the interface is no longer adhered to by
DslRater (which no longer implements it).
2019-10-24 10:30:14 -04:00
Mike Gerwitz e2edbfc7b2 src/numeric: New module
This beings to introduce compile-time safety for numeric values under the
assumption that they are enforced by the runtime.  See docblock for more
information.
2019-10-24 10:29:33 -04:00
Mike Gerwitz 7583cc1a71 RatingServiceSubmitNotify: Remove unused trait 2019-10-21 14:06:47 -04:00