sfcode
An Online Competing and Development Environment
|
Jobs is a high-order API that adds inputs, runtime type checking, sequencing, and other functionality on top of RxJS' Observable
s.
An Observable
(at a higher level) is a function that receives a Subscriber
, and outputs multiple values, and finishes once it calls the Subscriber.prototype.complete()
method (in JavaScript):
This, of course, can be typed in TypeScript, but those types are not enforced at runtime.
job handler
. The function that implements the job's logic.raw input
. The input observable sending messages to the job. These messages are of type JobInboundMessage
.raw output
. The output observer returned from the job handler
. Messages on this observable are of type JobOutboundMessage
.A JobHandler
, similar to observables, is a function that receives an argument and a context, and returns an Observable
of messages, which can include outputs that are typed at runtime (using a Json Schema):
This seems like a lot of boilerplate in comparison, but there are a few advantages;
A simpler way to think about jobs in contrast to observables is that job are closer to a Unix process. It has an argument (command line flags), receive inputs (STDIN and interrupt signals), and output values (STDOUT) as well as diagnostic (STDERR). They can be plugged one into another (piping), and can be transformed, synchronized and scheduled (fork, exec, cron).
JobInboundMessage
includes:
JobInboundMessageKind.Ping
. A simple message that should be answered with JobOutboundMessageKind.Pong
when the job is responsive. The id
field of the message should be used when returning Pong
.JobInboundMessageKind.Stop
. The job should be stopped. This is used when cancelling/unsubscribing from the output
(or by calling stop()
). Any inputs or outputs after this message will be ignored.JobInboundMessageKind.Input
is used when sending inputs to a job. These correspond to the next
methods of an Observer
and are reported to the job through its context.input
Observable. There is no way to communicate an error to the job.JobOutboundMessage
includes:
JobOutboundMessageKind.Ready
. The Job<>
was created, its dependencies are done, and the library is validating Argument and calling the internal job code.JobOutboundMessageKind.Start
. The job code itself should send that message when started. createJobHandler()
will do it automatically.JobOutboundMessageKind.End
. The job has ended. This is done by the job itself and should always be sent when completed. The scheduler will listen to this message to set the state and unblock dependent jobs. createJobHandler()
automatically send this message.JobOutboundMessageKind.Pong
. The job should answer a JobInboundMessageKind.Ping
message with this. Automatically done by createJobHandler()
.JobOutboundMessageKind.Output
. An Output
has been generated by the job.JobOutboundMessageKind.ChannelMessage
, JobOutboundMessageKind.ChannelError
and JobOutboundMessageKind.ChannelComplete
are used for output channels. These correspond to the next
, error
and complete
methods of an Observer
and are available to the callee through the job.channels
map of Observable.Utilities should have some filtering and dispatching to separate observables, as a convenience for the user. An example of this would be the Job.prototype.output
observable which only contains the value contained by messages of type JobOutboundMessageKind.Output
.
Because jobs are expected to be pure functions, they can be composed or transformed to create more complex behaviour, similar to how RxJS operators can transform observables.
Another way to compose jobs is to schedule jobs based on their name, from other jobs.
Jobs input, output and argument must be serializable to JSONs. This is a big limitation in usage, but comes with the benefit that jobs can be serialized and called across memory boundaries. An example would be an operator that takes a module path and run the job from that path in a separate process. Or even a separate server, using HTTP calls.
Another limitation is that the boilerplate is complex. Manually managing start/end life cycle, and other messages such as ping/pong, etc. is tedious and requires a lot of code. A good way to keep this limitation under control is to provide helpers to create JobHandler
s which manage those messages for the developer. A simple handler could be to get a Promise
and return the output of that promise automatically.