When runtimes are communicating with the Orchestrator, they do so by sending JSON messages.
The types of messages are defined in the following table:
The basic message format (derived from the generic ARENA message) is as follows:
{
"object_id": "50c2f088-a5b6-48c5-bbc7-4a693b0117a2",
"action": "create",
"type": "orch_resp",
"data": {...}
}| object_id: | contains a unique identifier for that request and is used to confirm transactions TODO: need to implement this in orchestrator! |
|---|---|
| action: | is one of: "create" | "update" | "delete", where "create" is used for module creation and runtime registration; "update" is used for keepalives and "delete" is used to terminate modules and notify termination of modules and runtimes |
| type: | indicates the message direction with respect to the orchestrator: "req" | "resp" |
| data: | contains the request/response-specific payload |
As Runtimes appear, they register with the Orchestrator, which responds with a confirmation message indicating what is the requested runtime keepalive interval.
Runtime → Orchestrator: Runtime sends registration message to {realm}/proc/reg/{runtime-uuid}.
Orchestrator → Runtime: Orchestrator confirms registration to {realm}/proc/reg/{runtime-uuid}.
RuntimeRegMsg:
{
"object_id": "50c2f088-a5b6-48c5-bbc7-4a693b0117a2",
"action": "create",
"type": "req",
"data": {
"type": "runtime",
"uuid": "5f937916-d29d-4f66-801e-3d69f57728e2",
"name": "rt1",
"runtime_type": "linux",
"max_nmodules": 128,
"apis": ["wasm", "wasi", "channels", "loopback", "delete_module"],
"platform": {...},
"metadata": {...}
}
}| object_id: | is the id of the request, and that the uuid field is the id of the runtime (which is generated by the runtime). |
|---|---|
| data[type]: | time of request: "runtime" |
| data[uuid]: | UUID of the runtime |
| data[name]: | name of the runtime |
| data[runtime_type]: | type of runtime |
| data[max_nmodules]: | maximum number of modules supported by a runtime |
| data[apis]: | list of apis supported by the runtime |
| data[platform]: | details the environment of the system on which the runtime is launched; platform manifest |
| data[metadata]: | runtime metadata; for information only |
After receiving a registration request, the Orchestrator will send a confirmation message (RuntimeRegConfMsg) back to the runtime (see next).
RuntimeRegConfMsg:
{
"object_id": "50c2f088-a5b6-48c5-bbc7-4a693b0117e5",
"type": "resp",
"data": {
"uuid": "5f937916-d29d-4f66-801e-3d69f57728e2",
"name": "rt1",
"ka_interval_sec": 60
}
}| object_id: | will be the same id as in the request TODO |
|---|---|
| uuid: | should match the UUID used to register |
| name: | is included for debugging / inspection purposes |
| ka_interval_sec: | indicates how often the runtime should send a heartbeat to Orchestrator, (0 = do not send) |
Periodically (with the periodicity - ka_interval_sec - indicated in the registration confirmation), runtimes send keepalive messages to the orchestrator.
Runtime → Orchestrator: Runtime sends message to {realm}/proc/keepalive/{runtime-uuid}.
KeepaliveMsg:
{
"object_id": "50c2f088-a5b6-48c5-bbc7-4a693b0117a2",
"action": "update",
"type": "req",
"data": {
"type": "runtime",
"uuid": "5f937916-d29d-4f66-801e-3d69f57728e2",
"name": "rt1",
"apis": ["wasm", "wasi", "channels", "loopback", "delete_module"],
"children": [
{
"uuid": "5f937916-d29d-4f66-801e-3d69f57728e2",
"active": "2020-08-25T15:46:20.545Z"
"cpu_usage_percent": 0.0065224038671030556,
"mem_usage": 1185840
}
]
}
}| children[active]: | indicates the last time a module saw an I/O request. If no activity on channels, active =-1 |
|---|---|
| children[cpu_usage_percent]: | module cpu usage percent |
| children[mem_usage]: | module memory usage bytes |
Used as an indication that the runtime exited (or is no longer connected to MQTT).
Runtime → Orchestrator: Runtime sends message to {realm}/proc/reg/{runtime-uuid}.
{
"object_id": "9d6a1011-5fd2-4e34-9d15-c4adde3c7679",
"action": "delete",
"type": "req",
"data": {
"type": "runtime",
"uuid": "5f937916-d29d-4f66-801e-3d69f57728e2",
"name": "rt1"
}
}| data[uuid]: | the runtime's UUID. |
|---|---|
| data[name]: | a display name for the runtime (not necessarily unique). |
This message should be set as a runtime's MQTT last will for non-managed runtimes.
Register a runtime manager with the orchestrator.
{
"object_id": "50c2f088-a5b6-48c5-bbc7-4a693b0117a2",
"action": "create",
"type": "req",
"data": {
"type": "manager",
"uuid": "5f937916-d29d-4f66-801e-3d69f57728e2",
"name": "mgr1",
}
}| data[uuid]: | the manager's UUID. |
|---|---|
| data[name]: | a display name for the manager (not necessarily unique). |
Runtime → Orchestrator: Runtime sends message to {realm}/proc/reg/{runtime-uuid}.
Used as an indication that the runtime manager exited (i.e. the entire node is down).
Runtime → Orchestrator: Runtime sends message to {realm}/proc/reg/{runtime-uuid}.
{
"object_id": "50c2f088-a5b6-48c5-bbc7-4a693b0117a2",
"action": "delete",
"type": "req",
"data": {
"type": "manager",
"uuid": "5f937916-d29d-4f66-801e-3d69f57728e2",
"name": "mgr1",
}
}This message should be set as a runtime's MQTT last will for the runtime manager on behalf of all child runtimes.
A create module request usually is sent to the Orchestrator by a program or a user. The Orchestrator decides where the module should be executed and forwards the request to the Runtime. It is possible (given the right permissions) that Users or, Program send requests directly to the runtime.
Program, User → Orchestrator: A Create module request to the Orchestrator is sent to: {realm}/proc/control
Orchestrator, Program or User → Runtime: A Create module request to the Runtime is sent to: {realm}/proc/control/{runtime-uuid}
CreateModuleMsg:
{
"object_id": "fcb2780b-abdd-43b6-bc13-895baa2075a3",
"action": "create",
"type": "req",
"data": {
"type": "module",
"uuid": "44c72c87-c4ec-4759-b587-30ddc8590f6b",
"name": "test",
"parent": "059ec980-c38f-4612-a655-cc4c1ef9624c",
"file": "stdinread.wasm",
"apis": ["wasm", "wasi", "channels", "delete_module", "profile:deployed"],
"args": {
"argv": ["arg1", "arg2"],
"env": ["ENV_VAR1=1", "ENV_VAR2=2"],
...
},
"channels": [
{
"path": "light",
"mode": "rw",
"topic": "realm/kitchen/light"
}
],
}
}| data[uuid]: | (optional) identifier for the module; must be unique. If not passed, is generated randomly by the orchestrator. |
|---|---|
| data[name]: | short, human-readable name; not necessarily unique. |
| data[parent]: | (optional) runtime that the module should run in. If not passed, is scheduled by the orchestrator. |
| data[file]: | path to program file. |
| data[apis] (optional): | allows for specifying the apis the module requires to run. If not passed, uses ["wasm", "wasi"]. |
| data[args]: | "arguments" passed to the module / runtime. |
| data[args][argv]: | argv to pass to the module. |
| data[args][env]: | environment variables; pass as "{variable}={value}". |
| data[args][...]: | other keys in data[args] are passed to the runtime running the module. |
| data[channels]: | specify channels the module has access to. Each object in the channels array contains a path visible to the module, a mode ("r", "w", "rw"), and a topic. When publishing or subscribing to a topic starting with path, the path prefix is replaced with topic. In this example, subscribing to light/status will subscribe to realm/kitchen/light/status and forward messages accordingly |
Requests to delete a module can be sent by the Orchestrator or a user or program (given the right permissions).
Orchestrator, User or Program → Runtime: A delete module request (DeleteModuleMsg) is sent to {realm}/proc/control/{runtime-uuid}
DeleteModuleMsg:
{
"object_id": "fcb2780b-abdd-43b6-bc13-895baa2075b4",
"action": "delete",
"type": "req",
"data": {
"type": "module",
"uuid": "44c72c87-c4ec-4759-b587-30ddc8590f6b",
}
}When a module exits (due to a regular exit, crash or a delete module request), a ModuleExitMsg is sent to notify the Orchestrator.
Runtime → Orchestrator: A module exit notification is sent to {realm}/proc/control/
ModuleExitMsg:
{
"object_id": "fcb2780b-abdd-43b6-bc13-895baa2075b4",
"action": "exited",
"type": "req",
"data": {
"type": "module",
"uuid": "44c72c87-c4ec-4759-b587-30ddc8590f6b",
}
}