The Dispatcher sends a request to the Resource Management Server (Arbiter), detailing what resources it needs, on the Kafka topic `REQUEST_DISPATCHER_2_ARBITER`.
Format:
```javascript
{
"id":"unique-transaction-id",
"memory":1024,// in MiB
...// Any other resources
}
```
The Arbiter finds a list of machines that will satisfy those resource demands and return it to the Dispatcher on the Kafka topic `RESPONSE_ARBITER_2_DISPATCHER`.
Format:
```javascript
{
"id":"unique-transaction-id",
// "port": 2343 --- NOT IMPLEMENTED YET
"grunts":["a","b",...]// List of machine IDs
}
```
## Dispatch Module (DM)
## Dispatch Module (DM)
The DM is divided into two submodules the **Dispatcher** and the **Dispatch Daemon**. The Dispatcher runs on the Master node while the Dispatch Daemon runs on each Worker nodes. When a request arrives at the dispatcher, it queries the RM for resources and on receiving the resource requests the Dispatch Daemon to run and execute the function on the specified worker node.
The DM is divided into two submodules the **Dispatcher** and the **Dispatch Daemon**. The Dispatcher runs on the Master node while the Dispatch Daemon runs on each Worker nodes. When a request arrives at the dispatcher, it queries the RM for resources and on receiving the resource requests the Dispatch Daemon to run and execute the function on the specified worker node.
...
@@ -61,28 +83,6 @@ The DM is divided into two submodules the **Dispatcher** and the **Dispatch Daem
...
@@ -61,28 +83,6 @@ The DM is divided into two submodules the **Dispatcher** and the **Dispatch Daem
- For Worker nodes modify the config.json in dispatch_daemon to provide an unique ID to each node.
- For Worker nodes modify the config.json in dispatch_daemon to provide an unique ID to each node.
- Run the Master and Worker server as `npm start` or `node index.js`
- Run the Master and Worker server as `npm start` or `node index.js`
### Inter-component Communication Interface
The Dispatcher sends a request to the Resource Management Server (Arbiter), detailing what resources it needs, on the Kafka topic `REQUEST_DISPATCHER_2_ARBITER`.
Format:
```javascript
{
"id":"unique-transaction-id",
"memory":1024,// in MiB
...// Any other resources
}
```
The Arbiter finds a list of machines that will satisfy those resource demands and return it to the Dispatcher on the Kafka topic `RESPONSE_ARBITER_2_DISPATCHER`.
Format:
```javascript
{
"id":"unique-transaction-id",
// "port": 2343 --- NOT IMPLEMENTED YET
"grunts":["a","b",...]// List of machine IDs
}
```
### Internal Communication Interfaces
### Internal Communication Interfaces
#### Dispatcher
#### Dispatcher
...
@@ -117,7 +117,47 @@ Format:
...
@@ -117,7 +117,47 @@ Format:
{address:'UID of the worker'}
{address:'UID of the worker'}
```
```
#### Resource Manager
### Interaction API
The platform works via a HTTP API based interface, the interface is divided into two parts:
- Deploy: The deploy interface is used to upload the function file and store on the server, and also setup containers and VM images.\
The POST request contains two parameters: 1. *runtime* which specifies the runtime to use viz. isolate, process, container or virtual machine and 2. *severless* which sends the serverless function as file via multipart/form-data.\
On successful deployment the API returns a function key which is to be for function execution.
- Execute: To execute the submitted function, we use the Execute API.\
The API takes a route value as the key returned by the deploy API and a runtime parameter specifying the runtime to be used.
## Resource Manager
### Dependencies
1. clang: version 9.0
2. librdkafka: version 0.11.6
### Internal Messages
Upon being launched, each Grunt sends a JOIN message to the Arbiter on the Kafka topic `JOIN_GRUNT_2_ARBITER`.
Upon being launched, each Grunt sends a JOIN message to the Arbiter on the Kafka topic `JOIN_GRUNT_2_ARBITER`.
Format:
Format:
...
@@ -167,35 +207,3 @@ The Grunts, recieve this message and send back their state on topic `RESPONSE_GR
...
@@ -167,35 +207,3 @@ The Grunts, recieve this message and send back their state on topic `RESPONSE_GR
The Arbiter waits for a certain amount of time for the Grunts; then, it sends a list of however many Grunts have replied affirmatively to the Dispatcher on topic `RESPONSE_ARBITER_2_DISPATCHER`, as described above.
The Arbiter waits for a certain amount of time for the Grunts; then, it sends a list of however many Grunts have replied affirmatively to the Dispatcher on topic `RESPONSE_ARBITER_2_DISPATCHER`, as described above.
### Interaction API
The platform works via a HTTP API based interface, the interface is divided into two parts:
- Deploy: The deploy interface is used to upload the function file and store on the server, and also setup containers and VM images.\
The POST request contains two parameters: 1. *runtime* which specifies the runtime to use viz. isolate, process, container or virtual machine and 2. *severless* which sends the serverless function as file via multipart/form-data.\
On successful deployment the API returns a function key which is to be for function execution.
- Execute: To execute the submitted function, we use the Execute API.\