@@ -4,7 +4,41 @@ The HPDOS project is an attempt to optimise the storage stack in hyperconverged
Here we plan to take advantage of SmartNIC technologies more specifically SoC based NICs to speed up the storage stack.
## Directory
### HPDOS NO_OFFLOAD/ FULL_OFFLOAD implementation
For details on the kernel stack based implementation on the NO_OFFLOAD or FULL_OFFLOAD HPDOS stack look into the [code](./code) folder
- For implementations on client cache design use the _client_cache_ branch
- For implementations using LSM tree _metadata_lsmtree_ branch
### MISC
-[code_p4_tryouts](./code_p4_tryouts) contain a bunch of early experimental designs in __P4__ on __caching__ for distributed KV stores based on Netcache design.
- For our efforts on __RAMCloud offloading__ checkout shashank-* branches
- For __documenation__, references and bunch of early phase designs look into [documentation](./documentation) directory.
-[mtcp-dpdk](./mtcp-dpdk) contains Bluefield compatible binaries and installation instructions for __mTCP and DPDK__.
### Setup
- For setting up the Bluefield NIC in __embedded mode__ read the [SmartNIC setup readme](./documentation/smartnic_setup.md).
- For setup related to __Mediated Devices__ in embedded mode read the [Mediated Devices readme](./documentation/mediated_devices.md).
- For setting up __separated mode__ or information regarding on __DPDK mTCP__ checkout [handover-material-rinku](./documentation/handover-material-rinku).
### Material in the Wild
These are a bunch of design/discussion documents created over the course of the project, which might help in understanding the course of development
1.[HPDOS Running doc - slides](https://docs.google.com/presentation/d/1NOxpgLvCdQnCP35Ia-PdAUEgfxyiBkhS5-TimaQIERg/edit?usp=sharing) A running document on the HPDOS R&D.
2.[HPDSO - Literature Survey](https://docs.google.com/spreadsheets/d/13Lu1AwV2tOMxN3JoQHOIlZtVhc9Bo7ePetriHrjGJqY/edit#gid=0) Earliest doc containing prior art references and reviews.
3.[Design Questions](https://docs.google.com/document/d/1zGzUerKbhf4gJFP484Bq4KHvSeUGYNCm6AwbDq64GNw/edit?usp=sharing) Initial design questions we faced.
4.[HPDOS Running doc](https://docs.google.com/document/d/1v7j-asEcF4CAOZ9NNXaPB1nZhbhZiy6RkC5gzWJCfEI/edit?usp=sharing) Even though it has the same name as P1, this is a separate document. Contains a detailed breakdown of storage operations and their resource affinity. Also contains some initial performance results on workload offloading on P4 based NICs.
5.[Workload Information](https://iitbacin-my.sharepoint.com/:o:/g/personal/20305r007_iitb_ac_in/ElVUeQYEgfRLsDXQX8vpAFsB2QDpg2g9sQ7tAItv-TFrWg?e=aGC9lH) Contains a detailed discussion on the benchmarking and the HPC application workloads thread.
## Directory Structure
```
├── code
│ ├── hpdos_client
...
...
@@ -40,21 +74,3 @@ Here we plan to take advantage of SmartNIC technologies more specifically SoC ba
│ └── offload.cpp
└── README.md
```
### HPDOS NO_OFFLOAD/ FULL_OFFLOAD implementation
For details on the kernel stack based implementation on the NO_OFFLOAD or FULL_OFFLOAD HPDOS stack look into the [code](./code) folder
- For implementations on client cache design use the client_cache branch
- For implementations using LSM tree metadata_lsmtree branch
### MISC
-[code_p4_tryouts](./code_p4_tryouts) contain a bunch of early experimental designs in __P4__ on __caching__ for distributed KV stores based on Netcache design.
- For our efforts on __RAMCloud offloading__ checkout shashank-* branches
- For __documenation__, references and bunch of early phase designs look into [documentation](./documentation) directory.
-[mtcp-dpdk](./mtcp-dpdk) contains Bluefield compatible binaries and installation instructions for __mTCP and DPDK__.
#### Setup
- For setting up the Bluefield NIC in __embedded mode__ read the [SmartNIC setup readme](./documentation/smartnic_setup.md).
- For setup related to __Mediated Devices__ in embedded mode read the [Mediated Devices readme](./documentation/mediated_devices.md).
- For setting up __separated mode__ or information regarding on __DPDK mTCP__ checkout [handover-material-rinku](./documentation/handover-material-rinku).
