# High Performance Distributed Object Store [HPDOS]
## About
The HPDOS project is an attempt to optimise the storage stack in hyperconverged setups in recognition of recent advances in hardware technologies like SmartNICs, NVMe SSDs and persistent memories. 

Here we plan to take advantage of SmartNIC technologies more specifically SoC based NICs to speed up the storage stack.


### HPDOS NO_OFFLOAD/ FULL_OFFLOAD implementation

For details on the kernel stack based implementation on the NO_OFFLOAD or FULL_OFFLOAD HPDOS stack look into the [code](./code) folder
- For implementations on client cache design use the _client_cache_ branch
- For implementations using LSM tree _metadata_lsmtree_ branch

For details on the DPDK and mTCP network stack based implementation on the NO_OFFLOAD or FULL_OFFLOAD HPDOS stack 
- For hash-table based server key-value cache implementation, look into the mtcp-dpdk/code folder of _metadata_lsmtree_ branch

### MISC

- [code_p4_tryouts](./code_p4_tryouts) contain a bunch of early experimental designs in __P4__ on __caching__ for distributed KV stores based on Netcache design.
- For our efforts on __RAMCloud offloading__ checkout shashank-* branches
- For __documenation__, references and bunch of early phase designs look into [documentation](./documentation) directory.
- [mtcp-dpdk](./mtcp-dpdk) contains Bluefield compatible binaries and installation instructions for __mTCP and DPDK__.

### Setup
- For setting up the Bluefield NIC in __embedded mode__ read the [SmartNIC setup readme](./documentation/smartnic_setup.md).
- For setup related to __Mediated Devices__ in embedded mode read the [Mediated Devices readme](./documentation/mediated_devices.md).
- For setting up __separated mode__ or information regarding on __DPDK mTCP__ checkout [handover-material-rinku](./documentation/handover-material-rinku).

### Material in the Wild

These are a bunch of design/discussion documents created over the course of the project, which might help in understanding the course of development

1. [HPDOS Running doc - slides](https://docs.google.com/presentation/d/1NOxpgLvCdQnCP35Ia-PdAUEgfxyiBkhS5-TimaQIERg/edit?usp=sharing) A running document on the HPDOS R&D.

2. [HPDSO - Literature Survey](https://docs.google.com/spreadsheets/d/13Lu1AwV2tOMxN3JoQHOIlZtVhc9Bo7ePetriHrjGJqY/edit#gid=0) Earliest doc containing prior art references and reviews.

3. [Design Questions](https://docs.google.com/document/d/1zGzUerKbhf4gJFP484Bq4KHvSeUGYNCm6AwbDq64GNw/edit?usp=sharing) Initial design questions we faced.

4. [HPDOS Running doc](https://docs.google.com/document/d/1v7j-asEcF4CAOZ9NNXaPB1nZhbhZiy6RkC5gzWJCfEI/edit?usp=sharing) Even though it has the same name as P1, this is a separate document. Contains a detailed breakdown of storage operations and their resource affinity. Also contains some initial performance results on workload offloading on P4 based NICs.

5. [Workload Information](https://iitbacin-my.sharepoint.com/:o:/g/personal/20305r007_iitb_ac_in/ElVUeQYEgfRLsDXQX8vpAFsB2QDpg2g9sQ7tAItv-TFrWg?e=aGC9lH) Contains a detailed discussion on the benchmarking and the HPC application workloads thread.

6. [HPDOS Handover document](https://docs.google.com/document/d/1j3ORGf8n0kQbjKvN56pLPv5Fa3vxHYegTH4zXq-Z78U/edit?usp=sharing) This is the handover document detailing the latest standing of the HPDOS project.


## Directory Structure
```
├── code
│   ├── hpdos_client
│   ├── hpdos_server
│   └── README.md
├── code_p4_tryouts
│   ├── client
│   ├── controller
│   ├── generator
│   ├── INSTRUCTIONS.txt
│   ├── LICENSE
│   ├── LiteratureSurvey.md
│   ├── MeetingMinutes.md
│   ├── mininet
│   ├── nc_config.py
│   ├── p4src
│   ├── README.md
│   └── server
├── documentation
│   ├── designs
│   ├── dfs_report.pdf
│   ├── handover-material-rinku
│   ├── HPDOS-presentation.pdf
│   └── references
├── mtcp-dpdk
│   ├── dpdk-19.11.8.tar.xz
│   ├── mtcp-devel.tar.xz
│   └── README
├── RAMCloud Offload
│   ├── client.cpp
│   ├── master.cpp
│   ├── no_offload_master.cpp
│   └── offload.cpp
└── README.md
```
