The HPDOS project is an attempt to optimise the storage stack in hyperconverged setups in recognition of recent advances in hardware technologies like SmartNICs, NVMe SSDs and persistent memories.
Here we plan to take advantage of SmartNIC technologies more specifically SoC based NICs to speed up the storage stack.
## Getting started
The folder code contains the HPDOS server and client implementation. To use them clone this repo.
HPDOS uses `gRPC` system calls with `protocol buffers` as the data exchange standard. The project is managed using `Gradle` project manager. As a first step, install the system requirements listed at the end of this section.
### Compile
To compile the codebase run `./gradlew build` separately from the client and server root directory. Gradle should take care of installing and managing any third party libraries used.
### Execute
To execute the server and client run `./gradlew run` in separate terminals.
### Clean
The project build generated files can be cleaned by running `./gradlew clean`
### System Requirements
- Java | OpenJDK 14.0+
- Gradle
- Protocol Buffer / Protobuf
Here we plan to take advantage of SmartNIC technologies more specifically SoC based NICs to speed up the storage stack. We found that the Mellanox Bluefield-1 smartNICs do not support hardware acceleration support.
For example, custom match+action table offload with P4 is not supported. In this branch we explore the following:
1. Recent kernel-bypass techniques such as DPDK and mTCP for metadata service acceleration
2. Metadata service implementation for LSMtree backend (high-speed IO write operations)
3. Preliminary checksum computation offload for Netronome smartNIC backend
## Table of Contents
This folder is arranged as follows.
1.**code**
- DPDK\_server\_MDcache: contains basic key-value server cache code for DPDK kernel-bypass network stack
- mTCP_server_MDcache: : contains basic HTTP-based key-value server cache code for mTCP kernel-bypass network stack
- kernel_MDS_lsmtree: contains metadata server code with LSMtree backend for traditional kernel network stack
- checksum-offload-p4: contains checksum computation offload code in p4 and the non-offload code (C++) for comparison
2.**net_stack_repos**
- Contains the package repositories for DPDK and mTCP. This repos are modified versions that are compatible with our Mellanox Bluefield-1 smartNIC (Ubuntu-20.10)
3.**documentation**
- Contains slide deck and videos that explain concepts and guidance to configure and execute DPDK/mTCP code
