Commit b1113349 authored by NILANJAN DAW's avatar NILANJAN DAW

added setup links

parent 3bd30b06
......@@ -52,5 +52,9 @@ For details on the kernel stack based implementation on the NO_OFFLOAD or FULL_O
- [code_p4_tryouts](./code_p4_tryouts) contain a bunch of early experimental designs in __P4__ on __caching__ for distributed KV stores based on Netcache design.
- For our efforts on __RAMCloud offloading__ checkout shashank-* branches
- For __documenation__, references and bunch of early phase designs look into [documentation](./documentation) directory.
- For __setting up__ the Bluefield NIC read the [SmartNIC setup readme](./documentation/
- [mtcp-dpdk](./mtcp-dpdk) contains Bluefield compatible binaries and installation instructions for __mTCP and DPDK__.
\ No newline at end of file
- [mtcp-dpdk](./mtcp-dpdk) contains Bluefield compatible binaries and installation instructions for __mTCP and DPDK__.
#### Setup
- For setting up the Bluefield NIC in __embedded mode__ read the [SmartNIC setup readme](./documentation/
- For setup related to __Mediated Devices__ in embedded mode read the [Mediated Devices readme](./documentation/
- For setting up __separated mode__ or information regarding on __DPDK mTCP__ checkout [handover-material-rinku](./documentation/handover-material-rinku).
\ No newline at end of file
### Setting up Mediated Devices
Follow this guide for more details: [the newer docs using mlx-sf did not work at the time of writing]
Move into root [`sudo su`], then
1. Decrease the number of maximum mediated devices[mdev] - This step seems to be optional
`echo 4 > /sys/bus/pci/devices/0000:03:00.0/mdev_supported_types/mlx5_core-local/max_mdevs`
To know the PCI device ID, run: `lspci`
2. Check if the max_mdevs have been set: `cat /sys/bus/pci/devices/0000:03:00.0/mdev_supported_types/mlx5_core-local/max_mdevs`
3. Generate a uuid using: `uuidgen`
4. Create a mdev device using the uuid generated: `echo bfced4bb-60f6-4a8a-9562-1d1616f756bf > /sys/bus/pci/devices/0000\:03\:00.1/mdev_supported_types/mlx5_core-local/create`
5. The created device must be un-binded. Reason quoting Mellanox Docs: _"By default, if the driver vfio_mdev is loaded, newly created mdev devices are bound to it. To make use of this newly created mdev device in order to create a netdevice and RDMA device, you must first unbind it from that driver"_.
Run: `echo bfced4bb-60f6-4a8a-9562-1d1616f756bf > /sys/bus/mdev/drivers/vfio_mdev/unbind`
6. Add a mac address to the newly created device: `echo 00:11:22:33:44:51 > /sys/bus/mdev/devices/bfced4bb-60f6-4a8a-9562-1d1616f756bf/devlink-compat-config/mac_addr`
7. Mediated devices like other network interfaces are created in pairs, one part to which applications must be bound and the other part to connect to the ovs-bridge.
To check the name of one end of the interface, run: `cat /sys/bus/mdev/devices/bfced4bb-60f6-4a8a-9562-1d1616f756bf/devlink-compat-config/netdev`
8. Bind the mediated device to the mlx5_core driver. run:
`echo bfced4bb-60f6-4a8a-9562-1d1616f756bf >/sys/bus/mdev/drivers/mlx5_core/bind `
9. To check the name of the other end run: `ls /sys/bus/mdev/devices/bfced4bb-60f6-4a8a-9562-1d1616f756bf/net/` and
`ls /sys/bus/mdev/devices/bfced4bb-60f6-4a8a-9562-1d1616f756bf/infiniband/`
Log out of sudo user [`CTRL + D`]
10. Add the pf side of the interface to the ovs-bridge, run: `sudo ovs-vsctl add-port ovsbr1 pf1sf1`
11. Assign an IP address to the eth side of the interface, run `sudo ifconfig eth2 up`
__Pings should start working at this point!__
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment