Commit 122aaf3e authored by Shah Rinku's avatar Shah Rinku

Update README.md

parent 3f3b0c10
## DPDK for Mellanox Bluefield-1 ARM cores ## DPDK for Mellanox Bluefield-1 ARM cores
### Install and configure DPDK
1. Download the required DPDK compressed version & decompress it. 1. Download the required DPDK compressed version & decompress it.
https://core.dpdk.org/download/ https://core.dpdk.org/download/
...@@ -12,7 +14,7 @@ ...@@ -12,7 +14,7 @@
a. Check if the Poll Mode Driver (PMD) for your Mellanox NIC is set; in our case it is MLX5 a. Check if the Poll Mode Driver (PMD) for your Mellanox NIC is set; in our case it is MLX5
b. Add “CONFIG_RTE_LIBRTE_MLX5_PMD=y”, if it does not exist b. Add “CONFIG_RTE_LIBRTE_MLX5_PMD=y”, if it does not exist
3. DPDK configuration alternatives 3. DPDK configuration & alternatives
- Alternative 1: For newer DPDK versions (>= version 18) - Alternative 1: For newer DPDK versions (>= version 18)
<path-to-dpdk-folder>$ meson build <path-to-dpdk-folder>$ meson build
...@@ -22,107 +24,75 @@ ...@@ -22,107 +24,75 @@
<path-to-dpdk-folder>$ ninja <path-to-dpdk-folder>$ ninja
<path-to-dpdk-folder>$ ninja install <path-to-dpdk-folder>$ ninja install
- Alternative 2: For older DPDK versions & for our mTCP setup - Alternative 2: For older DPDK versions & for our mTCP setup
- Run:
<path-to-dpdk-folder>/usertools$ sudo ./dpdk-setup.sh
- Choose
7: arm64-bluefield-linuxapp-gcc (NIC installation)
41: x86_64-native-linuxapp-gcc (Host installation)
Corresponding folder is created at DPDK root folder
- For both alternatives:
- Mellanox driver, mlx5 supports kernel and poll mode
- No need to unbind the interface from kernel
- Allocate huge pages for DPDK operation
$ sudo sysctl -w vm.nr_hugepages=4096
- Test huge page allocation
$ cat /proc/meminfo | grep -i huge
**Note: In case of numa.h not found error**
$ sudo apt-get install libnuma-dev
#### ### Run DPDK application
DPDK
#####
COMPILE 1. Identify PCI address of the NIC port which you want to bind to DPDK
vim dpdk/config/common_linuxapp
#@rinku ADD FOLL
CONFIG_RTE_LIBRTE_MLX5_PMD=y
In case of numa.h not found error $ lspci
sudo apt-get install libnuma-dev
RUN 03:00.0 Ethernet controller: Mellanox Technologies MT416842 BlueField integrated ConnectX-5 network controller => p0
ubuntu@linux:~/dpdk-stable-19.11.8/arm64-bluefield-linuxapp-gcc/build/app/test-pmd$
sudo ./testpmd -w 03:00.1 -- -i -a
OR
sudo ./testpmd -- -i -a
sudo ./testpmd -- --nb-cores=2 -i 03:00.1 Ethernet controller: Mellanox Technologies MT416842 BlueField integrated ConnectX-5 network controller => p1
testpmd> set fwd txonly 2. Run testpmd on the two smartNICs connected directly using a cable
testpmd> port start 0
testpmd> port start 1
testpmd> set eth-peer 0 0c:42:a1:df:ac:41
testpmd> set eth-peer 1 0c:42:a1:df:ac:40
testpmd> start
testpmd> show port stats all
################ <path-to-dpdk-folder>/arm64-bluefield-linuxapp-gcc/build/app/test-pmd$ sudo ./testpmd -w 03:00.1 -- --nb-cores=2 --txq=2 -i
MTCP
IRON OUT CODE ISSUES
ubuntu@linux:~/dpdk-stable-19.11.8/arm64-bluefield-linuxapp-gcc/build/app/test-pmd$ sudo ./testpmd -w 03:00.1 -- -i -a
ubuntu@linux:~/mtcp-devel$ sudo ./setup_mtcp_dpdk_env.sh /home/ubuntu/dpdk-stable-19.11.8 **Parameters explained:**
export RTE_SDK=/home/ubuntu/dpdk-stable-19.11.8 w: whitelist device; nb-cores: #forwarding cores; txq: #transmit queues (requires RSS); i: interactive mode; a: automatic traffic start
export RTE_TARGET=arm64-bluefield-linuxapp-gcc
wget https://github.com/mtcp-stack/mtcp/archive/refs/heads/devel.zip You can check for more configuration parameters: testpmd> help
core.c: In function ‘mtcp_create_context’: 3. In the interactive mode, some configurations before testing network bandwidth
core.c:1332:4: error: ‘lcore_config’ undeclared (first use in this function)
1332 | lcore_config[master].ret = 0;
ubuntu@linux:~/mtcp-devel$ vim mtcp/src/core.c **I have considered our servers "ub-05" and "user" to demonstrate example configuration**
if (master == whichCoreID(cpu)) { - “ub-05” server smartNIC
//@rinku
//lcore_config[master].ret = 0;
//lcore_config[master].state = FINISHED;
/usr/include/aarch64-linux-gnu/bits/string_fortified.h:106:10: error: ‘__builtin_strncpy’ output may be truncated copying 1023 bytes from a string of length 1023 [-Werror=stringop-truncation] testpmd> set fwd txonly
106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
ubuntu@linux:~/mtcp-devel$ vim mtcp/src/config.c testpmd> set eth-peer 0 0c:42:a1:df:ac:41
//@rinku
//strncpy(optstr, line, MAX_OPTLINE_LEN - 1);
memcpy(optstr, line, MAX_OPTLINE_LEN - 1);
ubuntu@linux:~/mtcp-devel$ vim mtcp/src/io_module.c testpmd> set eth-peer 1 0c:42:a1:df:ac:40
Comment the funct code for “probe_all_rte_devices” ; add return -1
COMPILE testpmd> start
export RTE_SDK=/home/ubuntu/dpdk-stable-19.11.8 testpmd> show port stats all //repeat
export RTE_TARGET=arm64-bluefield-linuxapp-gcc - “user” server smartNIC
ubuntu@linux:~/mtcp-devel$ ./setup_mtcp_dpdk_env.sh [<path to $RTE_SDK>] testpmd> set fwd rxonly
export RTE_SDK=/home/ubuntu/dpdk-stable-19.11.8
export RTE_TARGET=arm64-bluefield-linuxapp-gcc
ubuntu@linux:~/mtcp-devel$./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET CFLAGS="-DMAX_CPUS=8" //Specify the max cpus which will run MTCP; use --disable-hwcsum when working in virtualized environment
ubuntu@linux:~/mtcp-devel$ make
configure: error: Could not find gmp.h
SOLUTION: sudo apt-get install libgmp-dev testpmd> set eth-peer 0 0c:42:a1:df:ac:49
testpmd> set eth-peer 1 0c:42:a1:df:ac:48
Check the configurations in apps/example testpmd> start
epserver.conf for server-side configuration
epwget.conf for client-side configuration
Configure ~/mtcp-devel/apps/example/config/arp.conf testpmd> show port stats all //repeat
Configure ~/mtcp-devel/apps/example/config/route.conf
######
RUN THE MTCP APP
export RTE_SDK=/home/ubuntu/dpdk-stable-19.11.8
export RTE_TARGET=arm64-bluefield-linuxapp-gcc
sudo ./epserver -p /home/ubuntu/www -f epserver.conf
sudo ./epwget 192.168.220.35/example.txt 10000000 -N 8 -c 10000 -f epwget.conf
#######
REVERT BACK MTCP NIC CHANGES
./setup_linux_env.sh [<path to $RTE_SDK>]
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment