readme.txt 6.79 KiB
The files in this directory provide Xilinx PCIe DMA drivers, example software,
and example test scripts that can be used to exercise the Xilinx PCIe DMA IP.
This software can be used directly or referenced to create drivers and software
for your Xilinx FPGA hardware design.
Operating System Support:
=========================
Operating System Architecture: x86_64
Linux kernel: 3.10+
RHEL/CentOS: 7.6, 7.7, 7.8, 7.9,
8.1, 8.2
Ubuntu: 16.04.5 LTS, 16.04.6 LTS
18.04.1 LTS, 18.04.2 LTS, 18.04.4 LTS3, 18.04.5 LTS
20.04 LTS, 20.04.1 LT
HW Requirement:
===============
The PCIe DMA supports UltraScale+, UltraScale, Virtex-7 XT and 7 Series Gen2
devices.
Please refer to the Xilinx documentation "PG195 DMA/Bridge Subsystem for PCI
Express" for details of the IP.
Directory and file description:
===============================
- xdma/: This directory contains the Xilinx PCIe DMA kernel module
driver files.
- include/: This directory contains all include files that are needed for
compiling driver.
- tests/: This directory contains example application software to exercise the
provided kernel module driver and Xilinx PCIe DMA IP. This directory
also contains the following scripts and directories.
- load_driver.sh:
This script loads the kernel module and creates the necissary
kernel nodes used by the provided software.
The The kernel device nodes will be created under /dev/xdma*.
Additional device nodes are created under /dev/xdma/card* to
more easily differentiate between multiple PCIe DMA enabled
cards. Root permissions will be required to run this script.
- run_test.sh:
- dma_memory_mapped_test.sh, dma_streaming_test.sh:
- data/:
run_test.sh runs sample tests on a Xilinx PCIe DMA target and
returns a pass (0) or fail (1) result.
This script calls 2 other scripts in the same directory:
- dma_memory_mapped_test.sh for memory-mapped dma mode
- dma_streaming_test.sh for streaming dma mode
The data/ directory contains binary data files that are used
for DMA data transfers to the Xilinx FPGA PCIe endpoint device
with the run_test.sh.
!NOTE!:
=======
These scripts are intended for use with the PCIe DMA EXAMPLE
DESIGN ONLY.
- perform_hwcount.sh:
This script runs hardware performance for XDMA for both Host to
Card (H2C) and Card to Host (C2H). The result are copied to
'hw_log_h2c.txt' and hw_log_c2h.txt' text files.
For each direction the performance script loops from 64 bytes
to 4MBytes and generate performance numbers (byte size doubles
for each loop count).
You can grep for 'data rate' on those two files to see data
rate values.
Data rate values are in percentage of maximum throughput.
Maximum data rate for x8 Gen3 is 8Gbytes/s, so for a x8Gen3
design value of 0.81 data rate is 0.81*8 = 6.48Gbytes/s.
Maximum data rate for x16 Gen3 is 16Gbytes/s, so for a x16Gen3
design value of 0.78 data rate is 0.78*16 = 12.48Gbytes/s.
This program can be run on AXI-MM example design.
AXI-ST example design is a loopback design, both H2C and C2H
are connected. Running on AXI-ST example design will not
generate proper numbers.
If a AXI-ST design is independent of H2C and C2H, performance
number can be generated.
- scripts_mm/
This directory contains a set of scripts to check basic driver
loading/unloading and perform dma operations in memory-mapped
mode.
Compare with dma_memory_mapped_test.sh, the test is more
extensive with more dma size and it also utilizes fio tool in
addition to dma_from/to_device tools.
- xdma_mm.sh
top level script.
- io_sweep.sh, io.sh, unaligned
dma test via dma_from/to_device
- fio_test.sh fio_parse_result.sh
dma test via fio tool
- scripts_mm/ dependency
Some test in script_mm/ requires fio tool and python extension
Install fio:
- Centos/RHEL: yum install fio
- Ubuntu: apt install fio
Install python extension openpyxl, xlrd(version 1.2.0)
python --version
pip2 install openpyxl
pip2 install xlrd=1.2.0
Usage:
- Change directory to the driver directory.
cd xdma
- Compile and install the kernel module driver.
make install
- Change directory to the tools directory.
cd tools
- Compile the provided example test tools.
make
- Load the kernel module driver:
a. modprobe xdma
b. using the provided script.
cd tests
./load_driver.sh
- Run the provided test script to generate basic DMA traffic.
./run_test.sh
For more extensive memory mapped test:
assume the XDMA FGPA is at pci slot 0000:01:00.0
cd scripts_mm
./xdma_mm.sh 0000:01:00.0 | tee /tmp/xdma_mm.log
- Check driver Version number
modinfo xdma (or)
modinfo ../xdma/xdma.ko
Updates and Backward Compaitiblity:
- The following features were added to the PCIe DMA IP and driver in Vivado
2016.1. These features cannot be used with PCIe DMA IP if the IP was
generated using a Vivado build earlier than 2016.1.
- Poll Mode: Earlier versions of Vivado only support interrupt mode which
is the default behavior of the driver.
- Source/Destination Address: Earlier versions of Vivado PCIe DMA IP
required the low-order bits of the Source and Destination address to be
the same.
As of 2016.1 this restriction has been removed and the Source and
Destination addresses can be any arbitrary address that is valid for
your system.
Frequently asked questions:
Q: How do I uninstall the kernel module driver?
A: Use the following commands to uninstall the driver.
- Uninstall the kernel module.
rmmod -s xdma
Q: How do I modify the PCIe Device IDs recognized by the kernel module driver?
A: The xdma/xdma_mod.c file constains the pci_device_id struct that identifies
the PCIe Device IDs that are recognized by the driver in the following
format:
{ PCI_DEVICE(0x10ee, 0x8038), },
Add, remove, or modify the PCIe Device IDs in this struct as desired. Then
uninstall the existing xdma kernel module, compile the driver again, and
re-install the driver using the load_driver.sh script.
Q: By default the driver uses interupts to signal when DMA transfers are
completed. How do I modify the driver to use polling rather than
interrupts to determine when DMA transactions are completed?
A: The driver can be changed from being interrupt driven (default) to being
polling driven (poll mode) when the kernel module is inserted. To do this
modify the load_driver.sh file as follows:
Change: insmod xdma/xdma.ko
To: insmod xdma/xdma.ko poll_mode=1
Note: Interrupt vs Poll mode will apply to all DMA channels. If desired the
driver can be modified such that some channels are interrupt driven while
others are polling driven. Refer to the poll mode section of PG195 for
additional information on using the PCIe DMA IP in poll mode.