User Mode Register Debugger for AMDGPU Hardware Copyright (c) 2017 AMD Inc. Introduction ------------- umr is a userspace debugging and diagnostic tool for AMD GPUs using the AMDGPU kernel driver with limited support for driverless debugging (via PCI direct access). The tool allows reading/writing MMIO registers, analyzing wavefronts, ring contents, as well as performance tracking. It supports hardware from SI based hardware onwards and requires a v4.10 kernel or newer to function correctly. Older kernels (not older than v4.8) may work but with limited functionality/stability. Older kernels are not supported officially so please refrain from submitting bug reports in relation to them. The tool is open source and hosted at : https://cgit.freedesktop.org/amd/umr/ Users may report bugs, feedback, and submit patches to the amd-gfx mailing list at: : https://lists.freedesktop.org/mailman/listinfo/amd-gfx Building --------- To build umr you will need pciaccess and ncurses headers and libraries. Which are available in both Fedora and Ubuntu (as well as other distributions). To build simply invoke the make command $ mkdir build && cd build/ && cmake ../ $ make and then $ make install To install it. If you plan to use it as a non-root user you may want to add the setuid bit $ chmod +s `which umr` NOTE: Allowing untrusted users access to umr may lead to system comprimises, instability, and hardware damage. Do not setuid this tool on multi-user systems. Running umr ------------ umr is a command line which parses options and commands from the command line supplied. It processes arguments left to right which means that options specified after a command do not affect the command for instance $ umr -r *.dce100.mmFOO -O bits is not the same as $ umr -O bits -r *.dce100.mmFOO Users can get help with command options via $ umr -h or $ man umr Options can be stacked with commas. For instance: $ umr -O bits,follow,empty_log -ls Selecting Hardware ------------------- On machines with more than one AMDGPU device umr can be instructed which to look at. If the driver is loaded and display attached [*] the --instance option can select a device. The devices can be listed with the --enumerate option. [*] Kernels v4.11 and newer only require the amdgpu module to be loaded in order to work without '-f'. Once one is picked the number can be used for instance: $ umr -i 1 -s uvd6 Would scan the uvd6 block on the second AMDGPU device. If the AMDGPU driver is not loaded you can specify a device by name or PCI device ID for instance: $ umr -f tahiti -s uvd4 or $ umr -f 0x6780 -s uvd4 Would read the uvd4 block from the first tahiti device in the system. Note that the '-f' command does not support two instances of the same hardware. To choose between those the --instance command needs to be used. Quick Guide ------------ To read registers you can use various commands. To scan an entire IP block use the --scan command $ umr --scan uvd6 Would scan and print all of the MMIO registers for the uvd6 block (on the default instance 0 device). This can be specifed multiple times to read multiple blocks, for instance: $ umr --scan uvd6 --scan dce110 To read a specific register the --read command can be used: $ umr -r tahiti.uvd4.mmUVD_VCPU_CNTL Would read the mmUVD_VCPU_CNTL register of the uvd4 block. The asic name can be replaced with * to simplify scripting, for instance: $ umr -r *.uvd4.mmUVD_VCPU_CNTL With both --scan and --read the bits option can be used to decode bitfields. For instance, $ umr -O bits -s uvd6 Would scan the uvd6 block and print out all of the registers and their bitfield decodings. Registers can be written with --write and --writebit commands. To read ring contents the --ring command can be used. Optionally, ranges can be specified or by default it will read from 32 words before the read pointer upto the write pointer. The kernel supports a MMIO tracepoint and can be read with the --logscan command. Normally, one would want to follow it so the typical use would be $ umr -O bits,follow,empty_log -ls GPU Utilization ---------------- The GPU utilization can be summarize with the --top command. It is an interactive mode of umr which samples various registers and prints out the counts. When this mode is active the frequency of sampling can be changed by hitting '1'. It defaults to 100Hz but can be increased to 1000Hz. The frequency of reporting can be changed with '2' from 1Hz to 10Hz. Various fields can be enabled with keys indicated at the bottom of the screen. When all the desired options are enabled the 'W' (upper case) can be hit to write the configuration to disk. The fields can be logged to disk by hitting 'l' to toggle the logging on and off. By default, it will write to ~/umr.log in comma separated value format. It appends to the file so it can be toggled on and off as a test is instrumented. The first column includes an uptime counter so it can be used to sort the output. The use_colour (also: use_color) option can be specified to colourize the display making seeing busy bits easier to see. Wavefront Debugging -------------------- The status of valid wavefronts can be displayed with the --waves command. The bits option enables bitfield decoding. Normally, the columns would be pretty printed: $ umr -wa | column -t Note that wave fetching is unstable if GFX PG is enabled as reading some of the wave related registers can lead to GPU hangs while the GFX block is transitioning.