gpustat

Just less than nvidia-smi?

gpustat uses NVIDIA's official python bindings for NVML library
(pynvml). As of now gpustat requires nvidia-ml-py >= 11.450.129, which
is compatible with NVIDIA driver versions R450.00 or higher. Please
upgrade the NVIDIA driver if gpustat fails to display process
information.

The GPU ID (index) shown by gpustat (and nvidia-smi) is PCI BUS ID,
while CUDA uses a different ordering (assigns the fastest GPU with the
lowest ID) by default. Therefore, in order to ensure CUDA and gpustat
use same GPU index, configure the CUDA_DEVICE_ORDER environment variable
to PCI_BUS_ID .

This is a very simple program, you might want to add its output to
some status bar, such as byoby, i3-bar, or xfce4-generic-monitor.