This is a great milestone for Ampere GPU owners, but to get the most out of Ampere you’ll need to wait for PyTorch 1.8 which should have CUDA 11.1 - which has better performance. You can compile from source to get that today. Also CUDNN 8.04 is not Ampere optimized either. We’ll have to wait until November to see CUDNN 8.05 or 8.1 which will have better Ampere performance for 3090/3080 GPUs.