I’ve never seen an attack that scans all ports. Normally it just checks open ports and then tries common credentials and exploits. If that fails it moves on to the next IP.

Changing the default port on SSH probably isn’t going to do much as SSH is already pretty secure. However it is a good rule of thumb to change the defaults.