You have 100 zombies beating against your door. There is a chance one of them will beat his fists in the right spot to either smash a hinge or bump the handle in a way that the door springs open.
Your buddy across the street has only 5 zombies beating on his door, but is in the same predicament: they have a chance to smash and hinge or bump the handle.
Which door would you rather be behind?
If you choose the one with 5 zombies, then I’d say that is a less risky situation entirely because there are fewer zombies beating on the door.
If you move your SSH server from default port 22 to some obscure port like 38724, I can predict you will have fewer zombies beating on the door of your SSH server. You’ve lowered your risk. You’ve increased your security (depending on your definition of security).
(Obviously, I’m yet again annoyed at the insistence by some that there is no value in security through obscurity. Those people are confusing “security only through obscurity” as being the same as “no security value in obscurity.” I think most people say they like “security through obscurity” as an additive value to an overall posture. Not as the only measure.)