A long time ago I used to have a general purpose server on a DSL line with port 22 open. I thought everything was fairly secure, patched up, etc... I never thought brute forcers would be able to guess one of my accounts (which was a friend's account actually). The kicker? The password was the username. I found out about the rootkit a few days after they got in. I tell ya, kids these days just don't know how to hide a backdoor.
A really good port knocker is your best defense against rogue attackers of an external service (because there is no defense against a 0-day). For an iptables-supported system I recommend Knockknock by Moxie Marlinspike (http://www.thoughtcrime.org/software/knockknock/).
It doesn't specifically target distributed attacks, but I'm not really sure how something like that could know whether incoming requests are part of a botnet or ligitimate logins.
Or use only public-key authentication. I've run many internet-exposed servers for years without problems or kludgy obscurity measures like port knocking simply by disabling all forms of password authentication. This is faster, easier and has the advantages of not requiring non-standard client software, being observable by an attacker on the same network, and avoids the sort of rote password entry which causes mistakes and discourages long, complex passwords.
You only need the ability to write to a file in your home directory. In the case of a VPS you could do that before editing the sshd config (in large environments this is trivially automated as part of your install process, which is why e.g. Ubuntu EC2 images ship with public-key-only SSH by default).
In the case of a shared server you can't disable password auth, which is a powerful argument for using a unique password since that system is far more likely to be compromised.
A really good port knocker is your best defense against rogue attackers of an external service (because there is no defense against a 0-day). For an iptables-supported system I recommend Knockknock by Moxie Marlinspike (http://www.thoughtcrime.org/software/knockknock/).