/etc/ssh/sshd_config

Disable PasswordAuthentication and ChallengeResponseAuthentication to
disable password authentication.

Protocol 2
PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM no
RSAAuthentication no
# do not look up the remote host name: this will improve performance
UseDNS no

References