Restrict SSH to certain IP addresses, using iptables
By default SSH runs on port 22, so first we deny connections on port 22, then we add the allow rules for your address using iptables.
iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 22 -j DROP
iptables -I INPUT -p tcp -s youripaddress --dport 22 -j ACCEPT
iptables -I INPUT -p tcp -s your2ndipaddress --dport 22 -j ACCEPT
This will allow SSH on port 22 from youripaddress and your2ndipaddress.
It is recommended to save and run this as a script because if you run the first command without the others you will block yourself out and upon reboot you will need to enter it again.