The problem may be in the SSL-certificate on the mail server. A check with http://www.checktls.com/perl/TestReceiver.pl?FULL will easily confirm/deny this.
The recommended fix is of course to get a proper certificate on that mail server, but what if you can’t? Well… There is a quick/dirty fix:
You can modify
wp-includes/class-phpmailer.php. Search for:
public $SMTPOptions = array();
and replace this with:
public $SMTPOptions = array(
'ssl' => array(
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true
Keep in mind that this mod will revert after each and every (manual/automated) WordPress update!