Hello,
Since the default PowerShell endpoint works, and taking into account the difference in certificates, you can try binding the certificate of the default PowerShell endpoint to the Lync endpoint. To do this, bind the certificate to port 443 (HTTPS) of the IIS web site where the Ocspowershell web application resides. For information on how to do that, see the following guide, section Bind the Certificate to a website: https://www.sslshopper.com/article-inst ... s-7.0.html.
If that doesn't resolve your issue, try tracing the SSL traffic to understand where the cause for the issue may be. For information on how to do that, see the following article by Microsoft: https://blogs.technet.microsoft.com/tsp ... -failures/. Pay attention that when connecting to a Lync Server, the Ocspowershell endpoint is used (not the default WinRM endpoint), and the communication port is 443 (HTTPS).