Wednesday, August 12, 2020

Could not connect to one or more vCenter Server Systems: https://vCenterFQDN: 443/sdk"

When I have logged in vCenter 7 vSphere Client in my home lab, I have experienced the message
"Could not connect to one or more vCenter Server Systems: https://vCenterFQDN: 443/sdk"
Below is the screenshot from vSphere Client ...



The message is very clear but such an issue can be caused by various reasons, therefore vpxd.log in vCenter Server appliance should be checked to identify the specific reason causing the unavailability of vCenter Server service providing API endpoint for other services.

In my particular case, I have seen the following log messages ...

2020-08-05T00:07:38.663Z error vpxd[18559] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000
2020-08-05T00:07:38.746Z error vpxd[16483] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000
2020-08-05T00:07:38.821Z error vpxd[16578] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000
2020-08-05T00:07:38.999Z error vpxd[16549] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000
2020-08-05T00:07:39.045Z error vpxd[16506] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000
2020-08-05T00:07:39.122Z error vpxd[16503] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000
2020-08-05T00:07:39.311Z error vpxd[16553] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

This means that something exhausted the maximum number (2000) of HTTP sessions vCenter daemon VPXD is accepting. Now the question is who is the troublemaker. The other error messages appearing in vpxd.log were about invalid login of hmsuser

2020-08-05T00:00:39.651Z info vpxd[16492] [Originator@6876 sub=Default opID=3ff7839f] [VpxLRO] -- ERROR lro-16 -- SessionManager -- vim.SessionManager.impersonateUser: vim.fault.InvalidLogin:
--> Result:
--> (vim.fault.InvalidLogin) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>
-->    msg = ""
--> }
--> Args:
-->
--> Arg userName:
--> "hmsuser"
--> Arg locale:
--> "en"

Who is this hmsuser? Well, HMS stands for HMS service used by vSphere Replication and it was the culprit. After Power Off vSphere Replication, I was able to login to vCenter again. I had no time for further problem management, but because of the lab environment, I will most probably install brand new vSphere Replication on the latest version.

Hope this can help other folks in VMware community.
 

1 comment:

Anonymous said...

This was exactly the same issue we had in our Environment! After speaking with VMware over several sessions, I stumbled over your post here. Thank you for the insight and resolution!