iSCSI Best Practices - 2020 review

I have just listened to the Virtually Speaking podcast episode Back to Basics: iSCSI Back in 2014, I wrote a blog post about iSCSI Best Practices, but it was about general iSCSI best practices for any operating system or hypervisor. All these old best practices should be still considered in full-stack design but four design considerations have been highlighted in the above podcast. These four are

1. Jumbo Frames - more details in my blog post about iSCSI Best Practices
2. iSCSI Port Binding - more details here at VMware KB https://kb.vmware.com/s/article/2038869
3. Delay ACK - more details in my blog post and at VMware KB https://kb.vmware.com/s/article/1002598
4. NoOp Time out - in the article at https://www.jacobhopkinson.com/2019/05/10/iscsi-a-25-second-pause-in-i-o-during-a-single-link-loss-what-gives/

 

Update 2020-06-29:

Consider using custom named iSCSI IQN. See the justification at

https://www.yellow-bricks.com/2021/06/29/iscsi-iqn-may-have-changed-after-working-after-upgrade-to-7-0-u2/

 

Hope this info helps other folks in the VMware community.

Could not connect to one or more vCenter Server Systems: https://vCenterFQDN: 443/sdk"

When I have logged in vCenter 7 vSphere Client in my home lab, I have experienced the message

"Could not connect to one or more vCenter Server Systems: https://vCenterFQDN: 443/sdk"

Below is the screenshot from vSphere Client ...

The message is very clear but such an issue can be caused by various reasons, therefore vpxd.log in vCenter Server appliance should be checked to identify the specific reason causing the unavailability of vCenter Server service providing API endpoint for other services.

In my particular case, I have seen the following log messages ...

2020-08-05T00:07:38.663Z error vpxd[18559] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

2020-08-05T00:07:38.746Z error vpxd[16483] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

2020-08-05T00:07:38.821Z error vpxd[16578] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

2020-08-05T00:07:38.999Z error vpxd[16549] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

2020-08-05T00:07:39.045Z error vpxd[16506] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

2020-08-05T00:07:39.122Z error vpxd[16503] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

2020-08-05T00:07:39.311Z error vpxd[16553] [Originator@6876 sub=HTTP session map] Out of HTTP sessions: Limited to 2000

This means that something exhausted the maximum number (2000) of HTTP sessions vCenter daemon VPXD is accepting. Now the question is who is the troublemaker. The other error messages appearing in vpxd.log were about invalid login of hmsuser. 

2020-08-05T00:00:39.651Z info vpxd[16492] [Originator@6876 sub=Default opID=3ff7839f] [VpxLRO] -- ERROR lro-16 -- SessionManager -- vim.SessionManager.impersonateUser: vim.fault.InvalidLogin:

--> Result:

--> (vim.fault.InvalidLogin) {

-->    faultCause = (vmodl.MethodFault) null,

-->    faultMessage = <unset>

-->    msg = ""

--> }

--> Args:

-->

--> Arg userName:

--> "hmsuser"

--> Arg locale:

--> "en"

Who is this hmsuser? Well, HMS stands for HMS service used by vSphere Replication and it was the culprit. After Power Off vSphere Replication, I was able to login to vCenter again. I had no time for further problem management, but because of the lab environment, I will most probably install brand new vSphere Replication on the latest version.

Hope this can help other folks in VMware community.