A few days ago, I have updated my home-lab VCSA vCenter Server 7.0 GA (15952498) to vCenter Server 7.0.0a (16189094). Everything seemed ok from the vCenter (vSphere Client) perspective. I was seeing there vCenter build 16189207, which is obviously VCSA 16189094.
The only problem I had was the fact, that I was not able to log in to VCSA VAMI.
After user authentication into VAMI, I was getting a message ... "Update installation is in progress". During the problem isolation, I tested the REST API and was getting the same message through Appliance REST API.
To be honest, I had no idea, how to check VCSA update status and move it forward, therefore I did the log files troubleshooting.
In /var/log/vmware/applmgmt/vami.log I had regularly repeating messages... "Executing VMware vCenter Server SNMP post upgrade actions..."
The next step was parsing PatchRunner.log. There was nothing remarkable except info about the patch stageDir located at "/storage/core/software-update/updates/7.0.0.10300/patch_runner" where i had two empty directories and file "patch_phase_context.json"
Normally, the directory /storage/core/software-update/updates/ is empty, however I have there still subdirectory with patch … /storage/core/software-update/updates/7.0.0.10300
I tried to delete the whole directory /storage/core/software-update/updates/7.0.0.10300 but it did not help.
It seemed, that post-upgrade actions (specifically VMware vCenter Server SNMP) cannot finish.
I HAD NO IDEA ... I WAS STUCK ... the next step was the research on the internal VMware slack channel. I have not found anything, therefore asked for help. In a few days, somebody else had the same issue in his home lab and he got the tip from VMware Engineering guys.
The trick was in file /etc/applmgmt/appliance/software_update_state.conf
I had there the following content
{
"state": "INSTALL_IN_PROGRESS",
"version": "7.0.0.10300",
"latest_query_time": "2020-06-12T17:30:46Z",
"operation_id": "/storage/core/software-update/install_operation"
}
so the trick was to change the content into
{
"state": "UP_TO_DATE"
}
After the change, I was able to log in to VAMI and use it as usual. Well, almost as usual.
I realized, that I was not able to start another update as buttons [STAGE ONLY] and [STAGE AND INSTALL] are greyed out.
Nevertheless, this is only GUI problem and VCSA CLI update procedure works like a charm, so the update can be done through SSH session and command software-packages install --url or software-packages install --iso
Another positive thing is, that I was able to perform vCenter native backup, so if needed, I have the option to do a VCSA backup and restore, which would probably solve this cosmetic GUI issue.
Disclaimer: This exercise was done in my home lab. If you will experience a similar issue in a production system, please, contact VMware support before any changes in VCSA.
The only problem I had was the fact, that I was not able to log in to VCSA VAMI.
After user authentication into VAMI, I was getting a message ... "Update installation is in progress". During the problem isolation, I tested the REST API and was getting the same message through Appliance REST API.
To be honest, I had no idea, how to check VCSA update status and move it forward, therefore I did the log files troubleshooting.
In /var/log/vmware/applmgmt/vami.log I had regularly repeating messages... "Executing VMware vCenter Server SNMP post upgrade actions..."
The next step was parsing PatchRunner.log. There was nothing remarkable except info about the patch stageDir located at "/storage/core/software-update/updates/7.0.0.10300/patch_runner" where i had two empty directories and file "patch_phase_context.json"
Normally, the directory /storage/core/software-update/updates/ is empty, however I have there still subdirectory with patch … /storage/core/software-update/updates/7.0.0.10300
I tried to delete the whole directory /storage/core/software-update/updates/7.0.0.10300 but it did not help.
It seemed, that post-upgrade actions (specifically VMware vCenter Server SNMP) cannot finish.
I HAD NO IDEA ... I WAS STUCK ... the next step was the research on the internal VMware slack channel. I have not found anything, therefore asked for help. In a few days, somebody else had the same issue in his home lab and he got the tip from VMware Engineering guys.
The trick was in file /etc/applmgmt/appliance/software_update_state.conf
I had there the following content
{
"state": "INSTALL_IN_PROGRESS",
"version": "7.0.0.10300",
"latest_query_time": "2020-06-12T17:30:46Z",
"operation_id": "/storage/core/software-update/install_operation"
}
so the trick was to change the content into
{
"state": "UP_TO_DATE"
}
After the change, I was able to log in to VAMI and use it as usual. Well, almost as usual.
I realized, that I was not able to start another update as buttons [STAGE ONLY] and [STAGE AND INSTALL] are greyed out.
Nevertheless, this is only GUI problem and VCSA CLI update procedure works like a charm, so the update can be done through SSH session and command software-packages install --url or software-packages install --iso
Another positive thing is, that I was able to perform vCenter native backup, so if needed, I have the option to do a VCSA backup and restore, which would probably solve this cosmetic GUI issue.
Disclaimer: This exercise was done in my home lab. If you will experience a similar issue in a production system, please, contact VMware support before any changes in VCSA.
7 comments:
thanks! I ran into the same issue.
thanks! I ran into the same issue, was pulling my hair out.
=)
You are welcome. It is great if it helps at least one other VMware community member. Cheers.
Thank you so much!! I have had this issue several times unfortunately :)
https://kb.vmware.com/s/article/67179?lang=en_US
I had the same issue and tried the procedure listed in the above KB with no luck. Then I used the CLI and wsa successful per your suggestion. Thanks!!!
Thanks - helped me sa well. Also used the CLI to do the upgrade after the KB was not working for me.
Post a Comment