Be aware of backing up IIS MachineKeys
It’s quite difficult to tell you that I had had to spend 3 days looking into an issue when joining a machine to an existing SharePoint farm that I’m responsible for managing. This task shouldn’t have taken that long. Like every trivial workaround I have written in my blog, this post is just kindly warn you something that I experienced.
You may want to read a related article: Failed to connect to the configuration database when adding server to SharePoint farm
Recently one of the application servers in the farm went down and wasn’t accessible remotely. The infrastructure team came up with certificate issue on the remote desktop connection service. They did something I didn’t know but made the server work again. However, when I logged into SharePoint Central Administration website that was hosted in that machine’s IIS, SharePoint threw out several weird errors related to certificate. One of them was that I couldn’t change application pool identity either start IIS Admin service. SharePoint didn’t tell me where those certificates belong to. All of the errors were actually strange to me. I was afraid of the duplication of error made on other machines. Therefore, I decided to disconnect the machine out of the farm. And surely Central Administration service was started in the second application server first. Joining the first server to the farm was the next step then. However, this didn’t work as my expectation. Tracing log in ULS many times, I found that SharePoint always reported that “An exception occurred while committing IIS configuration changes: keyset does not exist” and “Unjoining farm”. It then terminated all processes of creating and assigning permissions to application pool or group. All web application began to be deleted. As a result, joining farm was failed. Reinstalling IIS role and running Preparation tool to automatically configure SharePoint didn’t help me.
In Google, folks suggest that the workaround is to assign permission to MachineKeys file in Program DataMicrosoftCryptoRSAMachineKey. I even set add Everyone to read it but didn’t work. I thought that the MachineKeys file wasn’t valid or got overwritten by another one. This could possibly be a root cause. I found one full backup done by Windows Server Backup feature in the server that I could use. I restored MachineKeys folder to the same path. Finally I was then able to join that server to the existing farm.
What is the MachineKeys folder?
IIS uses MachineKeys feature for encryption. You can find many keys that do encryption in Program DataMicrosoftCryptoRSAMachineKey. IIS Admin service replies on such a key to load and enumerate metabase. If you don’t have valid key, IIS Admin can’t be started.
Your application pool identity also replies on keys in this folder. If application pool can’t verify valid key, you can’t change to another identity.
Backing up IIS machine key is strongly recommended since it affects IIS configuration and even application pool identity that makes your SharePoint alive.
There are several ways to back up IIS machine key. The easiest way is to back up the MachineKeys folder. Another way is to use the following command.
aspnet_regiis -px "iisConfigurationKey" "C:iisConfigurationKey.xml" –pri
MachineKey has to be in the list of your SharePoint backup. Otherwise, you would have to deal with unknown errors that would steal your golden time. Below is the list of additional references I strongly recommend you to read.