Some notes on Office 365 synchronization error
One of my customers I have recently deployed Active Directory synchronization and single sign-on on Office 365 urgently called me he got message from the Microsoft Office 365 team that Windows Azure Active Directory did not register a synchronization attempt from the Directory Sync tool in the last 24 hours. I shortly suspected that the synchronization server might be down or its Internet connection got disconnected but this didn’t really make sense because a delta synchronization occurs every three hours.
I immediately logged into the synchronization server and then checked two sources: Directory Synchronization and FIMSynchronizationService in Event Viewer with the hope that it probably helped me in finding something helpfully.
At FIMSynchronizationService source, Event Viewer showed repeatedly every 3 hours that the Management Agent “TargetWebService” failed on run profile “Delta Confirming Import” because the server encountered errors. At Directory Synchronization source, the system only told that the user name and password was incorrect without pointing out which account that need to be verified.
For the FIM-related error, I opened FIM Synchronization Service Manager located in the path “C:Program FilesMicrosoft Online Directory SyncSYNCBUSSynchronization ServiceUIShell” on the synchronization server. Delta Confirming Import profile was not successfully run, with status “stopped-server-down”. This status really didn’t help identify what was happening in our synchronization server. Look at what Directory Synchronization source told, I was thinking that the error might be caused by the synchronization account. Basically, the synchronization account we use is Office365 Administrator account that has right privileges that provide the ability to synchronize changes from Active Directory directory service during the synchronization process of Synchronization service tool. I also checked Microsoft Online Services Directory Synchronization Service but it had completely been started.
To be more clearly in process of investigating, I opened Management Agents “TargetWebService” and opened Properties to look into a little more by clicking Management Agents tab and select Property on Actions panel. If you are not familiar with Forefront Identity Manager 2010, don’t adjust FIM setting especially Management Agents. In Properties windows, what you should be aware of is connection information because it contains crucial information of (synchronization) connection between Active Directory on-premises and Office 365.
Under Connection information section, there is the provisioning web service (https://adminwebservice.microsoftonline.com/ProvisioningService.svc) that is capable of provisioning users in Office 365. More information can be found in “Identity and Provisioning for Enterprise” whitepaper. Two things in my mind to check to make sure all connection information were correct, and worked.
- The synchronization server must successfully access to the provisioning web service.
- The password of the Office 365 administrator account must be valid.
When I logged into Microsoft Online Portal – Microsoft Online Services, the very first thing that beat on my eyes was the expiration warning of my administration account’s password.
So the expiration probably caused the error of synchronization service. I asked my customer to change password and then configure Synchronization configuration via Microsoft Online Services Directory Synchronization Configuration Wizard again. Make sure you run it under the Administration privilege unless you will get the error “Access to the registry key ‘HKEY_LOCAL_MACHINESoftwareMicrosoftMSOLCoExistence’ is denied” even the account you use is a member of the Enterprise Administration group in Active Directory.
After updating the Office 365 Administration credential, everything worked well. If you need to check the synchronization status, open FIM Synchronization Service Manager and look at Status column. Event Viewer is still a helper as well.
What is Management Agent?
As of reading this article, you might be asking yourself what Management Agent is. First, Management Agent is a connector that enable Forefront Identity Manager to seamlessly communicate with designated data sources supported by Microsoft. There are two types of Management Agents: Call-based and File-based. FIM supports various external data sources that would need to be connected with your internal application. That said, for example, let’s say you are planning to allow all users stored in IBM Tivoli directory service to be able to use Self-Service Password Reset functionality built on top of FIM 2010 R2, you will have to use Management Agent for IBM Directory Server and then FIM Synchronization service will be responsible for synchronizing IBM user profiles to FIM service database. As you can see, TargetWebService is an extensibility connectivity. TargetWebService management reads data stored in Metaverse (which stores aggregated user profiles synchronized from data source) and then maps those things to Microsoft Office 365 directory service. So if it is not running well, changed profiles in Active Directory directory service cannot be synchronized to Office 365. This great article covers in-depth principles of Office 365 synchronization. By the way, in SharePoint cases, you have an extensible connectivity named MOSS-<GUID>.
Extensibility connectivity allows you to customize a Management Agent that is not available in the list of supported management agents. If you need to play with Extensible Connectivity 2.0 (available on FIM 2010), read this article.
Throughout my error, I have introduced to a few things you should be aware of when encountering synchronization-related issues. I’m sure the check-list below is not enough, but at least to say it helps pretty much.
- Check if your synchronization gets disconnected to the Internet connection so that it cannot communicate with Provisioning service.
- Check if the password of your synchronization account (by default, Office 365 Administrator) is valid, unless it cannot be validated to synchronize changes made in Active Directory to Office 365. By default, after 90 days since you have registered Office 365, you must change administrator account’s password. If you need to set password to never expire, here is the workaround.
- Check if the Management Agent namely TargetWebService is well running by open FIM Synchronization Service Manager. If not, you must create a new one and properly configure. Note again, if you are not familiar with FIM, don’t edit anything even Connection information, unless you would have to re-deploy synchronization server.
- Check Microsoft Online Services Directory Synchronization Service (services.msc) to make sure it is already started.