Wednesday, September 14, 2011

ADAMSync Error -2146893813. Error code: 317

Today I discovered what may be a bug in ADAMSync, however I found a workaround. Let me go into detail sharing my findings with you. First... lets cover off my environment.

I have 4 forests, 10 domains. My AD LDS server is a member of subdomain.forest3.lan. Please see the diagram below.



- My LDS Server has multiple instances "LDS Databases" for each Active Directory domain. These instances are listening on 10001, 10002, 10003, 10004 etc... There is a 1 to 1 relationship between a single domain and its corresponding LDS instance.

- Each LDS Instance has only one application partition with the same distinguishedName mapping an Active Directory domain. For example dc=forest2,dc=lan or dc=bu2,dc=forest1,dc=lan.

- My ADAMSync configuration in each instance replicates the user class objects from Active Directory to "userProxy" class objects in each LDS Instance. Objects are successfully replicating.

- userProxy bind redirection is working successfully for all LDS Instances and the corresponding Active Directory domain (excluding forest4.lan due to no domain trusts existing). For more information on userProxy objects see http://clintboessen.blogspot.com/2011/04/userproxy-class-and-adam-lds.html

- There is no synchronization errors. I have validated all attributes declared in my adamsync.xml file are successfully synchronizing to the corresponding userProxy object in LDS for each user account.

- ADAMSync is successfully creating the same Organisational Unit structure matching that of LDS.

- The base distinguishedName of each LDS instance matches the base distinguishedName of the Active Directory domain.

- All child domains in forest1 synchronise using an ldapquery account located in the parent forest1.lan domain. This is configured in the XML file ldapquery and forest1.lan.

- forest2.lan synchronises using an account in forest2.lan

- forest3.lan synchronises using an account in subdomain.forest3.lan. No user accounts reside in the root domain forest3.lan. This is configured as an "empty root domain".

- forest4.lan synchronises using an account located in forest4.lan

The following Active Directory accounts were setup to make this solution work:

- subdomain.forest3.lan\LDSSVC. This account is only a member of "domain users" in subdomain.forest3.lan. This account has "Log on as service" rights assigned under "user rights assignments" on the LDS Server. This account provides the security context under which LDS Instance runs as a service. i.e. all LDS Services run as this account.

- subdomain.forest3.lan\LDSBoss. This account is only a member of "domain users" in subdomain.forest3.lan. During the installation of each LDS Instance using the "Active Directory Lightweight Directory Services Setup Wizard" this account was specified as the Administrator of each instance. This account has no access to the LDS Server itself, i.e. the account does not even have permissions to login to the LDS Server. It only has "Administrative privilages" inside each LDS Instance. All commands which manipulate LDAP data inside an LDS Instance such as ADAMSync must be run under the security context of this LDSBoss account.

- forest1.lan\ldapquery. This account is a member of "Domain Admins" and "Enterprise Admins"in the forest1.lan root domain. It is specified in the ADAMSync.xml file for each child domain in the forest1.lan forest. This account is used to perform LDAP Queries against the forest1.lan forest. If this account is not a Domain Admin or Enterprise Admin, ADAMSync fails to run.

- forest2.lan\ldapquery. This account is a member of "Domain Admins" in the forest2.lan domain. This is specified in the ADAMSync.xml file for the forest2.lan forest. It is used for LDAP queries against forest2.lan

- subdomain.forest3.lan\ldapquery. This account is a member of "Domain Admins" in the subdomain.forest3.lan domain. This is specified in the ADAMSync.xml file for the subdomain.forest3.lan forest. It is used for LDAP queries against subdomain.forest3.lan

- forest4.lan\ldapquery. This account is a member of "Domain Admins" in the forest4.lan domain. This is specified in the ADAMSync.xml file for the forest4.lan forest. It is used for LDAP queries against forest4.lan

So whats the bug you found clint?

I went and installed my ADAMSync XML configuration file running a command prompt under the security context of the user account SUBDOMAIN\LDSBoss with the following command:

adamsync /install localhost:10001 "C:\Windows\ADAM\AdamSync Configs\SUBDOMAIN\SUBDOMAIN-AdamSyncConf.XML" /passprompt

When prompted I entered the password for the subdomain.forest3.lan\ldapquery account which was specified in the ADAMSync XML file.

When I run ADAMSync.exe using my SUBDOMAIN\LDSBoss credentials (the credentials used to install the XML configuration file), the sync works successfully.

However I then created another account called SUBDOMAIN\boessenc_admin which I added to the configuration partition "Administrators" group in my LDS Instance. When I tried to sync using my SUBDOMAIN\boessenc_admin account the sync failed with ADAMSync.exe terminating. The following error is received:

Error occured fetching internationalized message number -2146893813. Error code: 317



Problem signature:
Problem Event Name: APPCRASH
Application Name: adamsync.exe
Application Version: 6.1.7600.16385
Application Timestamp: 4a5bc94c
Fault Module Name: ntdll.dll
Fault Module Version: 6.1.7600.16695
Fault Module Timestamp: 4cc7b325
Exception Code: c0000005
Exception Offset: 0000000000023b12
OS Version: 6.1.7600.2.0.0.272.7
Locale ID: 3081
Additional Information 1: dba5
Additional Information 2: dba5cead4302f0c0fa066ea618a55f8f
Additional Information 3: f135
Additional Information 4: f135fc65c6fa056fecabbbdf82720b5f


I created a memory dump of the ADAMSync process using the following MSDN article:
http://msdn.microsoft.com/en-us/library/bb787181.aspx

I submitted this memory dump to James Li from the Microsoft Directory Service Support Team. James analysed the memory dump carefully and suspects it might be caused by a bug in AdamSync.exe. Here is the stack information he provided me with:

00 00000000`0013e990 000007fe`fd479ce9 ntdll!RtlFormatMessageEx(
unsigned short * MessageFormat = 0x00000000`00000000,
unsigned long MaximumWidth = 0,
unsigned char IgnoreInserts = 0x10 '',
unsigned char ArgumentsAreAnsi = 0x00 '',
unsigned char ArgumentsAreAnArray = 0x00 '',
char ** Arguments = 0x00000000`0013f3b0,
unsigned short * Buffer = 0x00000000`ff7fb630,
unsigned long Length = 0x26d7,
unsigned long * ReturnLength = 0x00000000`0013f174,
struct _PARSE_MESSAGE_CONTEXT * ParseContext = 0x00000000`0013f1a8)+0x272

01 00000000`0013f110 000007fe`fd479a97 KERNELBASE!BaseDllFormatMessage(
unsigned char ArgumentsAreAnsi = 0x00 '',
unsigned long dwFlags = 0x1000,
void * lpSource = 0x00000000`0013f510,
unsigned long dwMessageId = 0x13d,
unsigned long dwLanguageId = 0x400,
unsigned short * lpBuffer = 0x00000000`ff7fb630,
unsigned long nSize = 0x2710,
char ** arglist = 0x00000000`0013f3b0)+0x239

02 00000000`0013f210 00000000`77024d88 KERNELBASE!FormatMessageW(
unsigned long dwFlags = 0,
void * lpSource = 0x00000000`ff7f91b0,
unsigned long dwMessageId = 0,
unsigned long dwLanguageId = 0xff797074,
unsigned short * lpBuffer = 0x00000000`ff7fb630,
unsigned long nSize = 0x2710,
char ** lpArguments = 0x00000000`0013f3b0)+0x37

03 00000000`0013f260 00000000`ff7942d0 kernel32!FormatMessageWStub(
unsigned long dwFlags = 0x13f510,
void * lpSource = 0x00000000`00002800,
unsigned long dwMessageId = 0,
unsigned long dwLanguageId = 0x5f7f20,
unsigned short * lpBuffer = 0x00000000`ff7fb630,
unsigned long nSize = 0x2710,
char ** lpArguments = 0x00000000`0013f3b0)+0x28


I continued investigating the issue... I then reinstalled the ADAMSync XML file by re-running the ADAMSync.exe process under SUBDOMAIN\boessenc_admin using the same command, and specifying the password for the SUBDOMAIN\ldapquery which was specified in the XML configuration file:

adamsync /install localhost:10001 "C:\Windows\ADAM\AdamSync Configs\SUBDOMAIN\SUBDOMAIN-AdamSyncConf.XML" /passprompt

I then ran the ADAMSync.exe program with the sync switch under the security context of SUBDOMAIN\boessenc_admin. It worked!

What did we learn?

Whatever account you used to install the ADAMSync.exe XML configuration file, i.e. the account the ADAMSync.exe process is running as during the config installation MUST also be used to perform the ADAMSync.exe synchronisation. If you attempt to perform a synchronisation using a different account other then the account used during the XML import, the ADAMSync.exe process will crash.

2 comments:

  1. Hi Clint,

    I was suffering this same error for 5 years, but thought it was "working as designed". Unfortunately we have automated syncing processes that currently must run under "privileged" user accounts. The real problem is that I don't really know where that config file credentials are stored so it's impossible to track any problem.
    I'm facing now a new challenge. I'm trying to sync through ssh remote command execution and all works fine with cygwin sshd daemon if you install that service to run under same user you installed your config file with. But if you try to run it from a different account it behaves as you noticed for other users, even if you login through ssh with the correct account with credentials access. I believe it's a problem with registry access when ssh public key authentication is used, because accessing using password behaves correctly.
    Maybe Microsoft should address this problem. Or tell users where the hell are those credentials stored to try to walk around it.
    Good luck.

    ReplyDelete
  2. Error is still there.
    This article saved me a lot of time.
    thx

    ReplyDelete