What are changes in active directory referred to as?

Answer 1

The Active Directory Changelog (v.2) Connector (hereafter referred to as ADCLV2) is a specialized instance of the LDAP Connector. It reports changed Active Directory objects so that other repositories can be synchronized with Active Directory.

The LDAP protocol is used for retrieving changed objects.

When run the Connector reports the object changes necessary to synchronize other repositories with Active Directory regardless of whether these changes occurred while the Connector has been offline or they are happening as the Connector is online and operating.

This connector also supports Delta Tagging, at the Entry level only.

The ADCLV2 Connector operates in Iterator mode.

Notes:

1. This Connector is a replacement for the Active Directory Changelog Connector; usage of the latter is deprecated.
2. This version of the Connector is able to process huge AD Servers (millions of entries) regardless of the administrative time limit for executing a query on AD (the MaxQueryDuration setting). In comparison the old version of the Connector could fail with TimeLimitExceeded error when run against big AD Servers.
3. It uses a simpler algorithm for retrieving changes and uses only one USN number to represent the synchronization state. In comparison the old Connector uses 4 USN numbers and a fairly complex algorithm.
4. It does not distinguish between "add" and "modify" operations - both are reported as "modify"; delete operations are reported as "delete". Not being able to distinguish between "add" and "modify" is not a serious restriction because the TDI Update Connector mode natively handles "add" and "modify" operations.
5. It might report "delete" operations for entries that have not been added to the repository being synchronized with AD (this will happen when an entry is added and deleted in AD while the Connector has been offline). It is something to be aware of, but it is not a serious restriction because TDI Delete Connector mode first checks if the entry to be deleted exists and if it does not exist, the "On No Match" hook is called - this is where you can place code to handle/ignore such unnecessary deletes.
6. The parameter Page Size specifies the size of the pages AD will return entries on (default value is 500).

Tracking changes in Active DirectoryActive Directory does not provide a Changelog as IBM Directory Server and some other LDAP Servers do.

The ADCLV2 Connector uses the uSNChanged Active Directory attribute to detect changed objects.

Each Active Directory object has an uSNChanged attribute that corresponds to a directory-global USN (Update Sequence Number) object. Whenever an Active Directory object is created, modified or deleted, the global sequence object value is increased, and the new value is assigned to the object's uSNChanged attribute.

On each AssemblyLine iteration (each call of the getNextEntry() Connector's method) it delivers a single object that has changed in Active Directory. It delivers the changed Active Directory objects as they are, with all their current attributes and also reports the type of object change - whether the object was updated (added or modified) or deleted. The Connector does not report which attributes have changed in this object and the type of attribute change.

Synchronization state is kept by the Connector and saved in the User Property Store - after each reported changed object the Connector saves the USN number necessary to continue from the correct place in case of interruption and restart; when started, the ADCLV2 Connector reads from the IBM(R) Tivoli(R) Directory Integrator's User Property Store this USN value stored from the most recent ADCLV2 Connector session.

Deleted objects in Active Directory

When an object is deleted from the directory, Active Directory performs the following steps:

• The object's isDeleted attribute is set to TRUE. Objects where isDeleted==TRUE are known as tombstones (not related to TDI tombstones).
• All attributes that are not needed by Active Directory are removed. A few key attributes, including objectGUID, objectSID, nTSecurityDescriptor, and uSNChanged are preserved.
• Moves the tombstone to the Deleted Objects container, which is a hidden container within the directory partition.

Tombstones or deleted objects are garbage collected some time after the deletion takes place. Two settings on the "cn=Directory Service,cn=Windows NT,cn=Service,cn=Configuration,dc=ForestRootDomain" object determine when and which tombstones are deleted:

• The "garbage collection interval" determines the number of hours between garbage collection on a domain controller. The default setting is 12 hours, and the minimum setting is 1 hour.
• The "tombstone lifetime" determines the number of days that tombstones persist before they are vulnerable to garbage collection. The default setting is 60 days, and the minimum setting is 2 days.

The above specifics imply the following requirements for synchronization processes that have to handle deleted objects:

• Synchronization has to be run on intervals shorter than the "tombstone lifetime" Active Directory setting.
• The objectGUID attribute has to be used for object identifier during synchronization. The object's distinguishedName attribute which uniquely identifies the position of an object in the directory tree, cannot be used because after the object is deleted it changes its place in the directory tree - it is moved in the Deleted Objects container and its old distinguished name is irrevocably lost. The objectGUIDattribute is however never changed. When a deleted object is found during synchronization, a search in the other repository for an object with the same objectGUID should be made and the found object should be deleted.

Moved objects in Active Directory

When an object is moved from one location of the Active Directory tree to another, its distinguishedName attribute changes. When this object change is detected based on the new increased value of the object's uSNChanged attribute, this change looks like any other modify operation - there is no information about the object's old distinguished name.

A synchronization process that has to handle moved objects properly should use the objectGUID attribute - it doesn't change when objects are moved. A search by the objectGUIDattribute in the repository which is synchronized will locate the proper object and then the old and new distinguished names can be compared to check if the object has been moved.

Use objectGUID as the object identifier

When tracking changes in Active Directory the objectGUIDattribute should be used for object identifier and not the LDAP distinguished name. This is so because the distinguished name is lost when an object is deleted or moved in Active Directory. The objectGUID attribute is always preserved, it never changes and can be used to identify an object.

When the ADCLV2 Connector reports that an entry is changed, a search by objectGUID value should be performed in the other repository to locate the object that has to be modified or deleted. This means that the objectGUID attribute should be synchronized and stored into the other repository.

BehaviorThe ADCLV2 Connector detects and reports changed objects following the chronology of the uSNChanged attribute values: changed objects with lower uSNChanged values will be reported before changed objects with higher uSNChanged values.

The Connector executes an LDAP query of type (usnChanged>=X) where X is the USN number that represents the current synchronization state. Sort and Page LDAP v3 controls are used with the search operation and provide for chronology of changes and ability to process large result sets. The Show Deleted LDAP v3 request control (OID "1.2.840.113556.1.4.417") is used to specify that search results should include deleted objects as well.

The Connector might report "delete" operations for entries that have not been added to the repository being synchronized with Active Directory - this will happen when an entry is added and deleted in Active Directory while the Connector has been offline. This is not a serious restriction because IBM Tivoli Directory Integrator's Delete Connector mode first checks if the entry to be deleted exists and if it does not exist, the "On No Match" hook is called - this is where you can place code to handle/ignore such unnecessary deletes.

The ADCLV2 Connector consecutively reports all changed objects regardless of interruptions, regardless of when it is started and stopped and whether the changes happened while the Connector was online or offline. Synchronization state is kept by the Connector and saved in the User Property Store - after each reported changed object the Connector saves the USN number necessary to continue from the correct place in case of interruption and restart.

The Connector will signal end of data and stop (according to the timeout value) when there are no more changes to report.

When there are no more changed Active Directory objects to retrieve, the Active Directory Connector cycles, waiting for a new object change in Active Directory. The Sleep Intervalparameter specifies the number of seconds between two successive polls when the Connector waits for new changes. The Connector loops until a new Active Directory object is retrieved or the timeout (specified by the Timeout parameter) expires. If the timeout expires, the Active Directory Connector returns a nullEntry, indicating there are no more Entries to return. If a new Active Directory object is retrieved, it is processed as previously described, and the new Entry is returned by the Active Directory Connector.

The ADCLV2 Connector delivers changed Active Directory objects as they are, with all their current attributes. It does not determine which object attributes have changed, nor how many times an object has been modified. All intermediate changes to an object are irrevocably lost. Each object reported by the Active Directory Connector represents the cumulative effect of all changes performed to that object. The Active Directory Connector, however, recognizes the type of object change that has to be performed on the replicated data source and reports whether the object must be updated or deleted in the replicated data source.

Note:

You can retrieve only objects and attributes that you have permission to read. The Connector does not retrieve an object or an attribute that you do not have permission to read, even if it exists in Active Directory. In such a case the ADCLV2 Connector acts as if the object or the attribute does not exist in Active Directory.

Using the Active Directory Changelog V2 ConnectorEach delivered entry by the Connector contains the changeType attribute whose value is either "update" (for newly created and modified objects) or "delete" (for deleted Active Directory objects). Each entry also contains 2 attributes that represent the objectGUID value:

• attribute objectGUID - contains a 16-byte byte array that represents the 128-bit objectGUID of the corresponding Active Directory object.
• attribute objectGUIDStr - contains the string representation of the hexadecimal value of the 128-bit objectGUID. It is delivered in the format {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}, where each x represents a hexadecimal digit.

If you need to detect and handle moved or deleted objects, you must use the objectGUID value as object identifier instead of the LDAP distinguished name. The LDAP distinguished name changes when an object is moved or deleted, while the objectGUIDattribute always remains unchanged. Store the objects' objectGUID attribute in the replicated data source and search by this attribute to locate objects.

Note:

Deleted objects in Active Directory live for a configurable period of time (60 days by default), after which they are completely removed. To avoid missing deletions, perform incremental synchronizations more frequently.

The ADCLV2 Connector can be interrupted any time during the synchronization process. It saves the state of the synchronization process in the User Property Store of the IBM Tivoli Directory Integrator (after each Entry retrieval), and the next time the Active Directory Connector is started, it successfully continues the synchronization from the point the Active Directory Connector was interrupted.

This Connector supports the IBM Tivoli Directory Integrator 6.1.1 Checkpoint/Restart functionality. When a restart is requested and restart data is passed, the Connector retrieves the USN number from the restart data and starts synchronization from this USN number.

ConfigurationThe Connector needs the following parameters:LDAP URLThe LDAP URL of the Active Directory service you want to access. The LDAP URL has the form ldap://hostname:port or ldap://server_IP_address:port. For example, ldap://localhost:389 Note:

The default LDAP port number is 389. When using SSL, the default LDAP port number is 636.

Login usernameThe distinguished name used for authentication to the service. For example, cn=administrator,cn=users,dc=your_domain,dc=com. Note:

If you use Anonymous authentication, you must leave this parameter blank.

Login passwordThe credentials (password). Note:

If you use Anonymous authentication, you must leave this parameter blank.

Authentication MethodThe authentication method to be used. Possible values are:

• Anonymous (use no authentication)
• Simple (use weak authentication (cleartext password))

Use SSLSpecifies whether to use Secure Sockets Layer for LDAP communication with Active Directory.Extra Provider ParametersAllows you to pass a number of extra parameters to the JNDI layer. It is specified as name:value pairs, one pair per line.Binary AttributesSpecifies a list of parameters that are to be interpreted as binary values instead of strings. The default value for this parameter is objectGUID objectSid.LDAP Search BaseThe Active Directory sub-tree that is polled for changes. The search base should be an Active Directory Naming Context if detection of deleted objects is required. For example, dc=your_domain,dc=com.Page SizeSpecifies the size of the pages AD will return entries on (default value is 500).Iterator State KeySpecifies the name of the parameter that stores the current synchronization state in the User Property Store of the IBM Tivoli Directory Integrator. This must be a unique name for all parameters stored in one instance of the IBM Tivoli Directory Integrator User Property Store.Start atSpecifies either EODor 0. EOD means report only changes that occur after the Connector is started. 0 means perform full synchronization, that is, report all objects available in Active Directory Service. This parameter is taken into account only when the parameter specified by the Iterator State Key parameter is not found in the User Property Store.State Key PersistenceGoverns the method used for saving the Connector's state to the System Store. The default is End of Cycle, and the choices are: After readUpdates the System Store when you read an entry from the Active Directory change log, before you continue with the rest of the AssemblyLine.End of cycleUpdates the System Store with the change log number when all Connectors and other components in the AssemblyLine have been evaluated and executed.ManualSwitches off the automatic updating of the System Store with this Connector's state information; instead, you will need to save the state by manually calling the ADCLV2 Connector's saveStateKey()method, somewhere in your AssemblyLine.Use Change NotificationsSpecifies whether to use notification when waiting for new changes in Active Directory. If not enabled, the Connector will poll for new changes. If enabled, the Connector will not sleep or timeout but instead wait for a Change Notification event (Server Search Notification Control (OID 1.2.840.113556.1.4.528)) from the Active Directory server.TimeoutSpecifies the maximum number of seconds the Connector waits for the next changed Active Directory object. If this parameter is 0, then the Connector waits forever. If the Connector has not retrieved the next changed Active Directory object within timeout seconds, then it returns an empty (null) Entry, indicating that there are no more Entries to return. The default is 5.Sleep IntervalSpecifies the number of seconds the Connector sleeps between successive polls.Detailed LogIf this field is checked, additional log messages are generated.CommentYour comments here. Migration from Active Directory Changelog EventHandler to Active Directory Changelog (v.2) ConnectorYou need to do the following to reproduce an old EventHandler's configuration into an ADCLv2 Connector's implementation:

1. Create a new AssemblyLine and insert the Active Directory Changelog(v.2) Connector in it.
2. Set the ldapUrl, ldapUsername, ldapPassword, ldapAuthenticationMethod, ldapUseSSL, ldapSearchBase and Debug Connector parameters to the values of the corresponding EventHandler parameters.
3. Set the iteratorStateKey Connector parameter to the value of the persistentParameterName EventHandler parameter.
4. Set the useNotifications Connector parameter to "true".
5. When implementing the AssemblyLine flow consider that the Connector reports newly added entries as modify.