1590 Wednesday, June 1, 2016 |
Suresh Maurya
Dynamics CRM Consultant at IBM

Understanding Entity Change Tracking in Microsoft Dynamics CRM

In CRM integration projects, your business requirements often dictate that you should only retrieve those transaction records that have been recently created or modified in your source system since last integration. When your CRM system is the source, you would typically use CRM FetchXML query to pull data from the CRM system by comparing the record's modifiedon field, which is not a solution that is reliable enough and may not perform well when the source entity has a significant number of records.

To address this particular challenge, CRM Online 2015 Update 1 release introduced the Change Tracking feature, which offers a reliable and efficient way to track transactional data changes for CRM entities.

Turn on Change Tracking

In order to track data changes of a particular entity, you have to first turn on "Change Tracking" option for the entity. You would do so by going to the entity's customization page, and tick "Change Tracking" option under "Data Services" section.

Retrieve Changes

After Change Tracking has been enabled for the entity, you can use CRM SDK RetrieveEntityChangesRequest message to pull changes from CRM server. The following is a snippet that demonstrates how to pull changes from the CRM server.

 using (var service = new OrganizationService(crmConnection))
{
var request = new RetrieveEntityChangesRequest();

// Set which entity to get changes for.
request.EntityName = "account";
request.Columns = new ColumnSet("accountnumber", "name", "creditlimit");

// Set paging preferences.
request.PageInfo = new PagingInfo() { Count = 5000, PageNumber = 1 };

// Set change token returned from last pull.
request.DataVersion = changeToken; // set to null or remove this line to do an initial pull

// Get the changes.
var response = (RetrieveEntityChangesResponse)service.Execute(request);

// TODO: Process all the changed records (see the code snippet below)

// Save the token somewhere for future use
changeToken = response.EntityChanges.DataToken;
Console.WriteLine(changeToken);
}

Walk through Changes

After retrieving the changes, you will get two types of results. The first type of result is the newly added or updated records, and the second one is for the records that have been deleted since the last pull.

// Replace the TODO line in the above snippet with the following code
foreach (var change in response.EntityChanges.Changes)
{
if (change.Type == ChangeType.NewOrUpdated)
{
var changedItem = (NewOrUpdatedItem)change;
Entity newOrChangedEntity = changedItem.NewOrUpdatedEntity;
// TODO: Process new or updated entity record
}
else if (change.Type == ChangeType.RemoveOrDeleted)
{
var deleteditem = (RemovedOrDeletedItem)change;
EntityReference deletedEntityReference = deleteditem.RemovedItem;
// TODO: Process deleted entity records
}
}

Note that the returned NewOrChangedEntity is an instance of Entity class, while the RemovedItem (the deleted record) is an instance of EntityReference class.

Some Closing Notes

There are a few things that you should be aware when using this feature:

  • The "Change Tracking" option is an entity-level setting. It has to be enabled for the entity before you can use RetrieveEntityChangesRequest to pull data. If the option is not enabled, and you try to use this feature, you will get an error message telling you "Entity: account isn't enabled for change tracking" where account can be any entity name that you are working with.
  • Due to the fact that the "Change Tracking" option works at entity level, you would need to keep track of the change token for each entity individually. There is no organization level tracking token.
  • There is one special situation that you should watch out for. Suppose there is a newly added record after last pull and it was deleted before the new pull, you will get the record in the Delete result set, which you may not have knowledge about it.
  • In the case that you have selected to return a lookup field, the field's Name property is not returned.
  • When change collection has more records than the page size that you have specified, you would have to page through the change collection.
  • Based on my preliminary testing, it appears that CRM keeps multiple token versions, which is really nice. However, I imagine keeping too many versions on the server side would not make sense as each version would consume a certain amount of database space. There must be a limit in terms of how many versions are kept on the CRM server side. It could be either a time-based limit (say 90 days probably, which could still be a lot of change versions if I keep pulling from the server every 30 seconds) or a number-based limit. This is something that has yet to be confirmed.
  • When you retrieve changes using this feature, you will get the records in their current status, the old values before the change are not returned.
  • The user account needs to have organization level read (or so-called "Root Read") privileges for the concerned entity in order to uses RetrieveEntityChangesRequest to retrieve changes.
  • More details about this feature can be found at the CRM SDK documentation page: Use change tracking to synchronize data with external systems.

Importing Data into CRM 2011

This video shows an example of importing Accounts and Contacts into CRM 2011. You can easily import records into almost any entity in CRM. You can create templates and use them over and over. But it's really pretty easy to just create them from scratch when you need them. I usually import direct from csv files.
 

 

Microsoft Dynamics CRM 2011: Implementing Claims and IFD: Part 5

This session will cover the setup of claims with an un-trusted domain.

Disassociate Records In Microsoft Dynamics CRM Using Early Bound

disassociate Method

To disassocite records in Microsoft Dynamics CRM use IOrganizationService.Associate(entityName, Guid, Relationship, EntityReferenceCollection) method.

Parameters

Name Type Comment
entityname String Entity logical name
guid Guid The ID of the record to which the related records are associated.
relationship Guid The name of the relationship to be used to create the link.
relatedEntities Guid A collection of entity references (references to records) to be associated..

Output

Void

This method is implemented by OrganizationService class and OrganizationServiceContext generated in previous chapter.

Using Early Bound 

Following example demonstarates how to disassociate a contact with three accounts in Microsoft Dynamics CRM using early bound

C#

// Associate the accounts to the contact record.

// Create a collection of the entities that will be 
// associated to the contact.
EntityReferenceCollection relatedEntities = new EntityReferenceCollection();
relatedEntities.Add(new EntityReference(Account.EntityLogicalName, _account1Id));
relatedEntities.Add(new EntityReference(Account.EntityLogicalName, _account2Id));
relatedEntities.Add(new EntityReference(Account.EntityLogicalName, _account3Id));

// Create an object that defines the relationship between the contact and account.
Relationship relationship = new Relationship("account_primary_contact");


//Associate the contact with the 3 accounts.
_service.Associate(Contact.EntityLogicalName, _contactId, relationship,
    relatedEntities);

Console.WriteLine("The entities have been associated.");

//Disassociate the records.
_service.Disassociate(Contact.EntityLogicalName, _contactId, relationship,
    relatedEntities);

Record Type: Quotes

Quotes

  • Represents a proposal or an estimate.
  • Converts to Sales Order.

 

If a customer accepts a presented quote, a sales representative can use Microsoft Dynamics CRM to create an order with the information contained in the Quote record with a single click.

 

Email Router Demystified in MS CRM Part -1

Troubleshooting Email Router Configuration Wizard issues

The first part of the Email Router to ensure you don’t run into any issues is around the Email Router Configuration Manager.  If you run into errors within the Email Router Configuration Manager, you will face these same errors when the Email Router is trying to process the email requests.  We need to ensure that the configuration is error free.

Load Data

The first step of ensuring the Email Router is configured properly is to click on Load Data.  When clicking on Load Data, it is going out to the Organization service and retrieving a list of CRM users that have their User Profile configured for E-mail Router for the Incoming/Outgoing E-mail Access Configuration.  If you run into an issue when clicking on Load Data, this indicates an issue accessing the CRM services, either the Discovery or Organization service endpoints.  To troubleshoot this issue, ensure you take the URL provided within the Deployment Profile and append it with the following: URL/XrmServices/2011/Discovery.svc?wsd
URL/XRMServices/2011/Organization.svc?wsdl

This should show an XML response like the following:

If you get any error accessing this URL, then you will need to troubleshoot those errors before the Load Data will ever work.  Since there are a vast number of problems that can come up when accessing this URL, we will not cover any of these. However, knowing that Load Data is directly related to the Deployment Profile should help you

Service Account

If you are able to access the Organization and Discovery services within Internet Explorer and not able to within Load Data, it could be an issue with the account running the Email Router service.  When clicking on Load Data, it is running under the context of the Email Router service account.  This account can be found within Services (Start > Administrative Tools > Services) with the name of Microsoft CRM Email Router.  By default, the account running the Email Router service is Local System.

NOTE: The service account needs to be a Microsoft Dynamics CRM user or it needs to be added to the PrivUserGroup for CRM OnPremise customers.

If you experience an error where the Email Router Configuration Manager is not able to Load Data, try changing the service account running the Microsoft CRM Email Router service to a domain user.  I have seen many situations where the Email Router is not able to connect to CRM due to proxy configuration.  Within these environments, the internet proxy configuration is pushed out to users and not applied to servers/workstations. 

If you still experience issues while clicking on Load Data after changing the service account to a domain user, then look at using a web debugger, called Fiddler, to try and give more information about what actual HTTP errors we are getting.  Again, when clicking on Load Data, we are trying to access a couple of HTTP endpoints from CRM.  In order for Fiddler to log the HTTP requests coming from the Load Data request, the Microsoft CRM Email Router service account must be the logged in user.  Fiddler will only capture HTTP requests of the user that opened the application.  We will cover using Fiddler in a later section but let’s view a situation where Fiddler can be used to help identify an issue with Load Data.

 

Fiddler 

In this scenario, I can see that when clicking on Load Data I am getting a very generic response in my OnPremise environment with Claims and IFD enabled.  When using the Auth endpoint for the Deployment Profile, the error I get from Load Data shows:

However, using Fiddler, I can get a better understanding of where the failure is occurring:

The error shows that it’s coming from ADFS.  This could help me with troubleshooting as I now know to look at the ADFS logs and see what exception is being raised there.

Test Access

After we have successfully loaded the users from the Load Data button, the next step to ensure the Email Router will work properly is to click on a user and then click on Test Access.  So when we click on Test Access, what exactly is the Email Router doing?  There is a lot of speculation of what this is exactly doing so let’s demystify and document what it actually does.

Incoming

  1. The first request being made is out to Exchange using EWS to do a GetFolder request for the default folder providing the user’s Mailbox email address and authenticating as the account within the Incoming Profile.  The response that comes back should be that of the Inbox.
  2. Now, as the account within the Deployment Profile, it logs into CRM and does a series of Organization service requests.  The first one is a WhoAmIRequest.  The WhoAmI response contains the user’s UserId, BusinessUnitId, and OrganizationId
  3. The next CRM request being made is CheckRouterCompatibility.  This is to verify that we are not using an older or newer version of the Email Router that is not supported by that version of Microsoft Dynamics CRM.
  4. Lastly, we do a CheckIncomingEmailRequest and checking if the result comes back with a ReasonCode of ShouldDeliver

 

Outgoing

  1. The first thing that happens is a WhoAmIRequest being made to the CRM server as the account specified in the Deployment Profile.  The WhoAmI response contains the user’s UserId, BusinessUnitId, and OrganizationId
  2. The next request being made is CheckRouterCompatibility.  This is to verify that we are not using an older or newer version of the Email Router that is not supported by that version of Microsoft Dynamics CRM.
  3. Lastly, it does one of the following:
    1. SMTP: The Email Router is doing a HELO request.  The router is looking for a ReplyCode of 220 within the response that comes back from the server.  We can emulate a similar request by using a Telnet as discussed HERE:

       
    2. EWS: The router is doing a GetFolder EWS request for the Default folder.  The response that comes back should be that of the Inbox.

As long as no error is presented during these requests, then you should see Succeeded responses within the Email Router Configuration Wizard.

 

The next part of this post is to cover Troubleshooting Outgoing email issues.

Visual Approval Workflow in Microsoft Dynamics Marketing

With Microsoft Dynamics Marketing 2015 Update the visual approval workflow provides a flexible, easy to use drag-and drop-area for approvals. Watch this short video to get an idea of how these workflows enable marketers to intuitively mimic internal processes in their Microsoft Dynamics Marketing instance.

What's New in CRM 2011 Workflows: Dialogs

Microsoft Dynamics CRM 2013 Application New Features - Access Teams

Conditional Formatting in the Outlook CRM Client

Microsoft Dynamics CRM 2011: Searchable Property

You can set the "Searchable" property of Dynamics CRM fields to fine-tune Advanced Find and create a better search experience for users.

Color Grid for Dynamics 365

Color Form for Dynamics 365