1317 Wednesday, June 1, 2016 |
Suresh Maurya
Dynamics CRM Consultant at IBM

Understanding Entity Change Tracking in Microsoft Dynamics CRM

In CRM integration projects, your business requirements often dictate that you should only retrieve those transaction records that have been recently created or modified in your source system since last integration. When your CRM system is the source, you would typically use CRM FetchXML query to pull data from the CRM system by comparing the record's modifiedon field, which is not a solution that is reliable enough and may not perform well when the source entity has a significant number of records.

To address this particular challenge, CRM Online 2015 Update 1 release introduced the Change Tracking feature, which offers a reliable and efficient way to track transactional data changes for CRM entities.

Turn on Change Tracking

In order to track data changes of a particular entity, you have to first turn on "Change Tracking" option for the entity. You would do so by going to the entity's customization page, and tick "Change Tracking" option under "Data Services" section.

Retrieve Changes

After Change Tracking has been enabled for the entity, you can use CRM SDK RetrieveEntityChangesRequest message to pull changes from CRM server. The following is a snippet that demonstrates how to pull changes from the CRM server.

 using (var service = new OrganizationService(crmConnection))
{
var request = new RetrieveEntityChangesRequest();

// Set which entity to get changes for.
request.EntityName = "account";
request.Columns = new ColumnSet("accountnumber", "name", "creditlimit");

// Set paging preferences.
request.PageInfo = new PagingInfo() { Count = 5000, PageNumber = 1 };

// Set change token returned from last pull.
request.DataVersion = changeToken; // set to null or remove this line to do an initial pull

// Get the changes.
var response = (RetrieveEntityChangesResponse)service.Execute(request);

// TODO: Process all the changed records (see the code snippet below)

// Save the token somewhere for future use
changeToken = response.EntityChanges.DataToken;
Console.WriteLine(changeToken);
}

Walk through Changes

After retrieving the changes, you will get two types of results. The first type of result is the newly added or updated records, and the second one is for the records that have been deleted since the last pull.

// Replace the TODO line in the above snippet with the following code
foreach (var change in response.EntityChanges.Changes)
{
if (change.Type == ChangeType.NewOrUpdated)
{
var changedItem = (NewOrUpdatedItem)change;
Entity newOrChangedEntity = changedItem.NewOrUpdatedEntity;
// TODO: Process new or updated entity record
}
else if (change.Type == ChangeType.RemoveOrDeleted)
{
var deleteditem = (RemovedOrDeletedItem)change;
EntityReference deletedEntityReference = deleteditem.RemovedItem;
// TODO: Process deleted entity records
}
}

Note that the returned NewOrChangedEntity is an instance of Entity class, while the RemovedItem (the deleted record) is an instance of EntityReference class.

Some Closing Notes

There are a few things that you should be aware when using this feature:

  • The "Change Tracking" option is an entity-level setting. It has to be enabled for the entity before you can use RetrieveEntityChangesRequest to pull data. If the option is not enabled, and you try to use this feature, you will get an error message telling you "Entity: account isn't enabled for change tracking" where account can be any entity name that you are working with.
  • Due to the fact that the "Change Tracking" option works at entity level, you would need to keep track of the change token for each entity individually. There is no organization level tracking token.
  • There is one special situation that you should watch out for. Suppose there is a newly added record after last pull and it was deleted before the new pull, you will get the record in the Delete result set, which you may not have knowledge about it.
  • In the case that you have selected to return a lookup field, the field's Name property is not returned.
  • When change collection has more records than the page size that you have specified, you would have to page through the change collection.
  • Based on my preliminary testing, it appears that CRM keeps multiple token versions, which is really nice. However, I imagine keeping too many versions on the server side would not make sense as each version would consume a certain amount of database space. There must be a limit in terms of how many versions are kept on the CRM server side. It could be either a time-based limit (say 90 days probably, which could still be a lot of change versions if I keep pulling from the server every 30 seconds) or a number-based limit. This is something that has yet to be confirmed.
  • When you retrieve changes using this feature, you will get the records in their current status, the old values before the change are not returned.
  • The user account needs to have organization level read (or so-called "Root Read") privileges for the concerned entity in order to uses RetrieveEntityChangesRequest to retrieve changes.
  • More details about this feature can be found at the CRM SDK documentation page: Use change tracking to synchronize data with external systems.

Microsoft Dynamics CRM 2011: Implementing Claims and IFD: Part 6

This session will cover some common troubleshooting questions related to Claims-Based Authentication and IFD in Microsoft Dynamics CRM.

 

CRM 2013 Quick Tip: Edited records are saved automatically in Microsoft Dynamics CRM

Microsoft Dynamics CRM 2011: Implementing Claims and IFD: Part 2

This session will cover how to install and configure the ADFS 2.0 server.

Microsoft Dynamics CRM 2015 Cortana Demonstration

Email Router Demystified in MS CRM Part -2

Troubleshooting Outgoing email issues

To do this, we need to reference the mail flow diagrams shown in the Explanation post previously.  We’ll first cover the Outgoing portion of the Email Router.

Troubleshooting Outgoing email issues is the not a difficult part of the Email Router to troubleshoot.  The reason is that there are only two different configuration options, SMTP or Exchange Online.  Realistically, you troubleshoot these the same way.  Obviously we have some restrictions during the troubleshooting steps when working with Exchange Online but these should be pretty obvious.  Let’s go ahead and get into troubleshooting at this point.

 

The first thing to know about troubleshooting the Email Router is to remember the traffic flow.  This was shown in the second post of this series but I have included it here as well:

 

Each step represents a potential break point.  The key to troubleshooting is to understand where the error occurs and what the error is telling you.  So how can we determine where the issue lies?

 

The first step to this is to understand, what’s the state of the email within CRM?  There are three status the email can be when in CRM. 

  1. Pending Send
  2. Sending
  3. Sent

What’s an easy way of understanding what emails are in which state?  The typical recommendation I have made to our customers is to use Advanced Find on the E-mail Messages entity.  When building your Advanced Find, you will want to ensure you have, at minimum, the following columns: Subject, From, To, Status Reason, No of Delivery Attempts.

Note: The one thing I want to ensure is pointed out is the last column of No. Of Delivery Attempts.  This tells us how many times the Email Router has tried to send the email.  Anytime it goes from Pending Send to Sending, it increments this attempt by 1.  We delay sending emails that have higher Delivery Attempt counts.

Each of the email statuses signify a different step within the mail flow diagram above.  Let’s review each of these now:

  1. Pending Send: Pending Send means that the email router has not processed this email message yet or there was an error when it did process it and put it back into a pending send status to try to send it again at a different time.  The key to knowing how to differentiate these is to look at the Advanced Find.
     
    One Pending email with the Subject of “Failing Email” shows that the router has tried sending the email five times and it’s still unable to do so since the Status Reason is still Pending Send.  This tells me that there is an issue between the Email Router and the Outgoing service (SMTP or Exchange Online).  The email router was able to retrieve the message from the CRM server, validated the user information within CRM, and submitted it to the Outgoing service to be delivered.   For some reason, the outgoing attempt failed for this email and I can review the Event Viewer Application Logs for more information.   The other email in a Pending Send state, shows the delivery attempts as 0.  This means the router has not yet picked up this email for processing.  This could be due to quite a few reasons. 
    1. The User’s profile does not list E-mail Router in the Email Access Configuration for Outgoing:
    2. The user is not listed in the Email Router Configuration Manager after clicking on Load Data:

      This could be due to the Email Access Configuration for the user not set to Email Router for Outgoing or the user’s email address has not been approved.
    3. The user does have Email Access Configuration set to Outgoing and is in the Email Router Configuration Manager but the user’s GUID <UserId> is not present in the Microsoft.Crm.Tools.EmailAgent.xml file:

      In this case, the user’s GUID is B8DD579B-501D-E211-AA2E-00155D51F43.  The reason could be that after the user’s settings were changed to Email Router (Email Access Configuration) no one went back in the Email Router Configuration Manager and clicked on Load Data and Publish.
    4. The user’s Personal Options do not allow the Email Router to do processing on their behalf.

      CRM 2011:

      CRM 2013:

      The setting for “Allow other Microsoft Dynamics CRM users to send e-mail on your behalf” is needed for those that generate emails and change the from field to use your account.  

      The setting “Allow E-mail Router to use my credentials to send and receive e-mail on my behalf” is required when the Outgoing profile within the Email Router Configuration Manager is set to User Specified for Access Credentials.
  2. Sending: Sending means the Email Router is currently processing the message.  It has already accepted it from CRM and is currently waiting for the Outgoing service to accept the message.  A failure here would send the email back into a Pending Send status and then Delivery Attempts would have been incremented.  If the issue is occurring at this step, then you are going to need to use some additional tools suggested below in the “Outgoing Protocols” section.
  3. Sent: This means that the Outgoing service has accepted the email for delivery and now it is up to the Email service to get the email to the To and CC addresses. 

Outgoing Protocols

We also need to understand the protocols being used for the Outgoing profile.  Each scenario dictates different types of tools and troubleshooting:

SMTP:

The main troubleshooting tool we will use for troubleshooting SMTP issues is some sort of network sniffer such as Microsoft Network Monitor (NetMon) or WireShark.  Being the company man that I am, let’s take a look at what NetMon will show us. 

 

By adding a Filter of “ProtocolName == “SMTP”, I can easily see all of the SMTP traffic.  Now the error the
problem email is the following:

 

Looking through the traffic, I can see that the Exchange server, for “some reason” is throwing Reset packets:

 

By looking at the NetMon traffic, I knew the issue was coming from the Exchange server and there must be some sort of configuration problem on that particular server.  If the issue was intermittent, maybe it was one of the servers in the environment having problems.  After doing further troubleshooting, it was found that, the firewall on the Exchange server was blocking the traffic from the CRM server. 

Exchange Online:

Exchange Online is using Exchange Web Services (EWS) to send emails.  In this case, we can use Fiddler to see what the response coming back to CRM is.

Additionally, we can use a tool called EWSEditor found on CodePlex.  EWSEditor allows us to emulate the same requests as what is being done by the Email Router.  In the screenshot above, you can see that CRM is making an EWS call to GetFolder.  You’ll notice that the Email Router is going to use the schema version for Exchange 2007 SP1, just a tip when using EWS Editor.  We’ll get into the actual usage in a tools post here.

The next part of this post is to cover Troubleshooting Incoming email issues.

Using Auditing - Microsoft Dynamics CRM 2011

In this video we demonstrate the 3 levels of control related to auditing.

We also demonstrate the Audit View and Audit Summary features.

Form events in MS CRM?

  • OnLoad Event
    • The OnLoad event occurs after the form has loaded.
    • It cannot prevent the window from loading.
    • Use the OnLoad event to apply logic about how the form should be displayed, to set properties on fields, and interact with other page elements.
  • OnSave Event
    • The OnSave event occurs when:
    • The user clicks the Auto save button button in the lower right corner of the form, even when there is no changed data to be saved.
    • Code executes the Xrm.Page.data.entity.save method, even when there is no changed data to be saved.
    • The user navigates away from the form and there is unsaved data in the form.
    • With auto-save is enabled, 30 seconds after data has changed and there is unsaved data in the form.
    • Code executes the Xrm.Page.data.save method and there is unsaved data in the form.
    • Code executes the Xrm.Page.data.refresh method passing a true value as the first parameter and there is unsaved data in the form.
  • Field OnChange Event
    • The OnChange event occurs when:
    • The data in a form field has changed and focus is lost.
    • Radio buttons or check boxes clicked.
    • Calling Xrm.Page.data.entity attribute.fireOnChange method.
    • The OnChange event does not occur if the field is changed programmatically using the setValue method. If you want call attribute.fireOnChange method.
  • Tab TabStateChange Event
    • The TabStateChange event occurs when the DisplayState of the tab changes due to user interaction or when the setDisplayState method is applied in code.
    • Use this event when you wish to change the src property of an IFRAME within the tab.
    • If you set the IFrame src property in the OnLoad event for an IFRAME within a collapsed tab, the value will be overwritten when the tab is expanded.
  • IFRAME OnReadyStateComplete Event
    • The OnReadyStateComplete event indicates that the content of the IFRAME has loaded and can be accessed in code.
    • Use this event when referencing IFRAME controls within your scripts.
  • Lookup Control PreSearch Event
    • The lookup control has a PreSearch event that occurs just before the control launches a dialog to search for records.
    • There is no UI to set event handlers for this event.
    • You must use the addPreSearch and removePreSearch methods on the lookup control to add or remove event handlers for this event.
  • OnProcessStatusChange Event
    • This event occurs when the status of a process instance changes.
    • Use the Xrm.Page.data.process.addOnProcessStatusChange method to add event handlers for this event
    • and the Xrm.Page.data.process.removeOnProcessStatusChange method to remove them.
  • OnStageChange Event
    • This event occurs when the user clicks the Next Stage or Move to previous stage buttons in the user interface
    • or when a developer uses the Xrm.Page.data.process.moveNext or Xrm.Page.data.process.movePrevious methods.
    • You can’t cancel the stage change using code in a handler for this event.
  • OnStageSelected Event
    • This event occurs when a stage of a business process flow control is selected.
    • You can’t cancel the stage selection using code in a handler for this event.
  • Knowledge Base Search Control Events
    • The knowledge base search control can only be added to forms for organizations that has the knowledge management feature enabled. This control has two events (OnResultOpened and OnSelection) that developers can programmatically assign event handlers to.
  • Subgrid OnLoad event
    • Subgrids load asynchronously from the form so you can’t reliably get access to the data in the subgrid in the form OnLoad event. But you can set an event handler for the OnLoad event of the subgrid.
    • This event will occur each time the data in the grid is refreshed, this includes when users sort the columns of the grid.
  • Editable Grid Events
    • OnRecordSelect
      • The OnRecordSelect event occurs when a single row (record) is selected in the editable grid. This event won't occur if a user selects different cells in the same row, or selects multiple rows.
    • OnChange
      The OnChange event occurs when a value is changed in a cell in the editable grid and the cell loses focus. This event can also occur when an attribute value is updated using the setValue method.
    • OnSave
      • The OnSave event occurs before sending the updated information to the server, and when any of the following occurs:
        There is a change in the record selection.
        The user explicitly triggers a save operation using the editable grid’s save button.
        The user applies a sort, filter, group, pagination, or navigation operation from the editable grid while there are pending changes.

MS CRM Many-to-Many Relationships of the "Native" Kind : CRM 4

Competitors

The competitor information management feature allows organizations to do the following:

  • Compile a repository of product literature, pricing structures, and product reviews for each competitor.
  • Track the products that competitors sell and compete with the organization’s products.
  • Add products or sales literature to help compete with the threat.
  • Maintain information on how to win against the competitor.
  • Track competitors by creating relationships between competitors and opportunities.
  • Track sales lost to competitors by specifying the competitor when a sale is lost. The organization can use this information to assess the opportunities lost to or won against each competitor.

Microsoft Dynamics CRM 2013 Setup and Upgrade New Features - Base and Extension Table Merge