Configuring Arm Templates (Deploying to different environments)

When developing around Azure, more specifically with Azure Logic Apps, Azure Functions, Azure API Management, and with almost no experience or none experience at all a good starting point it is using the browser.

This allows us to start doing our tasks, practising, testing and then deploying to a different environment.

I read many articles with different views of how to achieve this. Usually in any IT Company when we are developing we all need to use a source control tool, in this case it only applies to Microsoft Devops, and for some artifacts I think we need to be using Premium account.

I started the same way, I went to Azure Portal and created few Logic Apps to become proficient with the technology. Not to getting in here with a long story, I will cut things short now and if you need anything, comment in here or contact us.

Using Visual Studio and installing Azure extensions allowed me to start using my Visual Studio in conjunction with my TFS Online (Git in my case).

The reason of this post is more to have in mind a way to automate(not the focus of this post, but helps in terms of configuration) the deployment of the Resources to Azure.

Assumptions here:

• Use Visual Studio
• A resource group in Azure means an "Environment" 
• Full automation of Deployment trough Azure Dev Ops

High level steps below:

1. Created a Visual Studio 'Azure Resource Group' and used a Logic App template
   1. Created 2 files
      1. LogicApp.json
      2. LogicApp.parameters.json
   2. I changed the naming convention of the parameters file and created another file
      1. LogicApp.parameters.dev.json
      2. LogicApp.parameters.uat.json
   3. Opened the LogicApp.json design mode
      1. Configured with recurrence trigger
      2. Added an artifact 'Initialize variable' and set value to 'Hello World'
   4. Opened the LogicApp.json as json file and configured the following:
   5. Inside the parameters node:
      1. "Environment": { "type": "string", "defaultValue": "Dev" }
      2. "LogicAppName": { "type": "string", "defaultValue": "[concat('MyLogicAPp-', parameters('Environment'))]" }
   6. In the LogicApp.parameters.dev.json inside the parameters node:
      1. "Environment": { "type": "string", "value": "Dev" }
   7. In the LogicApp.parameters.uat.json inside the parameters node:
      1. "Environment": { "type": "string", "value": "Uat" }

This is a personal option of deploying to different environments, but as i saw there are many ways of doing the same. Hope it is clear.

The result in terms of naming convention would be something like:

1. Azure Resource Groups
   1. Dev
      1. Logic Apps
         1. MyLogicApp-Dev
   2. Uat
      1. Logic Apps
         1. MyLogicApp-Uat

Why this option? Because if you use Azure Dev Ops, you know for instance that navigating for example to Logic Apps area:

Without doing this naming convention the result would be something like.

Name	Resource Group
MyLogicApp	Dev
MyLogicApp	Uat

Doing this naming convention the result would be something like.

Name	Resource Group
MyLogicApp-Dev	Dev
MyLogicApp-Uat	Uat

If you have one or two logic apps that's probably fine doing without appending the environment to the name, but honestly I found that working in larger projects using Azure this is actually beneficial.

Any additional information or help please don't hesitate in contacting us.

CRM Transaction Mode operations (Dynamics 365 V9)

I hope I'm not late with this post. Who never had to do develop some code where the requirement would be "In Transaction"?

I don't think this will apply to all possible scenarios but it is a start.

Let's assume the following scenario: In CRM 365, someone asked you to apply a change to an entity based on a really easy filter. For example, if a customer address is in City "A", set a custom field with value "B", however, if you can't update one of the records for some reason (example, exception thrown), you need to rollback everything.

Here, and we all know that are many ways to achieve this, I'm giving one.

Hypothetically, let's assume we opt for a plugin at first sight, however, after some testing we start to get Timeouts because we need to update to many records and the 2 minutes just pass.

Another option would be with a Console Application, and this is a personal option, not 100% right nor 100% wrong, there is one of many ways. 

By not doing with a Plugin, brings me to a problem that needs to be addressed based on the requirements that is the requirement "Transaction". If one of the records fail, I need to undo the others, so, more code to deal with that, however, you can avoid that for this specific case only with CRM SDK, no need to reinvent the wheel.

Found in the SDK the message ExecuteTransactionRequest that do the Transaction I really needed, this allows me to perform multiple requests as one single database transaction. Using this message allows executing two or more organization services requests in a single database transaction. If one of the requests fails the transaction is rolled back.

For more information visit Microsoft page below:

https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/org-service/use-executetransaction

Any questions please reply trough this post or contact us.

Hope it helps.

Azure API Management Policies

 This post is about a scenario I had to overcome when connecting to a third party API trough Azure API Management.

Below I just have a small explanation on what are Policies in Azure API Management. that i just copied and pasted from Microsoft.

After will tell the issue and the solution i had to implement.

Transcript from Microsoft "https://docs.microsoft.com/en-us/azure/api-management/api-management-policies":

"Policies are a powerful capability of the system that allow the publisher to change the behavior of the API through configuration. Policies are a collection of Statements that are executed sequentially on the request or response of an API. Popular Statements include format conversion from XML to JSON and call rate limiting to restrict the amount of incoming calls from a developer. Many more policies are available out of the box.

Policy expressions can be used as attribute values or text values in any of the API Management policies, unless the policy specifies otherwise. Some policies such as the Control flow and Set variable policies are based on policy expressions. For more information, see Advanced policies and Policy expressions."

Scenario

I have 2 API's in my API Management service, one I called AUTH, the other, Services. From the Auth I just get a Token then after I will need to pass in all methods inside the Services API.

Another thing that worth to mention is, all needs to be deployed with Azure Devops Release Pipelines, don't want actually anything that needs to be deployed manually. 

Problem

Because of some import restrictions of importing API definitions to Azure API Management from a Swagger file, when importing the Swagger file, the required headers, that were defined in the file were not configured in the API.

Solution

I had to add a policy to check if a header was being passed from in my case a Logic App inside the API Management Service, but had to apply to all API's inside, so I could automate the deployment as said in the scenario. 

With this, I only had to use 2 artifacts on the release in DEV Ops, the first to create the API Management, and the other one to import an API from a swagger file to the API Management service created with the first artifact.

The condition inside the policy is important in here because as I said, I have 2 API's inside my API Management Service, however, i only want the policy to be applied in one of the cases. 

Hope it helps.

CRM 365 On Prem - Import CRM 2016 Organization

For the last days working on testing new features of CRM 365 Online, CRM 2016/365 On Prem and few more things.

And again, i had another issue, this one is really annoying, but let me explain what happened.

I have an environment in CRM 2016 with service pack 1.1 (8.1.1.1005) and we need to upgrade to V9 (365 Onprem). 

So, what i did? Backup of the database, restore of the database in the new V9 environment and imported the organization, expecting that the process would run smoothly, WRONG, it didn't, and that i should already expect based on my experience.

The error i got was: "09:24:11|Verbose| Error occured in UpdateRollupFieldStoredProceduresInternal: Microsoft.Crm.Metadata.MetadataAttributeMetadataNotFoundException: 'Attribute' metadata entity doesn't contain metadata attribute with Name = 'autonumberformat'."

I googled it, and nothing. So, in these cases what should we do? And again based on my experience, let's try to import into at least a same environment major version, in this case CRM 2016 (8.1) or CRM 2016 (8.2).

I imported into a VM with the same version of rollup and it worked. Next deleted the database, upgraded to version (8.2.2.0112), restored again the one from V8.1and imported the organization again. Voila, it worked. 

After doing this, imported this org into the V9 environment and it worked. Perfect, i could be happy, but not completely, needed more testing.

Again, deleted the database, upgradeed to version (8.2.9.19), last update in version 8.2, restored again the one from V8.1 and imported the organization again. And guess what, expecting to work, of course it didn't, was expecting a field from the systemuser table.

What can we learn from here, at least Microsoft is consistent, the same happened in CRM 2011, if not mistaken, we couldn't install update rollup 15, without 14, however, having the update rollup 14, we could go to update 21. What this means is, update rollup 14 was a major update release in the product, what it seams happening in here as well.

Hope it helps.

CRM 365 - Unified Interface - do not see Run Workflows On Demand in forms or views

 Ok, again one more time Microsoft strikes again with the new release, in this case Unified Interface.

I just enabled the Unified Interface in one of the CRM Organizations I am working in (Sandbox) and saw a couple of strange things happening. Being System Administrator and not being able to create new records or even updating records i am the owner (entity account), already sorted, please see my previous post.

So, at the moment i am just uploading the new versions of my Plugins/Workflows with the latest nuget pakages from Microsoft. What was my surprise when just added a process to Run On Demand, and couldn't see the "Process - Run Process" on the form and grid.

What i did next, set to run on Create, created a record, and saw that was still working as expected. After some investigation, i have found that as an extra, we have to enable few things in the Platform Admin Center, and we need to be sure that we have as well the Microsoft Flow license is enabled.

What the hell??!!? The product is not supposed to work as before? At least functionality that was even not said by Microsoft that would be deprecated?

The steps to follow are below:

1. Check if Licence is enabled

Open https://portal.office.com/adminportal#/users and select the user
Click vertical … (skinny burger menu?) and select Manage product licenses
Expand Apps and make sure Flow for Dynamics 365 is selected

2. Enable Microsoft Flow

Enable setting to show Microsoft Flow on the forms and in the sitemap
Open Power platform admin center (https://aka.ms/ppac)
Select Environments
Click … (horizontal one, ugh), select Settings
Select Behavior
Make sure Show Microsoft Flow on forms and in the site map is selected

Hope it helps.

CRM Dynamics 365 V9.0 - Web API improvements

 The following improvements are included in this release of the Web API, our OData v4 endpoint:

• Custom actions that return EntityReference, Entity, or EntityCollection types are available.

• Changes to API behaviors are available using the latest v9.0 version of the service, legacy behaviors remain available in the v8.x version. You don’t have to change your code when you upgrade.

• New messages: GrantAccess, ModifyAccess, and RetrieveSharedPrincipalsAndAccess messages are now available using the Web API.

• We have made the amount of service metadata smaller by not including annotations by default. If you need the annotations, you can use parameters to have it included.

Hope it helps.

Source: Microsoft

Issues connecting to CRM 365 from IIS only

Everything working with no issues with the latest Nuget Packages for CRM through a console application when connecting to CRM 365.

Now, we had to promote a service that is currently working and has been working already for long time to consume CRM 365 data to work with the new CRM ddls.

We added the dlls trough nuget packages, everything compiling but when this next line was called:

OrganizationServiceProxy serviceProvider = new OrganizationServiceProxy(config.OrganizationUri,
               config.HomeRealmUri, config.Credentials, config.DeviceCredentials);

We were having this next exception:

Metadata contains a reference that cannot be resolved: 'https://crm.domain/XRMServices/2011/Organization.svc?wsdl&sdkversion=9.0'.

After some research we found that the solution would be before calling this line, call first:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

I will investigate more when i have some spare time.

Hope it helps.

Importing Solution in CRM 2016 Issue

Yesterday i was deploying to another environment a solution and the import was failing.

The error in the log file was while importing a Process:

Failure

0x80040359

The Microsoft Dynamics CRM record could not be created.

So, i went back to the solution from the other environment and i have checked what could be wrong, apparently nothing was wrong, it was a standard CRM Process with a call to a custom workflow activity, an if condition and 2 Stop Workflows statements (Success & Cancelled). I double checked, yes, i was exporting in the same solution file the Workflow dll, and it has been working since ever.

Next, i removed the custom workflow call, tried again and nothing. What usually i do when i need some extra information, Diagtool, so I enabled the logging, checked the log file on the server and .....

and nothing. No more extra information to help me to see what was the problem.

I went back to the Process and instead of having the check box checked for "Record is deleted" on a Sync Process on a custom entity, i unchecked and checked the "Record is created".

Tried to import again, and yes, it worked.

Did it happen to any of you? Comments are welcome.

Hope it helps.

CRM 365 - Business Process Flow - Jump between stages

 The moment when something stops to work without any changes of any type. 

As it happened with me, i know it happened with many developers/customers more.

And as i got really annoyed more people got different feelings as well.

I will explain now what is the issue, in the opportunity entity it seemed the best for the client to use a Business Process Flow to help with a standard process, that is always a good idea, so that was developed against the Opportunity entity.

Everything was going great with the process, and then because of an exception we had to implement something that would allow to Jump from any stage to any stage, so, assuming the Business Process Flow had branches as well, thing could get more complicated. The solution was easy enough, implementing a "On Demand Process" that would change the Stage Id to the one necessary. Until that moment, everything was working with no issues.

However, at some stage, Microsoft just decided that the way the Business Process Flow was implemented was not the best/great, whatever....

What happened, or at least i understood from phone calls with Microsoft was, new tables were created for the purpose and a field "traversedpath" was about to become the Star of that.

Now let me explain what is the traversedpath. Traversedpath is a string field that contains all the guids of every stage until from the first until the current appended with the ',' comma. Trying to be on the brilliant minds that came to that for me it makes sense, so, what's the problem in here? Basically, opportunities created before the Update 9.0 (if i'm not mistaken). 

Everything seemed to work apparently at the beginning, people creating new Opportunity records, and playing around the stages, and jumping sometimes from the first stage to the last one, everything seemed to work more or less.

But, when people started to play around old opportunities, things started to become a nightmare, and now, believe in me, when you have a CRM implementation with many opened opportunities and everybody started to complain about that, then you starting to fill mad(here i don't want to put the name of my actual feeling, could be to strong).

What you do next, with no clue at all, you start to try things, to google and when you find that you not alone, you think, at least i'm not the only one, however, not me, i just thought, not again....

So Microsoft was called for that, and after few hours, something was becoming more clear, but even the code provided was not working 100% for the purpose, then i just tried few more things and i saw some improvements in the tests from the Test team and as well from the Client, scenarios were put on the table and tested in the Sandbox environment.

You can guess what happened, finally production environment was upgraded and then, with a full team of users using the system, more scenarios came into and more problems......

At that moment, and because is a Show Stopper, Microsoft was called again and few more experiences were tried, but at the end nothing conclusive, again i tried few more things and at the moment i believe that i'm really close to the end of the problems, but not 100% sure.

In the Custom Workflow Activity developed first thing done is getting the full path (all stages from the 1st until the current one) and update the opportunity before doing the actual jump of the stage. Below some code that can be seen as example:

Opportunity _opportunityToUpdate = new Opportunity();
_opportunityToUpdate.Id = opportunityId;

//all the guids should be appended in here from the 1st until the current stage
_opportunityToUpdate.TraversedPath = traversedPath;
_opportunityToUpdate.StageId = currentStageId;
_opportunityToUpdate.ProcessId = process.WorkflowId;
service.Update(_opportunityToUpdate);

I have tried to update the opportunity with the newStageId and the traversedpath to the new stage with no luck.

And then my solution, probably not the prettiest one, but it works: 

Going one by one from the current until the one we need.

foreach (Guid stageId in stagesToUpdate)
{
Opportunity opportunityToUpdate = new Opportunity();
opportunityToUpdate.Id = op.Id;
opportunityToUpdate.StageId = stageId;
opportunityToUpdate.ProcessId = process.WorkflowId;

service.Update(opportunityToUpdate);
}

Hope it helps.

CRM 2016 - Migrating Data Performance Issues (Important to read)

 I'm the kind o person that really likes to do Migration and Integration tasks into and from CRM.

Last time this happened to me i forgot to mention in here, and probably most of you already know, if no, just spend few minutes reading this.

I was importing some data into CRM 2016, the same applies to CRM 2015 and only for creating the Account entity records (few data fields filled) it was giving an average of 1.5 sec for each one. That's really an issue when you need to import thousands or millions of records, if you migrate or integrate only few hundred, you wouldn't loose time trying to understand why.

However, i'm really stubborn, so i had to figure out why.

What would you do at first? First you would question yourself, do i have any PrePlugin or Post-Plugin that would affect the performance? If yes, let's try to deactivate those steps and check if the performance is better.

I have to tell that i didn't have any plugin either Pre or Post Sync that would affect the performance, but then i remembered that CRM 2015 and CRM 2016 usually comes with the Activity Feeds Plugins activated. And that was the real problem, after deactivating them, instead of 1.5 sec in average it gave 80 ms (milliseconds).

My question is why Microsoft ??!!?!??!!? Leave those deactivated and let the partners or developers ask to the customer if they really need that active. 

I wouldn't mind at all to leave active if it wouldn't give any kind of performance problems, but it is not the case, it really gives.

Hope it helps

Dynamics 365 - Business Process Flow

This is a nice to have to guide users in terms of filling information based on stages, for instance:

The information needed to a contact to be a prospect, probably we only need First Name, Last Name and mobile phone. On the other hand, to become an opportunity, we will need some more details.

In this situation i would use this kind of approach, Business Process, because it will be really easy to see by opening the record in which phase of the process the contact is only by looking to the process bar. 

However, using this feature of CRM can be sometimes a little bit disappointing because customizing what the fields do (hiding based on rules, setting required based on others is not possible), i mean, configuring only with the CRM default functionality is not possible.

My personal opinion and because i've been working with CRM on my last 13 years already, using Javascript for this purposes it can be a nightmare in terms of Upgrades of the product.

But, because sometimes you get to a point where a system is already implemented and you don't have the time to redesign the feature, one thing i have to say, just customize with an approach of having some constants with the name of the field, and not use hard-coded all-around.

With this, if the DOM changes, you will need to change the new name only in one place, instead of in all the places you have been using this.

For this example i will give few lines of Javascript that can be used to Hide the next stage button on the Process flow, set the required level of a field and set the field readonly.

var probabilityHeaderFieldName = "header_process_new_probability_1";

//to set required

Xrm.Page.getControl(probabilityHeaderFieldName ).getAttribute().setRequiredLevel("required");

//to set readonly

Xrm.Page.getControl(probabilityHeaderFieldName ).setDisabled(true);

//to hide the next stage button

var nextStageButtonName = "stageAdvanceActionContainer";

document.getElementById(nextStageButtonName ).style.visibility = 'hidden';

Microsoft URL for the Business Process Flow, click here.

Hope it helps.