Dataverse REST Builder and XrmToolBox

The XrmToolBox version of Dataverse REST Builder is out, you can download it from the Tool Library inside XrmToolBox.

When creating a new tool for Dynamics 365/Dataverse, XrmToolBox is usually the first choice. It's a well known application, there are already several essential tools for it, you connect to your instance and you are ready.

When I created Dataverse REST Builder I went for a different path, a classic Managed Solution, but why?

Main reason is Xrm.WebApi, the client api created by Microsoft to deal with Web API is available only inside a Dynamics 365/Power Apps instance, it makes sense (at least for me) to create a tool running inside the instance in order to test this correctly.

Second reason is Windows Forms, although it's possible to recreate the same functionalities using the Windows Forms controls they would not have the same look and feel, (and I know I stressed others about this in the past, sorry) the dropdown library used inside DRB it's very convenient for me, I like to use it.

There are other reasons why you may prefer a Managed Solution (you use a Mac or you can't use XrmToolBox for a specific customer) and if you need the functionality you can't be too much picky on how you can get it.

This doesn't mean I don't care about people demanding an XrmToolBox version, I also prefer to have it if possible, however of one thing I was sure: I would not create a different codebase just for XrmToolBox.

There is another tool released as Managed Solution with an XrmToolBox version, Ribbon Workbench by Scott Durow. He may not recall this, but when we first met in London (June 2016) he anticipated me that he was working on the XrmToolBox version, and quickly described me the "bridge" required to make it works. I have fond memories of that trip, can't believe already 5 years passed.

Back to DRB, I knew it is technically possible to load a webpage inside XrmToolBox but it isn't the main problem. DRB is written using JavaScript + jQuery libraries, when I write some code I try to find a syntax available also for IE 11 but sometimes is not possible for everything (like some external libraries I use) and this is the case for DRB, it doesn't work inside IE 11.

Not big deal if you use the Managed Solution (just use a supported browser) but it's a big issue inside a Windows Forms application. Windows Forms has a control to display a webpage but uses the Internet Explorer engine, not the new Edge/Chromium one.

After doing some testing with WebView2 control and able to run DRB inside XrmToolBox, I shared my findings with Tanguy Touzard, he also needed this control for other functionalities and included it inside the release 1.2021.12.53.
If you are curious on how it works, the code of the tool is available inside this repository, inside DRB the code checks if the object coming from XrmToolBox is present.

I suggest this approach if you are planning to create a new tool? Absolutely not:

• You are limited only to connections using a token based authentication, nowadays it should be the most common way to connect but isn't always the case.
• You are forced to use the REST endpoint and not the official SDK, if you have the possibility to use the official SDK you should always prefer it, the new version of the SDK is still on public preview but it's mature enough in my opinion.

The XrmToolBox version of DRB comes with limitations, main one is of course the inability to execute Xrm.WebApi requests (but the code is generated), jQuery/XHR requests can be executed because behind the scenes I inject the authentication token.

One thing I expect is that probably more people will use DRB because is available inside XrmToolBox, this means more bugs discovered and more requests asked. If this is the case, please open a GitHub issue, comments in my blog or direct contact requests may be lost and if I receive them I will still ask you to open a GitHub issue.

I hope you find this tool useful.

File Columns and Web API

One of the recent and useful additions inside Dataverse is the File column. Prior to this type, files inside Dataverse were usually stored as a Note (annotation table) and having Notes is an option that customizers may prefer to don't enable (or you are dealing with a table like systemuser where you can't enable Notes).

You can find the official documentation regarding File columns here: link.

The page contains also some examples on how to deal with this type, both with REST requests and C# examples.

The latest version of Dataverse REST Builder (1.0.0.13) now includes a new request type called "Manage File Data" to deal with File Columns.

Note: This request is only available with jQuery and XMLHttpRequest, it should work also with Portals (and the safeAjax method) but I didn't test this scenario.

There are three operations available: Retrieve, Upload and Delete.

Retrieve and Delete are quite simple, they are just a GET and a DELETE request to the specific endpoints, the Upload one (it uses a PATCH) is quite tricky.

When Upload is selected a new textbox to specify the file name appears, there are two ways to pass the name using this operation, by url parameter or by header. I decided to use the url parameter, I encode the string before adding it to the main url and I implemented some checks to avoid special characters.

The content is another story, there are no REST examples inside the documentation so I did some tests.

First of all the expected format is binary, other types inside Dataverse, like the annotation table I mentioned above with the column documentbody or an Image column, require a Base 64 string.

Note: DRB allows the upload of images when dealing with Image columns, you can see this behavior also inside the demo with a Create/Update request to Custom Table and Image column.

I did some tests with a Base 64 string as source and converting the content to binary (with atob and other approaches) but the binary was always different from the original one (the Base 64 string was obtained with the FileReader object) probably due to encoding, so now DRB just shows an empty string, if in the future I can find a solution to this conversion I can add a Load button inside this request.

Note: With Postman you can select a file when the body is set to "Binary", the export from DRB handles this setting.

One of the reasons I had when I created DRB was to handle the new column types (choices, image, file), this version is another step forward to deal with them.

Web API Functions and Enum Types

Today my friend Natraj Yegnaraman tweeted a trick to get some organization details inside Power Automate, to be exact this one.

What caught my eyes was the syntax, I recognized it was a Web API Function (the parameter is listed inside the URL) but I didn't recognize the value format:
Microsoft.Dynamics.CRM.EndpointAccessType'Default'.

I checked the Web API Function reference and after the metadata, it was the infamous Enum Type.

Enum Types are special, they are internal (meaning you can't create them as input or output of a Custom API or a Custom Action) and the syntax on how to use them is not so precise, the only reference I found when I was developing Dataverse REST Builder is inside this page regarding structuralProperty.

There is also an example on how to use it but only for Xrm.WebApi, also they are quite rare (the metadata of a trial instance has only 55 Enum Types and many of them belong to Complex Types, another internal structure).

Inside previous versions of DRB you would see the following message, as it was an unknown type:

Natraj's tweet showed me the URL syntax to use with jQuery/XHR, in addition to the Xrm.WebApi syntax (the Microsoft documentation list all the values inside the enumProperties, but I found it just need the selected value and not the whole array).

After some changes I released a new version of Dataverse REST Builder (1.0.0.5) and now Enum Types are rendered correctly:

As I wrote before they are rare and probably you will not need to deal with, but it's nice to have a reference on how they can be used (the function RetrieveCurrentOrganization is now part of the DRB demo data).

Portals and Web API, why not?

After Portals Superhero and MVP Victor Dantas shared the news regarding Web API and Portals (link) I started to make the required changes to Dataverse REST Builder in order to support the new syntax, and a new version (1.0.0.2) is out:

This new functionality is in Preview, so if you are planning to use it make sure to read carefully the documentation, as syntax and functionalities may change. Inside the new tab "Portals" I added some comments with the current links.

In order to execute this new syntax DRB replaces the Portals endpoint ("/_api") with the standard Web API endpoint, this means you can generate the code also inside an instance where a portal is not installed, but this also means that is not 100% accurate, it's a trade-off.

You will find the new "Portals" tab for the following requests: Retrieve Single, Retrieve Multiple, Create, Update, Delete, Associate and Disassociate.

You can download the new version from GitHub: https://github.com/GuidoPreite/DRB

New DRB version, Web API and Postman

Two days ago I released Dataverse REST Builder (link) and I received many feedbacks about it, developers inside the Power Platform community found this new tool useful (and as I wrote before it would not exist without CRM REST Builder).

DRB generates Xrm.WebApi and jQuery/XHR code, the goal in my mind is to assist when writing JavaScript code, probably targeting a Model-Driven app. However I am well aware that not everybody works as I do and maybe they will use the tool in a different way.

When I was developing the tool I often checked the official Microsoft documentation regarding Web API (link) to understand some specific options and how the endpoint works but one section I noticed is Use Postman with the Web API. Postman is a tool I personally use but not too much for Dataverse, if I can use C# I prefer the official SDK and for JavaScript Xrm.WebApi is usually enough.

However Postman was a big inspiration, the idea to load/save a collection (and the possibility to create folders and requests at any level) is from there, I can't deny it.

After the first release I checked if it was possible to export a collection created in DRB to Postman, my json data structure holds more data but in a different format, however the generation code for jQuery/XHR is very similar to a Web API call: the url is exactly the same, differences are on how the headers are stored and how the body (for POST and PATCH requests) is written.

After some changes in my code I released the 1.0.0.1 version:

Beside the new "Export as Postman Collection (2.1)" functionality I also fixed a bug inside the "Create" and "Update" requestes for jQuery/XHR code generation.

The Postman collection is structured in a different way from the Microsoft tutorial, there are 4 variables and they are connected to the collection and not to an environment, the Authorization type is OAuth 2.0 and you need to generate a valid token after you imported the collection.

If you use a different authentication method and you think DRB can be expanded to include some settings during the export, open an issue and I will be happy to discuss there, I can't make promises (as this "Export as Postman Collection" is a totally separate functionality) but I am always interested on how people use the application.

The code is separated but some existing functions are still called during this export (for example the $filter clause for Retrieve Multiple is the same) so if something is not working inside Postman will be useful to know if the same problem happens inside Xrm.WebApi/jQuery/XHR or not.

New tool: Dataverse REST Builder

Note: new version has been released, read more here or download from GitHub

If you created Web API queries against CRM/Dynamics 365/Dataverse probably you are familiar with CRM REST Builder. The amazing tool created by Jason Lattimer helped me countless times in the past years and is a tool I still use today.

The ability to generate queries (especially with Xrm.WebApi when it was introduced) comes very handy, and CRM REST Builder has several functionalities but the one most used is probably the Retrieve Multiple one.

In the past years the development of CRM REST Builder has stopped and beside some bugs the other important missing thing is the compatibility with Unified Interface.

I decided to create a new tool to replace CRM REST Builder, and Dataverse REST Builder (DRB) is finally public at this repository:

https://github.com/GuidoPreite/DRB

 It comes with a Managed Solution to install inside your instance and it will popup inside the Apps list:

As I like to test my code offline, there is also a demo mode, so you can test the application before installing the solution: https://guidopreite.github.io/DRB/

The demo data is not complete but enough to understand how the application works and the demo table  called "Custom Table" contains the majority of columns available inside a Dataverse instance.

Main differences between CRM REST Builder and Dataverse REST Builder are:

• DRB does not support the 2011 Endpoint (so if you still need it you can continue to use CRM REST Builder)
• DRB allows to create, save and load a collection of requests, useful if you need to store somewhere the Web API queries you created
• Support for new columns like "Choices" (also known as Multi Select Option Sets) and better support for "Image" and "File" columns
• Ability to create requests to execute Custom API
• A specific request to execute a classic Workflow
• For Retrieve Single, Create, Update, Delete, it generates also the code for Xrm.WebApi.online.execute

DRB is not perfect and probably there are some bugs as well, but the main functionalities are available and I prefer to release it and gather some feedback.

Some may ask why I decided to create a new tool and not update the existing one. Is a valid concern and I thought about it too, in the end I decided to start from scratch. Main reason is that I had already a solid base to deal with dropdowns (as in my previous tool DOSM) and I designed the application with a different approach, the fact that you can save a collection of requests divides the tool in 3 main parts: a json data structure to hold the request, a UI to manipulate the json data structure and a code generation part accepting the json data structure to create the code.

As developers we need this tool? Honestly not too much, CRM REST Builder is still a great and working tool, and Dataverse REST Builder would not exist without it, I just felt that sometimes we need to deal with new column types or new requests and it's nice to have now a tool.

Hope you find Dataverse REST Builder useful and if something is not working you can open an issue inside the GitHub repo!

Power Automate Desktop and PDF Actions

Every month I receive my utility bill and because my supplier doesn't provide a way to pay it online, they send me a PDF with all the usage details including a page to pay the bill. It uses a system in Italy known as "Bollettino Postale", it can be paid in person in post offices or online (Poste Italiane website).
Example:

Because I pay it online I need to manually copy some information from it (amount, payment reference, ...) and fill a form. It takes me few minutes to do but inside the PDF I receive this "bollettino postale" is not horizonal but vertical, when you need to be sure that you copy some numbers rotating the head is not the best approach :)

So I asked around and my friend Joel Lindstrom suggested to check Power Automate Desktop because it has some PDF actions, like extracting a page from a PDF. The page that I need to extract and rotate is usually the last one.

Sadly Power Automate Desktop doesn't have the actions I need, so I searched and I found this idea regarding the page count (PLEASE UPVOTE IT !!!)

Count number of pages in attached PDF files

It should not be a difficult action, you provide as input the PDF file and it should return a variable containing the number of pages, in this way I can extract the last page or the previous one. There are some workarounds (like this one) but an action would be simpler.

My second requirement is to rotate a PDF (or a single page of it), I didn't find an idea so I created one (PLEASE UPVOTE THIS AS WELL !!!)

Rotate PDF Clockwise or Counterclockwise

I found that I am not the only one with a similar need (like this post), of course this is more complicated than just counting the pages but if they provide a way to rotate all the pages (instead of just a specific one) it could be beneficial.

So if you like these ideas please upvote them and if you have an idea check if someone else suggested it first and in case create it.

Personal note: there is a way to automatically pay utility bills with a bank account (like I do with my internet provider) but for this supplier I prefer to pay manually.

Writing plugins and If conditions

This may be a silly post but it's something bothered me enough to write it down, so I just start with a disclaimer:

if you are using a tool or some code to assist you writing a plugin go for it, this post is not about this.

Few weeks ago I watched a YouTube video about programming, it described the advantages to don't write else conditions, I don't recall the exact remarks but was something like this, so instead of writing:

if (condition == true) {
} else {
}

the author suggested to write something like

if (condition == true) {
}
if (condition == false) {
}

I don't agree by principle, some comments suggested that a switch can also be used, in the example the author probably put some else if statements and one thing I don't like is definitely else if.

Today I wrote a plugin, as usual I copied the wireframe from one that I previously created and I implemented the logic. It was working but the final part of my code was just a long list of closing brackets of several if conditions I had in the code. So I remember the YouTube video I watched and I told myself: Can I rewrite this plugin to avoid a concatenation of if conditions?

Usually my wireframe looks like:

if (context.InputParameters.Contains("Target") && context.InputParameters["Target"] is Entity) {
   Entity entity = context.InputParameters["Target"];
   if (entity.LogicalName == "account) {
      if (context.PostEntityImages.Contains("PostImage") && context.PostEntityImages["PostImage"] is Entity) {
          Entity postImage = (Entity)context.PostEntityImages["PostImage"];
          // rest of the code
      }
   }   
}          

When you start with this kind of style (copied from official documentation or tutorials) it's easy to follow the same structure, you go for the true path, and in the last if condition the main code is executed or else it fails, the majority of the previous if we don't care, because the plugin logic should probably be skipped.

Inside a plugin I usually forgot that I can use return if I want to close the plugin, it doesn't go to an error, just the main scenario should not be executed.

So I rewrote the plugin reversing the conditions, so my code looked like:

if (!context.InputParameters.Contains("Target") || !(context.InputParameters["Target"] is Entity)) { return; }
Entity entity = context.InputParameters["Target"];
if (entity.LogicalName != "account) { return; }
if (!context.PostEntityImages.Contains("PostImage") || !(context.PostEntityImages["PostImage"] is Entity)) { return; }
Entity postImage = (Entity)context.PostEntityImages["PostImage"];
// rest of the code     

One thing I noticed reversing the conditions is that I use often !string.IsNullOrWhiteSpace to check if a value is not empty and continue, inside this plugin I used string.IsNullOrWhiteSpace with a return (meaning I exit if the value is empty), made me curious to know which syntax the author(s) of this method may like.

Just a note, in the code I wrote above I retrieve the Image by name, my friend Daryl LaBar taught me that you can just check the existence and access the first item of the collection, because normally you register a single Image against your step.

This post is not a best practice one or similar, just writing down the thoughts I had in mind today.

Microsoft.PowerPlatform.Dataverse.Client and ClientSecret AuthType

.NET Core 3.1 is what I consider a good starting point if you plan to create a new project based on .NET Core, it's LTS (until end of 2022) and if you developed only with .NET Framework before, I usually suggest this version.

I know .NET 5.0 has been released for some months now but for what I am describing here it's not important, if you wish you can use this version.

If you are a developer and in the last years you worked with Dynamics CRM/CDS/Dataverse you probably know this NuGet package:
Microsoft.CrmSdk.XrmTooling.CoreAssembly

It's the package we usually install when we need to connect to Dataverse, it works but requires .NET Framework 4.6.2 or onward. It works also with ClientSecret AuthType, if you have this kind of connection string

AuthType='ClientSecret'; ServiceUri='https://mywonderfulorg.crm.dynamics.com';
ClientId='00000000-0000-0000-0000-000000000000'; ClientSecret='MyWonderfulClientSecret';

or if you are using Username & Password you may have something like this

AuthType='Office365'; Url='https://mywonderfulorg.crm.dynamics.com';
Username='username@mywonderfulorg.onmicrosoft.com'; Password='MyWonderfulPassword';

In your code you can connect with few lines:

CrmServiceClient service = new CrmServiceClient(connectionString);
WhoAmIResponse whoAmIResponse = (WhoAmIResponse)service.Execute(new WhoAmIRequest());
Console.WriteLine($"Connected with UserId: {whoAmIResponse.UserId}");

CrmServiceClient comes from the namespace Microsoft.Xrm.Tooling.Connector.

What if we want to use .NET Core? The NuGet package to use is currently in public preview:
Microsoft.PowerPlatform.Dataverse.Client

You can use this package also with .NET Framework and .NET 5.0 but the main reason is the compatibility with what we call .NET Core (and if you still want to say something like "but with .NET 5.0 we can bla bla bla" fine for me, also with .NET Core is possible to use .NET Framework packages but we would make the executable only compatible with Windows, I never suggested that kind of path but it's a possibility).

This package is not compatible with the Username & Password authentication, as stated inside the description: Note: that only OAuth, Certificate, ClientSecret Authentication types are supported at this time.

Meaning it's also a good time to move away from old forms of authentication.

Assuming you have a connection string with the ClientSecret AuthType how would our code changes? Just a type:

ServiceClient service = new ServiceClient(connectionString);
WhoAmIResponse whoAmIResponse = (WhoAmIResponse)service.Execute(new WhoAmIRequest());
Console.WriteLine($"Connected with UserId: {whoAmIResponse.UserId}");

ServiceClient comes from the namespace Microsoft.PowerPlatform.Dataverse.Client, that is also the name of the package, so make sure you are using it.

Hopefully next time I will remember that is called ServiceClient and not CrmServiceClient, DataverseClient or PowerPlatformClient, types I tried before checking what was inside the package.

Power Automate Desktop, opinions of others

Few days after the announcement of Power Automate Desktop for Windows 10 users (official link here) a famous Italian website focused on IT released a brief news about it (link here).

Microsoft is a big name so it is normal that something regarding Windows 10 made the news also inside this site (for example they published also the Microsoft-Bethesda agreement) but I was more interested in the comments.

First some numbers, the article contains 17 comments, I checked the recent news and except an article about Tesla Cybertruck (with 35 comments), the Power Automate Desktop one was the most commented (usually there are between 6 and 10 comments in other news).

This is very good in my opinion, people read the news and after wanted to discuss it, but what they are talking about?

A couple of comments are about the comparison to Mac (Apple Script is mentioned and the fact this is is coming to Windows years later), no surprises here, I also mentioned this in my previous post.

A user asked if can save something to PDF, rename it and after move it to a specific folder, another user promptly replied that is possible, that he is also checking the software and it looks useful.

Another user wrote that he downloaded the software, that is complex (meaning that you can do many things) and reported that maybe for his daily activities is not useful but he would like to check some demos.

Another comment was that in the download page there is the "Download now" option and "Buy" option, this made him confused.

A couple of users (in order to explain to others) compared Power Automate Desktop to IFTTT or to a Macro system.

Last comment was a bit more technical, the software was always intended as Business oriented, Powershell is also available for some automations under Windows.

In the end was good to see how this kind of product is received to a bigger audience, most of the comments were positive and they confirm my initial thought, having Power Automate Desktop included inside Windows 10 can be very beneficial.

Power Fx and Power Automate Desktop, my opinion

Microsoft announced a couple of things regarding Power Platform, Power Fx and Power Automate Desktop for Windows 10 users.

I start with Power Fx, you can find a couple of posts here:
Introducing Microsoft Power Fx: the low-code programming language for everyone
What is Microsoft Power Fx?

I am happy the language used in Canvas Apps got an official name (and it will be Open Source too) but I am not surprised, sooner or later (just for Documentation purposes) you need to call this agglomerate of functions in some way, so people can easily check or ask questions about it.

I am also happy that the code behind a Canvas App can be easily stored inside a source control system but let's be honest, nobody in a clear state of mind would be happy to edit that kind of YAML files directly :)

For how long we will continue to talk about low-code, citizen developers and stuff like that? When I was younger I developed applications with Visual Basic 6, I am sure that people developing in C++ thought about Visual Basic like it's a toy or something similar, the reality is that who used Visual Basic was probably a developer with less knowledge than the average C++ developer, nothing to be ashamed for, because there is a difference if I need to create a Windows driver for a PC component or a Windows Forms application to manage my customers.

As much as I appreciate the games developed by Scott Durow for Canvas Apps, using just a specific option (like a Canvas App or a writing a plugin) because it's technically possible should be (in my opinion) avoided. Like writing a 1000 lines of code plugin instead of 3 blocks inside a flow, or creating the biggest flow on earth just to avoid a 10 lines of code plugin. I think less of who write code in Canvas Apps? Definitely not, I call them "citizen developers"? Definitely not, if you know how to develop good for you, if you found a tool that encourage you to write better and lesser code good for you. For me Power Fx is not a low-code language, it's just a code language, like C#, JavaScript, VBA Macros, etc etc. Why I say this? because I can be proficient in one of the most common code languages, but I am sure that there are plenty of developers (maybe proficient in Assembler) developing more complicated stuff than I do. But the moment we write some code we are all developers, that's matter.

Moving to a more interesting topic, the new release of Power Automate Desktop for Windows 10 users is an AMAZING news. Announcement post:
Automate tasks with Power Automate Desktop for Windows 10—no additional cost

This is a game changer for Windows, if you are familiar with Mac, it is comparable to what Automator does. I quickly downloaded the application (link is https://aka.ms/GetStarted-PAD) and it's really something users should try.

Check this screenshot:

These are the actions related to FTP, imagine that your security camera saves the files in a specific FTP site and you want to copy automatically that files to a local folder (maybe synced with OneDrive), no manual steps, you can just create the automation and launch it.
Having a tool like this included in Windows will make users more familiar with the technology itself and the possibilities it offers, this means that when a user will need to automate some tasks in a work environment, already has the experience to do this kind of tasks.

Next time I need to create a repetitive task, I will definitely check if I can do it using Power Automate Desktop first.