Using the Data Management tools and Flow to Import Data From OneDrive
Using the Data Management tools and Flow to Import Data From OneDrive Murray Fife, September 2017
Page 1
Using the Data Management tools and Flow to Import Data From OneDrive Introduction The Data Management tools within Dynamics 365 for Operations open up a whole slew of different ways that you can get data into the system. And it also has an option to create batch import job that will allow us to watch for data. This opens up a whole slew of options for us because we can use these batch jobs to import data from other systems, because they can just package the data up for us and then send us the data. It creates a very easy way to create integrations because these data packages can also be sent to the Data Management batch job through a web service interface. Now if I were a coder, then I would just call the service from my other system and it would create the package and send it to Dynamics 365 for Operations, but there are also other options available to us. But what if you want to have a system that you don’t control integrate with Dynamics 365 for Operations, or maybe you are getting files from a partner that you want to automatically upload into the system? Then you can go a little old school and have it create a file based integration. But if you do that, how do you get the file to be picked up and processed within the Data Management service? The simplest answer is to use Flow to watch for the files and then grab them and send them to the Data Management service for you. And if you are looking for somewhere to All you need be put, and OneDrive is a great option.
is somewhere for the file to
In this walkthrough we will show you how to tie all of this together to create your very own file based integration,.
Topics Covered Creating a Secret Integration Key ................................................ 3 Create a OneDrive Dropbox ........................................................ 7 Creating an Import Batch Process ................................................. 8 Creating a Monitoring Process within Flow ....................................... 17 Conclusion ...................................................................... 35
Page 2
Using the Data Management tools and Flow to Import Data From OneDrive Creating a Secret Integration Key The first thing that we will need to do is to create an authorization key that we will be able to use to authenticate through into Dynamics 365 for Operations. To do this we will need to create a new Authorization Key for our Azure Active Directory tenant.
How to do it‌ To do this you will need to open up your Azure Portal and then click on the Azure Active Directory link within the services. This will open up all of the Azure Active Directory configuration options. we will want to click on the App registrations link.
Here
This will show us a list of all of the applications connected to our tenant and we will want to click on the app that is linked to our Dynamics 365 for Operations instance.
Page 3
Using the Data Management tools and Flow to Import Data From OneDrive
This will show us our application details. While we are here, note down the Application ID. in the setup.
We will need to use this later on
Now, click on the Keys link in the Settings panel.
This will open up the list of Keys that have been created for the application. of the key values are hidden, so we will want to create a new Application Key.
Page 4
All
Using the Data Management tools and Flow to Import Data From OneDrive
To do this, type in a Description for the key.
Then click on the Expires dropdown list and select the duration that you want the key to remain valid. Here we set it to Never expires.
Page 5
Using the Data Management tools and Flow to Import Data From OneDrive
Now click on the Save button, and you will see that the Value is populated with a value. Copy this down, because this is the Secret ID that we will be using to authenticate our flow integration.
Page 6
Using the Data Management tools and Flow to Import Data From OneDrive Create a OneDrive Dropbox Now that we have created our secret key for the integration we can start moving on. The next thing that we will want to do is to create a location where we can save our integration files to within OneDrive.
How to do it‌ So within OneDrive we created a new folder where we will put all of the import files. Also while we are here we also dropped in a sample file.
If we open up the file, then this is just a simple XML package for a Sales Order.
Page 7
Using the Data Management tools and Flow to Import Data From OneDrive Creating an Import Batch Process Now we can start configuring our data import process within the Data Management tools within Dynamics 365 for Operations.
How to do it‌ To do this, open up the Data management workspace, and then click on the Import tile to start creating our import job.
This will open up a new Import configuration screen.
Page 8
Using the Data Management tools and Flow to Import Data From OneDrive Give your Import a unique Name.
For example, SalesOrderXMLBatch.
Then click on the Source data format dropdown list and select the format of the file that we are using within our file drop folder. In this example we will select the XML-Element option.
Then click on the Entity name dropdown list and find the Entity that we will be importing with this job. For this example we will select the Sales orders composite entity.
Page 9
Using the Data Management tools and Flow to Import Data From OneDrive
Then click on the Upload button and find our sample import file, and then click on the Open button.
This will upload the file and then the Data Management tools will process the file and create a new data mapping for us. Now we will want to set up the batch process. recurring data job menu item.
Page 10
To do this, click on the Create
Using the Data Management tools and Flow to Import Data From OneDrive
This will open up the Create recurring data job pane.
Give the recurring job a unique Name. SalesOrderXMLJob.
In this example we set it to
Page 11
Using the Data Management tools and Flow to Import Data From OneDrive
Now copy the ID field and save it away in a safe place. We will need this later on when we create our integration job, because this is the link within the URL to this batch process where we can enqueue the data.
Now paste in the Application ID that we copied from the Azure Active Directory configuration into the Application ID field within the Setup authorization policy group.
Page 12
Using the Data Management tools and Flow to Import Data From OneDrive
And then check the Enabled flag. This will allow the application to use this job as an end point.
Now click on the Set processing recurrence link to open up the Define recurraence pane.
Page 13
Using the Data Management tools and Flow to Import Data From OneDrive
Here we will want to set the interval count that we want to run this job on. we set the Count to 1 to say that we want to run this process every minute.
Here
After we have done that we can just click on the OK button.
And then when we return to the recurring job pane, click on the OK button to create the job.
Page 14
Using the Data Management tools and Flow to Import Data From OneDrive
After we have done that, we will be asked if we want to enable the job, and we will want to click on the Yes button.
After we have done that we are done within the Data Management Import configuration and we can click on the Save button and then close the form.
Page 15
Using the Data Management tools and Flow to Import Data From OneDrive
Page 16
Using the Data Management tools and Flow to Import Data From OneDrive Creating a Monitoring Process within Flow Now we have an endpoint within Dynamics 365 for Operations that we can use to import in the data, we can tie everything together and create a Flow that will look for new files within the OneDrive drop folder and then send them to the import job.
How to do it‌ To do this, open up Flow and then click on the Create from blank link.
This will create a new blank for us and ask us for our trigger event,
Page 17
Using the Data Management tools and Flow to Import Data From OneDrive Filter down the services to the Schedule service and then click on the Schedule – Recurrence Trigger event.
This will create a Recurrence trigger for us.
For this example, we don’t want to run it every 5 minutes, so click on the Frequency dropdown list and select the Hour option, and then set the Interval to 1 so that this process will run once an hour.
Page 18
Using the Data Management tools and Flow to Import Data From OneDrive
Now click on the + New Step button and select the Add an action option.
Now we will want to search our folder for new files to be processed. So, when the actions are displayed, filter the service down to OneDrive for Business and then select the List files in folder action.
Page 19
Using the Data Management tools and Flow to Import Data From OneDrive
This will allow us to search a folder. icon to the right of the Folder field.
All we need to do is click on the folder
This will allow us to navigate OneDrive for Business. > for the Root folder.
Page 20
Start off by clicking on the
Using the Data Management tools and Flow to Import Data From OneDrive
Then we will be able to see the subfolders and can select the Dropbox that we created and put our integration file into.
This will populate the Folder name for us.
Page 21
Using the Data Management tools and Flow to Import Data From OneDrive
Now we will want to start processing the file. To do this we will want to create loop that will process all of the files that are found. To do this, click on the New Step tile, then click on the ‌ More item and select the Add an apply for each control.
This will create a new looping control for us. to loop through.
Page 22
All we need to do is tell it what
Using the Data Management tools and Flow to Import Data From OneDrive
To do this, click on the Select an output from previous step field and when the field explorer is displayed, select the Body element from the previous step.
After we have done that we will want get the file information for the item that we are processing. To do this, click on the Add an action link within the loop control, filter down to the OneDrive for Business service and select the Get file content using id action.
Page 23
Using the Data Management tools and Flow to Import Data From OneDrive
This will create a new action for us.
Click on the File field and then when the field explorer is displayed, select the Id element from the list, which links to the current file that is being processed.
Page 24
Using the Data Management tools and Flow to Import Data From OneDrive
Now we will want to send the contents of the file to the Data Management endpoint. To do this, click on the Add an action button again, filter down the services to the HTTP service and then select the HTTP action.
This will add a new action for us that allows us to post our file content.
Page 25
Using the Data Management tools and Flow to Import Data From OneDrive
Start off by clicking on the Method dropdown list and select the POST option.
Now we will want to specify our Uri which will point to the Data Management job end point. For this we will want to start pulling in all of the configuration keys that we saved away. The format for the Uri is: https:// [Cloud App URL] /api/connector/enqueue/ [Job ID] ?entity= [Entity Name] &company=USMF
Page 26
Using the Data Management tools and Flow to Import Data From OneDrive
Once we have done that we can click on the Body field and then find the File content element within the field explorer and add it. This will pass the contents of the file that we are working on to the post command.
Now click on the Show advanced options link and we will see that there are some additional options.
Page 27
Using the Data Management tools and Flow to Import Data From OneDrive
Click on the Authentication dropdown list and select the Active Directory Oauth option. This will allow us to use our authentication key we created at the beginning of the process to securely log in.
This will enable a few more fields that we will want to configure.
Page 28
Using the Data Management tools and Flow to Import Data From OneDrive
First we will want to specify our Active Directory Tenant.
Next, within the Audience field we will want to put in the URL for our Dynamics 365 for Operations instance. Note: Do not put a / at the end of the URL.
This will cause authentication errors.
Page 29
Using the Data Management tools and Flow to Import Data From OneDrive
Now click on the Client ID field and paste in the Application ID that we saved away from Active Directory.
Then within the Secret field, paste in the Secret Key that you created for the application in the very first step.
Page 30
Using the Data Management tools and Flow to Import Data From OneDrive
After you have done that, give your Flow a Flow name and then click on the Create flow link.
If everything is configured correctly, then the flow will save without any errors.
Page 31
Using the Data Management tools and Flow to Import Data From OneDrive
And then the flow will automatically kick off.
After the flow runs, return to the Data Management job that you created and then click on the Job history link in the menu bar.
Page 32
Using the Data Management tools and Flow to Import Data From OneDrive
This will open up a log of all of the jobs that have been ran.
And if we go over to the All Sales Orders page then we will see that the order has been created for us.
Page 33
Using the Data Management tools and Flow to Import Data From OneDrive
If you don’t believe it you can drill into the sales order and see that the lines have been created as well.
Page 34
Using the Data Management tools and Flow to Import Data From OneDrive Conclusion We just created a simple file interface that looks for new data packages and then uploads then directly into Dynamics 365 for Operations, without us having to write a single line of code. How cool is that?
Page 35