How to Extract Dynamics 365 Data & Save On Cloud Storage Costs
Large volumes of emails and attachments are the most frequent reasons why Dynamics 365 customers need to increase cloud storage capacity, but these can be stored in a more cost-efficient way.
By default, the data displayed in Dynamics 365 Online is held in Dataverse (formerly known as the Common Data Service) which securely stores and shares this across Microsoft Business Applications. Based on the type and volume of Dynamics 365 licenses purchased, each organisation has tenant-wide cloud storage entitlements which include defined capacities for database and files. If an organisation’s file data exceeds these accrued entitlements, add-on cloud storage will need to be purchased to ensure sufficient capacity.
Depending on your organisation’s data retention policy, it may not be possible to bulk delete some email messages and other rows/records. But, it may be more cost-effective to store this data in an Azure SQL database which can still be displayed on Dynamics 365 forms.
If your organisation has already exceeded its cloud storage entitlements, or is close to doing so, Preact can help by extracting emails, attachments and other data from Dynamics 365. In this post, we’ve outlined a solution which moves data from Microsoft Dynamics 365 into an Azure SQL database and presents this back using Virtual Tables.
For example, emails which are tracked on Contacts, Accounts, Leads and Opportunities may be considered ‘archive’ after two years, but must be retained for a longer time period. This creates an opportunity to place data into a cheaper storage model which is still accessible in Dynamics 365 via Dataverse. Depending on the data volumes and attachment file sizes, this can quickly free up a significant chunk of gigabytes of Dataverse storage capacity.
Extracted data can be displayed on an Account, Contact or another parent record. In the example shown below, archived emails are listed on a separate tab.
Out-of-the-box, Virtual Tables don't support search capabilities, but this can be configured. As shown above, users can search email terms on a sub-grid across all the data stored in a Virtual Table where the source data is stored in an Azure SQL database.
Clicking an individual email will open the message, including any associated file attachments.
In addition to an existing Dynamics 365 / Dataverse environment, this solution would require:
- An Azure subscription to host the Azure SQL database
- App Service Plan to host the App Service (OData) that will read data from the Azure SQL DB table and push this into Dataverse using a Virtual Table
- Data Export Service configuration for the tables / entities that will be extracted
To avoid unnecessary calls and cost, the OData service will only retrieve and display data when a Dynamics 365 user clicks on an ‘archive’ tab, rather than when the main form is opened. As the OData service is hosted in Azure, this can be scaled up and down based on specific usage needs.
Performance may be affected if large volumes of records need to be displayed. For example, if a request is made to retrieve several thousand emails, the service may take anywhere between 20-40 seconds to display this data.
Once the data has been extracted from Dataverse, a separate bulk deletion job would need to run by adding specific filters and criteria.
How Much Cost Can Be Saved?
This extraction process is designed to help Dynamics 365 customers reduce costs by benefiting from the lower-priced Azure SQL storage model when archived data is moved from Dataverse storage. The storage costs and potential savings will depend on an organisation's data volumes, and which Azure SQL database and OData App Service hosting plans are selected.
As an indicative guide, based on an example that extracts email and file attachment data totalling 50GB from Dataverse into an Azure SQL database, this could potentially reduce monthly storage costs from £1500 to approximately £250.
Contact us today, to understand if a bulk data extraction to an Azure SQL database would be feasible solution that enables your organisation to reduce storage costs and ensure compliance with your data retention policies.
Never miss a post! Subscribe to our blog
Receive our emails about the latest Microsoft Dynamics 365 updates and events