D365F&O: File-Based Integration Using Azure Blob Storage

 D365F&O: File-Based Integration Using Azure Blob Storage
x

Introduction

This article provides a detailed guide for technical specialists on integrating file-based systems with Dynamics 365 for Finance and Operations (D365F&O) using Azure Blob Storage. This solution leverages the power of Azure Blob Storage to replace traditional FTP-based integrations, offering a modern and cloud-based alternative.

Requirements

To follow along with this guide, you should have:

• Proficiency in X++ language.
• Basic understanding of the .NET Framework.

Problem Overview

D365F&O offers a robust out-of-box integration framework known as the Data Management Framework (DMF). This framework is ideal for handling various types of integrations, including file-based integrations.

When upgrading from AX 2012 to D365F&O, file-based integrations via FTP servers often need to be replaced. Managing FTP servers for cloud-based ERP systems is not ideal due to:

1. The need to maintain an FTP server despite moving to the cloud.
2. The requirement for third-party libraries to connect D365F&O to FTP.
3. The outdated nature of FTP in modern integrations.

Solution: Azure Blob Storage

Azure Blob Storage offers a better alternative for file-based integrations with the following advantages:

1. Lower subscription costs compared to maintaining FTP servers.
2. No need for third-party libraries.
3. Modern, cloud-hosted storage system.
4. Integration with business events for more intelligent operations.
5. Many external systems can work with Azure Blob Storage out-of-the-box.

Terminology

• Connection String: Path to the Azure Blob Storage, including the address and access token.
• Container: A storage unit in Azure Blob Storage, similar to a folder.
• Blob Block: Metadata and reference to the actual file stored in Azure Blob Storage.

Implementing Azure Blob Storage Integration

AzureBlobStorageHelper Class

This class handles all operations with Azure Blob Storage, such as establishing connections, uploading/downloading files, and moving files between containers.

Class Definition

csharp
using Microsoft.WindowsAzure.Storage;

/// <summary>
/// A helper class for Azure BLOB storage actions
/// </summary>
class AzureBLOBStorageHelper implements System.IDisposable
{
    protected Microsoft.WindowsAzure.Storage.CloudStorageAccount storageAccount;
    protected str connectionString;
    protected boolean silentMode;

    // Constructor method
    public static AzureBLOBStorageHelper construct(str _connectionString = '', boolean _silentMode = false)
    {
        AzureBLOBStorageHelper storageHelper = new AzureBLOBStorageHelper();

        if (_connectionString != '')
        {
            storageHelper.connectionString = _connectionString;
        }
        storageHelper.silentMode = _silentMode;
        storageHelper.init();
        return storageHelper;
    }

    // Initialization method
    protected void init()
    {
        if (!connectionString)
        {
            connectionString = AzureBLOBStorageHelper::getDefaultConnectionString();
            if (!connectionString)
            {
                throw error("Azure BLOB storage connection string is not specified");
            }
        }
        storageAccount = CloudStorageAccount::Parse(connectionString);
        if (!storageAccount)
        {
            throw error("Cannot establish connection to Azure BLOB storage");
        }
    }

    // Method to download a file
    public System.IO.Stream downloadFile(Filename _fileName, str _containerName)
    {
        if (!storageAccount)
        {
            throw error("System does not have active Azure BLOB storage connection");
        }
        if (!_containerName)
        {
            throw error(Error::missingMethodParameter(classStr(AzureBLOBStorageHelper), methodStr(AzureBLOBStorageHelper, downloadFile), 'ContainerName'));
        }

        // Create a reference to the file container
        Blob.CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
        Blob.CloudBlobContainer blobContainer = blobClient.GetContainerReference(_containerName);

        if (!blobContainer.Exists(null, null))
        {
            throw error(strFmt("Container %1 is not found.", _containerName));
        }

        // Create a reference to the file client
        Blob.CloudBlockBlob blobBlock = blobContainer.GetBlockBlobReference(_fileName);

        if (!blobBlock || !blobBlock.Exists(null, null))
        {
            throw error(strFmt("File %1 is not found.", _fileName));
        }

        System.IO.Stream fileStream = new System.IO.MemoryStream();
        blobBlock.DownloadToStreamAsync(fileStream).Wait();

        if (!silentMode)
        {
            blobBlock.FetchAttributes(null, null, null);
            Blob.BlobProperties blobBlockProperties = blobBlock.Properties;

            if (blobBlockProperties.Length == fileStream.Length)
            {
                info(strFmt("File %1 is downloaded", _fileName));
            }
        }

        return fileStream;
    }
}

AzureBLOBStorageFileInfo Class

This class acts as a container for file information in Azure Blob Storage.

csharp
/// <summary>
/// A file info class for Azure BLOB storage objects converting to X++
/// </summary>
class AzureBLOBStorageFileInfo
{
    public Filename fileName;
    public System.DateTime dateTime;
    public str fileUri;
}

Method to Retrieve File Information

This method returns a list of AzureBLOBStorageFileInfo objects from a specified container.

/// <summary>
/// Gets list of files (file infos) for specific container
/// </summary>
/// <param name="_container">A container</param>
/// <returns>A List of <c>AzureBLOBStorageFileInfo</c></returns>
public List getFileInfoList(str _container)
{
    List ret = new List(Types::Class);

    if (!storageAccount)
    {
        throw error("System does not have active Azure BLOB storage connection");
    }
    if (!_containerName)
    {
        throw error(Error::missingMethodParameter(classStr(AzureBLOBStorageHelper), methodStr(AzureBLOBStorageHelper, getFileInfoList), 'ContainerName'));
    }

    Blob.CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
    Blob.CloudBlobContainer blobContainer = blobClient.GetContainerReference(_container);

    if (!blobContainer.Exists(null, null))
    {
        throw error(strFmt("Container %1 is not found.", _container));
    }

    System.Collections.IEnumerable list = new System.Collections.Generic.List<str>();
    list = blobContainer.ListBlobs(null, false, 0, null, null);

    System.Collections.IEnumerator listEnumerator = list.getEnumerator();

    while(listEnumerator.moveNext())
    {
        Blob.CloudBlockBlob blobBlock = listEnumerator.get_Current();

        if (blobBlock.Exists(null, null))
        {
            Blob.BlobProperties blobBlockProperty = blobBlock.Properties;
            AzureBLOBStorageFileInfo fileInfo = AzureBLOBStorageFileInfo::construct();

            fileInfo.fileName = blobBlock.Name;
            fileInfo.fileUri = blobBlock.StorageUri.PrimaryUri.AbsoluteUri;
            fileInfo.dateTime = blobBlockProperty.LastModified.HasValue ? blobBlockProperty.LastModified.Value.DateTime : nullValueBaseType(Types::UtcDateTime);

            ret.addEnd(fileInfo);
        }
    }

    return ret;
}

Conclusion

This guide provides a foundational approach to integrating D365F&O with Azure Blob Storage, replacing traditional FTP-based integrations. The use of Azure Blob Storage offers a modern, efficient, and cost-effective solution for managing file-based integrations in cloud environments.