Azure Data Factory is a scalable, trusted, cloud-based solution for building automated data integration solutions with a visual, drag-and-drop UI. Moving on-premises SSIS workloads to Azure can reduce the operational costs of managing infrastructure, increase availability with the ability to specify multiple nodes per cluster and deliver rapid scalability. With the release of Azure Data Factory V2 Integration Runtimes (ADFv2 IR), deployment to the Azure cloud is now possible for SSIS projects that use CData Components.
In this article, we walk through the steps for deploying CData SSIS Components to Azure.
Deployment Prerequisites
The deployment of CData Components requires the following:
- CData SSIS Component License and RTK
To deploy to the Azure cloud, you will need a license (full or trial) and a Runtime Key (RTK) for each of the CData components being deployed. For more information on obtaining this license (or a trial), contact our sales team. - Azure Subscription
This can be managed in the Azure Portal. More information on Azure Subscriptions is available from the Microsoft website. - AzureRM PowerShell 6.2.0 or higher
With PowerShellGet you may install using Install-Module -Name AzureRM -AllowClobber, or by using a .msi installer from Microsoft Releases. You can find other detailed instructions on obtaining the supported Azure PowerShell from Microsoft's Web Documentation. - SSMS 2012 or higher
We recommend using the latest version available from Microsoft.
Deployment Guide
Overview
- Create an Azure Resource Group
- Create an Azure SQL Server (logical server)
- Run CData PowerShell Script AzureDeploy.ps1 to set up and create an Integration Runtime
- Deploy SSIS Project From Visual Studio to Azure
- Use SSMS to manage and execute a deployed project
Step-by-Step Guide
- Create an Azure SQL Server (logical server) and Resource Group or reuse an existing one.
If the existing SQL Server already has an Integration Runtime (and SSISDB), then you may overwrite it by specifying its name with the -InterationRuntimeName parameter. Each SQL Server may only have one Integration Runtime. - Allow IP addresses on the SQL Server Firewall as necessary.
- Run the AzureDeploy.ps1 PowerShell script, and provide the required parameters.
The default location (example Salesforce 2018) is C:\Program Files\CData\CData SSIS Components for Salesforce 2018\lib
You may use powershell –ExecutionPolicy Bypass to bypass execution policy restrictions for the current session.
(Optional) Deploying Multiple CData Components
To deploy multiple CData components, copy all additional 2017 CData .dll and .Design.dll files to the current directory. - Log into Azure in the generated pop-up.
- After logging into Azure, the script creates the resources necessary for deployment and starts the Integration Runtime.
- You are now ready to deploy your SSIS Package.
In Visual Studio, right-click the project -> Deploy.
Use the Deployment Wizard to select your project for deployment and authenticate to the Azure Server as your destination.
(Optional) After this step, your package is deployed and accessible in the Azure Data Factory web UI at: https://adf.azure.com
Managing the Project in SSMS
By default, connection settings with sensitive information (passwords, security tokens, etc.) are redacted when deploying to Azure. You may provide this information again through SSMS.
- Connect to the Azure SQL Server
In Options -> Connection Properties, set the 'Connect to Database' field to SSISDB. - Configure the SSIS project in the Integration Services Catalog (right-click the project -> Configure).
(Note: If you do not see Integration Services Catalogs, you may need to upgrade your SSMS version or set 'SSISDB' in step 1.)- Add connection information as necessary in the Connection Manager tab. This is where you can add your RTK.
- Execute the project (right-click the project -> Execute).
From here, the project is executed in the configured Azure Data Factory, and the results of executions can be viewed by right-clicking the project -> Reports -> All Executions.