Skip to main content

CData Sync

Share product questions and get support for CData Sync

  • 97 Topics
  • 177 Replies
97 Topics
S
Stanley L.Employee
posted in CData Sync
Replicate Salesforce Data to SAP Datasphere with CData Sync

Hey CData Community!Looking to bring your Salesforce data into SAP Datasphere without the headaches? Our latest post, Replicate Salesforce Data to SAP Datasphere with CData Sync, walks you through the full setup — from database user creation to job execution — so you can get your external data flowing into Spaces fast.No native connector? No problem. With CData Sync, you can plug in Salesforce (or any of 270+ sources) and start building with real-time, reliable data. Check out the full guide here: https://www.cdata.com/kb/articles/sync-sap-datasphere.rst

40
S
D
DibyenduEmployee
posted in CData Sync
CData Sync - Change Tracking in Dynamics 365 Entities for CDC Replication

Hey CData Community!Manually enabling Change Tracking for every Dynamics 365 entity? That is a lot of effort.Good news. CData Sync can do it for you using a stored procedure.It automatically enables Change Tracking for the right entities so you can set up incremental replication without the manual work.Want to see how it’s done? Check it out here: https://www.cdata.com/kb/articles/dynamics365-change-tracking.rst

60
D
S
Stanley L.Employee
posted in CData Sync
Enhanced CDC in CData Sync: How much faster is it?

Hey CData Community!Just dropped: a quick deep-dive on the new Enhanced CDC feature in CData Sync• 9× faster replication (10 k changes: 6 s vs 54 s)• Always-on log reading + staging area = reduced wait at run time• Step-by-step setup, benchmarks, and diagrams inside Refer to the following link to read the post: https://www.cdata.com/kb/articles/sync-enhanced-cdc-performance.rst

80
S
B
BI-ModApprentice
asked in CData Sync
How to setup Incremental Replication

Hi,I have 3 years worth of data. All the data stay the same but the latest 3 months are dynamic. I need to replace the old data with the latest data. But if I do “Drop Table”, it drops all the data and bring all 3 years worth of data and sometimes the system dont seem to be able to handle it.  I would like to keep everything and just have the last 3 months tables dropped and replaced. I dont want the system to have to bring ALL data from the past 3 years. I just need the last 3 months dropped and replaced.  What setup should I do?

231
P
D
DibyenduEmployee
posted in CData Sync
CData Sync Enhances Snowflake Integration with Optimized High Volume Data Handling

Hey CData Community!Moving large volumes of data into Snowflake just got a major performance upgrade. With the release of CData Sync version 24.3, high-volume data replication is now significantly faster and more efficient, cutting job completion times nearly in half for optimized processes.In our recent article, we looked at the enhancements made to CData Sync’s processing engine and explored how these improvements play out across datasets ranging from 1 million to 10 million records.Learn More: https://www.cdata.com/kb/articles/sync-snowflake-performance.rst

90
D
J
JTapioCollaborator
asked in CData Sync
Sync Kafka source and max rows

Hi,I have Sync version 24.3.9120.0 and I’m using Apache Kafka as source. It’s working fine, except little irritation. When I run task it will replicate only first 150 000 rows and stops after that. Next time I run the task, it replicates 179 000 rows and so on. Row count will increase. Seem to be randomI can run job multiple times and it will continue where it was stopped earlier. Is there way to change this behavior, so that all rows will be replicated at once?

1127
J
J
JackolasApprentice
asked in CData Sync
Sync – “[0] Failed to cache row [XXXX] null"

Since the update to Sync version 25.1.9207 we are getting some strange errors from different tasks.All these tasks worked without problems before the update.So far i have not found any difference (except for the tables) between tasks that work and that won’t work. It even fails on small datasets (19 in this example, a 16.000.000+ tasks works without problem)And the log also is not showing a lot of info:2025-03-25T12:34:16.494+0100    2    [ |Q-Id| T-Id]    [ |Q-Id]    [SQL |Bulk  ] 15 rows are being committed.2025-03-25T12:34:16.494+0100    5    [    2|    0| 1164]    [CORE] [RealDB:SQLServer]Starting WriteToServer2025-03-25T12:34:16.494+0100    5    [    2|    0| 1164]    [CORE] [RealDB:SQLServer]Invoke WriteToServer2025-03-25T12:34:16.508+0100    1    [    2|    0| 1164]    [CORE] [RealDB:SQLServer]Invoke WriteToServer is failed. null2025-03-25T12:34:16.508+0100    5    [    2|    0| 1164]    [CORE] [RealDB:SQLServer]Invoking rollbackTransaction. 2025-03-25T12:34:16.509+0100    0    

701
J
Jaclyn
JaclynEmployee
posted in CData Sync
🌼 Greetings from CData! 🌼 CData Sync Q1 2025 Release Highlights

Spring is here, bringing fresh updates to CData Sync! Our latest enhancements are designed to streamline your data integration and management, helping you bloom with efficiency this season: 📂 Workspaces for Better Organization: Easily group Jobs, Connections, and Transformations under Workspaces for improved management and streamlined workflows. Users now have greater control over the processes and data, ensuring a more structured and efficient Sync experience. ⚡ Enhanced CDC Catalog Expansion: You asked, we delivered—now supporting DB2 LUW and Informix for even broader data replication capabilities! 🔄 Reverse ETL Catalog Expansion: We are adding Oracle as source and destinations such as Kintone, HubSpot and Zoho CRM to enhance your reverse ETL capabilities! 🛠️ Destination Performance Improvements: Experience faster data processing with increased performance for our Databricks & Redshift destinations! 📚 Full Feature Documentation: For a comprehensive understanding of all new fe

770
Jaclyn
C
Cameron LEmployee
posted in CData Sync
dbt Transformations with CData Sync

CData Sync supports the execution of ELT processes flexibly using dbt Core or dbt Cloud. This integration gives users complete control over defining and executing complex data transformation workflows in dbt, while CData Sync provides a pipeline for users to run those transformations in your chosen destination. Read more about using dbt Core or dbt Cloud with CData Sync here:dbt Core: https://www.cdata.com/kb/articles/sync-dbt-core.rstdbt Cloud: https://www.cdata.com/kb/articles/sync-dbt-cloud.rst

90
C
D
David FosterApprentice
asked in CData Sync
CData Sync - Loading/Replicating data in a looping batch

Hi there.Is there a way of replicating data in batches.We are looking at a lambda approach with small incremental replication, followed up with a weekly full reload to catch late-arriving data and physical/hard deletes.The challenge is that the full reload data volumes are too large for the server hosting our CData Sync. Even with the JAVA heap adjustments.I was wondering if we could do the reload/replication in a looping process.  The source is Snowflake the target is Azure ADLS2 parquet files.

901
A
C
Cameron LEmployee
posted in CData Sync
Deploying CData Sync in a Kubernetes Environment

CData Sync can be deployed in a Kubernetes environment. This allows users to take advantage of Kubernetes’ orchestration functionality, including high availability (no downtime), scalability of loads, and backup-and-restore capabilities.Read more about this and follow our guide for this deployment here:https://www.cdata.com/kb/articles/sync-kubernetes.rst

130
C
D
DibyenduEmployee
posted in CData Sync
Slowly Changing Dimensions (SCD): A No-Code Approach

Hey CData Community!Slowly Changing Dimensions (SCDs) are essential in data warehousing to manage and track changes in dimension data over time.With CData Sync, users can easily implement these SCD models. In our recent article on how to implement SCD using CData Sync, we introduce the most common types—Types 1 to 3—and explain how CData Sync enables you to configure and automate the management of these SCD types without writing any code, making it accessible to users of all technical levels.Read here to know more: https://www.cdata.com/kb/articles/slowly-changing-dimensions.rstIf you have additional questions, please let us know!

70
D
C
Cameron LEmployee
posted in CData Sync
Deploying CData Sync to Microsoft Internet Information Services (IIS)

CData Sync can be deployed to Microsoft Internet Information Services (IIS), which enables businesses to host Sync as an externally facing website. This allows organizations to utilize the advanced features of IIS, all while keeping the simplicity and functionality of CData Sync's embedded web server.Read more about the two methods you can use to deploy CData Sync in IIS here: Sync IIS Java deployment

250
C
Aleksander P
Aleksander PCommunity Manager
posted in CData Sync
CData Sync automatic deployment in AWS via Terraform

Table of Contents Introduction Overview of CData Sync Overview of Terraform Benefits of CData Sync automation with Terraform Prerequisites Terraform configurations VPC module EC2 module Parent module DEMO: CData Sync automatic deployment on AWS Conclusions Key Considerations IntroductionIn today's rapidly evolving cloud infrastructure landscape, automation has become essential for maintaining efficiency and reliability in deployments. By combining the powerful data integration capabilities of CData Sync with the infrastructure automation prowess of Terraform, organizations can achieve a streamlined and repeatable deployment process that significantly reduces operational overhead.This article demonstrates how to automate the deployment of CData Sync in AWS using Terraform. We'll explore the step-by-step process of setting up a fully automated infrastructure, including network configuration, security groups, and EC2 instances. This automation approach significantly reduces manual i

840
Aleksander P
C
Cameron LEmployee
posted in CData Sync
How to Access Live Data from the WQL for a Workday Report

CData Workday Drivers and Connectors now allow users to create tables that execute specific Workday Query Language (WQL) statements for Workday reports. The new stored procedure, CreateWQLSchema, processes the output from Workday's "Convert Report to WQL" task and retrieves the same report data using the WQL statement. Read more about this Workday Driver enhancement here:Workday: How to Access Live Data from the WQL for a Workday ReportIf you have additional questions, please let us know!

470
C
S
sooryaprakash_parthibanf3Apprentice
asked in CData Sync
MSSQL to Snowflake CDC Load Task failing

Hi all,Our team is using CData Sync to extract and load data from SQL Server to Snowflake. The source MSSQL table is CDC-enabled, and we have created a CData Job with Change Data Capture enabled. We started testing the task with smaller tables (containing 100K - 300K records), and the task was successful without any issues. However, when trying with larger tables (containing 10M - 35M records), we faced failures every time.I have attached the error output for your reference. Has anyone come across this scenario? Any replies or help would be much appreciated. Thanks in advance.Error output: "2024-11-14T02:25:19.555+0000 2 [32|Q-Id] Failed to cache row [501097] The query was canceled."We tried running the same task with a few manipulations, like reducing the source table records to 1M over 10M but had no luck. We are not facing any other issues besides "Failed to cache row." What does caching mean in a CData task? Does this have something to do with the AWS AMIs? Thanks heaps,S.

701
A
R
Renukachari_KaseeEnthusiast
asked in CData Sync
Unable to retrive Jira Issues table customes fields

Hi Team, kindly help me on Jira custom fields are not visible in Cdata connector. ThanksChari

754
E
B
bboesdoApprentice
asked in CData Sync
Missing Salesforce Objects When Creating New Sync Task

I’ve successfully created dozens of tasks to sync certain Salesforce objects to an Oracle destination.  However there is a group of Salesforce objects that are missing when I go to create a new task.  All are created/used by the same 3rd-part ISV app, but the account I am using for connecting to Salesforce is admin and I have no issue interacting with those objects outside of CData Sync.  I’ve read where this could be a cache issue, but I have not been able to find a cache refresh anywhere in CData Sync.  Would anyone have suggestions?  Thanks.

301
Ethem Q
M
Marcus_SApprentice
asked in CData Sync
Cdata Sync SQL Server Permissions for data loads

Hey All,I am trying to adhere to the principle of least permissions. I created a SQL Server Authenticated Login for cdata to use as a service account to SQL Server. I’m not sure what permissions it needs at the database level though? I wasn’t able to find any documentation on the site about minimum permissions cdata sync service account for SQL Server would need.I allow the cdata sync service account to drop and create tables. My guess is that I only need to add the cdata sync service account user to the following database level roles in my data lake.ddl_admindb_datareaderdb_datawriterWhat I am not sure about is whether or not Cdata Sync Service Account needs to have permissions to create Schemas.Thanks

331
Ethem Q
M
marodsalApprentice
asked in CData Sync
Suggestions in how to load tables over 15 millions rows using CData Sync

What are the suggestion in how to load tables over 15 millions rows using CData Sync? The data source is ServiceNow and the destination is SQL Server. I changed the sync.exe.config to<?xml version="1.0" encoding="utf-8" ?><configuration>  <startup>    <supportedRuntime version="v4.0" />   </startup>  <!-- You can add some Java JVM parameters here. This is for Java application only.   <appSettings>    <add key="JAVA_OPTS" value="" />  </appSettings>  --><appSettings>     <add key="JAVA_OPTS" value="-Xms2g -Xmx50g"/> </appSettings>    </configuration>Also, I changed the timeout of the jobs to 6200.

891
M
Jaclyn
JaclynEmployee
posted in CData Sync
🎄 Season's Greetings from CData! 🎄 Data Sync Q4 2024 Release Highlights

We're delighted to spread some holiday cheer with the latest enhancements to CData Sync, designed to elevate your data integration and management capabilities this festive season: 🔐 Custom Roles for Enhanced Security: Gain greater control over user access and permissions with our new Custom Roles feature. Tailor roles to fit your organization's needs and restrict the usage of customer-created objects for improved security. 🔄 Dynamics 365 Reverse ETL Destination: Seamlessly sync data back into Dynamics 365, enhancing your CRM and ERP workflows during this busy holiday period. ⚡ Enhanced CDC: Experience faster and more reliable Change Data Capture, ensuring your data is always up-to-date with improved efficiency—just in time for year-end reporting. ❄️ Snowflake Performance Improvements: Enjoy significant performance boosts in our Snowflake integration for quicker data processing and analytics, helping you unwrap insights faster. 📚 Full Feature Documentation: For a comprehensive unders

750
Jaclyn
C
CJIWApprentice
asked in CData Sync
Multiple schedules for a single Sync job?

Is it possible to create multiple schedules for a single Sync job?The use case here is that I'd like to run a job only once per day on some days of a month but then multiple times per day for some other days.One idea that comes to my mind is that I can create two jobs with different schedules, but the problem here is that if a destination table is incrementally synced (which is required for large tables), two jobs will not be able to coordinate.

633
Ethem Q
M
Marcus_SApprentice
asked in CData Sync
How to Use API to enable Automatic Job Retry

How do I use the REST API to enable the automatic job retry option? I’m not seeing that option in the API REST documentation.I am using CData Sync cross platform.

581
Ethem Q
J
JackolasApprentice
asked in CData Sync
CDATA Sync: add semicolons in a transformations SQL script with SQL server as target?

I’m trying to do a transformation after a dataload to a SQL server.The query used is: (the query is tested and works on the SQL server)MERGE INTO [S_Material_Data] AS tgtUSING ( SELECT HASHBYTES('SHA2_256', mat.MatNr) AS HSH_Material, CR_date AS Creation_Date, CASE WHEN mat.CH_Date IS NOT NULL THEN mat.CH_Date ELSE mat.CR_date END AS Changed_On, mat.Mat_Type AS Material_Type, mat.Mat_Grp AS Material_Group, mat.PRD_Hier AS Product_Hierarchy, Mat_Disc AS Material_Description, Mat_DiscU AS Material_DescriptionUp FROM Materials mat LEFT OUTER JOIN Descriptions dsc ON mat.MatNr = dsc.MatNr AND dsc.Lang = 'E' ) AS src ON tgt.HSH_Material = src.HSH_MaterialWHEN MATCHED THEN UPDATE SET tgt.Creation_Date = src.Creation_Date, tgt.Changed_On = src.Changed_On, tgt.Material_Type = src.Material_Type, tgt.Material_Group = src.Material_Group, tgt.Product_Hierarchy = src.Product_Hierarchy, tgt.Material_Description = src.Material_Description, tgt.Material_DescriptionUp = src.Materi

652
J
J
JelleStasNewcomer
asked in CData Sync
Amazon Marketplace Connector

Hi CDATA community, I’m Jelle from Belgium and very new to CDATA Sync. I’m very excited about all what CDATA Sync has to offer. I’m currently working to get our Amazon Marketplace data loaded in our Snowflake instance via Sync.Are there other Sync users who are working with the Amazon Marketplace connector and are willing to discuss and share their experiences?Thanks!RegardsJelle

280
J
Vibe Querying with MCP

Badge winners

  • Product Expert
    Stanley L.has earned the badge Product Expert
  • Astronomical Ideas
    Mikhas earned the badge Astronomical Ideas
  • Product Expert
    Mikhas earned the badge Product Expert
  • Product Expert
    BI-Modhas earned the badge Product Expert
  • Product Expert
    natykarinahas earned the badge Product Expert
Show all badges
Terms and ConditionsCookie settings