How to setup Incremental Replication

Data Integration CData Sync
, ,
1

Hi,

I have 3 years worth of data. All the data stay the same but the latest 3 months are dynamic.

I need to replace the old data with the latest data. But if I do “Drop Table”, it drops all the data and bring all 3 years worth of data and sometimes the system dont seem to be able to handle it.

I would like to keep everything and just have the last 3 months tables dropped and replaced. I dont want the system to have to bring ALL data from the past 3 years. I just need the last 3 months dropped and replaced.

What setup should I do?

2

It is possible if:
1. We create a destination Table A having all the data upto last 3 months(for example: March 2025) by setting the ReplicationEndDate in the advanced job options. Please refer the following documentation: CData Sync - Advanced Job Options | 25.3.9396
2. Create another task having destination Table B where ReplicationStartDate set to 3 months back(for example: April 2025) and perform the full load every time.
3. Create a Transformation to merge Table A and Table B in destination and it should be scheduled after performing the 2nd Step.

I hope this will work in your use-case. Let us know if you have further questions or else you can drop a mail on support@cdata.com