I have 3 years worth of data. All the data stay the same but the latest 3 months are dynamic.
I need to replace the old data with the latest data. But if I do “Drop Table”, it drops all the data and bring all 3 years worth of data and sometimes the system dont seem to be able to handle it.
I would like to keep everything and just have the last 3 months tables dropped and replaced. I dont want the system to have to bring ALL data from the past 3 years. I just need the last 3 months dropped and replaced.
What setup should I do?
Best answer by Prateek Jain
It is possible if: 1. We create a destination Table A having all the data upto last 3 months(for example: March 2025) by setting the ReplicationEndDate in the advanced job options. Please refer the following documentation: https://cdn.cdata.com/help/ASM/sync/Advanced-Job-Options.html#additional-options:~:text=avoid%20replication%20errors.-,ReplicateEndDate,-None 2. Create another task having destination Table B where ReplicationStartDate set to 3 months back(for example: April 2025) and perform the full load every time. 3. Create a Transformation to merge Table A and Table B in destination and it should be scheduled after performing the 2nd Step.
I hope this will work in your use-case. Let us know if you have further questions or else you can drop a mail on [email protected]
It is possible if: 1. We create a destination Table A having all the data upto last 3 months(for example: March 2025) by setting the ReplicationEndDate in the advanced job options. Please refer the following documentation: https://cdn.cdata.com/help/ASM/sync/Advanced-Job-Options.html#additional-options:~:text=avoid%20replication%20errors.-,ReplicateEndDate,-None 2. Create another task having destination Table B where ReplicationStartDate set to 3 months back(for example: April 2025) and perform the full load every time. 3. Create a Transformation to merge Table A and Table B in destination and it should be scheduled after performing the 2nd Step.
I hope this will work in your use-case. Let us know if you have further questions or else you can drop a mail on [email protected]
