CDC/CT - Resynchronize from Date

Ideas
, , , ,
1

I’ve been using Sync to pull from MS SQL Server, from a couple of different sources that have either CDC or Change Tracking enabled.

Occasionally, there is some kind of hiccup where things get out of sync.  There are changes in the source that never replicated, but Sync thinks it’s already processed them, so it doesn’t try to catch up.

For CDC-based jobs, I’d like an option to “Run with Resync”.  Supply a date, and have that be used as the basis point for Sync to look back.  Not whatever high water mark is stored in the internal Sync database.

It would be OK if it applied the same look-back date to all the Tasks in the job.  It would be just for that one job execution.

2

(I know you can reset the job completely in a couple of different ways, but this causes the entire data set to be resynced, the full tables.  This could take hours or days.)

3

Hey Doug,

In the Advanced Job Options, there is a configuration for Replicate Start Value, which sets the minimum date/ int value (date in the form yyyy-MM-dd) from which to begin replication.

Is this what you are looking for?

4

That doesn’t solve this problem fully, because one of my use cases requires forcing CData to reset its internal high-water mark.  I.e. Sync “thinks” it’s up-to-date, when in fact it is not.  Adding a start date wouldn’t really resolve that.

Having said that, it would resolve some other scenarios.  I had inquired with tech support about a start date for replication, but they weren’t aware of any.  And the parameters in that help doc (Replication Interval, Replication Start Date) do not show up in my UI.  I’m on the latest version - Cross-Platform Edition - 23.3.8731.0      I confirmed my job type is “change data capture”.

5

Hey Doug,

To follow up on the conversation with Ankit and your feature request:

Replicate Start Value, Replicate Interval, Replicate Interval Unit are options available only for Standard Job Types , hence you are not able to see or use them with CDC Job type.

There is no way to achieve what you’ve asked for with CDC jobs at the moment in a straightforward way; however, we could discuss this with the development team if it aligns with our current implementation and our Product roadmap.

I do see having hiccups as the bigger issue though and something that must be addressed.

From the support history, you seem to have been in touch with the support team lately, but please feel free to reach out if you’ve not done so regarding this matter, so we can have a look into it.

1 Like
6

Thank you.  Yes, fixing the underlying bumps with my CDC/CT jobs would be the preferred solution.  :slight_smile:

For others - there is a parameter “'IgnoreUsingStatusTable” you can apply at the Job level, contact tech support for more info.  That worked for my use case.