Hi,
I have Sync version 24.3.9120.0 and I’m using Apache Kafka as source. It’s working fine, except little irritation. When I run task it will replicate only first 150 000 rows and stops after that. Next time I run the task, it replicates 179 000 rows and so on. Row count will increase. Seem to be random
I can run job multiple times and it will continue where it was stopped earlier. Is there way to change this behavior, so that all rows will be replicated at once?
Hi @JTapio , could you please share the logs so we can investigate this issue further and assist? To do this, please go into the Job settings, set the Logfile Verbosity to “Verbose”, run the job again, and share the resulting logs with us.
Here is one example. Job finished successfully, but didn’t replicate all rows.
Hi @JTapio , In the source log file, we can see only 33,000 rows being fetched (screenshot attached). However, all these fetched rows are replicating successfully, so from the replication standpoint, it appears to be working correctly. The most probable cause I can identify relates to the source logs: the page size is set to 1000, and I suspect this size might be insufficient to fetch all the data.
Additionally, please try using the incremental replication feature. This might work, as this issue shouldn’t typically occur during full replication.
I don’t think I can configure incremental replication, because source is Kafka. Kafka consumer offset value keeps track of incremental replication by default (Records can be consumed only once)
Hi @JTapio , just to clarify, whenever you perform full replication in CData Sync, the destination data is dropped, and a fresh set of data is fetched from the source.
Since the issue is persistent, I suggest that you contact our support team to help resolve your query.. CData Support: https://www.cdata.com/support/submit.aspx
ok, I’ll do it.
If I count rows from the source and from the destination, values are very close, so I don’t think that all the data is fetched every time replication occurs
