Data Integration

Ask questions, get answers, and engage in discussions about data integration solutions from CData.

77 Topics

J
JTapioApprentice
asked in CData Sync

CData Sync and Windows Event viewer

My CData Sync servers are running on Azure. I’d like to use Azure Monitor Agent to monitor if CData Sync service is in stopped state. Usually Windows Service monitoring can be queried from Windows System logs finding Event ID 7036. CData Sync seem not to write anything to System Log (or I just not find it). Where should I try to find information if service is stopped?

2
E
2 hours ago
Aleksander P
Aleksander PEmployee
posted in CData Sync

Seamless monitoring between CData Sync and Datadog

Given the increasing reliance of companies on their IT infrastructure to maintain competitiveness and fulfill customer needs, effectively managing and monitoring their IT environments can pose a significant challenge in the absence of appropriate tools.Datadog is a cloud-based monitoring and analytics platform that helps organizations gain real-time visibility into their IT infrastructure.This article consists of detailed steps related to enablement of the integration between CData Sync and Datadog.NOTE: This article can be taken into account for other CData applications that auto-generate logfiles from their operational activities (i.e. “CData Arc”, “CData Connect Server”) Step 1: Install the latest version of Datadog Agent (assuming that you already have a Datadog account)On the UI of your Datadog’s instance, hover into “Integrations” and then click “Agent”, in order to access the page that will help you with the installation of the agent.Once you are on the “Agent” page, select your

0
Aleksander P
1 day ago
J
JTapioApprentice
asked in CData Sync

Task data source settings grayed out in the Sync version 23.4.8843.0

I’ve updated our test environment to latest version 23.4.8843.0 and I noticed that when I try to configure data source settings, I can’t and I get information “Settings are not editable”. Same issue with new tasks and previously created tasks. I can however make settings directly to query-tab and it will work ok. Our production environment is version 23.3.8700.0 and that issue is not there. Data source which I’m using is ElasticSearch

3
Ethem Q
14 days ago
N
Newbie CdataApprentice
asked in CData Sync

How to set Additional Job Options via API Call

We are currently using cdata version .NET Edition - 23.3.8700.0 in our organization and we create jobs through API call. Through API call we are able to create the JOB fine, but we are unable to set StoresUppercaseIdentifiers=true Additional Options property.Any help on this topic is greatly appreciated.  

4
N
14 days ago
A
abhi_quadwaveApprentice
asked in CData Sync

CDC in Cdata Sync creates new record in Destination Table for a record update in Source table

[META|Schema] Skip the drop column operation for column [_cdatasync_deleted]. Can anyone help me understand this issue please. As in the destination table there is a column  _cdatasync_deleted which is being set to null for this new row inserted(Which was supposed to be a UPDATE instead of INSERT in first place)

1
E
22 days ago
Jaclyn
JaclynEmployee
posted in CData Sync

CData Sync Q1-2024 Release

📢 New Year, New Features! CData Sync Q1 2024 Release Highlights   Kickstarting 2024 with a bang, we're excited to announce the latest features in the Q1 2024 CData Sync release! Check out what's new:  API Connector Query Slicer - Elevate your API interactions! This innovative feature allows more flexible and efficient API call executions. Ideal for all users, making API integrations smoother than ever.  Reverse ETL - Big news for Salesforce users! Now push data directly into your Salesforce instance, integrating your database insights seamlessly. A major leap forward in CRM data management.  Enhanced Usability Features - Based on your valuable feedback, we've improved usability aspects across our platform, ensuring an even more intuitive and user-friendly experience.  📖 Discover More in Our Q1-2024 Sync Release Blog - Dive into the details of these exciting updates in our latest blog post.  🎥 Visual Learner? Watch our tutorials on the new API Connector Query Slicer and Salesforc

2
P
1 month ago
C
CJIWApprentice
asked in CData Sync

Using Sync pre events to run sync loops

Hello!I know that I can define an out environment variable in pre-job event and use that in the task replication query (documented here and here excellently).I’m probably stretching here but can I define that out-variable to read from a file or any other source and then run the sync task for ALL values of that variable? Kind of a loop? Thanks in advance for you answers!

0
C
1 month ago
C
CharleyGApprentice
asked in CData Sync

Exception happened: Failed to start database

Exception happened:Failed to start database 'C:\ProgramData\CData\sync\db\cdata_sync' with class loader WebAppClassLoader{CData Sync}@20ce78ec, see the next exception for details.ServletPath: /login.rst ContextPath:  

2
C
1 month ago
S
Steve RApprentice
asked in CData Sync

CData Sync Oracle CDC using logminer fails

I am having an issue with my sync installation. I am able to successfully start logminer on the oracle instance and query the results of that logminer session.However, when I run a CDC job from Sync, it will complete the initial data load successfully but then fail on the incremental loads after the initial one with the below error:logminer failed to start If I have it delete the table ot truncate the data then it reloads just fine. It just won’t start logminer to run the CDC loads.Has anyone run into this issue or found a way to resolve this so I can get CDC loads from oracle to sql server working correctly? 

0
S
1 month ago
A
aniruddhmaitraApprentice
asked in CData Sync

Customized Query throwing ambigious column error

 Hey! I have written a query in CData Sync to load rows from TransactionAccountingLine only for a date range, Feb 2024.REPLICATE [TransactionAccountingLine]SELECT tranacc.* FROM [TransactionAccountingLine] tranaccLEFT JOIN [transaction] tranON tranacc.transaction = tran.idWHERE tran.trandate between '2024-02-01' and '2024-02-29'Upon running this query, I am getting the error, [0] Column 'lastmodifieddate' is ambiguous. But I should not be getting this error since  I am selecting all the columns only from the TransactionAccountingLine table, where there is only one lastmodifieddate.Please help me with this issue.

3
D
1 month ago
C
CharleyGApprentice
asked in CData Sync

After upgrading CData Sync my Aha connector is not working anymore

Hi Team, Script manager is not registered for schema 'Aha' check whether the scheme is valid  I’m getting this error while trying to save and test the Aha connector Regards,Charitha

3
C
2 months ago
S
siriusApprentice
asked in CData Sync

All the transaction lineitems not loading in Netsuite suiteql

Hey tribe,I am running into a major problem while using Cdata sync to load tables through suiteql. For transactions table as an example, Many line items are not loading for a particular date range even though I made the relevant configuration and added the filters in my query.Sometimes I have to specifically type the tranid and load the same to get those lineitems. Doing this over and over again has become tiresome and is taking away the automated nature of the Fact tables we’ve built surrounding this. Is there something we can do to resolve this? Like adjusting the batch size or something? Looking out for your help :)

3
Ethem Q
2 months ago
S
srchuApprentice
asked in CData Sync

Download Oracle connector into cdatasync application

I am trying to download the Oracle db connector. in CDataSync application. I am getting the following error message An error occurred during downloading connector [Oracle]: Download provider error: Cannot conclude ssl handshake. Cause: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Any one tried downloading and configuring Source Oracle DB?

2
S
2 months ago
D
DougN NouriaEnthusiast
asked in CData Sync

Source CSV file attributes as columns

I have 3 different CSV files to load each week, and they always have the same filename.I set up a “Sync All” job to read these files and load them into my destination.However, I would like to pass the files’ creation dates or modify dates, as columns into the destination target table.  Is this possible to do? 

3
J
2 months ago
N
Newbie CdataApprentice
asked in CData Sync

JIRA Worklog Table Timeout

Trying to bring JIRA data onto snowflake using CData. All the tables loaded successfully except Worklog. The worklog table failed with below error. [0] [500] Could not execute the specified command: Failed to cache row [3652696] Invalid HTTP response! Timeout.. StackTrace: at aso230l.nT.Next() at aso230l.mH.Next() at aso230l.mb.Next() at aso230l.mY.Next() at aso230l.mQ.Next() at aso230l.Kz.q(DbCommand , DataTable , IResultSet , ColumnInfo[] , Int32[] , jsb ) Any help on this topic is greatly appreciated.

1
A
2 months ago
A
aniruddhmaitraApprentice
asked in CData Sync

transactionStatus table not loading CData Sync

Hi!I am working with a client who uses NetSuite ERP. We are trying to get Sales reports for them, which we are doing by using the transaction table from SuiteQL schema.We also need the transactions’ status, for which we need the transactionStatus table to load through Sync. On doing so, I am getting the following error :“[0] [INVALID_PARAMETER] Invalid search query. Detailed unprocessed description follows. Search error occurred: Record 'transactionStatus' was not found. Bad Request.” Please help me load this table.​​​​​​​

2
A
2 months ago
John
JohnCommunity Manager
posted in CData DBAmp

CData DBAmp Winter 2024 Release is here!

🚀 Exciting News from the DBAmp Team! The CData DBAmp Winter 2024 Release is here, bringing groundbreaking enhancements to your experience!  🔍 What's New?  Significantly improved OpenQuery Performance: We've optimized the TDS daemon process, ensuring a 50% boost in efficiency for OpenQuery operations! 📈  Refined internal operations: With smarter processing of SMP request packets and timely SMP scheduling thread notifications, say goodbye to waiting times. ⏱️  Enhanced system alignment: The execution times in SSMS are now closely synced with the Salesforce driver method. 🔄  💡 Why This Matters?  Experience unparalleled efficiency and reliability with CData DBAmp.   Our commitment to quality and customer satisfaction shines through in every update.   👉 Don't miss out on the latest advancements that CData DBAmp has to offer. Download the newest version and take advantage of a free trial today. Your feedback is what drives our innovation – be a part of our community and share you

1
H
2 months ago
D
DougN NouriaEnthusiast
asked in CData Sync

Sync pre/post Events - reuse existing db Connection info

You can use post-job Events to run some arbitrary SQL (like a stored procedure) at the end of your Job.The documentation says you have to make your own connection, as below.  But I’d really rather not embed cleartext credentials in multiple jobs.  Is there a way to specify a CData connection (which we’ve already established at this point), in the Event?What I’d really like is the ability to run arbitrary SQL commands against the target table, after each Task completes, not just at the end of the Job.  Seems like this could be a parameter that we supply as we define each Task.<api:set attr="db.driver" value="System.Data.SqlClient"/> <api:set attr="db.conn" value="Data Source=sqlserver;Initial Catalog=Test;User ID=user;Password=pass;"/> <api:set attr="db.query">INSERT INTO MyLog (JobName, RunDate, Results) VALUES ('[_input.JobName]', '[null | now()]', '[_input.JobStatus]')</api:set>

2
Ankit Singh
3 months ago
J
joshphillipssrApprentice
asked in CData Sync

Workday WQL Error

Using CData Sync Cross Platform with Workday WQL connection. Some queries work successfully, but a bunch fail with this message:[0] Received error from Workday: Invalid WQL query: !**! This error occurred during a request to calculate the field !**!, for instance !**!.. Cause: com.workday.exceptions.ExternalFieldException: The Increment or Decrement Date field, !**!, has encountered a blank date..  Thanks.

1
Ethem Q
4 months ago
J
joshphillipssrApprentice
asked in CData Sync

Min.io S3 Bucket

Trying to connect to Min.io on-prem S3 bucket. Confirmed API is working:  However, I don’t know how to point to https://nas2.piorl.us:9000/bucket1/ in CDataSync Cross-Platform Edition. I’ve tried defining the full path in the Bucket field as well as “Path” in advanced options, but neither want to work. There is a reference to a “Custom URL” field in the advanced connection properties, but I can’t find the variable anywhere:  Thanks.

6
Ethem Q
4 months ago
N
NeilHApprentice
asked in CData DBAmp

Data transport when using DBAmp remotely

If we configure DBAmp to use a remote server (e.g. Azure managed SQL), does all data flow through the server where DBAmp is installed?  And if so, what is the transport-layer security between the DBAmp-server and the remote server?

1
D
4 months ago
R
RahmanfpApprentice
asked in CData Sync

Failed to use backup setting

Hi, So I have a system with CDataSync to sync my data from SQL Server to Azure Synapse with below detail: Old System:CDataSync Version: 22.0.8342.0CData AzureSynapse Provider Version: 22.0.8326.0 New System:CDataSync Version: 23.3.8704.0CData AzureSynapse (23.0.8616.0) I backup the old configuration and restore it to new configuration. However, it keeps getting an error like this Error: The SQL Error Number is 103010, State is 1, Severity is 16, Error is 'Parse error at line: 1, column: 126: Incorrect syntax near '.'.', Server name is server1, Line number is 1. The Token Type is 170. I found that the task is successful when I use this conditions:I dont use a new scheme for the destination, but I checked the Drop Table/Truncate Table I use a new scheme, but I unchecked the Drop Table/Truncate TableIs there a new query or options that changed in the new version? What can I do to fix the error? Thank you

1
Ankit Singh
4 months ago
R
RajeshApprentice
asked in CData Sync

Error adding task to job

I’m attempting to add a task to an existing job and get an error:“The job <job-name> contains an unlicensed connection: Only 10 tasks are allowed in the Job” I also tried creating a job from this existing job and got an error:The connection [Salesforce_abc] is unlicensed: Unlicensed connections cannot be shared across multiple jobs. We have an Enterprise license, and my job is using SalesForce as a source and Azure SQL as a destination. I tried adding a task to a different job that uses Salesforce as source and Azure SQL as destination, and it worked.

2
R
4 months ago
S
SatyendraApprentice
asked in CData Sync

Issue with CDATA Sync Job Using Incremental Replication Despite Removal

Hello CDATA Community,I hope this message finds you well. I am currently facing an issue with the CDATA Sync tool and would appreciate any insights or assistance from the community.Problem Description: I have configured a job with a single task, where the source is Salesforce, and the destination is an SQL database. Initially, I ran the task without incremental replication, and it successfully synchronized all records for a specific Salesforce object. Later, I implemented incremental replication and verified that it worked as expected. However, when I decided to remove incremental replication and attempted to sync the data again, the job seems to persist in using incremental replication, and not all data is being synchronized. Steps Performed:Created a job with Salesforce as the source and SQL database as the destination. Ran the task without incremental replication, successfully syncing all records. Implemented incremental replication and verified its functionality. Removed incrementa

3
Ankit Singh
4 months ago
D
DougN NouriaEnthusiast
asked in CData Sync

Snowflake - specify temp table location/schema

When bulk loading to Snowflake, data is uploaded to a temp table (named by CData) and then insert/merged into the final target table.Is there a way to specify, either at the Connection level, or the Job level, what Database or Schema should be used to store those tables?  By default it just puts them in the target schema, but it would be nice to split those out so we can more quickly clean anything up after an aborted job.

3
D
5 months ago
Virtual Event: Data Virtualization, Reimagined Ma”arch 14, 2024 style=

Badge winners

Show all badges
Terms and ConditionsCookie settings