Data Integration

Change Data Capture (CDC) is a crucial technique in modern data management that allows you to track and capture changes made to a database. It provides real-time insights into data modifications, enabling businesses to react swiftly to evolving trends and requirements. In this article, we'll explore what CDC is and how to implement it using CData Sync.Understanding Change Data Capture (CDC)CDC revolves around capturing data changes at the source using a log file, eliminating the need for constant querying. Instead, CData Sync reads the log file for events (Insert, Update, or Delete) and extracts these changes in near real time, ensuring accurate replication for future use.Supported CDC Sources:MySQL: Utilizes binary logs. Oracle: Leverages Oracle Flashback. PostgreSQL: Employs logical replication. SQL Server: Uses either change tracking or change data capture.

Is it possible to import PDF data into QuickBooks Online (QBO) with CData Arc? If so, can someone please provide a step-by-step guide?

INFO: Get app data directory form /Users/mac/Downloads/CDataSync /sync.properties file. nullAug 18, 2023 3:51:40 PM cdata.sync.SyncServer aINFO: The server log directory is /Users/mac/Downloads/CDataSync /logs/httpserverAug 18, 2023 3:51:40 PM b.d qINFO: Init Jetty log with default settings.Aug 18, 2023 3:51:40 PM b.d bINFO: Starting server from /Users/mac/Downloads/CDataSync /sync.properties configuration.Aug 18, 2023 3:51:40 PM d.a aINFO: The properties deserializes success.Aug 18, 2023 3:51:41 PM b.d cINFO: Enabling embedded Jsp support...Aug 18, 2023 3:51:41 PM b.d cINFO: Successfully enabled embedded Jsp support.2023-08-18 15:51:41.719:INFO :oejs.Server:main: jetty-10.0.14; built: 2023-02-22T23:12:32.272Z; git: 976721d0f3e903a243584d47870ad2f2c1bf9e55; jvm 17.0.8+9-LTS-2112023-08-18 15:51:49.684:WARN :oejw.WebAppContext:main: Failed startup of context o.e.j.w.WebAppContext@7d3a22a9

 What should I put in the “”Perform this query to select the UPSERT key” box?

Hi Community, new to CData Sync here.When am trying to extract a certain table from Workday through the inbuilt Workday source connection, an getting the error - [0] Cannot invoke "java.util.List.iterator()" because "" is null Could anyone help with the error please. Thanks in advance.  

One of the most common causes for AS2 connection errors is not having the correct certificates configured for your trading partner within Arc. In an AS2 connection, your partner’s SSL server certificate is validated against your system’s security settings to proceed with an SSL handshake, and their S/MIME certificate is used for the validation of signed messages and encryption of outgoing messages. Ordinarily, these certificates are exchanged as part of the setup of a new AS2 connection, but certificates are commonly replaced, and you may be left with a connection that is not working after an update from your partner. The following methods are not foolproof, but there are possible methods for obtaining these certificates without direct contact from your partner.  Obtaining the SSL Server certificate  This is straightforward - place your partner’s receiving URL in a browser: 

How to define a cross reference table or codelist in XML to X12  mapping? Collect = CCPrepaid = PPPer Contract = DE

Dear all, apologizes in advance for the probably silly question, but I am new at this and I could not find any examplea customer of mine is sending me a file always with the same filename, and I need to be able to save it and be sure that it will not be overwrittenfor example adding a timestamp would be perfect (but preserving the extension)EUC2313.xml → EUC2312_20230828101211.xmlEUC2313.xml → EUC2312_20230828101313.xmlhow could I do this ? I can not use filepath or filename

Hi Everyone!The latest release of CData Sync has arrived! Take a closer look at what's new and how it can benefit you. Visit these resources to learn more:BlogWhat’s New in the Q3 Sync releaseKnowledge Base ArticlesEnterprise features Transformation librariesExperience what’s new yourself!Start a 30 day trial 

Executing SF_MirrorAll results in an error as indicated below.  It occurs both with FullCopy or DeltaCopy.  I have performed various searches but haven’t found a solution.  --- Starting SF_Mirror for WorkStepStatus V8304 [SQLSTATE 01000]01:53:03: Parameters: SALESFORCE_Med_ARCH_FULL WorkStepStatus  Version: V8304 [SQLSTATE 01000]01:53:04: DBAmpAZ 22.0.8304.0 Copyright c 2021 CData Software, Inc. [SQLSTATE 01000]01:53:04: Obtaining Connection Props [SQLSTATE 01000]01:53:04: Obtaining User Information [SQLSTATE 01000]01:53:04: Using settings from Registry. [SQLSTATE 01000]01:53:05: Using FullCopy because the WorkStepStatus table does not exist. [SQLSTATE 01000]01:53:05: Server: WUIT-S-14261, Database: MedARCH_SF_FULL, Linked Server: SALESFORCE_Med_ARCH_FULL [SQLSTATE 01000]01:53:05: Using the Salesforce BulkAPI. [SQLSTATE 01000]01:53:06: Create WorkStepStatus_Previous20230828015305925 with new structure. [SQLSTATE 010

I started using the new Branch connector with an existing AS2 connector. The Branch connector will be used to save specific messages based on the Filename to a different folder.I've setup the Branch connector as shown in the image provided in the Help for the Branch connector.IF Filename matches glob CUS*.*But instead of the output being send to the provided File Connector to save the message in a different folder, the message is forwarded to the ELSE clause and forwarded to that attached connector.The log-file shows the following information (not really helpful at the moment):[2023-08-25T15:36:47.403] [Info] CUSDEC325-0000000000062014 is routed to File_SHPP2MEP_Test (which is the ELSE clause). [2023-08-25T15:36:47.403] [Info] Message CustomerAS2_TEST-20230825-153646575-eCIr is finalized. Name: CUSDEC325-0000000

It is often the case in a document mapping that a source file can contain special characters that cause problems later down the line - notably in EDI mappings where non-printable characters cannot be allowed.  Arc will treat strings as UTF-8 when reading from files, but sometimes you may want a bit of code to replace the non-printable characters in a string. The following Script: <arc:set attr="file.file" value="[FilePath]" /><arc:call op="fileRead" in="file" out="fileo"> <arc:set attr="output.data" value="[fileo.file:data | regexreplace('\[^ -~\]','')]" /></arc:call><arc:set attr="output.filename" value="[filename]" /><arc:push item="output" /> Can be used to replace any characters outside of the printable ASCII range (here, defined as anything not between the whitespace and tilde characters in the ASCII set) with an empty string. This s

Is #EDI really a tough cookie to crack for you? Are you new to EDI and not sure where to start?If yes, then this one is for you.Join our CData Tech Evangelist @Matt S and me for a three-part webinar series that helps you understand #EDI automation from the ground up. Register now!You can find the details of the three-part webinar series here: 

My DBAMP version is 22.0.8354.0I have DMAMP installed on a non OS Drive.   M:\Program Files\Cdata\CData DBAmpTrying to upgrade the license using the Config Utility.  Help → Licensing ->Update LicenseGetting following error.  The keys.reg file does not exist on my installation. Work Around Create the folder + File the error was complaining about. It can be an empty text file.“c:\CData\CData DBAmp\keys.reg”Re-Run the Update License command and seems to work fine now. Question Is this fixed in a later version of dbamp?  I am aware I dont have the most recent “patch”.

Hello ALL,  I have new requirement, From Customer they use API to send data and i need to receive the data, convert to XML ( CDM Customized data format) send it SWAG system via AS2. And is same way, i will be receiving the XML file from SWAG system via AS2, need to convert to customer JASON format and POST the data to Customer using API.  Please help to advice how i should be use the API, i never used API before, all i was familiar with connections like AS2 SFTP FTP’s.  Hence i cant able to understand API in term of connection method.  Thanks for your time and consideration.  Thanks Pradeep 

Hi CData Community,    I am trying to add another layer of security for a webhook. I would like to do the following:    1. Pass @authtoken query parameter for CData API auth    2. Pass Authorization via headers    Unfortunately, I am getting an error Error: Authentication is required for access to this resource.It seems like the webhook first checks the Authorization header for CData API auth before checking the query parameter @authtoken.But I need the Authorization header for my additional security. Is this implementation possible?Thanks!

The standard documentation in https://cdn.cdata.com/help/AZH/mft/op_HTTPReceive.html shows the elements needed to use the httpReceive operation in scripting. Is there documentation or examples available to show the structure or syntax on how to use this operation?

EDI acknowledgments provide an important aspect of the EDI standard: a non-repudiation guarantee. In other words, since you’re alerted when your business partners receive your EDI documents, they can’t later claim that they didn’t. Arc simplifies the process of generating and processing EDI acknowledgements by handling the details behind-the-scenes. Here's all you need to understand to ensure that Arc is configured correctly. EDI Connector pairs Arc’s EDI Connectors come in pairs – one for inbound EDI documents and one for outbound EDI documents. Each connector in this pair is configured for the same trading partner, but some settings are mirrored (e.g. the ‘Receiver ID’ from one is the ‘Sender ID’ for the other). Your EDI Connector pairs need to work together to automate EDI acknowledgements. When your inbound EDI connector receives a document, it needs to alert its outbound counterpart to send out an acknowledgement. Similarly, when you

This script is co-authored in part by Russel with Jersey Post:https://community.cdata.com/members/russell-jerseypost-190 The following code will enumerate the lines of a multiline file, such as a txt or csv, and output a new file with only the unique lines returned: <arc:set attr="file.file" value="[FilePath]" /><arc:call op="fileReadLine" item="file"> <!-- enumerate the file and add the rows to a collection --> <arc:check attr="newrows.[file.file:data|md5hash(false)]"> <arc:else> <arc:set item="newrows" attr="[file.file:data|md5hash(false)]" value="[file.file:data]" /> <arc:set attr="tmp.keys#" value="[file.file:data|md5hash(false)]" /> </arc:else> </arc:check></arc:cal

Hi, I’ve been successfully using Sync to replicate data from Eleasticsearch index to Azure Data Lake. Now I have faced an issue one of index that is bigger than others (not much, but obviously enough bigger). In replication log there are following errors: [META|Cache ] Cache failed: Updating Row is failed. ERROR: Invalid HTTP response! Cannot upload PostData: System error: An existing connection was forcibly closed by the remote host., AffectedCount: 5200000 [SQL |Messag] Query Failed: [CACHE [uspa/spaiderloki/arkisto/2023] SELECT [_id], [Aika], [Atunnus], [Caller], [Callsite], [cdata_timestamp], [Duration], [Exception], [Filter], [guid], [ID], [kafka_partition], [Logger], [LogLevel], [Message], [Thread], [Version] FROM [Elasticsearch].[uspa-spaiderloki_arkisto_20233x]]. Error: Updating Row is failed. ERROR: Invalid HTTP response! Cannot upload PostData: System error: An existing connection was forcibly closed by the remote host., AffectedCo

I’d like to do this Will it work? I get an error ”Error parsing Date value [now()].”

Is it possible to use the xpath formatter to get the name of the current element?Currently I am interested to get the name of the root element and get the content of an attribute on said element.

We upgraded DBAMp version on our SQL Server.Followed the instructions and created a new Linked Server. Stored procedures are running fine, but when I execute a SOQL query; we get the following error An invalid schema or catalog was specified for the provider "MSOLEDBSQL" for linked server "SalesForce". Example SOQL queriesSelect * from SalesForce...user__cSelect * from SalesForce...Account

Can i get the Twitter Connector API details for the Cdata platform ? 

The sysExecute operation in ArcScript can be used to call processes on the local system as long as you have access to the process, and this includes VBScript files that you create yourself.  sysExecute To call a VBScript, you’ll want to use cscript as the name of the process to execute, and pass the path to the vbs file as an argument. If there is whitespace in that path, you’ll want to include quotes around the full path so that the path is passed as a singular argument, e.g.:<arc:set attr="sys.Name" value="cscript" /><arc:set attr="sys.Arguments" value='"C:\\Program Files\\CData Arc\\www\\bin\\test.vbs"' /><arc:call op="sysExecute" item="sys">  <arc:set attr="output.data" value="[sys.sys:output]" />  <arc:set attr="output.fi