Data Integration

One issue that comes up infrequently but can be inscrutable to troubleshoot is a situation where you are unable to start the web server hosting CData Arc on a port that you have previously used for the embedded web server provided with the application. For the purposes of this thread, we will refer to port 8001 here, since this is the default port assigned to the embedded web server distributed with CData Arc. Most commonly, issues with binding to a specific port are due to an existing process that is already listening on that port, but you may find that even after you run a check for ports in the listening statenetstat -a You do not see anything listening on that port. One possibility is that the URL for the port you are selecting has already been reserved on the system and cannot be assigned to another server. To check for this, run:  netsh http show ur

OverviewSometimes, a requirement arises where you may need to receive data from a trading partner via the Webhook connector but that partner might require a bit more security other than just the basic auth username/password that is used to authenticate inbound webhook requests - or maybe you just want to add an additional layer of security on inbound webhook messages.Using the custom response feature of the webhook connector, you can actually create your own HTTP signature authentication logic to perform some additional authentication on inbound webhook requests, via the use of an HMAC signature value assigned to a header on the request.GitHub has a good writeup about this, and this is what has been used to make the below example within CData Arc - https://docs.github.com/en/webhooks-and-events/webhooks/securing-your-w

In CData Arc 2022, a new feature was added to the application which has been coined “backoff”. You may or may not have seen this mentioned in the application log of your instance, but if you haven’t, an entry in the application log would look something like this:~/AutomationService    Warning    The [ConnectorID] connector has failed more than 3 times and has entered a backoff state. The automation service will skip this connector until [date/time]. The connector will remain in the backoff state until a message is successfully sent manually, the connector settings change, or all messages are sent successfully during an automated send attempt.This is a feature where the automation for the connector will be temporarily paused if the connector fails to send a file successfully with automation three times in a row. If this happens, a 60 minute delay will be added to the automation of that connector before the connector will try to send any files via automation aga

Our newest Sync release is now generally available! In this release, as part of our ongoing work to improve the Sync user experience, we employed a Voice of the Customer methodology, soliciting and listening to customer feedback about navigation flows, tasks that would benefit from a more streamlined approach, and the need for improved schema support for common use cases. Read the blog to learn more.

I have a stored procedure that I want called once per day and the data pulled sent to a Send Email Connector. When I manually click the Receive button on the Output table of the Sql Server Connector, everything works as desired.  However, I set up the Receive Interval on the Automation Settings to Receive the data once per day, but it doesn’t run.  Am I doing something wrong? Or is there another way to automate the execution of this stored procedure?  Thanks,Chris

I have a flow that processes multiple files into multiple Sql Server tables. Once all those database tables are updated with the imported data, I want to run a stored procedure that uses that data to populate other tables. I was thinking I would just add another Sql Server connector after the import_pricing connector, but can I be sure that all the other import connectors have successfully completed first?  Is there a proper way to do this in Arc, so that when I run the final stored procedure all the needed import tables are populated appropriately first? Thanks!

Hello - I am having troubles establishing an sFTP ssh key pair connection. I want to set the server thumbprint to the servers public key which I have saved to a certificate but I just keep getting a “thumbprint format” error. Does anyone know exactly how to accomplish this?  

Whenever I try to use the bulkapi, or pkchunk, or even just use SF_Mirror on a large table, it says that SF_Mirror fails. The message output looks like this:--- Starting SF_Mirror for Account V830410:53:19: Parameters: salesforce2022 Account bulkapi Version: V830410:53:19: DBAmpAZ 22.0.8304.0 Copyright c 2021 CData Software, Inc.10:53:19: Using options specified in Options parameter: bulkapi10:53:19: Obtaining Connection Props10:53:28: Obtaining User Information10:53:31: Using settings from Registry.10:53:33: Error: DBAmpWork directory does not exist.10:53:33: DBAmpAZ Operation FAILED.10:53:33: Error: MirrorCopy program was unsuccessful.10:53:33: Error: Command string is C:\"Program Files"\CData\"CData DBAmp"\bin\DBAmpAZ.exe MirrorCopy "Account"  "DL-7KVJRL3"  "salesforce_backups2022"  "salesforce2022"  "bulkapi"--- Ending SF_Mirror. Operation FAILED.Msg 50000, Level 16, State 1, Procedure sf_mirror, Line

I’ve upgraded to the CData version of DBAMP, and I see that there is no longer a separate procedure for replicating the KnowledgeBase.  Is there a replacement process for this sync?

OverviewBearer token authentication can be one of the more finicky authentication mechanisms to implement within Arc. The reason is because the token lifetime is very short and automatic token refresh isn’t something that is built into that mechanism, like it is with OAuth 2.0. Thus, a new token needs to be generated and manually re-entered every time the old one expires. This isn’t really practical in Arc since many flows rely on automation to continually process files or make requests to APIs. Luckily, in Arc t is possible to create a custom script that gets a new token for each request that you send and then the token is dynamically evaluated within the REST connector.The ScriptThe script that should be used here is going to need to be responsible for two things:1) Obtaining a refresh token from the token endpoint of your API2) Adding the token as a header onto the message within ArcThis will fire every time your input file passes thro

CData Arc’s enterprise edition includes a built-in tool to generate custom reports tracking EDI messages, file exchanges with partners, and much much more. To learn how you can start building multi-dimensional reports out of CData Arc’s transactions in mere minutes, simply watch our 4-minute video tutorial:  As always, feel free to ask any questions or provide any feedback as you generate your own reports! -- Matt S

Hello,Thought I’d post this question here to help build content for the community.I have a situation where I have a workspace dedicated to  complete document flows for all EDI trading partners (832, 850, 855, 810, 856).  I have a single X12 connector where all documents either send or receive files.I have another workspace where I have specific  1-off flows created where my application interacts with Arc via API.An ongoing issue (in my industry) has always been customers asking us to re-send 810 Invoices. My main workspace is setup to send 810 files in a batch at the end of the day.  So I have a flow I created in my “API” workspace where my application allows the user to generate a single 810 file and makes an HTTP POST to the workspaceid and connectorid using /api.rsc/receiveFile through the Admin API.My question is this:  Since I already have these trading partners X12 connectors (and corresponding AS2 or SFTP connectors) setup in another workspa

I need to be able to ingest the following string and remove everything but the financial values in an xml map connectorCIGNA;QPA=242.06any direction is appreciated. This is for an x12 837 file

While our talented Arc support team is always happy to help resolve errors that you are seeing, it can certainly be nice to handle simpler issues on your own. To that end, we’ve compiled a list of common Arc errors and their solutions to arm you against those pesky red bars. If your error isn’t included in the list and you think it belongs, feel free to post it below! We’re always looking to expand our self-service resources and would love to hear your feedback. -- Matt S

When I run SF_Refresh on one of my tables, I get the error "EXCEEDED_ID_LIMIT: there are '600000' or more records deleted in the specified time period" and it won’t complete. How can I run SF_Refresh on that table?

When I run SF_Replicate on my tables with lots of fields, like Account and Case, it fails. SF_Mirror also fails, and I can’t run SF_Refresh because there’s been a field added so it just tries to run SF_Replicate anyway. The message output looks like this:--- Starting SF_Replicate for Account V5.1.812:58:41: Parameters: Salesforce Account  Version: V5.1.812:58:41: DBAmpAZ 5.1.8.0 Copyright c 2021 CData Software, Inc.12:58:41: Using options specified in Options parameter: soap12:58:42: Using settings from Registry.12:58:42: Using FullCopy to replicate the Account table.12:58:42: Server: DL-7KVJRL3, Database: olddbamp, Linked Server: Salesforce12:58:42: Using the Salesforce SOAP API.12:58:43: Opening SQL Server rowset12:58:43: Error: Commit failed.Source: Microsoft SQL Server Native Client 11.0Description: The statement has been terminated.Source: Microsoft SQL Server Native Client 11.0Description: Can

Salesforce added a couple new fields to the Contact and Lead objects, GenderIdentity and Pronouns. I can find them in Data Loader and in Workbench, but DBAmp is not bringing them down. How do I get DBAmp to find those new fields? I have DBAmp v22, if that makes a difference.

Hi,I was wondering if it was possible to write and save custom code snippets which would be available for use with any script block.  I have a lot of custom scripts that I re-use often and was wondering it there was a way to build upon the default snippet list.  Either through the Arc interface or by adding my own code files within one of the Arc directories.  Thanks!

My organization is trying to enable MFA in Salesforce. Will this affect DBAmp's ability to function?

Take a look at CData Sync capabilities.

I know I have data in my source but there a 0 records returned in my destination!

I’m currently in the process of changing machines and I do have a lot of jobs and connections in my Sync machine, does Sync have a migrate feature that could help me transfer all the jobs and connections in an easier way or do I need to manually recreate all the connections and jobs?

When I try to configure a new connection in CData Sync, I click the “Create & Test” button, then an error is displayed:“Add the connection failed: attempt to write to a readonly database”

This is a continuation of the discussion in Mapping a document with parent-child relationships to CSV (small files)In the previous discussion, relative XPaths were used to map header elements to line values in the XML Map, but this can become impractical if the input XML is very large, as is often the case with large inventory reports and health care claim reports. Loading the XML document model for large files in memory can often require more memory than is available, or at least significantly degrade performance.  Fortunately, there is a feature of the XML Map that allows for efficient processing of XML files, available in the Advanced tab called XML Streaming:  When XML Streaming

I’m having an error with Sync timing out Databricks clusters when starting up, I’ve tried modifying the job timeout and connection timeout but haven’t been able to solve it.I’m also getting “Failed to check the directory status” 500 errors, what’s going on?