Data Integration

I am still running Arc Version 23.3.8725.0 and want to upgrade to 24, but I am hesitant.  There’s been a lot of upgrades to the UI and I’m concerned about my workspaces blowing up - as I have them all carefully laid out for ease of use.Another concern I have is deprecated script functions.  For instance, I use the After Send events on dozens of connectors to trigger other flows the “old fashion way”<arc:set attr="target.ConnectorId" value="SOME_CONNECTOR" /><arc:call op="portReceiveFile" in="target" />From what I understand, this method will not work in the latest release.  This has been replaced with a more secure token-based approach:<arc:call op="api.rsc/receiveFile?" ConnectorID="SOME_CONNECTOR" httpmethod="POST" authtoken="admin:xxxxxxxxxxxxxxx" />I have so much ArcScript written I’m concerned there are other places throughout

Hello!I know that I can define an out environment variable in pre-job event and use that in the task replication query (documented here and here excellently).I’m probably stretching here but can I define that out-variable to read from a file or any other source and then run the sync task for ALL values of that variable? Kind of a loop? Thanks in advance for you answers!

When trying to mirror a salesforce object using the following: EXEC SF_Mirror 'SFPROD', 'c2g__codaBankAccount__c'; I get a failure as follow: --- Starting SF_Mirror for c2g__codaBankAccount__c V877411:34:52: Parameters: SFPROD c2g__codaBankAccount__c  Version: V877411:34:52: Running DBAmpAZ.exe.11:34:52: DBAmpAZ 22.0.8775.0 Copyright © 2023 CData Software, Inc.11:34:52: Obtaining Connection Props11:34:52: Obtaining User Information11:34:53: Using settings from Registry.11:34:53: Retrying describeSObject call to Salesforce.11:34:53: Retrying describeSObject call to Salesforce.11:34:53: Retrying describeSObject call to Salesforce.11:34:53: System.Web.Services.Protocols.SoapException: INVALID_TYPE: sObject type 'c2g__c' is not supported. If you are attempting to use a custom object, be sure to append the '__c' after the entity name. Please reference your WSDL or the describe call for the appropriate

Hey everyone!  We are thrilled to introduce the new Reporting and Visibility Package for CData Arc! This powerful feature, previously available only on our enterprise tier, is now accessible as a standalone package.  Here is what you can expect with this new package:  • Enterprise-Level Reporting: Generate detailed reports on B2B data exchange activities, including workflow performance, transaction volumes, and expected SLAs per trading partner. Gain valuable insights into your integration health. • Scheduled Reporting: Automate report generation and delivery, ensuring timely access to critical information for stakeholders across your organization. • Customizable Dashboards: Visualize key B2B gateway metrics on customizable dashboards. Monitor trends and identify potential issues proactively.  This package empowers you to make informed decisions based on real-time data, optimizing your B2B integra

How can i write to sub-directories in AzureDataLake. I am able to write to parent folder but sub-directories are not working

Is there a way to discard an empty message?I have a misbehaving HTTP endpoint which is causing me to get intermittent empty messages going through the flow.These end up clogging up the logs.It would be good if I could detect empty messages in the flow and discard them before they affect anything downstream

Hi I need to pass some XML through an XMLMap connector, but I also want to preserve the original XML input, for later use.My thought was to do this in a script before it hits the XMLMap But I don’t know how to copy the Script connector input, straight out to the output copy the Script connector input into a message headerThoughts

We have a workspace for each customer we have and there we mostly do a mapping from their interface messages to our own format.  We are not using views, but I am thinking of starting to use them to make a difference between the incoming and outgoing flow to have a better overview. Are you using views, and yes, how are you using them?  Can you also move connectors from 1 view to another view?

I have spent hours spinning my wheels on how to process an x12 278 and don’t even know where to start. My brain is fried so I really don’t have more details than that.

Hi all,I am receiving an error when I am trying to replicate a table from QuickBooks to a local SQL Server.“[0] Failed to cache row [126934] The iteratorID "{4853b873-e75a-4f1a-a4d6-2c32253237ad}" is not valid.”See the screenshot below. To try and address this I have: Decreased the Pagesize property in my Quickbooks connection to 50 from 500. Enabled Persistent connections and increased the idle timeout in my Quickbooks Remote Connector from 30 to 120.I restarted the services but none of that worked. Any suggestions?  

Hi,When I try to create a SQLServer connection for a local SQL Express install, CData Sync is throwing the following error: System error: UnknownHostException - No such host is known (DATA-DEV2\SQLEXPRESS)I am able to connect to the server with SSMS but for some reason CData cannot find it. CData and SQL Express are on the same server. What am I missing? Thanks.

(Title should read 404 not 4040) HiIf I use <arc:call op="httpGet" in="http"> , will a 404 response raise an exception which I need to catch or can I check the http:statuscode ?

HiI’d like to file a bug report for CData Arc™ 2023 - 23.4.8789.0I have an XML Connector (PrepareForJSONConversion) which was originally created using the output from one connector (MapLorealIFCSUMToET) as an Source File.That source connector was no longer needed so it was detached and then subsequently deleted at a much later date.Now the mappings in the XML Connector have gone.I have no idea if the connector is still performing the expected mappings   I can also no longer export the XMLMap connector. Expected Operation:not lose mappings if preserving mappings is not

HiIn the REST Connector, under Advanced, there is the Option “Allow ArcScript in URL”. I have two questionsDoes this also apply to the Access Token URL when using OAuth2.0 ? Would it support dynamic endpoints and formatters like this[Vault('ENVIRONMENT') | equals('TEST', 'https://test-api.domain.com/api/SendTracking', 'https://api.domain.com/api/SendTracking')] 

Hi,I have a flow and would like to take in and forward a csv file in said flow. In the flow, the end product should however have the 2nd column values translated. Say we have a python dict for translations:translations: Dict[str, str] = { "08.0": "S", "08.1": "O", "08.3": "S", "08.4": "O", "08.6": "S3", "09.0": "T", "10.5": "LS", "10.6": "LS"}What I would like to do would be to check each row’s 2nd column if a key from this dictionary is present in the said 2nd column (substring, doesn’t have to be 100% match), and then change the row so that the 2nd column would be rewritten to the value of the key found from the translation dictionary.If the row’s 2nd column’s text doesn’t match to any of the dictionary keys, the row should be omitted completely. I can do this really easily in Python. But how can I achieve

I am currently using cData Sync to replicate data from NetSuite Tables to Snowflake. However, I am encountering issues with two critical tasks within my job, which consists of 24 tasks in total.The error messages for the failing tasks are as follows: 1.[0] [500] Could not execute the specified command: Can not view record 2.[0] [500] Could not execute the specified command: Failed to cache row [5247] Can not view record StackTrace: at aso230l.nT.Next() at aso230l.mH.Next() at aso230l.mb.Next() at aso230l.mY.Next() at aso230l.mQ.Next() at aso230l.Kz.q(DbCommand , DataTable , IResultSet , ColumnInfo[] , Int32[] , jsb ) These errors are hindering the successful completion of my data replication process, and it is critical for my project that these tasks run smoothly.Please advise .

I am having an issue with my sync installation. I am able to successfully start logminer on the oracle instance and query the results of that logminer session.However, when I run a CDC job from Sync, it will complete the initial data load successfully but then fail on the incremental loads after the initial one with the below error:logminer failed to start If I have it delete the table ot truncate the data then it reloads just fine. It just won’t start logminer to run the CDC loads.Has anyone run into this issue or found a way to resolve this so I can get CDC loads from oracle to sql server working correctly? 

Replicating Workday Data: The ChallengesReplicating Workday HCM and Financials data can be a challenging task in itself, especially when the Workday APIs are so vast, support a variety of data models, and offer different types of authentication mechanisms.With CData Workday connectivity solutions, we simplify the complexities of the backend APIs and offer you a relational view of all your Workday that can be queried using SQL. CData Workday connectors support a variety of Workday connections through:Workday SOAP APIs Workday REST APIs Workday RaaS (Reports-as-a-Service) Workday WQLTo learn more about the seamless Workday connectivity, CData offers, please refer here: https://cdn.

I’m working with a few different GraphQL endpoints where are becoming much more common it seems in B2B applicaitons. While GraphQL still uses just REST, formatting payloads in Arc is a bit difficult other than just using plain text and script port:For example:https://shopify.dev/docs/api/admin-graphql/unstable/queries/order?language=cURL GraphQL looks like: query { node(id: "gid://shopify/Order/148977776") { id ... on Order { name } }} but to support this in Arc, I need to use a script port with a string of: {"query": "query { node(id: "gid://shopify/Order/148977776") { id ... on Order { name } } }Wh

Given the increasing reliance of companies on their IT infrastructure to maintain competitiveness and fulfill customer needs, effectively managing and monitoring their IT environments can pose a significant challenge in the absence of appropriate tools.Datadog is a cloud-based monitoring and analytics platform that helps organizations gain real-time visibility into their IT infrastructure.This article consists of detailed steps related to enablement of the integration between CData Sync and Datadog.NOTE: This article can be taken into account for other CData applications that auto-generate logfiles from their operational activities (i.e. “CData Arc”, “CData Connect Server”) Step 1: Install the latest version of Datadog Agent (assuming that you already have a Datadog account)

How do I convert a JSON string to XML in ArcScript / Script connector?

I have the files going into the connector every 10 mins and that create a ton of logs in the Input tab.I already have the Save to Send Folder and Log Messages disabled for the MySQL connector but it is still taking up disk space.  what would be the best way to handle the clean up process for the MySQL Connector besides the Cleanup Option in the Advanced settings?

We're thrilled to announce the latest enhancements to CData Sync, now with even more powerful features to supercharge your data integration and management:  🌐 Sync Cloud - Our renowned Sync product, previously only available on-premise, has now ascended to the cloud! Managed directly by us and unlimited volume allow... experience the full power and scalability of cloud hosting.  💾 S3: Enhanced Incremental Replication - We've significantly enhanced our S3 Incremental Replication Load from Folder feature by fully supporting directory paths in S3. This update allows for more precise and flexible data management, enabling you to efficiently manage and synchronize your data across even the most complex folder structures.  🔄 CDC from Dynamics 365 - Keep your data fresh with our new Change Data Capture (CDC) support for Dynamics 365. Capture and sync changes in real-time, ensuring your data remains current and accurate.  📚

Hello,A PDF file was made from the XLSX document. Then I created a PDF template with form fields (Adobe Acrobat), using the “Text field” function Multi-line. By filling in the form manually, the text in the field moves to a new line, but the text taken from the XML doesn't move to a new line. Why the Multi-line functionality doesn't work in CData Arc? 

My CData Sync servers are running on Azure. I’d like to use Azure Monitor Agent to monitor if CData Sync service is in stopped state. Usually Windows Service monitoring can be queried from Windows System logs finding Event ID 7036. CData Sync seem not to write anything to System Log (or I just not find it). Where should I try to find information if service is stopped?