Data Integration

Is it possible to create my own Operations that I can call via <arc:call>? Or simple reusable methods/functions using <arc:include> perhaps that can return data? Or?

I have a trading partner who’s sending EDI with a file extension (.xml) and a Content-Type: text/plain and they’re also sending EPCIS xml files which are also of Content-Type: text/plain.  I need to determine which files are legit xml/edi and route them accordingly.I use a script on my EDI files to add additional headers, but I don’t think this will work on the xml EPCIS files because they are not EDI.<!-- code I use to add additional headers to edi files --><arc:set attr="edi.File" value="[FilePath]" /><arc:call op="x12Scan" in="edi" out="edi"> <arc:enum item="edi"> <arc:set attr="output.header:[_attr]" value="[_value]" /> </arc:enum></arc:call><arc:set attr="output.filepath" value="[filepath]" /><arc:push item="output" />Since the EPCIS files are true xml, I feel like this will bomb on the op=”x12Scan”.  A

Fall is here and so is CData Sync’s Q3 2025 release! Our latest updates are designed to help you unlock more value from your data, faster and more efficiently. ⚡ Enhanced CDC Catalog Expansion: You asked, we delivered Sync now supports DB2 protocol AS400/iSeries and MySQL for even broader data replication capabilities! 🔄 Reverse ETL Catalog Expansion: New destinations now supported, including Salesforce Pardot, Salesforce Marketing Cloud, Sage Intacct, and Veeva Vault to enhance your reverse ETL capabilities ✨ Open Delta Table Support: Now available for all your favorite data lake destinations such as: Amazon S3, Azure Blob Storage, Google Cloud Storage, and Azure Data Lake Storage powering your delta integrations across Databricks, Fabric, and more! ⚡ Faster File Replication: CSV, Avro, Parquet, and One Lake Mirroring are now 90% faster for smoother large-workload handling. 🛠 Generic JDBC Connector: Unlock source-only connectivity and schema disco

I am trying to use httpGet to retrieve binary files, then push them through via a Script connector: <arc:set item="httpGetInput" attr="url" value="[params.pdfUrl]" /><arc:call op="httpGet" in="httpGetInput" out="response"> <arc:set attr="output.data" value="[response.http:content]" /> <arc:set attr="output.filename" value="test.pdf" /> <arc:push item="output" /></arc:call>It downloads the file, and sends it through, however the file is garbled, I assuming this is a encoding problem somewhere.Downloading the files directly from the source URL, the files render fine.Said connector is inside an API Request, with Response content type set to “application/pdf”:There must be something I am missing.

I am new to arcESB and we have a basic requirement where we need to call a rest service, get the data in json format and same data i need to use to later while using smtp connector (filename need to be modified). I am unable to map the data, i tried using arcScript in event section of rest connector but it seems to be not working. Any suggestion will be really appreciated. thanks in advance. 

We are trying the map multiple fields in a XML (EDI) into one field in another XML in a XML Map connector. We use the implode formatter for this. But it isn’t working, we only get the first item. But we want all the NTE02 fields joined with a newline as separator. This is the script we use:[xpath("FunctionalGroup/TransactionSet/TX-00401-940/NTE/NTE02") | implode] Source XML:<FunctionalGroup>            <TransactionSet>        <TX-00401-940 type="TransactionSet">            <NTE type="Segment">                <!--Note                Reference Code-->                <NTE01><!--Warehouse                    Instruction-->WHI</NTE01>                <!--Description-->                <NTE02>PALLETS COVERED WITH STRETCH HOOD</NTE02>            </NTE>            <!--Note/Special            Inst

Is there a better way to check for null results from a dbQuery, I feel dirty for coming up with this: <arc:set attr="db.query">select null as something;</arc:set><arc:call op="dbQuery" in="db" out="results"> <arc:if exp="'[results.*]' == '<table></table>'"> <!-- handle null --> </arc:if> ...</arc:call> 

This works with a REST connector just fine: However with a SOAP connector it does not (fails with  _message specified in the formatter, does not exist): Both connectors do have the correct header payload. Both have Allow Script in Url checked. Any ideas what I am doing wrong?

I’m trying to deploy an SSIS package to my server, and receiving the error “file:FileService SFTP Client Error during operation 'DOWNLOAD': The specified directory could not be created” server side.This runs without any issues from my developer. The service account has access to the folder (I also tried running the package with my AD account and got the same error). I’m kind of running out of ideas where to look. Does anyone else have ideas where to look?

Most ETL/ELT tools provide connectivity to Workday; however, the connection is limited because it is only through Reports as a Service (RaaS). With the CData Sync Workday connector, users get more connection options, including support for:Prism Data Catalog Workday Query Language (WQL) RaaS REST API SOAP APIOur latest article reviews how the Workday connector for CData Sync delivers full connectivity when compared to other ETL/ELT tools' connectors.Read the full article: https://www.cdata.com/kb/articles/workday-sync.rst

CData Arc supports cloud deployment on Microsoft Azure through Azure Virtual Machines and Azure Web Apps. Run CData Arc with the flexibility to choose between full control on VMs or simple management with Web Apps. You can follow our guide for both deployment options here: https://arc.cdata.com/kb/articles/azure-deployment.rst

Error:Hi everyone, New to the community, within our team we are responsible for a ARC server (CData Arc™ 2024 - 24.3.9159.0) and I am running into the following issue: “Connection open failed: [500] Unable to connect to databasename.database.windows.net:1433: System error: No such host is known” (Azure SQL)Strange thing is that it works normally, but just sometimes it doesn't. This connector has almost 6000 entries in the last 5 days, only 8 have this error.These 8 are 4 Messages, with a retry of 60 minutes and a retry count of 1. Making that these 4 messages were tried 2 times in total. Last time 1,5 hour prior to this post. Trying manual send and boom all messages processed.SQL Database show no health warning and I can't find any metric that would indicate a problem with the database. Connector type is a script.

We’ve recently upgraded to the latest version of dbamp. Since doing so we are constantly seeing the following error on one of sf objects when we bring it down to SQL via the “exec sf_refresh” command. Executing the query "DECLARE @IncludeArchiveDeleted  BIT, @Table VARCHA..." failed with the following error: "Cannot fetch a row from OLE DB provider "MSOLEDBSQL" for linked server "SalesForce_ls".". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.   We only get this error on this one particular object. Task is probably the largest object in salesforce whose data we bring down locally to our SQL environment. Any help would be very much appreciated. Thanks and have a great day.  Running DBAMP 22.0.8991.0 version

I am interested in learning ARC Script for cdata arc and I am from non programming background,  Can you please suggest me any pre requisite (like any programming concepts should know )required before starting learning ARC Script 

When attempting to use SF_Metadata to retrieve metadata from CustomField, the MetadataXML column is never populating. The Member and Id columns are populating when I use the List operation, and executing the proc with the Retrieve operation appears to succeed, but the MetadataXML column remains NULL.CustomObject and the other tooling objects I’ve worked with have worked fine.This behavior is not confined to a single org. I have confirmed it against three orgs so far.I’ve tried querying CustomField directly in a query using the four-part object identifier, but I get an errorMsg 7314, Level 16, State 1, Line 1The OLE DB provider "MSOLEDBSQL" for linked server "<linked server name here>" does not contain the table ""CData"."Salesforce"."CustomField"". The table either does not exist or the current user does not have permissions on that table.I am able to successfully query the CustomField object in the

Hi Team, I have requirement, where  I need to send back the response back to customer, Below is message we will be receiving from customer, via Webhook connector.    {     "transId": "PUTO300000308",     "whId": "OMEGA-TEST",     "ownerCode": "199501010666",      "jobType": "11",     "jobCode": "010000000408",     "status": "ACTIVE",     "eventAt": "2025-07-03 12:34:56",     "containerId": "00493460370000410856",     "jobGroupCode": "03072026"    }  The response customer is expecting are like below, {"transId": " PUTO300000308","status": "ok","message": ""}  I need pick the transId value from the received data and built the response back with status “Ok” to them. Please help to advice how I need achieve this.

Hi, since a month we are getting this error:08:06:39: Using the Salesforce SOAP API. [SQLSTATE 01000]08:06:39: Create AuraDefinitionInfo_Previous20250630080639716 with new structure. [SQLSTATE 01000]08:06:40: Error System.Web.Services.Protocols.SoapException: EXCEEDED_ID_LIMIT: AuraDefinitionInfo does not support queryMore(), use LIMIT to restrict the results to a single batch [SQLSTATE 01000]at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall) [SQLSTATE 01000]at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters) [SQLSTATE 01000]at DBAmpAZ.sforce.SforceService.query(String queryString) [SQLSTATE 01000]at afo220A.iA.s() [SQLSTATE 01000]08:06:40: Drop if it exists. [SQLSTATE 01000]08:06:40: Error: Unable to drop the table. [SQLSTATE 01000]08:06:40:

The CData Arc Team is excited to announce our Q2 2025 release, with powerful enhancements to connectivity, visibility, and security! What’s New? The Q2 2025 release focuses on improving EDI metadata awareness, enhancing API integration, and reinforcing platform governance. Our overall goal with this release is to accelerate workflow design and give you greater visibility into your trading partner network. Key Highlights:  Enhanced EDI control: EDI workflows are now metadata-aware, allowing you to track and route documents based on information in document headers.  Improved REST API integration: Use Swagger doc imports to easily integrate with REST APIs.  Extended governance and security: We’ve added SAML 2.0 integrations with identity providers, extended RBAC, and

We’re excited to share the latest updates in CData Sync, now live with powerful new features designed to elevate your data integration experience: 🔁 Microsoft Mirroring as a Destination In partnership with Microsoft, Sync now supports OneLake Mirroring, enabling direct ingestion into the Microsoft Fabric ecosystem. With native Parquet support and catalog integration, it's a seamless way to accelerate your lakehouse strategy. 📈 CDC for SAP ERP Deliver near real-time replication from SAP ERP systems using RFC, expanding our enterprise-grade CDC coverage. 🔐 Stronger Identity & Access Management Enjoy a smoother, more secure experience with revamped SAML integration and 2FA 🌟 Coming in July!  New Reverse ETL Destinations: Pardot, Sage Intacct, and Salesforce Marketing Cloud  Open Table Format Support: support for DeltaLake on AWS, Azure, and GCP.  📚 View full release details in th

Hey CData Community!Looking to bring your Salesforce data into SAP Datasphere without the headaches? Our latest post, Replicate Salesforce Data to SAP Datasphere with CData Sync, walks you through the full setup — from database user creation to job execution — so you can get your external data flowing into Spaces fast.No native connector? No problem. With CData Sync, you can plug in Salesforce (or any of 270+ sources) and start building with real-time, reliable data. Check out the full guide here: https://www.cdata.com/kb/articles/sync-sap-datasphere.rst

Hey CData Community!Manually enabling Change Tracking for every Dynamics 365 entity? That is a lot of effort.Good news. CData Sync can do it for you using a stored procedure.It automatically enables Change Tracking for the right entities so you can set up incremental replication without the manual work.Want to see how it’s done? Check it out here: https://www.cdata.com/kb/articles/dynamics365-change-tracking.rst

Hey CData Community!Just dropped: a quick deep-dive on the new Enhanced CDC feature in CData Sync• 9× faster replication (10 k changes: 6 s vs 54 s)• Always-on log reading + staging area = reduced wait at run time• Step-by-step setup, benchmarks, and diagrams inside Refer to the following link to read the post: https://www.cdata.com/kb/articles/sync-enhanced-cdc-performance.rst

Hi Everyone,I have a scenario to use Azure Managed SQL Instances and these are not supported for DBAmp because they do not allow the installation of 3rd-party linked servers on the Instance. So I am trying to install dbamp remotely - to a on-prem SQL server virtual machine. We will create a Azure SQL managed instance where data will be stored. - DBInstanceDbAmp will be configured as linked server  in a on premise virtual machine with SQL server installed. - DBAmpServer I have two questions:Can we run DBAmp from DBAmpServer to move the data from DBInstance to Salesforce? The article DBAmp - Running DBAmp Remotely (cdata.com) mentions the other way around, but we need a bi-direction data transfer between DBInstance and Salesforce. The article mentions that a subset of stored procedures will be av

Hi,I have 3 years worth of data. All the data stay the same but the latest 3 months are dynamic. I need to replace the old data with the latest data. But if I do “Drop Table”, it drops all the data and bring all 3 years worth of data and sometimes the system dont seem to be able to handle it.  I would like to keep everything and just have the last 3 months tables dropped and replaced. I dont want the system to have to bring ALL data from the past 3 years. I just need the last 3 months dropped and replaced.  What setup should I do?

<arc:set attr="input.uri" value="[FilePath]"/><arc:call op="xmlOpen" input="input">  <arc:set attr="xml.handle" value="[handle]" /></arc:call><arc:set attr="xml.map:lastupdate" value="/Items/gtm_req_pr_details_ib_pr/lastupdate"/><arc:set attr="xml.map:till_date"     value="/Items/gtm_req_pr_details_ib_pr/till_date"/><arc:try>  <arc:call op="xmlDOMGet" in="xml" out="out1">    <arc:set attr="data.lastupdate" value="[out1.lastupdate]"/>    <arc:set attr="data.till_date"     value="[out1.till_date]"/>  </arc:call>  <arc:set attr="http.URL" value="https://api.com/purchaserequisitions?approvalLeftDate=[data.lastupdate]&approvalRightDate=[data.till_date]" />  <arc:set attr="http.header:name#" value="X-AUTH-TOKEN" /><arc:set attr="http.header:value#" value="xcxcxcxcdxcxcccccccc"/><arc:set attr="http.header:name#" valu