Skip to main content

B2B Integration

Ask questions, get answers, and engage in discussions about B2B integration solutions from CData.

247 Topics
D
David FosterApprentice
asked in CData Sync
CData Sync - Loading/Replicating data in a looping batch

Hi there.Is there a way of replicating data in batches.We are looking at a lambda approach with small incremental replication, followed up with a weekly full reload to catch late-arriving data and physical/hard deletes.The challenge is that the full reload data volumes are too large for the server hosting our CData Sync. Even with the JAVA heap adjustments.I was wondering if we could do the reload/replication in a looping process.  The source is Snowflake the target is Azure ADLS2 parquet files.

10
D
C
Cameron LEmployee
posted in CData Sync
How to Access Live Data from the WQL for a Workday Report

CData Workday Drivers and Connectors now allow users to create tables that execute specific Workday Query Language (WQL) statements for Workday reports. The new stored procedure, CreateWQLSchema, processes the output from Workday's "Convert Report to WQL" task and retrieves the same report data using the WQL statement. Read more about this Workday Driver enhancement here:Workday: How to Access Live Data from the WQL for a Workday ReportIf you have additional questions, please let us know!

180
C
A
andrew.wilsonApprentice
asked in CData DBAmp
DBAMP - Unable to apply new license to server

Hi, We recently renewed our license for another year, i have applied it to all dev and Test SQL servers, and the offline node in Prod, but our online node in prod gets the below error.  Can you please advise what to do to resolve this??  We only have a couple of days before our current license expires.ThanksAndrew

381
Ethem Q
G
gordonApprentice
asked in CData Arc
Option to remove all empty JSON nodes

Hello,I am currently working on building certain JSON messages.For doing this I am starting with an XML message that has the desired structure and place the data into this XML. Then I use the JSON Connector to convert my XML into the JSON format.So far so good.Due to the fact that sometimes the data of a node does not exist, I would like to know, if there is an option in the JSON Connector that will automatically remove all nodes that are empty in the underlying XML.For example the following XML Structure shows that some fields have values and other do not:<?xml version="1.0" encoding="UTF-8"?><Items xmlns:json="http://arc.cdata.com/ns/jsonconnector" json:array="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">  <Item>    <field1>value1</field1>    <field2>value1</field2>    <field3></field3>    <subnode1>      <subfield1>subvalue1</subfield1>      <subfield2></subfield2>    </subnode1>

524
G
S
sooryaprakash_parthibanf3Apprentice
asked in CData Sync
MSSQL to Snowflake CDC Load Task failing

Hi all,Our team is using CData Sync to extract and load data from SQL Server to Snowflake. The source MSSQL table is CDC-enabled, and we have created a CData Job with Change Data Capture enabled. We started testing the task with smaller tables (containing 100K - 300K records), and the task was successful without any issues. However, when trying with larger tables (containing 10M - 35M records), we faced failures every time.I have attached the error output for your reference. Has anyone come across this scenario? Any replies or help would be much appreciated. Thanks in advance.Error output: "2024-11-14T02:25:19.555+0000 2 [32|Q-Id] Failed to cache row [501097] The query was canceled."We tried running the same task with a few manipulations, like reducing the source table records to 1M over 10M but had no luck. We are not facing any other issues besides "Failed to cache row." What does caching mean in a CData task? Does this have something to do with the AWS AMIs? Thanks heaps,S.

391
A
A
Anonymous
posted in CData Arc
New Features Alert: CData Arc's Q1 2024 Release is Here! 🌟

Hey CData Community!We're excited to share the latest CData Arc update for Q1 2024, bringing innovative features that manyof you have been anticipating. Let's dive into what's new and start some discussions around these updates:  Enhanced User Role Granularity & Access Policies: We've upgraded user roles and access policies for better management and security. How do you see these changes impacting your workflow?   Service-Level Agreements (SLAs) for Connectors: With SLAs for connectors, you can now monitor transaction volumes more effectively. What are your thoughts on utilizing SLAs to maintain partner relationships?   Workspace-Specific Access Controls: Tailored access in different workspaces is now possible. We'd love to hear how you plan to implement this in your setup. Customizable Access Policies: This feature allows for detailed control over user permissions. What specific policies are you looking forward to setting up? SLA Implementation and Alerts: Keep track of tr

1015
R
B
bala_bkApprentice
asked in CData Arc
how to add multiple invoice in a single EDI file

Hi,I am working on EDI 856(CSV to X12).Please help me on this issue.Wanted to generate multiple transactions set in single X12 fileEXAMPLE:ISA*...GS*...ST......SE...ST.....SE...GE...IEA...

281
lohith
R
Renukachari_KaseeEnthusiast
asked in CData Sync
Unable to retrive Jira Issues table customes fields

Hi Team, kindly help me on Jira custom fields are not visible in Cdata connector. ThanksChari

674
E
B
bala_bkApprentice
asked in CData Arc
How to remove the last line in flat file using Arc script ??

How to remove the last line in flat file using Arc script ??The below script is to add the “END” on the last line but I need to remove the END which is present in the last line of a Flat-file??<arc:set attr="file.file" value="[filepath]" /><arc:call op="fileRead" item="file">  <arc:set attr="tmp.originalBody">[file.file:data]</arc:set></arc:call><arc:unset attr="output.data">[tmp.originalBody]END</arc:set><arc:unset attr="output.filename" value="940_[now()]" /><arc:push item="output" />

562
Rohit
B
bboesdoApprentice
asked in CData Sync
Missing Salesforce Objects When Creating New Sync Task

I’ve successfully created dozens of tasks to sync certain Salesforce objects to an Oracle destination.  However there is a group of Salesforce objects that are missing when I go to create a new task.  All are created/used by the same 3rd-part ISV app, but the account I am using for connecting to Salesforce is admin and I have no issue interacting with those objects outside of CData Sync.  I’ve read where this could be a cache issue, but I have not been able to find a cache refresh anywhere in CData Sync.  Would anyone have suggestions?  Thanks.

191
Ethem Q
B
bala_bkApprentice
asked in CData Arc
Arc script

how to remove the first five lines in flat file using Arc script 

402
B
J
jussihiApprentice
asked in CData Arc
Join data from 2 CSV files

Hi.I have incoming invoice CSV files with invoicer’s ID as the only metadata of the invoicer. These are single-line csv files. Let’s call them “invoice.csv”.I have other single CSV file with all of our invoicers’ metadata (including the ID). This file has some 90+k lines. Let’s call this file “invoicers.csv”How can I enrich the invoice.csv file with the correct row from the invoicers.csv file so that the invoice.csv file would contain all the metadata of the invoicer in question?Also, one of the fields in the invoice.csv is a html link to said invoice. How can I download this file with CData Arc?

674
J
M
Marcus_SApprentice
asked in CData Sync
Cdata Sync SQL Server Permissions for data loads

Hey All,I am trying to adhere to the principle of least permissions. I created a SQL Server Authenticated Login for cdata to use as a service account to SQL Server. I’m not sure what permissions it needs at the database level though? I wasn’t able to find any documentation on the site about minimum permissions cdata sync service account for SQL Server would need.I allow the cdata sync service account to drop and create tables. My guess is that I only need to add the cdata sync service account user to the following database level roles in my data lake.ddl_admindb_datareaderdb_datawriterWhat I am not sure about is whether or not Cdata Sync Service Account needs to have permissions to create Schemas.Thanks

181
Ethem Q
ericbux
ericbux
asked in CData Arc
Is the Email Send connector the end of the line?

I have a flow where I need to create two csv files and email them using the Email Send connector.I’ve done these many times and it’s super easy...until now.  I need to create and email the first CSV then set a delay (Schedule Connector?) and use a tracked header value from the first part of the flow to create and email another CSV with data from a stored procedure.In other words, once the CSV hits the Input folder of the first Email Notify it sends the email and stops.  Nothing ever shows up in the Output of the Email Notify.I suppose I could use the After Send event to trigger the Schedule, but is their another way to send the first email and continue to follow the flow and send the next email.  The key to this is that I have a tracked header value that must remain through the entire flow, as it’s my stored procedures incoming parameter.Appreciate any input!

251
A
N
nrapozaApprentice
asked in CData DBAmp
DBAmp upgrades and licensing questions

I just noticed the EOL announcement for  CData DBAmp versions prior to v22 on this page.  My organization is still on version 4.1.8. I’m interpreting this announcement to mean that version 4.1.8 will continue to work as long as we have a valid license key.  So it will still be running on say, 1/31/2025.  Is this correct? I keep getting surprised by these announcements.  Besides coming to the CDATA site and browsing around, how can I stay up to date on important news?I WILL be talking to my manager about upgrading to the most current version ASAP, but we will need time for lots of testing.

671
Ethem Q
M
marodsalApprentice
asked in CData Sync
Suggestions in how to load tables over 15 millions rows using CData Sync

What are the suggestion in how to load tables over 15 millions rows using CData Sync? The data source is ServiceNow and the destination is SQL Server. I changed the sync.exe.config to<?xml version="1.0" encoding="utf-8" ?><configuration>  <startup>    <supportedRuntime version="v4.0" />   </startup>  <!-- You can add some Java JVM parameters here. This is for Java application only.   <appSettings>    <add key="JAVA_OPTS" value="" />  </appSettings>  --><appSettings>     <add key="JAVA_OPTS" value="-Xms2g -Xmx50g"/> </appSettings>    </configuration>Also, I changed the timeout of the jobs to 6200.

511
M
A
Anonymous
posted in CData Arc
New CData Arc Q4 Release: Streamlined Partner Management, Enhanced UX & More!

Hey everyone, We’ve just released the Q4 2024 version of CData Arc, and it’s packed with features designed to make managing your trading partners and building workflows even smoother. From a centralized partner management interface, to a revamped UI for quicker workflow creation, to improved connectors and routing capabilities—this release has something for everyone. Key Highlights:• Centralized Trading Partner Management: View and manage all partner connections, documents, and dashboards in one place, making onboarding and optimizing relationships simpler than ever.• Enhanced User Experience: A refreshed UI, built-in disk usage info, and new messageOps capabilities help you work more efficiently. Expect faster workflows, clearer visibility, and a more intuitive platform.• Improved Connectivity: XML Map interface upgrades, more flexible Workspace Send options, and Dropbox Team Folder support give you greater control and flexibility in how you move data through your workflows. Your Turn

250
A
Jaclyn
JaclynEmployee
posted in CData Sync
🎄 Season's Greetings from CData! 🎄 Data Sync Q4 2024 Release Highlights

We're delighted to spread some holiday cheer with the latest enhancements to CData Sync, designed to elevate your data integration and management capabilities this festive season: 🔐 Custom Roles for Enhanced Security: Gain greater control over user access and permissions with our new Custom Roles feature. Tailor roles to fit your organization's needs and restrict the usage of customer-created objects for improved security. 🔄 Dynamics 365 Reverse ETL Destination: Seamlessly sync data back into Dynamics 365, enhancing your CRM and ERP workflows during this busy holiday period. ⚡ Enhanced CDC: Experience faster and more reliable Change Data Capture, ensuring your data is always up-to-date with improved efficiency—just in time for year-end reporting. ❄️ Snowflake Performance Improvements: Enjoy significant performance boosts in our Snowflake integration for quicker data processing and analytics, helping you unwrap insights faster. 📚 Full Feature Documentation: For a comprehensive unders

660
Jaclyn
C
CJIWApprentice
asked in CData Sync
Multiple schedules for a single Sync job?

Is it possible to create multiple schedules for a single Sync job?The use case here is that I'd like to run a job only once per day on some days of a month but then multiple times per day for some other days.One idea that comes to my mind is that I can create two jobs with different schedules, but the problem here is that if a destination table is incrementally synced (which is required for large tables), two jobs will not be able to coordinate.

463
Ethem Q
M
Marcus_SApprentice
asked in CData Sync
How to Use API to enable Automatic Job Retry

How do I use the REST API to enable the automatic job retry option? I’m not seeing that option in the API REST documentation.I am using CData Sync cross platform.

491
Ethem Q
luciaonmatteo
luciaonmatteoApprentice
asked in CData Arc
What are the security benefits of using Intel Core M vPro with CData's cloud data connectivity solutions?

Hello everyone,I’m curious about the security advantages of combining Intel Core M vPro technology with CData’s cloud data connectivity solutions. We know that Intel vPro is built with security features like Intel Active Management Technology (AMT) and hardware-based encryption, which offer enhanced protection for enterprise devices. But how does this integrate with CData's connectors, especially for cloud-based applications and databases?For instance, CData’s JDBC and ODBC drivers provide seamless connectivity to a wide range of cloud data sources. Using these in tandem with Intel security features could potentially streamline secure data access and reduce the risk of unauthorized access during data transactions.I’m particularly interested in how Intel’s built-in remote management and secure boot options could enhance the security of data connections when using CData’s cloud solutions. How do these technologies work together to ensure encrypted, secure data flows, especially in hybrid

270
luciaonmatteo
L
liostyApprentice
asked in CData Arc
What is ERP Implementation?

ERP Implementation refers to the process of deploying an Enterprise Resource Planning (ERP) system within a business. ERP systems integrate and manage core business processes, such as finance, human resources, supply chain, and customer relationship management, into a unified platform.

350
L
P
PradeepApprentice
asked in CData Arc
Accessing Files from Server

Hi Team, I was calling files from XML map connector to do cross reference from Input values to pick the respective values from list. This function working fine, and from yesterday i am facing the below error “System error: Access to the path 'C:\Users\ligEDIAdmin\Documents\SCJ\SCJCode.xlsx' is denied”  Same function works fine in Production environment, but in Test system i am facing the issue, We have worked with all network / server team and there where no changes made with access / permission levels. Still i am facing the issue, could anyone please help me out by pointing what is to be done to fix this issue. 

431
lohith
M
Mauricio BorgesApprentice
asked in CData Arc
Get MDN logs to email

How to get the log file from the folderC:\\ProgramData\\CData\\Arc\\workspaces\\[WorkspaceId]\\[ConnectorId]\\Logs\\Sent\\[Date]\\[MessageId]\\[LOG FILE NAME].mdnand send it by email after running the AS2 connector?I tried using the AS2 success connector linked to a "notify" or "send email" or "scripts"I tried in the AS2 connector in the Events tab in "Before send" to find the file and attach it to an emailI tried to read the .mdn file in batch mode and have the result in the output so I could create the attachmentall of them without success, is it possible to do this?

512
M
L
LNaylorApprentice
asked in CData Arc
Handling special characters in xpath

I’m using an Excel connector to pass a spreadsheet into a Script connector for further processing.The Excel input contains columns of the form:  A: B CIn the Script connector, I try to reference it in an xpath like this:  xpath([A:_B_C])But I get the “attribute does not exist” error. I need to deal with or escape that colon character somehow, I just haven’t been able to find where in the docs how to do that yet.Any suggestions?Thank you!

281
L
Virtual Event: Data Virtualization, Reimagined Ma”arch 14, 2024 style=

Badge winners

  • Product Expert
    David Fosterhas earned the badge Product Expert
  • Product Expert
    Marijanhas earned the badge Product Expert
  • Product Expert
    Shelly_Chenhas earned the badge Product Expert
  • Product Expert
    Doctorhas earned the badge Product Expert
  • Product Expert
    Mohsinhas earned the badge Product Expert
Show all badges
Terms and ConditionsCookie settings