The Jobs feature, formerly known as the Caching feature, is a key component of CData Connect Cloud. It allows users to select tables for caching, saving them to a PostgreSQL database. Once the tables are selected, CData Connect Cloud takes over, retrieving data for these tables on a regular schedule (previously set). This process ensures that users always have swift and reliable access to their data.  

This feature is particularly beneficial when dealing with drivers that may be slow in retrieving live data, as it significantly enhances the performance of data connections. By retrieving and storing data in advance, it eliminates the need for real-time data retrieval, thereby reducing latency and improving efficiency. 

Jobs  

The Jobs (Caching) feature of CData Connect Cloud offers a dedicated page for managing and executing cached jobs. When data caching is enabled, CData Connect Cloud performs database queries against the cached data instead of the live data, ensuring efficient retrieval. The Jobs page displays a list of caching jobs that can be run. Users can select multiple jobs and initiate them instantly by clicking on Run.

Additionally, users can delete multiple jobs at once by selecting them and clicking on Delete. It’s important to note that deleting a job also removes the corresponding cached data.  

Incremental Updates 

This feature enables incremental updates, thereby enhancing performance. Upon creating a caching job, CData Connect Cloud attempts to identify columns in the database that are date- or datetime-based, which indicate the rows that are new or have been modified since the last execution. Consequently, Connect Cloud can execute the caching job solely for these rows. If this is not possible, Connect Cloud performs a complete update.

Steps to configure Jobs in CData Connect Cloud 

This section outlines the detailed processes for caching any source data into the client’s PostgreSQL database using CData Connect Cloud. This approach shows how the client data can easily be queried and accessed without directly querying the source for live data, thereby saving execution time, and improving efficiency in data analysis. The steps also demonstrate how the caching feature can be scheduled, and further edits can be made according to customer-specific use cases. 

1. Configure PostgreSQL to store cached data 

To utilize the caching feature, you must set up a PostgreSQL database to store your cached data. Here’s how to establish a PostgreSQL connection for your cache: 

1. Select Setup Cache Connection. This will lead you to the Add PostgreSQL Connection page.  
2. Configure authentication for your PostgreSQL connection. For detailed instructions, refer to the PostgreSQL connection guide. 
3. Choose Save & Test. You’re now ready to cache data to the PostgreSQL database.

Remember, you can modify your PostgreSQL connection whenever you wish by selecting Manage Cache Connection. Be aware that any changes to your connection will clear any existing data in the cache. 

NOTE: The caching feature does not apply to relational databases. 

2. Add a Job 

To add a new job, complete the following steps: 

1. On the Jobs (Caching) page, click Add Job. This action displays the Add Jobs dialog. There are three steps to adding a job: Select Connection, Add Tables and Schedule Jobs. You have the option to go back to any of the earlier stages by either selecting the stage from the header or by pressing the Back button.  
2. Select a connection from the list of your connections in the dialog (Step 1), and then click Next.
    
3. Choose the tables from your connection that you wish to include in the job (Step 2). If the root connection is selected, it will automatically select all the tables.
   CData Connect Cloud shows the total count of selected datasets. Proceed by clicking Next.
4. Choose the frequency for data updates, which can be set in terms of hours, days, weeks, or months (This is Step 3). 
5. Choose a Cache Scheme, either full or incremental, and set the Time Check. 
   The Time Check Column is exclusively for incremental updates. Select a column in your table that CData Connect Cloud can utilize to identify new rows or rows that have been modified since the last execution. This is usually a column with date- or datetime-based data. 
   NOTE: In the absence of a time check column in your data, CData Connect Cloud will only carry out full updates. 
6. Once the job is created, click on Confirm to save it. If you wish to make modifications, you can either click on Back or select one of the prior steps in the header.
   The job you created will be displayed in the caching queue. Each table that you have chosen for a specific data connection is stored as an individual job in the caching queue.

3. Edit a Job 

At any given time, you can alter or remove a job by selecting the edit or delete symbols in the list. Also, you can manually execute a job whenever you wish by clicking on Run Now.

On the Edit Job page, you can: 

• Change the caching scheme, such as changing full updates to incremental updates.  
• View the run history and status, making sure the job was successful. 
• Change the log verbosity.  
• Run a single job manually. You can run multiple jobs in the main Jobs (Caching) screen. 
• Turn off caching. Note that if you turn off caching, queries go to the live data source until you turn caching back on.  
• Refresh the cache. Refreshing the cache deletes all previously stored data and can take some time to complete. 
• Delete the job. This deletes all the cached data in your database. You can delete multiple jobs in the main Caching screen.

Advantages of caching 

Some advantages of using the caching feature include (but are not limited to) the following: 

• Speed: Caching significantly accelerates data retrieval by storing a subset of data in a high-speed storage layer, making it faster than accessing the data’s primary storage location.  
• Reduced server load: By storing frequently accessed data in cache, the load on the server is reduced as it minimizes the need to constantly access the database. 
• Cost efficiency: Caching can lead to cost savings by reducing the need for additional resources to support the same scale as traditional databases and disk-based hardware.  
• Improved user experience: Faster data retrieval and reduced server load lead to quicker response times, providing a smoother and more satisfying experience for users. 
• Scalability: Caching helps in building scalable, high-performance systems by offloading database resource usage while improving data retrieval speed.

Caching use cases 

Some popular use cases of caching include: 

• Web applications: Caching is extensively used in web applications to store the results of database queries, computationally intensive calculations, API requests/responses, and web artifacts such as HTML, JavaScript, and image files. This significantly reduces latency and improves IOPS for many read-heavy application workloads.  
• Content delivery networks (CDNs): CDNs utilize caching to store static content such as images, videos, and installation files across distributed edge servers closer to users. This enhances the speed of content delivery and improves user experience. 
• Microservices: In a microservices architecture, caching improves performance by storing frequently accessed data, reducing the need for constant database access. This leads to faster response times and a more efficient system.  
• Database caching: Database caching involves storing frequently accessed or computationally expensive data from a database in a cache to improve the performance and efficiency of data retrieval operations. This reduces the need to repeatedly query the database for the same data, leading to faster response times and lower load on the database.

Try CData Connect Cloud today 

To get governed access to hundreds of SaaS, Big Data, and NoSQL sources for live data consumption and analysis and to utilize the caching feature for faster data access, query execution, and analysis with your favorite tools, sign up for a free 30-day trial of CData Connect Cloud!