Understanding CData Log Files – Drivers

  • 21 February 2024
  • 0 replies


About CData Driver Log Files 

CData Driver log files capture communications between client applications and data sources, providing valuable information for troubleshooting error messages or addressing unexpected behavior. These logs serve as detailed records of the driver's activities, documenting each step of the process. Reviewing these log files allows users to gain a deeper understanding of the driver's actions, facilitating the identification and diagnosis of issues. Therefore, the log files play a crucial role in enhancing the overall troubleshooting and maintenance capabilities of the CData driver.

How to Generate Log Files 

To generate log files, the user sets the Logfile property in the driver to a valid file location. The driver creates the file automatically if it doesn’t exist, but it won’t generate the directory for the specified file; hence, ensure that the directory already exists. Additionally, the driver requires permission to write to the specified file location. Ideally, the user should designate a directory without any access restrictions. A log file location must offer read/write access and be easily accessible to all users without encountering permission issues. For instance, an advisable log file location could be: 

For Windows: C:/Users/Public/Documents/log.txt  

For Linux/Mac: /tmp/log.txt 

Fig: Assign the file location in the Logfile property of Advanced Connection settings  

Characteristics of CData Log Files

Basic Logging 

You simply need to set two connection properties to begin capturing driver logging. 

Logfile: The file path designates the name and location of the log file. 

Verbosity: This is a numerical value (1-5) that determines the amount of detail in the log. 

MaxLogFileSize: When the system reaches the limit, it creates a new log in the same folder with the date and time appended to the end. The default limit is 100 MB. If the specified value is lower than 100 kB, the system uses 100 kB as the value instead. 

MaxLogFileCount: The string specifies the maximum file count of log files. When the limit is reached, the system creates a new log in the same folder, appending the date and time to the end, and deletes the oldest log file. The minimum supported value is 2. A value of 0 or a negative value indicates no limit on the count. 

Once you set these properties, the driver populates the log file while performing various tasks, such as authentication or query execution. If the specified file doesn't exist, it creates it.

Advanced Logging 

Refine the exact information recorded in the log file. This can be accomplished using the LogModules property. This property allows you to filter the logging using a semicolon-separated list of logging modules. 

All modules are four characters long. Please note that modules containing three letters have a required trailing blank space. The available modules are: 

  • EXEC: (Query Execution) Includes execution messages for original SQL queries, parsed SQL queries, and normalized SQL queries. Query and page success/failure messages appear here as well. 
  • INFO: (General Information) Includes the connection string, driver version (build number), and initial connection messages. 
  • HTTP: (HTTP Protocol messages) Includes HTTP requests/responses (including POST messages), as well as Kerberos-related messages. 
  • SSL: SSL certificate messages. 
  • OAUTH: OAuth-related failure/success messages. 
  • SQL: Includes SQL transactions, SQL bulk transfer messages, and SQL result set messages. 
  • META: Metadata cache and schema messages. 
  • TCP: Incoming and Ongoing raw bytes on TCP transport layer messages. 

An example value for this property would be: LogModules = INFO; EXEC; SSL; SQL; META

Note that these modules refine the information as it is pulled after taking the Log Verbosity into account (explained in the next section).

Log Verbosity 

The verbosity level determines the amount of detail that the driver reports to the Logfile. Verbosity levels from 1 to 5 are supported. These are described in the following list: 

1. Setting Verbosity to 1 log the query, the number of rows returned by it, the start of execution the time taken, and any errors. 

2023-11-30T11:32:20.819+0530   1   [6|Q-Id]   [INFO|Connec] Opened Snowflake connection. Version: 23.0.8609.0. Edition: [JDBC Driver]. 
2023-11-30T11:32:20.821+0530 1   [6|Q-Id]   [INFO|Connec] Connection String: authscheme=Password;cachelocation="%APPDATA%\CData\Snowflake Data Provider"; location="C:\Users\DibyenduDatta\AppData\Roaming\CData\Snowflake Data Provider\Schema";logfile=D:\attach5\snowflakelog1.txt;;verbosity=1;warehouse=SAMPLE_WAREHOUSE 
2023-11-30T11:32:21.196+0530 1 [6|Q-Id] [EXEC|Messag] Executed query: [SELECT VALUE FROM sys_sqlinfo WHERE NAME = 'NUMERIC_FUNCTIONS'] Success: (3 ms)
2023-11-30T11:32:21.199+0530 1 [6|Q-Id] [EXEC|Messag] Executed query: [SELECT VALUE FROM sys_sqlinfo WHERE NAME = 'STRING_FUNCTIONS'] Success: (3 ms)
2023-11-30T11:32:21.201+0530 1 [6|Q-Id] [EXEC|Messag] Executed query: [SELECT VALUE FROM sys_sqlinfo WHERE NAME = 'TIMEDATE_FUNCTIONS'] Success: (1 ms) 
2023-11-30T11:32:21.603+0530 1 [6|Q-Id] [META|Schema] Executed  catalog query Success: 10 results (400 ms)

2. Setting Verbosity to 2:   logs everything included in Verbosity 1, including cached queries, along with additional data related to the request, such as the driver's interaction with system tables for metadata retrieval, the transmission of HTTP requests, reception of responses, and other relevant information.

2023-11-30T11:35:43.617+0530	1   [11|Q-Id]   [INFO|Connec] Opened Snowflake connection. Version: 23.0.8609.0. Edition: [JDBC Driver]. 

2023-11-30T11:35:43.618+0530 1 [11|Q-Id] [INFO|Connec] Connection String: authscheme=Password; cachelocation="%APPDATA%\CData\Snowflake Data Provider";credentialslocation="%APPDATA%\CData\Snowflake Data Provider\CredentialsFile.txt";location="C:\Users\DibyenduDatta\AppData\Roaming\CData\Snowflake Data Provider\Schema";logfile=D:\attach5\snowflakelog1.txt; oauthsettingslocation="%APPDATA%\CData\Snowflake Data Provider\OAuthSettings.txt"; verbosity=2; warehouse=SAMPLE_WAREHOUSE

2023-11-30T11:35:43.618+0530 2 Connection Property: user = CDataSoftware
2023-11-30T11:35:43.620+0530 2 [11|Q-Id] [HTTP|Req: 57] POST https://il76494.us-central1.gcp.snowflakecomputing.com/queries/v1/query-request?requestId=f2624788-0746-4271-a1fa-deb5781f2ed6
2023-11-30T11:35:43.978+0530 2 [11|Q-Id] [HTTP|Res: 57] HTTP/1.1 200 OK, 1559 Bytes Transferred
2023-11-30T11:35:43.978+0530 2 [11|Q-Id] [HTTP|Res: 57] Request completed in 359 ms.
2023-11-30T11:35:43.980+0530 2 [11|Q-Id] [EXEC|Parsed] Executing query: [SELECT VALUE FROM sys_sqlinfo WHERE NAME = 'NUMERIC_FUNCTIONS']
2023-11-30T11:35:43.981+0530 1 [11|Q-Id] [EXEC|Messag] Executed query: [SELECT VALUE FROM sys_sqlinfo WHERE NAME = 'NUMERIC_FUNCTIONS'] Success: (2 ms)
2023-11-30T11:35:43.985+0530 2 [11|Q-Id] [META|Schema] Executing catalog query
2023-11-30T11:35:43.987+0530 2 [11|Q-Id] [HTTP|Req: 58] POST https://il76494.us-central1.gcp.snowflakecomputing.com/queries/v1/query-request?requestId=d2713fe5-1c0d-47b7-9964-9b2489ec96ea
2023-11-30T11:35:44.357+0530 2 [11|Q-Id] [HTTP|Res: 58] HTTP/1.1 200 OK, 1729 Bytes Transferred
2023-11-30T11:35:44.358+0530 2 [11|Q-Id] [HTTP|Res: 58] Request completed in 371 ms.
2023-11-30T11:35:44.361+0530 1 [11|Q-Id] [META|Schema] Executed catalog query Success: 10 results (376 ms)
2023-11-30T11:35:44.361+0530 1 [11|Q-Id] [EXEC|Messag] Executed query: [SELECT * FROM sys_catalogs] Success: (376 ms)

3.Setting Verbosity to 3 includes everything above, plus HTTP headers and the body of the request and the response.

2023-11-30T11:39:06.287+0530	1  [18|Q-Id]  [INFO|Connec] Opened Snowflake connection. Version: 23.0.8609.0. Edition: [JDBC Driver]. 

2023-11-30T11:39:06.288+0530 1 [18|Q-Id] [INFO|Connec] Connection String: cachelocation="%APPDATA%\CData\Snowflake Data Provider"; cachemetadata=false; cachetolerance=600;credentialslocation="%APPDATA%\CData\Snowflake Data Provider\CredentialsFile.txt";customstage=''; location="C:\Users\DibyenduDatta\AppData\Roaming\CData\Snowflake Data Provider\Schema";logfile=D:\attach5\snowflakelog3.txt;logmodules='';maxlogfilecount=-1;maxlogfilesize=100MB;verbosity=3;warehouse=SAMPLE_WAREHOUSE

2023-11-30T11:39:06.290+0530 2 [18|Q-Id] [HTTP|Req: 76] POST https://il76494.us-central1.gcp.snowflakecomputing.com/queries/v1/query-request?requestId=c2954eaa-925a-4bff-ae89-01400ebfd14d

2023-11-30T11:39:06.291+0530 3 [18|Q-Id] [HTTP|Req: 76] [Reused]
[HTTP Headers]
HTTP Auth Scheme: 2
POST /queries/v1/query-request?requestId=c2954eaa-925a-4bff-ae89-01400ebfd14d HTTP/1.1
Host: il76494.us-central1.gcp.snowflakecomputing.com
Authorization: ******
Accept: application/snowflake
Accept-Encoding: gzip, deflate
User-Agent: CData Data Provider Engine - www.cdata.com – gzip
Content-Type: application/json
Content-Length: 183
Content-encoding: gzip

2023-11-30T11:39:06.631+0530 3 [18|Q-Id] [HTTP|Res: 76]
[HTTP Headers]
HTTP/1.1 200 OK
Date: Thu, 30 Nov 2023 06:09:05 GMT
Content-Type: application/json
Cache-Control: no-cache, no-store
Vary: Accept-Encoding, User-Agent
X-Country: India
X-Frame-Options: deny

2023-11-30T11:39:06.633+0530 3 [18|Q-Id] [HTTP|Res: 76] {"data" : {"parameters" : [{"name":"CLIENT_PREFETCH_THREADS","value":4},{"name":"TIMESTAMP_OUTPUT_FORMAT","value":"YYYY-MM-DD HH24:MI:SS.FF3 TZHTZM"},{"name":"TIME_OUTPUT_FORMAT","value":"HH24:MI:SS"},{"name":"CLIENT_RESULT_CHUNK_SIZE","value":160},{"name":"TIMESTAMP_TZ_OUTPUT_FORMAT","value":""},{"name":"CLIENT_SESSION_KEEP_ALIVE","value":false},name":"ENABLE_STAGE_S3_PRIVATELINK_FOR_US_EAST_1","value":false},{"name":"CLIENT_RESULT_PREFETCH_THREADS","value":1},{"name":"TIMESTAMP_NTZ_OUTPUT_FORMAT","value":"YYYY-MM-DD HH24:MI:SS.FF3"}]

4.Setting Verbosity to 4 includes everything above plus transport-level communication with the data source. This includes SSL negotiation. 

2023-11-30T17:21:09.208+0530	 1   [35|Q-Id]   [INFO|Connec] Opened Snowflake connection. Version: 23.0.8609.0. Edition: [JDBC Driver]. 

2023-11-30T17:21:09.209+0530 1 [35|Q-Id] [INFO|Connec] Connection String: authscheme=Password; cachelocation="%APPDATA%\CData\Snowflake Data Provider"; credentialslocation="%APPDATA%\CData\Snowflake Data Provider\CredentialsFile.txt";location="C:\Users\DibyenduDatta\AppData\Roaming\CData\Snowflake Data Provider\Schema";logfile=D:\attach5\snowflakelog4.txtoauthsettingslocation="%APPDATA%\CData\Snowflake Data Provider\OAuthSettings.txt";oauthtokentimestamp='';verbosity=4; warehouse=SAMPLE_WAREHOUSE

2023-11-30T17:21:04.292+0530 4 [35|Q-Id] [SSL |Certif] Certificate validation complete. User Authorized SSL Cert: [null] Server Cert:[LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tDQpNSUlIcGpDQ0JvNmdBd0lCQWdJUUJyU0E3SlFOaWd4SVhsa2pmcEdJUkRBTkJna3Foa2lHO] Accepted: True

5.Setting Verbosity to 5 includes everything above, plus communication with the data source and additional details that may be helpful in troubleshooting problems. This includes interface commands. e.g., it exposes commands like FTP/SFTP captured in the log file.

2022-07-22T13:43:11.872+05:30	  1   [1|Q-Id]	  [INFO|Connec] Opened CSV connection. Version: 21.0.8236.0. Edition: [ADO.NET Provider .NET 3.5]. 

2022-07-22T13:43:11.874+05:30 1 [1|Q-Id] [INFO|Connec] Connection String: authscheme=FTP; cachelocation="%APPDATA%\CData\CSV Data Provider"; location="C:\Users\cdata\AppData\Roaming\CData\CSV Data Provider\Schema"; logfile="D:\CData\CData ADO.NET Provider for CSV 2021\FTP_Success.txt"; logmodules='';maxcelllength=-1;maxlogfilecount=-1;maxlogfilesize=100MB;maxrows=-1; verbosity=5;

2022-07-22T13:43:12.112+05:30 3 Info: Connecting to FTP server at
2022-07-22T13:43:12.626+05:30 3 Server: 220 Good day! Welcome to the ftp site of kasoftware!
2022-07-22T13:43:12.627+05:30 3 Client: USER tempuser
2022-07-22T13:43:12.860+05:30 3 Server: 331 Password required for tempuser
2022-07-22T13:43:12.860+05:30 3 Client: PASS ******
2022-07-22T13:43:13.090+05:30 3 Server: 230 Logged on
2022-07-22T13:43:13.090+05:30 3 Client: TYPE I
2022-07-22T13:43:13.320+05:30 3 Server: 200 Type set to I
2022-07-22T13:43:13.320+05:30 3 Client: CWD /JohnDoe/
2022-07-22T13:43:13.547+05:30 3 Server: 250 CWD successful. "/JohnDoe" is current directory.
2022-07-22T13:43:13.552+05:30 3 Client: PASV
2022-07-22T13:43:13.839+05:30 3 Server: 227 Entering Passive Mode (115,29,191,22,125,145)
2022-07-22T13:43:13.847+05:30 3 Client: LIST
2022-07-22T13:43:14.346+05:30 3 Server: 150 Opening data channel for directory listing of "/JohnDoe
2022-07-22T13:43:14.346+05:30 3 Server: 226 Successfully transferred "/JohnDoe"
2022-07-22T13:43:14.373+05:30 1 [Connection: 1] The IncludeFile property supports glob patterns. For example, if you want to specify the .csv file, you need to set the property as '*.csv' to avoid errors in future versions.

NOTE: For normal operation, set the Verbosity to a value not greater than 1. Higher verbosities can log substantial amounts of data, potentially causing delays in execution times. 

You can refine the logged content further by adjusting specific categories of information using the LogModules property in the Advanced Logging section.

Sensitive Data 

Level 3 and higher of log verbosity may capture information that you do not want to share outside of your organization. The following lists information of concern for each level: 

Verbosity 3: The driver returns all the data in the full body of the request and the response. 

Verbosity 4: The log returns the SSL certificates. 

Verbosity 5: Includes any additional transfer data not covered at Verbosity 3, such as non-human-readable binary transfer data.

Basic structure of a Log File 

Each entry in the log file includes a timestamp for that entry, the required logging level to view this information, and typically, indicates the connection number of the connection that generated this entry. 

2022-11-03T16:56:52.106-0400   1   [2|Q-Id]   [INFO|Connec]

Moreover, the system appends a tag following the connection number to signify the purpose of the entry and its respective action. The tag [INFO|Connec] indicates that the entry encompasses general information, particularly related to setting up the connection. Here are some tags commonly used: 

[EXEC|…] - employed during query execution or processes such as paging results. 

[HTTP|…] - utilized for handling HTTP requests and responses. 

[META|Schema] - employed when the driver retrieves metadata information.

Upon opening a connection, the driver records fundamental details. Specifically, information about the data source, version, and edition is logged. The subsequent entry also includes details about the connection string used, and if Java is employed, some information about the Java Virtual Machine (JVM) is included. 

2022-11-03T16:56:52.106-0400   1   [2|Q-Id]   [INFO|Connec]   Opened Exchange connection.  Version: 22.0.8257.0.  Edition: [JDBC Driver].
2022-11-03T16:56:52.108-0400   1   [2|Q-Id]   [INFO|Connec]   Connection String: authscheme=OAuth;autocache=false,….
2022-11-03T16:56:52.115-0400   1   [2|Q-Id]   [INFO|Connec]   JVM Start Args              -> -javaagent:C:\Program Files\JetBrains\ItteliJ IDEA…
2022-11-03T16:56:52.115-0400   1   [2|Q-Id]   [INFO|Connec]   JVM Heap                    -> Init: 532676608 Used: 641728512 Committed: 65534464

Understanding different variations of Log Files

1. HTTP Logs 

For drivers employing HTTP requests, at Verbosity 3, you can observe the requests and responses. These lines are marked with tags [HTTP|Req: #] or [HTTP|Res: #], respectively. The number in the tag increases with each new request or response, meaning the response labeled [HTTP|Res: N] corresponds to the Nth request tagged [HTTP|Req: N], (technically, the (N+1)th request, as the count starts at 0). 

Request In every driver request, there is the Method and URL of the request, the Headers, and (optionally) the Request Body. These elements consistently follow the same order and share a common tag in each request.

2022-12-05T16:59:32.980-0500   3   [2|Q-Id]   [HTTP|Req: 1]   POST https://outlook.outlook365.com/ews/exchange.asmx 
2022-12-05T16:59:32.982-0500 3 [2|Q-Id] [HTTP|Req: 1] [Reused]
[HTTP Headers]
2022-12-05T16:59:32.982-0500 3 [2|Q-Id] [HTTP|Req: 1] <?xml version=“1.0” encoding=“utf-8”?>
<soap:Envelope xmlns:soap= “http://schemas.xmlsoap.org/soap/envelope/”
xmlns:t= “http://schemas.microsoft.com/exchange/services/2006/types”
xmlns:m= “http://schemas.microsoft.com/exchange/services/2006/messages”
xmlns:xsi= “http://www.w3.org/2001/XMLSchema-instance”>
<t:RequestServerVersion Version “Exchange2013_SP1” />
<m:FindItem Traversal=”Shallow”>

The request may not always contain a body, and if it does, the body typically includes filters for the query. In cases where these filters are directly incorporated into the URL, such as with OData, the body is usually empty. 


The server's response includes the Headers and, optionally, the Response Body. Typically, the response includes a response body, but in certain situations, the response may have Content-Length: 0, returning only the headers. If the response body is extensive, the driver might also segment it into multiple lines. 

2022-12-05T16:59:33.209-0500   3   [2|Q-Id]   [HTTP|Res: 1]
[HTTP Headers]
Date: Mon, 06 Feb 2024 21:59:25 GMT
2022-12-05T16:59:33.215-0500   3   [2|Q-Id]   [HTTP|Res: 1]  <?xml  version=“1.0”  encoding=“utf-8”?><s:Envelope  xmlns:s=  “http://schemas.xmlsoap.org/”>
2022-12-05T16:59:33.215-0500   3   [2|Q-Id]   [HTTP|Res: 1]  AQMkl……

2. Database Logs 

Dealing with drivers that connect to databases such as SQL Server, MySQL, DB2, MongoDB, etc., can pose a challenge in extracting valuable information from the log file. Many providers encode the transmitted information as HEX values, which can then be decoded into ASCII characters.  

2024-01-18T14:47:18.523+0530	3   [5|Q-Id][TCP ] SSLStatus: Starting handshake sequence. 
2024-01-18T14:47:18.609+0530 3 [5|Q-Id] [TCP ] SSLStatus: Using platform security API.
2024-01-18T14:47:18.927+0530 2 [5|Q-Id] [TCP ] SSLServerAuthentication
2024-01-18T14:47:18.927+0530 3 [5|Q-Id] [TCP ] CertEncoded:
2024-01-18T14:47:18.927+0530 3 [5|Q-Id] [TCP ]

2D 2D 2D 2D 2D 42 45 47 49 4E 20 43 45 52 54 49
46 49 43 41 54 45 2D 2D 2D 2D 2D 0D 0A 4D 49 49
46 76 6A 43 43 41 36 59 43 46 41 6C 6C 70 38 32
6B 75 53 31 35 49 47 7A 38 63 57 39 44 4E 74 6C
6A 2F 7A 39 73 4D 41 30 47 43 53 71 47 53 49 62
33 44 51 45 42 43 77 55 41 4D 49 47 5A 4D 51 73
77 43 51 59 44 0D 0A 56 51 51 47 45 77 4A 4A 54
6A 45 53 4D 42 41 47 41 31 55 45 43 41 77 4A 53

The HEX code can be translated in ASCII as:

The given converter can be used to achieve this HEX to ASCII translation: https://www.rapidtables.com/convert/number/hex-to-ascii.html 

Common Errors

While the drivers are generally reliable, users may encounter common errors due to various reasons. Here are some potential issues and errors associated with CData drivers.

Connection Errors

These errors relate to the basic connection established in most network transmissions. 

  1. System error: Connection refused 
    Cause: This network error indicates a general connectivity issue, occurring when the system attempts to connect to a server that is not actively listening. It may suggest server downtime or an incorrect URL/port in the connection attempt. 
    Resolution: Ensure the accuracy of the target URL and port. If the error persists, reach out to the server administrators of the target system to determine whether the server is down or if they possess additional information about the issue. 

  2. Connection time-out 
    Cause: This network error occurs when the server does not respond to the connection attempt within a specified time frame (usually 60 seconds). It commonly arises due to firewall interference but could also suggest incorrect connection parameters. 
    Resolution: Try to ensure the accuracy of the fundamental connection parameters (remote host, port, URL, etc.).

HTTP Errors

These errors pertain to processing an HTTP request and can be significant for any protocols constructed atop the HTTP foundation. 

  1. 500 Internal Server Error 
    Cause: This HTTP error signifies a general server-side failure. While the connection is successfully established, an error occurs at the HTTP processing level, and there is no immediate access to additional debugging information. 
    Resolution: As this error lacks debugging information, contacting the remote user becomes necessary to inspect server-side logs that offer context to the failure. These logs help identify the root cause of the issue. 

  2. 400 Bad request 
    Cause: A 400 Bad Request error signals that the server considers the incoming request to be malformed. This frequently occurs due to incorrect or absent headers or unexpected data in the HTTP body. 
    Resolution: Since the server is expressing concerns regarding the client's request, the server-side probably contains supplementary debugging logs containing extra details about the failure. This may suggest the existence of additional connection prerequisites for this server, such as the need for HTTP authentication headers. The administrator on the server side should verify the relevance of such requirements and, if applicable, configure them in the connector responsible for the outbound request. 

  3. 404 Not Found 
    Cause: This HTTP error signifies the absence of any resource at the target URL. Typically, it suggests that the first portion of the URL (host, port) is accurate, but the second part (the resource path) is unrecognized. 
    Resolution: Ensure the accuracy of the resource path in the target URL. 

  4. 403 Forbidden 
    Cause: The HTTP 403 Forbidden error signifies that the remote server identifies the sender of the request but denies access to the requested resource for this client. This could occur due to either directing the request to the wrong URL (e.g., a sensitive server endpoint instead of the publicly accessible one) or insufficient permissions on the server side to grant access to the client. 
    Resolution: Firstly, verify the accuracy of the target URL. If it is correct, consult the remote server to determine whether additional permissions must be granted to the client or if any part of the request is unsupported, leading to the server denying access. Resolving this matter is possible only from the server side.

Log File Checklist 

A log file contains a wealth of valuable information. Here are some initial recommendations for checking if the solution is not immediately evident: 

1. Inspect the log for any error messages in the HTTP responses. 

2. Compare connection strings, especially if you have a working one. 

3. Verify the validity of the query. 

4. Check the driver's version; if it's an older version, contact our customer support.

Try CData Drivers Today 

Explore our library of drivers for 250+ data sources and download a 30-day free trial today.

0 replies

Be the first to reply!