Trying to connect to Min.io on-prem S3 bucket. Confirmed API is working:
However, I don’t know how to point to https://nas2.piorl.us:9000/bucket1/ in CDataSync Cross-Platform Edition. I’ve tried defining the full path in the Bucket field as well as “Path” in advanced options, but neither want to work. There is a reference to a “Custom URL” field in the advanced connection properties, but I can’t find the variable anywhere:
Thanks.
While we have had to test scenarios with Mini.io instances before, we have not tested this specifically in Sync. However we would be happy to look into this.
One thing you could try is navigating down to the Other property in the Advanced settings and setting the following:
StorageBaseURL=https://nas2.piorl.us:9000
It is important to specify the URL in the defined format http://<HOSTNAME>:9000 and set the name of the bucket you want to connect to in the Bucket property. Also, ensure that you are selecting the correct AWS Region when testing the connection.
Let me know if it works for you.
Since the AWS Region is list only, I’m assuming I’ll have to set an “AWS” region in Min.io so they will match?
Almost got it. Had to set Virtual Hosting to False (though I could have created a DNS entry instead I guess). Now I’m getting a “authorization header is malformed; the region is wrong”
Here’s the full log where is seems region is matching up:
2023-12-19T17:01:01.891-0500 1 [6|Q-Id] [INFO|Connec] Connection String: authscheme=AwsRootKeys;autocache=false;awsaccesskey=*****;awscognitoregion=NORTHERNVIRGINIA;awsexternalid=‘’;awsidentitypoolid=‘’;awsregion=NORTHERNVIRGINIA;awsrolearn=‘’;awssecretkey=*****;awssessiontoken=*****;awsuserpoolclientappid=‘’;awsuserpoolclientappsecret=*****;awsuserpoolid=‘’;batchsize=0;browsableschemas=‘’;bucket=bucket1;cacheconnection=‘’;cachedriver=‘’;cachelocation=“%APPDATA%\CData\AmazonS3Sync Data Provider”;cachemetadata=false;cachetolerance=600;catalog=CData;cdataapp=DataSync;charset=UTF-8;connectionlifetime=0;connectonopen=false;credentialslocation=“%APPDATA%\CData\AmazonS3Sync Data Provider\CredentialsFile.txt”;customdelimiter=,;customheaders=‘’;customurlparams=‘’;datetimeformat=‘’;excludefileextensions=True;fileformat=CSV;firewallpassword=*****;firewallport=0;firewallserver=‘’;firewalltype=NONE;firewalluser=‘’;includecolumnsheaders=False;insertmode=FilePerBatch;location=‘’;logfile=/opt/sync/logs/Minio_test_2023-12-19-17-01-01.log;logmodules=‘’;maxlogfilecount=-1;maxlogfilesize=100M;maxrows=-1;mfaserialnumber=‘’;mfatoken=*****;oauthencryptkey=*****;offline=false;other=“cdataapp=DataSync;storagebaseurl=https://nas2.piorl.us:9000”;path=‘’;poolidletimeout=60;poolmaxsize=100;poolminsize=1;poolwaittime=60;proxyauthscheme=BASIC;proxyautodetect=True;proxyexceptions=‘’;proxypassword=*****;proxyport=80;proxyserver=‘’;proxyssltype=AUTO;proxyuser=‘’;pseudocolumns=‘’;quotecharacter=‘’;quoteescapecharacter=‘’;readonly=false;rowscandepth=100;schema=bucket1;sslclientcert=*****;sslclientcertpassword=*****;sslclientcertsubject=*;sslclientcerttype=USER;sslservercert=*****;ssoexchangeurl=‘’;ssologinurl=‘’;ssoproperties=‘’;storagebaseurl=https://nas2.piorl.us:9000;tables=‘’;temporarytokenduration=3600;timeout=60;typedetectionscheme=RowScan;useconnectionpooling=False;userdefinedviews=‘’;usevirtualhosting=False;verbosity=5;views=‘’
2023-12-19T17:01:01.891-0500 1 [6|Q-Id] [INFO|Connec] JVM Start Args →
2023-12-19T17:01:01.891-0500 1 [6|Q-Id] [INFO|Connec] JVM Heap → Init: 29360128 Used: 100869704 Committed: 180236288 Max: 448069632
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: ssoproperties =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: other = cdataapp=DataSync;storagebaseurl=https://nas2.piorl.us:9000
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: ssoexchangeurl =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: views =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsuserpoolclientappid =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: quoteescapecharacter =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: mfatoken =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: sslclientcerttype = USER
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: sslclientcertsubject = *
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: customurlparams =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: fileformat = CSV
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: insertmode = FilePerBatch
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: firewalltype = NONE
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxyexceptions =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: oauthencryptkey =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxyserver =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awssessiontoken =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxyuser =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: sslservercert =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsuserpoolclientappsecret =
2023-12-19T17:01:01.891-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: logmodules =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: sslclientcert =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: location =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: customheaders =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: mfaserialnumber =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: maxrows = -1
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxyport = 80
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: pseudocolumns =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awssecretkey = *****
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: temporarytokenduration = 3600
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: typedetectionscheme = RowScan
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsexternalid =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxypassword =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: rowscandepth = 100
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: user =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: verbosity = 5
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: password =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: bucket = bucket1
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsidentitypoolid =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsaccesskey = *****
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxyautodetect = True
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: includecolumnsheaders = False
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: charset = UTF-8
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: useconnectionpooling = False
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsrolearn =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: datetimeformat =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: sslclientcertpassword =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: firewallpassword =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: maxlogfilesize = 100M
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: browsableschemas =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: customdelimiter = ,
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: quotecharacter =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: oemkey = *****
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: userdefinedviews =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: firewalluser =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: logfile = /opt/sync/logs/Minio_test_2023-12-19-17-01-01.log
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: path =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: ssologinurl =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxyssltype = AUTO
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: timeout = 60
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: tables =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsregion = NORTHERNVIRGINIA
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: usevirtualhosting = False
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: proxyauthscheme = BASIC
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awscognitoregion = NORTHERNVIRGINIA
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: firewallport = 0
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: excludefileextensions = True
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: authscheme = AwsRootKeys
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: awsuserpoolid =
2023-12-19T17:01:01.892-0500 2 [6|Q-Id] [INFO|Connec] Connection Property: firewallserver =
2023-12-19T17:01:01.894-0500 5 [6|Q-Id] [MDUL|SbMDUL] PrepareStatement: [SELECT * FROM SYS_SQLINFO]
2023-12-19T17:01:01.894-0500 5 [6|Q-Id] [MDUL|SbMDUL] PrepareStatement: [SELECT * FROM SYS_SQLINFO]
2023-12-19T17:01:01.894-0500 5 [6|Q-Id] [EXEC|Origin] Prepare: [SELECT * FROM SYS_SQLINFO]
2023-12-19T17:01:01.894-0500 5 [6|Q-Id] [EXEC|Origin] Execute Prepared Query: [SELECT * FROM SYS_SQLINFO]
2023-12-19T17:01:01.894-0500 2 [6|Q-Id] [EXEC|Parsed] Executing query: [SELECT * FROM SYS_SQLINFO]
2023-12-19T17:01:01.895-0500 2 [6|Q-Id] [EXEC|Messag] Executed query: [SELECT * FROM SYS_SQLINFO] Success: (1 ms)
2023-12-19T17:01:01.895-0500 5 [6|Q-Id] [MDUL|SbMDUL] PrepareStatement: [SELECT * FROM sys_tabletypes]
2023-12-19T17:01:01.895-0500 5 [6|Q-Id] [MDUL|SbMDUL] PrepareStatement: [SELECT * FROM sys_tabletypes]
2023-12-19T17:01:01.895-0500 5 [6|Q-Id] [EXEC|Origin] Prepare: [SELECT * FROM sys_tabletypes]
2023-12-19T17:01:01.896-0500 5 [6|Q-Id] [EXEC|Origin] Execute Prepared Query: [SELECT * FROM sys_tabletypes]
2023-12-19T17:01:01.896-0500 2 [6|Q-Id] [EXEC|Parsed] Executing query: [SELECT * FROM sys_tabletypes]
2023-12-19T17:01:01.896-0500 2 [6|Q-Id] [EXEC|Messag] Executed query: [SELECT * FROM sys_tabletypes] Success: (0 ms)
2023-12-19T17:01:01.901-0500 3 [HTTP] URL: https://nas2.piorl.us:9000/bucket1/?location=
2023-12-19T17:01:01.901-0500 3 [HTTP] UseFIPSCompliantAPI: false
2023-12-19T17:01:01.902-0500 3 [HTTP] Authorization: ******
2023-12-19T17:01:01.902-0500 3 [HTTP] HTTP Method: GET
2023-12-19T17:01:01.902-0500 3 [HTTP] HTTP Version: 1.1
2023-12-19T17:01:01.902-0500 3 [HTTP] UsePlatformHTTPClient: false
2023-12-19T17:01:01.903-0500 3 [HTTP] UseInternalSecurityAPI: false
2023-12-19T17:01:01.903-0500 2 [HTTP] Sending Request: GET (HTTP 1.1) to https://nas2.piorl.us:9000/bucket1/?location=
2023-12-19T17:01:02.105-0500 3 [HTTP] Remote Host IP: 10.16.1.102
2023-12-19T17:01:02.106-0500 3 [HTTP] Sending Headers
2023-12-19T17:01:02.114-0500 3 [HTTP] GET /bucket1/?location= HTTP/1.1
Host: nas2.piorl.us:9000
Authorization: ******
Accept-Encoding: gzip, deflate
User-Agent: CData Data Provider Engine - www.cdata.com - Accepts: gzip
Date: Tue, 19 Dec 2023 22:01:01 GMT
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20231219T220101Z
x-amz-s3-version: 2009-02-01
2023-12-19T17:01:02.115-0500 3 [HTTP] Request Sent
2023-12-19T17:01:02.116-0500 3 [HTTP] Response Received
2023-12-19T17:01:02.131-0500 3 [HTTP] Status Line: HTTP/1.1 200 OK
2023-12-19T17:01:02.132-0500 3 [HTTP] Accept-Ranges: bytes
2023-12-19T17:01:02.133-0500 3 [HTTP] Content-Length: 144
2023-12-19T17:01:02.134-0500 3 [HTTP] Content-Type: application/xml
2023-12-19T17:01:02.135-0500 3 [HTTP] Server: MinIO
2023-12-19T17:01:02.137-0500 3 [HTTP] Strict-Transport-Security: max-age=31536000; includeSubDomains
2023-12-19T17:01:02.137-0500 3 [HTTP] Vary: Origin
2023-12-19T17:01:02.138-0500 3 [HTTP] Vary: Accept-Encoding
2023-12-19T17:01:02.139-0500 3 [HTTP] X-Amz-Bucket-Region: NORTHERNVIRGINIA
2023-12-19T17:01:02.143-0500 3 [HTTP] X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
2023-12-19T17:01:02.146-0500 3 [HTTP] X-Amz-Request-Id: 17A25AA246E0B115
2023-12-19T17:01:02.148-0500 3 [HTTP] X-Content-Type-Options: nosniff
2023-12-19T17:01:02.149-0500 3 [HTTP] X-Xss-Protection: 1; mode=block
2023-12-19T17:01:02.151-0500 3 [HTTP] Date: Tue, 19 Dec 2023 22:01:02 GMT
2023-12-19T17:01:02.151-0500 3 [HTTP] <?xml version=“1.0” encoding=“UTF-8”?>
<LocationConstraint xmlns=“http://s3.amazonaws.com/doc/2006-03-01/”>NORTHERNVIRGINIA</LocationConstraint>
2023-12-19T17:01:02.152-0500 3 [HTTP] Transfer Complete
2023-12-19T17:01:02.154-0500 2 [HTTP] Response: HTTP/1.1 200 OK, 144 Bytes Transferred
2023-12-19T17:01:02.161-0500 3 [HTTP] URL: https://nas2.piorl.us:9000/bucket1/?location=
2023-12-19T17:01:02.162-0500 3 [HTTP] UseFIPSCompliantAPI: false
2023-12-19T17:01:02.167-0500 3 [HTTP] Authorization: ******
2023-12-19T17:01:02.167-0500 3 [HTTP] HTTP Method: GET
2023-12-19T17:01:02.168-0500 3 [HTTP] HTTP Version: 1.1
2023-12-19T17:01:02.169-0500 3 [HTTP] UsePlatformHTTPClient: false
2023-12-19T17:01:02.171-0500 3 [HTTP] UseInternalSecurityAPI: false
2023-12-19T17:01:02.175-0500 2 [HTTP] Already Connected. Sending Request: GET (HTTP 1.1) to https://nas2.piorl.us:9000/bucket1/?location=
2023-12-19T17:01:02.176-0500 3 [HTTP] Sending Headers
2023-12-19T17:01:02.198-0500 3 [HTTP] GET /bucket1/?location= HTTP/1.1
Host: nas2.piorl.us:9000
Authorization: ******
Accept-Encoding: gzip, deflate
User-Agent: CData Data Provider Engine - www.cdata.com - Accepts: gzip
Date: Tue, 19 Dec 2023 22:01:02 GMT
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20231219T220102Z
x-amz-s3-version: 2009-02-01
2023-12-19T17:01:02.199-0500 3 [HTTP] Request Sent
2023-12-19T17:01:02.200-0500 3 [HTTP] Response Received
2023-12-19T17:01:02.215-0500 3 [HTTP] Status Line: HTTP/1.1 200 OK
2023-12-19T17:01:02.217-0500 3 [HTTP] Accept-Ranges: bytes
2023-12-19T17:01:02.218-0500 3 [HTTP] Content-Length: 144
2023-12-19T17:01:02.219-0500 3 [HTTP] Content-Type: application/xml
2023-12-19T17:01:02.221-0500 3 [HTTP] Server: MinIO
2023-12-19T17:01:02.224-0500 3 [HTTP] Strict-Transport-Security: max-age=31536000; includeSubDomains
2023-12-19T17:01:02.225-0500 3 [HTTP] Vary: Origin
2023-12-19T17:01:02.226-0500 3 [HTTP] Vary: Accept-Encoding
2023-12-19T17:01:02.228-0500 3 [HTTP] X-Amz-Bucket-Region: NORTHERNVIRGINIA
2023-12-19T17:01:02.233-0500 3 [HTTP] X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
2023-12-19T17:01:02.235-0500 3 [HTTP] X-Amz-Request-Id: 17A25AA24BD8DEA2
2023-12-19T17:01:02.237-0500 3 [HTTP] X-Content-Type-Options: nosniff
2023-12-19T17:01:02.239-0500 3 [HTTP] X-Xss-Protection: 1; mode=block
2023-12-19T17:01:02.242-0500 3 [HTTP] Date: Tue, 19 Dec 2023 22:01:02 GMT
2023-12-19T17:01:02.242-0500 3 [HTTP] <?xml version=“1.0” encoding=“UTF-8”?>
<LocationConstraint xmlns=“http://s3.amazonaws.com/doc/2006-03-01/”>NORTHERNVIRGINIA</LocationConstraint>
2023-12-19T17:01:02.243-0500 3 [HTTP] Transfer Complete
2023-12-19T17:01:02.246-0500 2 [HTTP] Response: HTTP/1.1 200 OK, 144 Bytes Transferred
2023-12-19T17:01:02.254-0500 3 [HTTP] URL: https://nas2.piorl.us:9000/bucket1/?fetch-owner=true&list-type=2&max-keys=1
2023-12-19T17:01:02.255-0500 3 [HTTP] UseFIPSCompliantAPI: false
2023-12-19T17:01:02.257-0500 3 [HTTP] Authorization: ******
2023-12-19T17:01:02.257-0500 3 [HTTP] HTTP Method: GET
2023-12-19T17:01:02.257-0500 3 [HTTP] HTTP Version: 1.1
2023-12-19T17:01:02.258-0500 3 [HTTP] UsePlatformHTTPClient: false
2023-12-19T17:01:02.258-0500 3 [HTTP] UseInternalSecurityAPI: false
2023-12-19T17:01:02.261-0500 2 [HTTP] Sending Request: GET (HTTP 1.1) to https://nas2.piorl.us:9000/bucket1/?fetch-owner=true&list-type=2&max-keys=1
2023-12-19T17:01:02.420-0500 3 [HTTP] Remote Host IP: 10.16.1.102
2023-12-19T17:01:02.420-0500 3 [HTTP] Sending Headers
2023-12-19T17:01:02.421-0500 3 [HTTP] GET /bucket1/?fetch-owner=true&list-type=2&max-keys=1 HTTP/1.1
Host: nas2.piorl.us:9000
Authorization: ******
Accept-Encoding: gzip, deflate
User-Agent: CData Data Provider Engine - www.cdata.com - Accepts: gzip
Date: Tue, 19 Dec 2023 22:01:02 GMT
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20231219T220102Z
x-amz-s3-version: 2009-02-01
2023-12-19T17:01:02.422-0500 3 [HTTP] Request Sent
2023-12-19T17:01:02.422-0500 3 [HTTP] Response Received
2023-12-19T17:01:02.438-0500 3 [HTTP] Status Line: HTTP/1.1 400 Bad Request
2023-12-19T17:01:02.438-0500 3 [HTTP] Accept-Ranges: bytes
2023-12-19T17:01:02.438-0500 3 [HTTP] Content-Length: 426
2023-12-19T17:01:02.439-0500 3 [HTTP] Content-Type: application/xml
2023-12-19T17:01:02.439-0500 3 [HTTP] Server: MinIO
2023-12-19T17:01:02.439-0500 3 [HTTP] Strict-Transport-Security: max-age=31536000; includeSubDomains
2023-12-19T17:01:02.440-0500 3 [HTTP] Vary: Origin
2023-12-19T17:01:02.440-0500 3 [HTTP] Vary: Accept-Encoding
2023-12-19T17:01:02.440-0500 3 [HTTP] X-Amz-Bucket-Region: NORTHERNVIRGINIA
2023-12-19T17:01:02.441-0500 3 [HTTP] X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
2023-12-19T17:01:02.441-0500 3 [HTTP] X-Amz-Request-Id: 17A25AA25936CDF3
2023-12-19T17:01:02.442-0500 3 [HTTP] X-Content-Type-Options: nosniff
2023-12-19T17:01:02.442-0500 3 [HTTP] X-Xss-Protection: 1; mode=block
2023-12-19T17:01:02.443-0500 3 [HTTP] Date: Tue, 19 Dec 2023 22:01:02 GMT
2023-12-19T17:01:02.443-0500 3 [HTTP] <?xml version=“1.0” encoding=“UTF-8”?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region is wrong; expecting 'NORTHERNVIRGINIA'.</Message><BucketName>bucket1</BucketName><Resource>/bucket1/</Resource><Region>NORTHERNVIRGINIA</Region><RequestId>17A25AA25936CDF3</RequestId><HostId>dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8</HostId></Error>
2023-12-19T17:01:02.443-0500 3 [HTTP] Transfer Complete
2023-12-19T17:01:02.444-0500 2 [HTTP] Response: HTTP/1.1 400 Bad Request, 426 Bytes Transferred
2023-12-19T17:01:02.446-0500 2 AuthorizationHeaderMalformed: The authorization header is malformed; the region is wrong; expecting ‘NORTHERNVIRGINIA’.
XcoreXamazons3syncX230X8669.kde
at XcoreXamazons3syncX230X8669.at.a(Unknown Source)
at XcoreXamazons3syncX230X8669.ct.a(Unknown Source)
at XcoreXamazons3syncX230X8669.at.a(Unknown Source)
at XcoreXamazons3syncX230X8669.ct.a(Unknown Source)
at XcoreXamazons3syncX230X8669.ct.c(Unknown Source)
at XcoreXamazons3syncX230X8669.at.q(Unknown Source)
at XcoreXamazons3syncX230X8669.at.ag(Unknown Source)
at XcoreXamazons3syncX230X8669.uwd.i(Unknown Source)
at XcoreXamazons3syncX230X8669.tee.a(Unknown Source)
at XcoreXamazons3syncX230X8669.tee.a(Unknown Source)
at XcoreXamazons3syncX230X8669.yvc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.yvc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.yvc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.itc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.itc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.hxc.b(Unknown Source)
at XcoreXamazons3syncX230X8669.hxc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.hxc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.hxc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.muc.b(Unknown Source)
at XcoreXamazons3syncX230X8669.jub.a(Unknown Source)
at XcoreXamazons3syncX230X8669.jub.a(Unknown Source)
at XcoreXamazons3syncX230X8669.lxc.b(Unknown Source)
at XcoreXamazons3syncX230X8669.jzc.b(Unknown Source)
at XcoreXamazons3syncX230X8669.stc.a(Unknown Source)
at XcoreXamazons3syncX230X8669.rxc.g(Unknown Source)
at XcoreXamazons3syncX230X8669.zuc.g(Unknown Source)
at XcoreXamazons3syncX230X8669.vwb.g(Unknown Source)
at XcoreXamazons3syncX230X8669.dpd.r(Unknown Source)
at XcoreXamazons3syncX230X8669.dpd.g(Unknown Source)
at XcoreXamazons3syncX230X8669.fac.g(Unknown Source)
at XcoreXamazons3syncX230X8669.xub.g(Unknown Source)
at XcoreXamazons3syncX230X8669.kqb.n(Unknown Source)
at cdata.jdbc.amazons3sync.AmazonS3SyncConnection.testConnection(Unknown Source)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at XcoreXcdata.realdb.zrd.testConnection(Unknown Source)
at XcoreXcdata.syncX233X8700.fha.testConnection(Unknown Source)
at XcoreXcdata.syncX233X8700.gha.c(Unknown Source)
at sync.core.QueryExec$4.a(Unknown Source)
at sync.core.QueryExec.a(Unknown Source)
at sync.core.QueryExec.g(Unknown Source)
at rssbus.syncops.CallStackKeeper.TestConnection(Unknown Source)
at rssbus.syncops.TestConnection.exec(Unknown Source)
at XcoreXcdata.syncX233X8700.wra.a(Unknown Source)
at XcoreXcdata.syncX233X8700.wra.a(Unknown Source)
at XcoreXcdata.syncX233X8700.wra.a(Unknown Source)
at XcoreXcdata.syncX233X8700.ipa.a(Unknown Source)
at rssbus.src.testConnection_rsb.a(Unknown Source)
at XcoreXcdata.syncX233X8700.eta.Exec(Unknown Source)
at rssbus.src.testConnection_rsb.execRSB(Unknown Source)
at XcoreXcdata.syncX233X8700.ppa.execRSB(Unknown Source)
at XcoreXcdata.syncX233X8700.zra.a(Unknown Source)
at XcoreXcdata.syncX233X8700.cra.a(Unknown Source)
at XcoreXcdata.syncX233X8700.cra.a(Unknown Source)
at XcoreXcdata.syncX233X8700.spa.call(Unknown Source)
at XcoreXcdata.syncX233X8700.mma.call(Unknown Source)
at XcoreXcdata.syncX233X8700.mma.call(Unknown Source)
at rssbus.RSBScript.a(Unknown Source)
at rssbus.RSBScript.a(Unknown Source)
at rssbus.RSBScript.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1410)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1665)
at b.c.doFilter(Unknown Source)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:538)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
at org.eclipse.jetty.server.Server.handle(Server.java:563)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:558)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:379)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:146)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:416)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:385)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:272)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:140)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
at java.base/java.lang.Thread.run(Unknown Source)
2023-12-19T17:01:02.462-0500 5 [ |Q-Id] [META|Schema] Engine Invalid object name ‘sys_disconnect’
2023-12-19T17:01:02.462-0500 4 [6|Q-Id] [INFO|Connec] Executed sys_disconnect: Success: (0 ms)
2023-12-19T17:01:02.462-0500 1 [6|Q-Id] [INFO|Connec] Closed AmazonS3Sync connection
In the screenshot you have provided it looks like you are setting the AWS Region to “us-orl-nas2” when testing the API in Postman. If this is the correct Region, can you try adding the same in the Other property:
StorageBaseURL=https://nas2.piorl.us:9000;Region=us-orl-nas2;
Let me know if this changes anything.
That did the trick. I had tried renaming the Minio Region to NORTHERNVIRGINIA so they would match up, then tried changing it in both Minio and CDataSync to “UAE” just in case it was an issue with the length, same error. Then renamed back to us-orl-nas2 in Minio and added the region variable above, and viola.
Glad to hear that worked for you. Generally speaking the driver should autodetect the AWS Region in most cases however since this is a Mini.io instance I believe the driver has failed to autodetect the region for some reason.
In case you were wondering, Region is a hidden property that can be used in scenarios like this one where the driver is unable to detect the AWS Region automatically from the URL provided which means we can get around this by explicitly setting the Region property in Other.
Let us know if you have any other questions.
