Skip to main content

How can i write to sub-directories in AzureDataLake. I am able to write to parent folder but sub-directories are not working

Hi @obathla 

When you create an ADLS connection in CData Sync, under the Advanced section, there should be a property called Path, where you can define this root directory of where you’d like to replicate the data in a file system:
 

Please try and see if this helps!


This provides one directory and even that fails when I try to write. use case is to have multiple sub-directories - one per source system.


Hi @obathla , If it is for reading data you might do so by setting the IncludeSubDirectories=True in the connection and specify the correct directory in the Path connection property. In the case of the ADLSSync destination component, it is not possible to have or specify multiple sub-directories in the same connection. In order to do that you will need to create separate connection and specify the directory you want to write to for each connection in the Path connection property.

If the job is failing for you, it would be helpful to know what error are you receiving. Kindly share the details on the error you are getting and we can further assist you.

You can also reach out to our support team at [email protected] if you would like to raise a support ticket with them regarding this.


I was able to get it to work. Here’s summary of what worked for me

Set Connector Properties:

Filesystem - define root directory

Under Advanced > Path Separator  Add / 

 

Replicate Query Properties:

  1. Open the task in your job
  2. Navigate to the query tab and click edit
  3. Write the subfolder name in the query. e.g if my subfolder is “workday” here’s the query

Replicate >workday/tablename] select * from etablename]


Reply