Announcing the SAVIYNT KNOWLEDGE EXCHANGE unifying the Saviynt forums, documentation, training, and more in a single search tool across platforms. Click HERE to read the Announcement.

SFTP connector not sending file but FileTransfer job is completed succesfully

ekorh
Regular Contributor
Regular Contributor

Hi,

We have a following set-up in place:

1. SQL analytics saved to elastic search and saved to file (with path: /saviynt_shared/reports/admin directory) with name 'Me_Managers'. We are saving only 1 version at a time.
2. SFTP connector in place with following configs in FILES_TO_PUT 
[
{
"eic_src_dir":"Reports",
"target_dir":"/Customer_Receive/",
"regex_list":["^Me_Managers.*\\.zip"]
}
]

3. FileTransfer job, with the correct connection and Action as 'Upload'

 

We have followed these instructions: https://docs.saviyntcloud.com/bundle/SFTP-Certified-v23x/page/Content/Configuring-the-Integration-fo...

After running analytics, we tried to run the job and job completed successfully. However, customer reported that no file was received. 

Next we tried with following config in FILES_TO_PUT:
[
{
"eic_src_dir":"Reports",
"target_dir":"/Customer_Receive/",
"regex_list":["^Me_Managers.*\\.csv"]
}
]

But still job was shown as successful but no file was sent. 

After that we changed the Analytics save to file path to 'Analytics/', which is the default one, but still no luck.
The connection works ok and our connectors login to the customer sftp server is logged also as successful.

Are we missing some steps in the configuration?

BR,
Ella

19 REPLIES 19

DaanishJawed
Saviynt Employee
Saviynt Employee

Hi @ekorh ,

Thanks for reaching out.

Can you try sending one file and see if that works?

Add the below line and remove the regex line.

 "file_list":["file1.csv","file2.xlsx"]

Thanks.

ekorh
Regular Contributor
Regular Contributor

Hi! I tried that but got following error: 

 

java.io.FileNotFoundException: /saviynt_shared/reports/admin/Me_Managers.csv (No such file or directory)

Please read and comment my following troubleshooting intempts:
When I look at the logs after running the analytics, the logs always show the file was saved and the name was appended with some additional info. This file name for example from yesterday: "Me_Managers_<hkey also here but omitted for the forum>_2023-05-09_10-49-42(UTC).csv".

This is why I wanted to use regex.

Now, analyzing logs, I also found one interesting thing.
When I configured the Analytics 'Save to file' as path </saviynt_shared/reports/admin> and ran the analytics, following info was found in the logs:
Directory not exists, creating new directory: /saviynt_shared/reports/adminMe_Managers/
2023-05-10T11:49:32+03:00-ecm-2023-05-10T08:49:31.87130637Z stdout F 2023-05-10 08:49:31,871 [http-nio-8080-exec-1]

Interestingly, in the FD info it says that the put_files json in the connector can only contain value 'Report' for eic_src_dir, and it will always refer to path /saviynt_shared/reports/admin. This is why I configured this path for the Analytics 'save to file'.
Reference here: https://docs.saviyntcloud.com/bundle/SFTP-Certified-v23x/page/Content/Permissible-Values-for-File-Tr...
If I try with any other value or path, the job will fail.


Then I started to dig into the analytics info and found following document: https://docs.saviyntcloud.com/bundle/EIC-Admin-v23x/page/Content/Chapter17-EIC-Analytics/Managing-An...
It states following: By default, the Save To File path will be the path set in thejasper.dir.reportsparameter in externalconfig.properties file.

This attribute was missing in our externalconfig.properties file. So, I added the attribute there with the path and restarted our environment: 
jasper.dir.reports=/saviynt_shared/reports/admin


Even after this, the file sending is still not working and I got the same file/directory does not exist with these settings when I tried with the suggestion in the previous comment.


How should we proceed from here?

mbinsale
Saviynt Employee
Saviynt Employee

Please try the below

eic_src_dir = Reports

The above is case sensitive 

target_dir = This is the directory on your SFTP server. Ensure that this directory is already created on your SFTP Server and then specify the path here

regex_list = pattern of the filenames.

 

[
 {
    "eic_src_dir":"Reports",
    "target_dir":"/home/reports2/",
    "regex_list":["*MedalliaManagers3*.zip"]
  }
]

 

ekorh
Regular Contributor
Regular Contributor

Hi,

This is exactly what we have already tried and it did not work. 

The customer sftp server folder path has been verified, we have used the 'Reports' as the saviynt directory and used regex to pull the file. 

Is there any specific path that should be configured for the analytics 'save to file' part?
If you read my troubleshooting, the job is not able to fetch the file when I leave it as default neither when I specify it to be '/saviynt_shared/reports/admin'. This path '/saviynt_shared/reports/admin' is said to be the only one that the connector can use when you specify sav directory as reports. Source: https://docs.saviyntcloud.com/bundle/SFTP-Certified-v23x/page/Content/Permissible-Values-for-File-Tr...

 

Hi @ekorh,

Can you try below recommendations -

  • In your externalconfig.properties file, you added the path as below.
jasper.dir.reports=/saviynt_shared/reports/admin

Can you change it to the below path, restart and check. Here 'R' in report should be uppercase and an extra '/' after admin.

jasper.dir.reports=/saviynt_shared/Reports/admin/
  • And also update the config JSON to the below where we do not need an '*' at the start.
[
 {
    "eic_src_dir":"Reports",
    "target_dir":"/home/reports2/",
    "regex_list":["MedalliaManagers3*.zip"]
  }
]

 Thanks.

ekorh
Regular Contributor
Regular Contributor

Hi,

I tried with these instructions but still no file is uploaded to the customers sftp server.

I'm still wondering, what path should I define in the Analytics 'save to file' part?

  1. If I leave the 'save to file' as blank, so that it would be saved to the default path defined in the jasper.dir.reports=/saviynt_shared/Reports/admin/, I can see following in the logs:
    DEBUG analytics.AnalyticsService - Directory not exists, creating new directory: /saviynt_shared/Reports/admin/MedalliaManagers3/

  2. If I define 'save to file' as the default what it suggests (Analytics/), logs show this:
    DEBUG analytics.AnalyticsService - Directory not exists, creating new directory: /saviynt_shared/Reports/admin/Analytics/MedalliaManagers3/

  3. If I define 'save to file' as /saviynt_shared/Reports/admin/, logs show this:
    DEBUG analytics.AnalyticsService - Directory not exists, creating new directory: /saviynt_shared/Reports/admin//saviynt_shared/Reports/admin/MedalliaManagers3/

In each and every case, the file is saved to a directory that the connector presumably can't handle. And when I run the job, it shows success but nothing is really sent.

Also troubleshooting is quite difficult since I don't see any errors in the logs (log viewer, job logs, etc.).

Hi @ekorh ,

Thank you for trying all our recommendations. We appreciate all your efforts.

There is one last thing that we want you to try before we move this to Freshdesk Portal in order to log a ticket in.

Can you try the below path in the externalconfig.properties file, restart and check.

jasper.dir.reports=/saviynt_shared/Reports/

Thanks.

ekorh
Regular Contributor
Regular Contributor

Hi,

I did the change and tried with <blank> in the Analytics 'save to file'. Once again, in the logs following can be seen:
2023-05-23T11:10:33+03:00-ecm-2023-05-23T08:10:33.172887792Z stdout F 2023-05-23 08:10:33,172 [http-nio-8080-exec-5] DEBUG analytics.AnalyticsService - Directory not exists, creating new directory: /saviynt_shared/Reports/MedalliaUsers2/

And when I run the job, it is marked as 'success' but when I check the SFTP server no file has been uploaded there. I tried to upload a csv format and zip format but nothing.

Also no errors can be seen in the logs for any of the attempts (ecm, connectorms, ecm-worker).

Hi @ekorh ,

Currently this is being tracked on the support FD ticket - https://saviynt.freshdesk.com/a/tickets/1627870.

Thanks.

@DaanishJawed @ekorh @mbinsale 

Any solution to this?. We are facing the same issue. We have tried the methods suggested in this ticket as well as in the Saviynt documentation. The job is showing failure in our case. Pease guide.

Thanks

ekorh
Regular Contributor
Regular Contributor

Hi,

No, the SFTP connector is still not working for us either.

But please, if you can try with the community sourced guide using JAR connector:

https://docs-be.saviyntcloud.com/bundle/SSM-Connectors/page/Content/Resources/Attachments/SFTPIntegr...


You probably have to modify the JAR code to accept zip-files as well.

SinghAtul
Regular Contributor
Regular Contributor

@ekorh 

We tried something like below for FILES_TO_PUT json:

[
{
"eic_src_dir":"Reports",
"target_dir":"/SAPIDMFinalReport",
"regex_list":["SAPIDMReport/SAPIDMReport_HKey*.csv"]
}
]

I added "SAPIDMReport/" with the filename because I could see in logs that whenever I am running the Analytics ..report is getting saved in path:

/saviynt_shared/reports/admin/SAPIDMReport/SAPIDMReport_*.csv

Also, I observed that:

-> If we put anything other than 'Reports' in eic_src_dir. It doesn't accept it.

-> And, 'Reports' is being evaluated internally by Saviynt to path: /saviynt_shared/reports/admin/

When I ran the job,

The job says Success but no file is uploaded to the remote location. Also, the job not even runs for a second. It starts and then ends in less than a second.

I am using this config for FILES_TO_GET:

[
{
"eic_dest_dir":"DataFiles",
"src_dir":"/SAPIDMFinalReport",
"file_list":["sample_report.txt"]
}
]

I was able to download the file from the remote SFTP location using this config. 

I am not sure why file download is working but file upload is not working. If download is working it means there is no firewall issue. And even if there is any issue the job should show as Failure and we should get something in logs. We see no error messages in logs.

Really weird!. 😐

 

 

 

ekorh
Regular Contributor
Regular Contributor

Yeah, with the SFTP connector we have had exactly the same issues. 😞 

But the community sourced JAR connector has been working now for us. Link in my previous comment.

mbinsale
Saviynt Employee
Saviynt Employee

Based on the above feedback, we have opened a BUG ticket for this to be fixed. Appreciate your patience

SinghAtul
Regular Contributor
Regular Contributor

@mbinsale @DaanishJawed 

Hi, We have created a ticket with Saviynt regarding this issue. This is the ticket Id: #1635964

As you are aware that this seems to be an existing bug, can you please help speed up the process.

We have raised this ticket and probably we will have to answer multiple questions asked by the Saviynt Team and we will answer that and all this will delay our timelines greatly. 

If possible can you reply to our ticket in Saviynt that this is an existing bug and will be fixed in a later upgrade?. This will help speed up the resolution for us.

Regards,

Atul Singh

 

SinghAtul
Regular Contributor
Regular Contributor

Hi, 

I followed all the steps mentioned in the document and setup the Community Sourced JAR.

But when I run the Import job, nothing is happening..no file download , no file upload.

Can you please check if I have configured this Json correctly?. Please help

{
"fullyQualifiedClassName": "com.saviynt.connector.SFTPConnector.FileTransferUtilitywithargs",
"methodName": "saviyntFileTransfer_SFTP",
"arguments": {
"SFTPHOST": "${connectionJSON.SFTPHOST}",
"SFTPPORT": "${connectionJSON.SFTPPORT}",
"SFTPUSER": "${connectionJSON.SFTPUSER}",
"SFTPPASS": "${connectionJSON.SFTPPASS}",
"ACTION": "UPLOAD",
"SFTP_DIRECTORY":"/SAPIDMFinalReport/",
"SAVIYNT_DIRECTORY":"/saviynt_shared/saviynt/Reports/",
"FILE_NAME_REGEX": "SAPIDMReport*",
"SFTP_LOGFILE_DIRECTORY":"/saviynt_shared/saviynt/Import/",
"LOG_FILE_AGE":"2"
}
}

We have created the Security System, Endpoint, Attached AutoApprovalWorkflow.

The job type is Application Data Import(Single Threaded) and Full Accounts Import.

Also, based on above recommendations I had added a line in externalConfig.properties file:

jasper.dir.reports=/saviynt_shared/reports/admin/

Should I remove this line?. Is it even required?

Any help would be appreciated. We are stuck with this since ages.

Thanks

SinghAtul
Regular Contributor
Regular Contributor

@ekorh 

Can you please confirm what you have used in these two places:

1). jasper.dir.reports in externalConfig file

2) 'Save to file' option in Analytics

I am really having a tough time trying to figure this out.

Also, in our case the filename is starting with "SAPIDMReport". So I am using Regex like:

SAPIDMReport[A-Za-z0-9-_\\\\\\\\(\\\\\\\\)]*

Is this the right way?. What regex is working for you?.

Thanks.

ekorh
Regular Contributor
Regular Contributor

Hi,

The path in jasper.dir.reports we have: jasper.dir.reports=/saviynt_shared/Reports/admin  
But my wild guess is that the jasper.dir.reports might not be that important for the JAR connection.

Then in Analytics 'save to file' we kept the default suggestion: /Analytics
And lastly this is a sample of the importAccountJSON in the JAR connector: 

{
  "fullyQualifiedClassName": "com.saviynt.connector.SFTPConnector.FileTransferUtilitywithargs",
  "methodName": "saviyntFileTransfer_SFTP",
  "arguments": {
    "SFTPHOST": "${connectionJSON.SFTPHOST}",
    "SFTPPORT": "${connectionJSON.SFTPPORT}",
    "SFTPUSER": "${connectionJSON.SFTPUSER}",
    "SFTPPASS": "${connectionJSON.SFTPPASS}",
    "ACTION": "UPLOAD",
    "SFTP_DIRECTORY": "<specify here target path>",
    "SAVIYNT_DIRECTORY": "/saviynt_shared/Reports/admin/Analytics/<Analytics Name, e.g. SAPIDMReport>/",
    "FILE_NAME_REGEX": "^<analytics name here again, e.g. SAPIDMReport>.*"
  }
}

SinghAtul
Regular Contributor
Regular Contributor

Hi,

Thanks for the details.

We ended up creating a custom SFTP Code by using the community JAR as base because the community JAR doesn't allow .zip files upload but we needed to transfer a zip file.