Upload Datafile via REST API

[petra_bremer]

Hi,

We would like to periodically upload a data file via Saviynt's REST interface so that a schema job can then perform the import of the data.
We followed the documentation in Postman:

Saviynt Enterprise Identity Cloud API Reference v23.4 (getpostman.com)

Unfortunately, the response we get is:  {"msg": "file not found", "errorCode": "1"}

The documentation refers to a file InternalConfig.groovy that does not exist in our system.

We were able to find the parameter job.ecm.imp.file.path in in externalConfig.properties, where it is set to the file extension that we want to use for the upload.

So the questions are:

• Do we need InternalConfig.groovy ?  If yes: is there any template that can be used?
• Are there any other settings we need to make?

Thanks for any help.

Kind regards,

Petra Bremer

[petra_bremer]

Hello,

is there anyone who has already successfully imported a data file into the Saviynt folder /saviynt_shared/Import/Datafiles/ via the REST interface with uploadSchemaFile?

Did any special settings have to be made for this?

[rushikeshvartak]

yes 

[pmahalle]

Hi @petra_bremer ,

Saviynt rest api to upload file expect file present in the body rather than file path. 

In postman we can directly attached file in the body that’s why it will work in postman.

[petra_bremer]

I tried it with a Powershell script and followed the documentation stored in Postman:

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"

$headers.Add("Authorization", "Bearer "+$token)

$headers.Add("Content-Type", "application/json")

$multipartContent = [System.Net.Http.MultipartFormDataContent]::new()

$multipartFile = "./TTA_Partner2022/data/orgDataUpload.xlsx"

$FileStream = [System.IO.FileStream]::new($multipartFile, [System.IO.FileMode]::Open)

$fileHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$fileHeader.Name = "file"

$fileHeader.FileName = "orgDataUpload.xlsx"

$fileContent = [System.Net.Http.StreamContent]::new($FileStream)

$fileContent.Headers.ContentDisposition = $fileHeader

$multipartContent.Add($fileContent)

$stringHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$stringHeader.Name = "pathLocation"

$stringContent = [System.Net.Http.StringContent]::new("Datafiles")

# /saviynt_shared/Import/Datafiles/

$stringContent.Headers.ContentDisposition = $stringHeader

$multipartContent.Add($stringContent)

$body = $multipartContent

$url = $parameterJSON.url

$urlPost=$url+"/ECM/api/v5/uploadSchemaFile"

$response = Invoke-RestMethod -Uri $urlPost -Method 'POST' -Headers $headers  -Body $body

$uploadJSON = $response | ConvertTo-Json

Write-Host $uploadJSON

I get the following response:

{
"msg": "file not found",
"errorCode": "1"
}

[shilpab]

Hi Peter,

Are you able to solve this error ? {
"msg": "file not found",
"errorCode": "1"
}

We are also writting powershell script. Its giving same error. 
Could you please guide if you able to solve this ?

[petra_bremer]

we are still working on this.  We hope for further support from Saviynt support or Saviynt users in the forum.  We will share the solution with the forum as soon as there is anyone.

[petra_bremer]

I have recreated the situation in Postman - again without result, but with a different error message:

[rushikeshvartak]

Please check shared directory path in externalconfig

[petra_bremer]

That was precisely the aim of my initial question.
The Postman documentation talks about
property job.ecm.imp.file.path in InternalConfig.groovy.
We only found this property in externalConfig.properties (what is InternalConfig.groovy??).
in the externalConfig.properties our setting looks like this:

#job.ecm.savfile.path=/saviynt_shared/Import/SAV/
job.ecm.imp.file.path=/saviynt_shared/Import/Datafiles/
#job.ecm.imp.file.importprocess.path=/saviynt_shared/Import/Process/

We have already tested the upload with  an without activated path parameter - both without success.
The only consequence of the activated entry was that the menu item Datafiles under Settings=> File Directory is now missing.

[petra_bremer]

In Log, we find following message:

"ecm","2023-05-03T11:24:10.570+00:00","{"log":"2023-05-03 11:24:10,337 [http-nio-8080-exec-7] DEBUG ws.Restfulv5Controller - Exception occurred while uploading the file : \n","stream":"stdout","time":"2023-05-03T11:24:10.337703356Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"java.io.FileNotFoundException: /saviynt_shared/Import/Datafiles/orgDataUpload.xlsx (No such file or directory)\n","stream":"stdout","time":"2023-05-03T11:24:10.337722526Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at java.io.FileOutputStream.open(FileOutputStream.java:270)\n","stream":"stdout","time":"2023-05-03T11:24:10.337738074Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at java.io.FileOutputStream.\u003cinit\u003e(FileOutputStream.java:213)\n","stream":"stdout","time":"2023-05-03T11:24:10.337740856Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at java.io.FileOutputStream.\u003cinit\u003e(FileOutputStream.java:162)\n","stream":"stdout","time":"2023-05-03T11:24:10.337747072Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at org.apache.commons.fileupload.disk.DiskFileItem.write(DiskFileItem.java:417)\n","stream":"stdout","time":"2023-05-03T11:24:10.337752944Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at com.saviynt.ws.Restfulv5Service.saveFileSuccessInEcm(Restfulv5Service.groovy:302)\n","stream":"stdout","time":"2023-05-03T11:24:10.337785517Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at com.saviynt.ecm.ws.Restfulv5Controller$_closure210.doCall(Restfulv5Controller.groovy:18774)\n","stream":"stdout","time":"2023-05-03T11:24:10.33779166Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at grails.plugin.springsecurity.rest.RestTokenValidationFilter.processFilterChain(RestTokenValidationFilter.groovy:118)\n","stream":"stdout","time":"2023-05-03T11:24:10.337798076Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at grails.plugin.springsecurity.rest.RestTokenValidationFilter.doFilter(RestTokenValidationFilter.groovy:84)\n","stream":"stdout","time":"2023-05-03T11:24:10.33780434Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter.doFilter(GrailsAnonymousAuthenticationFilter.java:53)\n","stream":"stdout","time":"2023-05-03T11:24:10.337810145Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at com.saviynt.webservice.SaviyntRestAuthenticationFilter.doFilter(SaviyntRestAuthenticationFilter.groovy:144)\n","stream":"stdout","time":"2023-05-03T11:24:10.337815012Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter.doFilter(MutableLogoutFilter.java:62)\n","stream":"stdout","time":"2023-05-03T11:24:10.33781912Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at grails.plugin.springsecurity.web.SecurityRequestHolderFilter.doFilter(SecurityRequestHolderFilter.java:59)\n","stream":"stdout","time":"2023-05-03T11:24:10.337823066Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at com.mrhaki.grails.plugin.xframeoptions.web.XFrameOptionsFilter.doFilterInternal(XFrameOptionsFilter.java:69)\n","stream":"stdout","time":"2023-05-03T11:24:10.337837536Z"}"
"ecm","2023-05-03T11:24:10.570+00:00","{"log":"\u0009at com.brandseye.cors.CorsFilter.doFilter(CorsFilter.java:82)\n","stream":"stdout","time":"2023-05-03T11:24:10.337841682Z"}"

[RakeshMG]

Please try following path :  /saviynt_shared/saviynt/Import/Datafiles

[petra_bremer · Accepted solution]

Hello  Rakesh,

thank you for this tip. It really helped!!!

We have set the parameters in the externalConfig.properties as follows:

job.ecm.imp.file.path=/saviynt_shared/saviynt/Import/Datafiles/

In the TTA partner environment as well as at the customer environment, we were able to upload a data fiel via Postman.

So far we had always taken the file path that was commented out in the externalConfig: /saviynt_shared/Import/Datafiles/ and have nowhere found a hint in the documentation that the path must be different.

So again, many thanks for this very useful tip.

[stalluri]

@RakeshMG  and @rushikeshvartak 
Can we upload the files in bulk instead of single file upload.

[rushikeshvartak]

Single file

[aro]

It now works via Postman.

But the CURL call from the documentation

curl --location -g --request POST '{{url}}/ECM/api/v5/uploadSchemaFile' \

--header 'Authorization: Bearer {{token}}' \

--form 'file=@"C:\temp\Testuser1.csv"' \

--form 'pathLocation="Datafiles"' \

still gives the error:
{ "msg": "file not found", "errorCode": "1" }

What's wrong with the CURL call?

[rushikeshvartak]

curl --location -g --request POST '{{url}}/ECM/api/v5/uploadSchemaFile' \

--header 'Authorization: Bearer {{token}}' \

--form 'file=@"C:\\temp\\Testuser1.csv"' \

--form 'pathLocation="Datafiles"' \

[aro]

It doesn't work