Rest connector- account import

[sk]

Hello experts,

We are trying to import accounts, 

This is the first call, POST method  and the url is https://xyz-it-ops-search.abc.az.net/services/search/jobs/ and in body raw we should pass 

search= | inputlookup ad_signin_logs.csv so we will be getting the SId in response 

And that SID we need to pass in another GET call and url is  https://xyz-it-ops-search.abc.az.net/services/search/jobs/ SID/results?output_mode=json

curl for GET call

curl --location --request GET 'https://xyz-it-ops-search.abc.az.net/services/search/jobs/ SID/results?output_mode=json' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic hWdJlJGlnbmluTG9ncw==' \
--data 'search= | inputlookup ad_signin_logs.csv'

curl for POST call

curl --location 'https://xyz-it-ops-search.abc.az.net/services/search/jobs/ ' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic hWdJlJGlnbmluTG9ncw==' \
--data 'search= | inputlookup ad_signin_logs.csv'

currently using json

{
"accountParams": {
"connection": "acctAuth",
"showLogs": true,
"createUsers": false,
"processingType": "SequentialAndIterative",
"statusAndThresholdConfig": {
"accountThresholdValue": 100,
"correlateInactiveAccounts": true,
"inactivateAccountsNotInFile": false
},
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "https://xyz-it-ops-search.abc.az.net/services/search/jobs/ SID/results?output_mode=json",
"httpHeaders": {
"Authorization": "Basic hWdJlJGlnbmluTG9ncw==",
"Accept": "application/json"
},
"httpContentType": "application/json",
"httpMethod": "GET"
},
"listField":"results",
"keyField": "accountID",
"colsToPropsMap": {
"accountID": "userPrincipalName~#~char",
"name": "userPrincipalName~#~char",
"customproperty11": "last_usage~#~char"
},
"disableDeletedAccounts": true

}
}
}
}

Please let me how can we achieve this, thanks in advance

[sk]

Tried with below json 

{
"accountParams": {
"connection": "acctAuth",
"showLogs": true,
"createUsers": false,
"processingType": "SequentialAndIterative",
"statusAndThresholdConfig": {
"accountThresholdValue": 100,
"correlateInactiveAccounts": true,
"inactivateAccountsNotInFile": false
},
"call": [
{
"callOrder": 0,
"name": "call1",
"connection": "acctAuth",
"showResponse": true,
"url": "https://xab/services/search/jobs",
"httpMethod": "POST",
"httpParams": "search=%20%7C%20inputlookup ad_signin_logs.csv",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"httpContentType": "application/json",
"successResponses": {
"statusCode": [
201,
200
]
}
},
{
"call2": {
"callOrder": 1,
"stageNumber": 1,
"http": {
"url": "https://xacg.net/services/search/jobs/${response.call1}/results?output_mode=json",
"httpHeaders": {
"Authorization": "Basic nOkB6dXlnbmluTG9ncw==",
"Accept": "application/json"
},
"httpContentType": "application/json",
"httpMethod": "GET"
},
"listField": "results",
"keyField": "accountID",
"colsToPropsMap": {
"accountID": "userPrincipalName~#~char",
"name": "userPrincipalName~#~char",
"customproperty11": "last_usage~#~char"
},
"disableDeletedAccounts": true
}
}
]
}
}

job is failing with this error 

[NM]

Hi @sk is the first call working .. seperately?

[sk]

@NM , second call is working 

[rushikeshvartak]

• This will not be supported as colsToPropsMap is mandatory in account Import JSON.
• Please raise idea ticket

[sk]

@rushikeshvartak / @NM -

We have modified the json as below and call1 is working fine, we need to pass the sid in call2 , how can we achieve this

{
"accountParams": {
"connection": "acctAuth",
"createUsers": true,
"adminName": "admin",
"processingType": "SequentialAndIterative",
"statusAndThresholdConfig": {
"deleteLinks": false,
"accountThresholdValue": 30,
"correlateInactiveAccounts": false,
"inactivateAccountsNotInFile": false,
"deleteAccEntForActiveAccounts": true
},
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "https://xxxa?output_mode=json",
"httpHeaders": {
"Authorization": "Basic abhxxxxxxxxxxxxx",
"Content-Type": "application/json"
},
"httpParams": "search=%20%7C%20inputlookup%20ad_signin_logs.csv",
"httpContentType": "application/json",
"httpMethod": "POST"
},
"listField": "response",
"keyField": "accountID",
"colsToPropsMap": {
"accountID":"sid~#~char",
"name": "sid~#~char"
},
"disableDeletedAccounts": true
},
"call2": {
"callOrder": 1,
"stageNumber": 1,
"http": {
"url": "https://xxa/${response.call1.message.sid}/output_mode=json",
"httpMethod": "GET",
"httpHeaders": {
"Authorization": "Basic abhxxxxxxxxxxxxxxxxxxxxxx",
"Accept": "application/json"
},
"inputParams": {
"dependentCall": true
},
"httpContentType": "application/json",
"listField": "results",
"keyField": "accountID",
"colsToPropsMap": {
"accountID": "userPrincipalName~#~char",
"name": "userPrincipalName~#~char",
"customproperty11": "last_usage~#~char"
}
}
}
}
}
}
please advice, if we are missing anything 

[rushikeshvartak]

• What is error you are getting in logs ?
• Did you checked call1 response in saviynt logs ?
• "url": "https://xxa/${response.call1.sid}/output_mode=json",

[sk]

@rushikeshvartak , could you please advice how can we add call1 response in call2 url

[sk]

@rushikeshvartak 

We are getting below error in logs 

2024-08-05T15:31:24+05:30-ecm-worker--null-9c85b--java.lang.NullPointerException: Cannot invoke method trim() on null object at com.saviynt.provisoning.rest.RestProvisioningService.persistAccounts(RestProvisioningService.groovy:4944) at com.saviynt.provisoning.rest.RestProvisioningService$_processAccounts_closure28_closure107.doCall(RestProvisioningService.groovy:4197) at com.saviynt.provisoning.rest.RestProvisioningService$_processAccounts_closure28.doCall(RestProvisioningService.groovy:4195) at com.saviynt.provisoning.rest.RestProvisioningService.processAccounts(RestProvisioningService.groovy:4194) at com.saviynt.provisoning.rest.RestProvisioningService.processAccountsFinal(RestProvisioningService.groovy:1689) at com.saviynt.provisoning.rest.RestProvisioningService.processAccountsFullBySequentialAndIterative(RestProvisioningService.groovy:1653) at com.saviynt.provisoning.rest.RestProvisioningService.importAccountsFull(RestProvisioningService.groovy:1475) at com.saviynt.provisoning.rest.RestProvisioningService.doImport(RestProvisioningService.groovy:139) at com.saviynt.ecm.integration.ExternalConnectionCallService.invokeExternalMethod(ExternalConnectionCallService.groovy:233) at SapImportJob.execute(SapImportJob.groovy:109) at org.quartz.core.JobRunShell.run(JobRunShell.java:199) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)
2024-08-05T15:31:24+05:30-ecm-worker-rest.RestProvisioningService-quartzScheduler_Worker-7-9c85b-ERROR-Invalid record not processed : [preview:false, init_offset:0, messages:[[type:INFO, text:Successfully read lookup file '/opt/splunk/etc/apps/search/lookups/azure_ad_signin_logs.csv'.]],

Yes we checked the call1 response in logs, below is the response

2024-08-05T15:31:23+05:30-ecm-worker-rest.RestProvisioningService-quartzScheduler_Worker-7-9c85b-DEBUG-Got Webservice API Response: [headers:[Date: Mon, 05 Aug 2024 10:01:23 GMT, Expires: Thu, 26 Oct 1978 00:00:00 GMT, Cache-Control: no-store, no-cache, must-revalidate, max-age=0, Content-Type: application/json; charset=UTF-8, X-Content-Type-Options: nosniff, Link: <1722852083.187184>; rel=info, Content-Length: 27, Location: /services/search/jobs/1722852083.187184, Vary: Cookie, Authorization, Connection: Keep-Alive, X-Frame-Options: SAMEORIGIN, Server: Splunkd], responseText:{"sid":"1722852083.187184"}, cookies:[], statusCode:201]

Tried with this 

• "url": "https://xxa/${response.call1.sid}/output_mode=json", but still the same error

[sk]

@rushikeshvartak , After troubleshooting now we are getting below error 

2024-08-06T10:01:53+05:30-ecm-worker-rest.RestProvisioningService-quartzScheduler_Worker-3-9c85b-DEBUG-Got Webservice API Response: [error:Error Illegal character in path at index 69: https://xxx/${response.call1.sid}/results/?output_mode=json]
2024-08-06T10:01:53+05:30-ecm-worker-rest.RestProvisioningService-quartzScheduler_Worker-3-9c85b-ERROR-Exception in getting response in pullObjectsByRest :
2024-08-06T10:01:53+05:30-ecm-worker--null-9c85b--java.lang.Exception: NullResponseFromTarget at com.saviynt.provisoning.rest.RestUtilService.checkForErrorMsg(RestUtilService.groovy:1463) at com.saviynt.provisoning.rest.RestProvisioningService.pullObjectsByRest(RestProvisioningService.groovy:4551) at com.saviynt.provisoning.rest.RestProvisioningService.processAccountsByPagination(RestProvisioningService.groovy:4256) at com.saviynt.provisoning.rest.RestProvisioningService.processAccounts(RestProvisioningService.groovy:4183) at com.saviynt.provisoning.rest.RestProvisioningService.processAccountsFinal(RestProvisioningService.groovy:1689) at com.saviynt.provisoning.rest.RestProvisioningService.processAccountsFullBySequentialAndIterative(RestProvisioningService.groovy:1653) at com.saviynt.provisoning.rest.RestProvisioningService.importAccountsFull(RestProvisioningService.groovy:1475) at com.saviynt.provisoning.rest.RestProvisioningService.doImport(RestProvisioningService.groovy:139) at com.saviynt.ecm.integration.ExternalConnectionCallService.invokeExternalMethod(ExternalConnectionCallService.groovy:233) at SapImportJob.execute(SapImportJob.groovy:109) at org.quartz.core.JobRunShell.run(JobRunShell.java:199) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)
2024-08-06T10:01:53+05:30-ecm-worker-rest.RestProvisioningService-quartzScheduler_Worker-3-9c85b-DEBUG-Error while getting Account Import response for url- https://xxx/${response.call1.sid}/results/?output_mode=json is: null
2024-08-06T10:01:53+05:30-ecm-worker-rest.RestProvisioningService-quartzScheduler_Worker-3-9c85b-DEBUG-Getting response statusCode null, so failing Account Import Job

[rushikeshvartak]

hardcoded sid is working ?

[sk]

@rushikeshvartak - No , it was also not working, but it is working in postman.