Unable to Populate UserCustomproperty based on SAV4SAV REST connection

[AuvikDey]

Hi Team,

We have a requirement where we need to populate the CITY 3 letter abbreviated name from a Dataset for each user in one of the Users customproperty. I have created an analytics with the query to select the city shortname as per user's city attribute and trying to call the analytics in an UserImport JSON in SAV4SAV rest connection, but still the User CustomProperty is not getting populated.

=================================================================================

Here is the Analytics Query I Used :
SELECT U.USERNAME AS "USERNAME" ,DV.ATTRIBUTE2 AS "CUSTOMPROPERTY5" from USERS U,DATASET_VALUES DV
WHERE DV.DATASETNAME='City_Shortname' AND (U.CITY)=DV.ATTRIBUTE1
AND ((U.CUSTOMPROPERTY5 IS NULL) OR (U.CUSTOMPROPERTY5!=DV.ATTRIBUTE2)) AND U.STATUSKEY=1

=================================================================================

SAV4SAV REST Connection JSON:
{
"authentications": {
"userAuth": {
"authType": "oauth2",
"url": "https://@@BASEURL@@/ECM/api/login",
"httpMethod": "POST",
"httpParams": "{\"username\": \"XXXXXXXXXX\",\"password\": \"XXXXXXXXXXXXXX\"}",
"httpContentType": "application/json",
"httpHeaders": {
"Content-Type": "application/json"
},
"authError": [
"USER_AUTHENTICATION_FAILED",
"PARTNER_AUTHENTICATION_FAILED",
"AuthenticationFailed"
],
"errorPath": "errorCode",
"maxRefreshTryCount": 5,
"tokenResponsePath": "access_token",
"tokenType": "Bearer",
"authHeaderName": "Authorization",
"accessToken": "Bearer abcd",
"retryFailureStatusCode": [
401
]
}
}
}

=================================================================================
USER IMPORT JSON:

{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/fetchRuntimeControlsData",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1515\"}",
"httpHeaders": {
"Authorization": "${access_token}"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "result"
}


Please tell me, where is the issue as I am unable to troubleshoot it for a long time

[AuvikDey]

Never mind :

Changed the UserImport JSON to :
{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/fetchControlDetailsES",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1515\"}",
"httpHeaders": {
"Authorization": "${access_token}"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "result"
}


and the analytics was missing default actions (Accept, Revoke, Further Review) 

[AuvikDey]

Hi Team,
Please help here, the solution here is not updating for more than 50 users at a time and also I checked other forum discussion where they are requesting to use the URL as : 
"url": "XXXXX/ECM/api/v5/FetchRuntimeControlsData" -- but I am not getting any response in Postman.

I am getting response in Postman in  : "url": "XXXXX/ECM/api/v5/fetchControlDetailsES".

Currently I am using this Import User JSON, but this is not working:

{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/fetchControlDetailsES",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1520\"}",
"httpHeaders": {
"Authorization": "${access_token}"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "result",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "displaycount",
"batchSize": 50,
"totalCountPath": "completeResponseMap.total"
}
}
}

Please help here @rushikeshvartak , @AashishD.

[Raghu]

can you try remove v5 in url and try pleae

[AuvikDey]

It does not give any response in Postman and i hope it will not give any response in Saviynt2Saviynt connector as well. Please see attached screenshot

Any other suggestions please

[rushikeshvartak]

Does it works from postman ?

[AuvikDey]

@rushikeshvartak ,
No the below URL is not working from Postman :
 https://@@BASEURL@@/ECM/api/v5/fetchRuntimeControlsData

However, the URL below is giving results in Postman, but only after the analytics is run manually on the console:

https://@@BASEURL@@/ECM/api/v5/fetchControlDetailsES


Moreover, the SAV4SAV rest connection is not able to update for more than 50 users at a time.
I have included Pagination in the userImport JSON, still it is not working:
{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/fetchControlDetailsES",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1520\"}",
"httpHeaders": {
"Authorization": "${access_token}"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "result",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "displaycount",
"batchSize": 50,
"totalCountPath": "completeResponseMap.total"
}
}
}

[rushikeshvartak]

It seems your pagination is not working properly and you should use runtime report instead of normal report

[AuvikDey]

Hi @rushikeshvartak ,
Thanks for your suggestion for RunTime Analytics report.
I created a Runtime Analytics report with same query and it is working in Postman with the below URL:
URL : https://@@BASEURL@@/ECM/api/v5/FetchRuntimeControlsDataV2
METHOD : POST
However, I believe since I am not able to add the "Allowed Action" in analytics like ("Accept"), it is not populating the data in User custom property. Please see attached screenshot.

Here is the updated UserImport JSON that I am using, but still user customproperty5 is not populating:
{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/FetchRuntimeControlsDataV2",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1521\"}",
"httpHeaders": {
"Authorization": "${access_token}"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "result",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "displaycount",
"batchSize": 50,
"totalCountPath": "completeResponseMap.total"
}
}
}

Please help with your suggestion how can we populate the customproperty5 by making the analytics work.

Also if you have a working pagination code, please send that as well and I can try that

[AuvikDey]

Hi @rushikeshvartak , 
I have tried the following JSON, it is populating the customproperty5 with required value, but it is not going in recursive manner, and stopping in the 1st 500 batch of user.


Here is the USER IMPORT JSON i tried : 
{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/FetchRuntimeControlsDataV2",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1521\", \"max\":\"500\", \"offset\":\"0\"}",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "results",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "displaycount",
"batchSize": 500,
"totalCountPath": "completeResponseMap.total"
}
}
}

I am getting around 11.5k entries in the Postman for the same analytics, but all of them are not getting stamped in one go. Is there a way to make it recursive and populate for all the users in 1 go.

regards,
Auvik

[AuvikDey]

I tried the following USERIMPORT JSON following this DOCS(https://docs.saviyntcloud.com/bundle/AzureAD-v24x/page/Content/Configuring-the-Integration-for-Impor...), as part of pagination, but still it is not working - It is still updating only 500 users(which is the max limit):

Here is the USERIMPORT JSON:

=====================================================================

{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/FetchRuntimeControlsDataV2",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1521\", \"max\":\"500\", \"offset\":\"0\"}",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "results",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "displaycount",
"batchSize": 500,
"totalCountPath": "completeResponseMap.total"
},
"nextUrl": {
"nextUrlPath": "${(response?.completeResponseMap?.get('@odata.nextLink')==null)? null : response?.completeResponseMap?.get('@odata.nextLink')}"
}
}
}

Please help me here, this has been stuck for a while now.

[AuvikDey]

Hi @rushikeshvartak ,

I believe the pagination is kind of working but however it is not updating more than 500 users(refer screenshot : JobLog) , do you know the reason.

The below is the latest Updated JSON  after which I am getting the logs (refer screenshot : Logviewer_Pagination)


User Import JSON:
==========================================================================
{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/FetchRuntimeControlsDataV2",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1521\", \"max\":\"500\", \"offset\":\"0\"}",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "results",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "displaycount",
"batchSize": 500,
"totalCountPath": "completeResponseMap.totalcount"
}
}
}

====================================================================================


Please can anyone help me on this.

@Saviynt employees, if you have an answer please check and help

[rushikeshvartak]

Refer https://forums.saviynt.com/t5/identity-governance/sav4sav-rest-account-import-through-analytics-pagi...

[naveenss]

Hi @AuvikDey  try changing the "batchParam": "displaycount", to "batchParam": "max", in your pagination logic

[AuvikDey]

Hi @naveenss , @rushikeshvartak 

I tried changing the logic in Pagination and set it to as below, still only 500 users are getting updated and rest of the users are not getting updated.

Please find the JSON below which I am trying :
===================================================================================

{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/FetchRuntimeControlsDataV2",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"1521\", \"max\":\"10000\", \"offset\":\"0\"}",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "results",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "max",
"batchSize": 500,
"totalCountPath": "completeResponseMap.totalcount"
}
}
}

==================================================================================

Please let me know how  this can be fixed.

[AuvikDey · Accepted solution]

The below JSON finally worked and the pagination works:

Please find the JSON below which I am trying :
===================================================================================

{
"connection": "userAuth",
"url": "https://@@BASEURL@@/ECM/api/v5/FetchRuntimeControlsDataV2",
"httpMethod": "POST",
"httpContentType": "application/json",
"httpParams": "{\"analyticsid\":\"XXXX\"}",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"colsToPropsMap": {
"username": "USERNAME~#~char",
"customproperty5": "CUSTOMPROPERTY5~#~char"
},
"userResponsePath": "results",
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "max",
"batchSize": 500,
"totalCountPath": "completeResponseMap.totalcount"
}
}
}

==================================================================================