Improve Create user API performance by inlineruleevaluation, Batchidentifier & ProcessRules

[Darshanjain]

Use Case

Saviynt create and update user APIs are used to create/update user identities in Saviynt. This means there will large number of API calls as there can be large number of user records and for every record there will be an API call.

Moreover there can also be multiple sources from where Saviynt is receiving updates from different feeders via API Calls of different volumes at different time intervals. 

Pre-requisites

Basic Knowledge on the processRules API - This API allows the Saviynt Admin to run rules for unprocessed data in userrulerundata table manually

{
 "jobidstoprocess" : "0",   /**mention the parentjobid to run unprocessed rulerun for importjobentries**/
"runzerodayrules" : "true",  /** param to runzerodaytechrules **/
"totaljobstorun" : "2",        /** Mention the number of threads of rulerun job to be created **/
"batchidentifier" : "testbatch1"  /** batchidentifier for processing **/
 }

jobidstoprocess - Should be 0 for processing webservice related entries in the userrulerundata table. Should be the parent jobid for processing the User Import related entries in the userrulerundata tablebatchidentifier - This is not a mandatory parameter. If we are using batch identifier in the createuser and updateuser API only then, it will make sense to use it in the processrules API.totaljobstorun - This count should not increase the maximum count of rulerunjobcount in the externalconfig.properties

Applicable Version(s)

All

Solution

For every API call, Saviynt will try to evaluate all the rules configured in the solution. Since there can be large number of API call, even a small delay in each call can add up to cause overall performance issue.

To avoid evaluation of rules add the parameter inlineruleevaluation=false in the Create User and Update User API call

Once all the new/updates for user record are completed from the external system calling Saviynt API and the Rule Run Entries for each user is being added into the userrulerundatatable. Then invoke /api/processrules API - this will trigger the rule evaluation for the users.

So we are creating and updating users via API with inlineruleevaluation=false. The Rule Run Entries for each user is being added into the userrulerundatatable. Finally once the Feeder has dumped all the data, /api/processrules API, is called to run the rules for the users. 

In case you have multiple sources/feeders sending API request at different timeslots and managing the /api/processrules is difficult as every feeder calls this api at the end; then in that case make use of batchidentifier parameter in createUser API and updateUser API. 

The External application will call the createUser and updateUser API with inlineruleevaluation=false and sets this wsbatchidentifier  for each and every create User and update User call
This value will be persisted against a column wsbatchidentifier in the userrulerundata table directly.
For example:

CreateUser API:

Update User API:

The rule run table will store the batch identifier information passed in the API. Refer below example

Userrulerundata table:

Now pass the batch identifier information in the ProcessRule API as follows:

   

Summary -

After making all the updates through create/update API, the External application should run the rules by calling /api/processrules API, with the same batchidentifier, so that only for the entries with this batchidentifier in the userrulerundata table will be picked up by the Rule Run jobs for Rule Evaluation .

• In case of bulk user create/update using API, use the option inlineruleevaluation=false to improve API performance
• Call the processrules API to evaluate the rules
• In case you want to restrict processrules API to only certain users, then make use of batchidentifier parameter so that the change will be added into userrulerundata table populating the wsbatchidentifier parameter
• Now, when the processRules API is called, mention the batchidentifier parameter in the API call to process only the selected entries.

References

https://docs.saviyntcloud.com/bundle/TPAG-Guide-v2020x/page/Content/Create-Users-using-API.htm

https://documenter.getpostman.com/view/1797923/SzzgAepY#54b28cfc-0c7f-40e4-998e-21abe6186820

[Paul_Meyer]

@Darshanjain 

Thank you for publishing this article. Very useful.

There is however contradicting statements between the article and the API documentation:

The article states :

• In case of bulk user create/update using API, use the option inlineruleevaluation=true to improve API performance.
• To avoid evaluation of rules add the parameter inlineruleevaluation=true in the Create User and Update User API call.
  

The API documentation for both /createUser and /updateUser states: "It's highly recommended to pass inlineruleevaluation:"false" for bulk load and call processrules api to evaluate and run the rules after the load is done."

The example API payloads show inlineruleevaluation as "false".

Please advise.

[Darshanjain]

Hi @Paul_Meyer 

Thanks for notifying it. Yes it should be false and i have corrected it as well.

Thanks

Darshan

[mmomin-xalient]

Hello, this API is not working on v5.5 SP12. When we are calling the API, we are getting 403 forbidden error. Below is the API request from Postman. I tried using api/v5 as well and same error. Was able to test this in v24.3 and it is working fine there. 

curl --location 'https://SUB_DOMAIN.saviyntcloud.com/ECM/api/processrules' \

--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJwcmluY2lwYWwiOiJINHNJQUFBQUFBQUFBSlZTUDA4VVFSUlwvZXh5QmdGRWcwWVFDRzdFemM0bVcxd2lJUnJNY3h2TWFURFJ6dTQ5MVlIWm1uSm1GdThaY2hRVUZCQ1F4OFN2d1RiVHhBeGd0YktsdGZiTWM3R2xEbkdyM3pXOVwvXC85NmVuc0c0c1wvQWdzMXhJeDR3c01xR1lNMWFvekdGU1dPSDdySEJvVVwvUWw0a2tKN05BRXprOVVneWlHbWtnOXpNVmJmSWMzSkZkWlk3MjdoWWx2OWl6YzF6WWJNbTVhbnVPdXR0dnNranZSRnY4U3FLaWpoeldZMklCWm5pUzZVTDZsMVdyUENJdnBCc3hVczFnbjIyRjBNNkViVkY1dzZVYWhFNmg0VjJJYXd6UXZcL0Z0TnFnS2RoeHZuWmdzdlpLT052aG5EcE9IT2tidFwva3JSOXNCN3VnMDFGQ2Q3QmU2ajNURVNIdXJzYm9DendzQlV0SmFVV1dybkZqc3AxS2paRkVDZit3Y0xSdDRQUGcwNE5nRHE1ZFwvVTMxWHgrR1FaZlh2KytYUllkSlI1dWpWaXZZTTJlSVRlekZmTkxpMEg1KzZmbnh5ZG5IMTZOa1hKQVBQN1wvZlN3dURadnJyK2pjY011OUh0a1IwZTdXd3pPUkwxOU5mckdGUG11TDNFaWtQMHA1VEM4bEttS0tXN2RhWHZUdFllckZlcno2WnVuUjJ0TldlTDBtZU02TllEek5oU0x4NjJYMnNEUVdhMXJaXC9xXC9EcndkM2ZoRFJNeGpmNGJKQXFuNm1BcldLdkl0MjdcL1JrWWZyanpcLzB5eVBDbm52d0RpSWVVV2hnREFBQT0iLCJzdWIiOiJpYW1hcGkuYWRtaW4iLCJleHAiOjE3MjIwMDIxNjQsImlhdCI6MTcyMjAwMDM2NCwicm9sZXMiOlsiUk9MRV9BRE1JTiJdfQ.gb43iV' \

--header 'Content-Type: application/json' \

--data '{

    "jobidstoprocess": "7736566",

    "runzerodayrules": "true",

    "totaljobstorun": "2",

    "fetchroles": "true"

}'