Provisioning wsretry thread count?

[Community_User]

Originally posted on November 15 2021 at 11:38 UTC

Hi

What this wsretry thread count means in practise?

Is it always better to set threads to 20 than 2?

Is there, and what is the downside of using 20 instead of 2?

How can i figure out, should i set this to 2 or 5 or 20?

What happens to tasks 291, 292, 293, if configured to have 2 threads. Are those tasks 291, 292, 293 waiting for next job to run, since 2 threads can handle 290 tasks?

Found from freshdesk:

WSRETRY Multi-Threading Overview 

Starting from Release 5.4.0, the WSRETRY Job provides the capability to use multi-threading to process tasks. The advantage of using multi-threading is one thread can parallelly process multiple tasks (1 thread processes 149 tasks). Hence, for example, if you want to process 290 tasks, you can specify the thread count as two. 

Additionally, you can filter the tasks based on the task type. Secondly, you can also use a customized SQL query to filter task type using Advanced option. The maximum number of threads that can be configured is 1-20 (default thread=1, maximum thread=20).  

Regards

Riku

[Community_User · Accepted solution]

Originally posted on November 15 2021 at 19:00 UTC

Hi Riku,

To Answer your question, we would need to understand how wsthread works in Saviynt.

The following configurations influence the way multi-threading works.

1. When the WSRetry job picks the tasks from the pending tasks list, it will first create a parent trigger (say Level-1(L1) trigger). 

2. The parent trigger checks if the number of tasks is less than or equal to wsretry batch size (wsretrybatch) in externalconfig.properties,

a. If less the parent trigger (L1 trigger) processes the tasks

               b. Else the system creates child triggers (L2 triggers) equal to WSRetry Thread Count mentioned in UI. Now for each L2 trigger:

1. If the task count <= wsretrybatch the child trigger processes the tasks
2. Else system will create sub-child triggers (say L3 triggers) equal to WSRetry Thread Count for each L2 trigger and check if task count <= wsretrybatch  for L2 trigger. If it’s less the L2 triggers process the tasks else it will create next level child triggers (L3 triggers) similar to the above logic and continue the process until each sub-child trigger has maximum of tasks equal to batch size

Below is a depiction of the above logic for a sample set:

Hope this helps.

Regards,

Avinash Chhetri

[Community_User]

Originally posted on November 16 2021 at 06:20 UTC

Hi

Thank you very much for this great answer.

Still i am wondering what my WSRetry Thread Count should be? How to find out optimal number and what is downside setting it to 20 which seems to be fastest, any example cases? 

And should i in some situations change wsretrybatch up or down from 500, any example cases?

Regards

Riku

[Community_User]

Originally posted on November 17 2021 at 00:05 UTC

Hi Riku,

Since wsthread count is dependent on a lot of things, it is not easy to say and would probably depend on your environment. 

What is the number of average tasks created on each provisioning run ? How frequently do they run ?

How many tasks are created during the peak time of the day ?  

How are your user/account imports scheduled, how much is the time taken for these imports ? Do they run concurrently with the provisioning ? etc

Let's say if you have org.quartz.threadPool.threadCount=20 and WSRETRY Thread Count in UI as 20 and WSRetry job is running and has spawned 20 threads and they are all under execution:

1. At the same time if a user import job is triggered, it will wait for one of the threads consumed by WSRETRY to complete
2. At the same time if an account import job is triggered, it will wait for one of the threads consumed by WSRETRY to complete

You will have to factor in these considerations when you set your values.

Regards,

Avinash Chhetri

[Community_User]

Originally posted on November 17 2021 at 06:15 UTC

Hi

Thank you again. Now i can start planning my wsretry configurations based on your answers. 

This explained to me clearly why i should not set thread count in UI to 20.

Regards

Riku