cancel
Showing results for 
Search instead for 
Did you mean: 

Did you know? We have the ability to run parallel and multi-threaded application data import jobs

Community_User
Saviynt Employee
Saviynt Employee
Originally posted on June 5 2020 at 23:42 UTC

1. When do we run parallel import jobs?

When customers want to reconcile multiple applications at the same time

2. Which connectors support parallel import jobs?

All out of the box connectors

3. Which connectors support multi-threaded import jobs?

Box, Office365, GoogleApps

4. What is the need to run multi-threaded import jobs?

When Saviynt was required to perform file imports for scanning from cloud providers(Box, Office365, GoogleApps) and number of files ~ 1 million, it took days to import these files. This caused delay in overall import and scan completion time

By introducing the multi-threaded import capability, we are able to successfully import multiple files simultaneously from target applications like Box, O365 and GoogleApps within few hours

5. Are there any pre-configurations necessary to set up parallel import jobs?

jobs.enableparallelimport=true

sap.usefilebasedapproach=true // This setting is only needed for SAP parallel import jobs

Work with your respective technical partners/ team to get this configured

6. How can the parallel import jobs be executed?

Application Data Import (Parallel import) jobs need to be scheduled to be executed successfully like below




image


7. How can the multithreaded import jobs be executed?

Application Data Import (Multi-Threaded import job can be scheduled to be executed successfully like below. Important note is to add the maximum number of threads that can run at a time per server at any given point in time

To avert risk of running more number of threads than allowed, we should configure max threads/server threshold in quartz.properties org.quartz.threadPool.threadCount parameter

image


This message was previously posted on Saviynt's legacy forum by a community user and has been moved over to this forum for continued exposure.
5 REPLIES 5

Community_User
Saviynt Employee
Saviynt Employee
Originally posted on June 8 2020 at 08:28 UTC

Thanks Aarthi.

This message was previously posted on Saviynt's legacy forum by a community user and has been moved over to this forum for continued exposure.

Community_User
Saviynt Employee
Saviynt Employee
Originally posted on July 14 2021 at 07:32 UTC

HI @Aarthi / Ajay - I have few questions around this :

  1. Is providing the number of threads mandatory for the job to be multi threaded?
  2. Doesn't it automatically take the available threads?
  3. What happens if i mention the thread count = 10 and other processes ( Analytics / provisioning / email jobs are stalled because the threads are consumed by application import job?
  4. Are there any best practices about scheduling the Multi threaded account/ access import jobs?? WE have seen jobs getting stuck with the status - "This job was suspended as there is already an import running for the same endpoint"
  5. Can the account import and access import jobs for the same endpoint overlap?
This message was previously posted on Saviynt's legacy forum by a community user and has been moved over to this forum for continued exposure.

Community_User
Saviynt Employee
Saviynt Employee
Originally posted on February 1 2022 at 09:00 UTC

Hi Aarthi,


We are unable to locate "Application Data Import (Parallel import) job". We are currently on v5.5SP3.7.16. Is there any specific configuration that needs to be enabled?


Regards,

Naveen

This message was previously posted on Saviynt's legacy forum by a community user and has been moved over to this forum for continued exposure.

Community_User
Saviynt Employee
Saviynt Employee
Originally posted on February 1 2022 at 12:59 UTC

Naveen, you should check 

Application Data Import (Multi Threaded) - this is the job meant for parallel imports. 

This message was previously posted on Saviynt's legacy forum by a community user and has been moved over to this forum for continued exposure.

Community_User
Saviynt Employee
Saviynt Employee
Originally posted on February 1 2022 at 13:17 UTC

Thanks for the confirmation, Kirti. May I please ask you if this job can be used to run imports parallelly? For example, running AD and Salesforce import parallelly at the same time. 


Thank you very much for your help in advance!


This message was previously posted on Saviynt's legacy forum by a community user and has been moved over to this forum for continued exposure.