REST Connector Multi-thread settings / default value

[AmitM]

HI Team,

Is there a way to know default thread value for REST connector application import job (multi-thread)? Can it be modified? 

Job control doesn't seem to have any option to do that.

We have three REST connectors and for all of it, account import and access import performance is not ideal. It takes sometimes 4 hours for 4k records.

Have anyone tried increasing thread count ? if product allow.

There is a relevant knowledge article where it shows below option but it is not available for REST

[sahil]

Multi threading is currently not supported for REST. In case the job is taking time, can you run the api call from postman and see if that is also taking time or is it just from the Saviynt app.

Also, can you use Application data import (single Threaded) for your job trigger instead.

[AmitM]

HI , this is the multi-thread documentation - https://docs.saviyntcloud.com/bundle/SSM-Admin-v55x/page/Content/Chapter10-Job-Control-Panel/Job-Cat...

2nd Point - Saviynt knows which connector is REST which is not, so when we create a JOB trigger multi thread and connector type is not supported. It should not let us create it, right?

[sahil]

Let me verify on this but in the mean time can you select the type to Single threaded and see if it helps. Also, do check in case there are other jobs running at the same time that might be affecting the performance. 

[AmitM]

In postman , an API call takes 330ms which will turn out to be less than an hour for 4k records

[rushikeshvartak]

Multi threading is supported by AzureAD Connector, Default thread count is 4.

4k records for 4 hours seems configuration issue. what is processingType used ?

[AmitM]

We are not using it for AzureAD connector. THe Saviynt does says it is available for REST connectors and it lets me create multi thread job too for REST conn.

Processing type is SequentialAndIterative 

{
"globalSettings": {
"dateFormat": "yyyy-MM-dd'T'HH:mm:ss"
},
"accountParams": {
"createUsers": false,
"adminName": "admin",
"connection": "acctAuth",
"processingType": "SequentialAndIterative",
"statusAndThresholdConfig": {
"statusColumn": "customproperty11",
"activeStatus": [
"true"
],
"deleteLinks": false,
"accountThresholdValue": 100,
"correlateInactiveAccounts": true,
"inactivateAccountsNotInFile": false,
"deleteAccEntForActiveAccounts": false
},
"successResponses": {
"statusCode": [
200
]
},
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "https://abc.com/api/v1/users?filter.includeInactive=true",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"httpContentType": "application/json",
"httpMethod": "GET"
},
"listField": "records",
"keyField": "accountID",
"colsToPropsMap": {
"accountID": "id~#~char",
"name": "userName~#~char",
"displayName": "displayName~#~char",
"description": "emailAddress~#~char",
"customproperty1": "domainId~#~char",
"customproperty2": "isApplicationAccount~#~char",
"customproperty3": "lastLogin~#~date",
"customproperty4": "created~#~date",
"customproperty5": "loginFailures~#~char",
"customproperty6": "isLockedOut~#~char",
"customproperty7": "isApplicationAccount~#~char",
"status": "enabled~#~char",
"customproperty11": "enabled~#~char"
},
"makeProcessingStatus": true,
"pagination": {
"offset": {
"offsetParam": "skip",
"batchParam": "take",
"batchSize": 100,
"totalCountPath": "completeResponseMap.total"
}
}
},
"call2": {
"callOrder": 1,
"stageNumber": 3,
"http": {
"url": "https://abc.com/api/v1/users/${accountName}",
"httpHeaders": {
"Authorization": "${access_token}"
},
"httpContentType": "application/json",
"httpMethod": "GET"
},
"inputParams": {
"dependentCall": true
},
"listField": "",
"keyField": "accountID",
"nextApiKeyField": "accountID",
"colsToPropsMap": {
"accountID": "id~#~char",
"name": "userName~#~char",
"displayName": "displayName~#~char",
"description": "emailAddress~#~char",
"customproperty1": "domainId~#~char",
"customproperty2": "isApplicationAccount~#~char",
"customproperty3": "lastLogin~#~char",
"customproperty4": "created~#~char",
"customproperty5": "loginFailures~#~char",
"customproperty6": "isLockedOut~#~char",
"customproperty7": "isApplicationAccount~#~char",
"customproperty8": "userLcid~#~char",
"customproperty9": "lastSessionActivity~#~char",
"customproperty10": "radiusUserName~#~char",
"customproperty12": "twoFactor~#~char",
"customproperty13": "radiusTwoFactor~#~char",
"customproperty14": "isEmailVerified~#~char",
"customproperty15": "mustVerifyEmail~#~char",
"customproperty16": "verifyEmailSentDate~#~char",
"customproperty17": "passwordLastChanged~#~char",
"customproperty18": "isEmailCopiedFromAD~#~char",
"customproperty19": "adGuid~#~char",
"customproperty20": "adAccountExpires~#~char",
"customproperty21": "resetSessionStarted~#~char",
"customproperty22": "oathTwoFactor~#~char",
"customproperty23": "oathVerified~#~char",
"customproperty24": "duoTwoFactor~#~char",
"customproperty25": "fido2TwoFactor~#~char",
"status": "enabled~#~char",
"customproperty11": "enabled~#~char"
}
}
}

[sahil]

Were you able to find if there are any other jobs that are running at the same time and might be impacting the job run. 

[AmitM]

Hi Sahil, No other job scheduled at same time as this job. I did raise a freshdesk ticket and from past 10 runs average time would be 1.5 hr. I am not sure who did the magic but it seems to work fine now.

Though sometimes it takes close to 3 hour (rarely ) and some time 45 mins. Mostly 1.5 hr.

[sahil]

Probably the support team would have worked internally to enhance the environment performance.