User import via REST connection

[Harish7]

Hi,

I am trying to import users using 'User Import via connection' job.

This was working fine and importing users from the target - until last month.

But, now the new data which is sent by the target is not getting imported into Saviynt.

Is there any limit for importing users? if yes how can we increase the limit?

No recent changes done to connection and also the job is running successfully, but new users are not getting imported to Saviynt.

Or is there anything else i need to check?

UserImportJSON:

{
"connection": "userAuth",
"url": "https://api.COMPANYNAME.de/v1/company/employees",
"httpMethod": "GET",
"httpHeaders": {
"contentType": "application/x-www-form-urlencoded",
"Authorization": "${access_token}",
"Accept": "application/json"
},
"colsToPropsMap": {
"statuskey": "attributes.status.value~#~char",
"customproperty1": "attributes.status.value~#~char",
"companyname": "attributes.subcompany.value.attributes.name~#~char",
"departmentname": "attributes.department.value.attributes.name~#~char",
"email": "attributes.email.value~#~char",
"firstname": "attributes.first_name.value~#~char",
"lastname": "attributes.last_name.value~#~char",
"jobcode": "attributes.position.value~#~char",
"jobcodedesc": "attributes.position.value~#~char",
"startdate": "attributes.hire_date.value~#~char",
"termdate": "attributes.termination_date.value~#~char",
"enddate": "attributes.last_working_day.value~#~char",
"location": "attributes.office.value.attributes.name~#~char",
"employeeid": "attributes.dynamic_5737.value~#~char",
"employeetype": "attributes.dynamic_6743.value~#~char",
"systemusername": "attributes.id.value~#~char",
"manager": "attributes.supervisor.value.attributes.id.value"
},
"statusConfig": {
"active": [
"active",
"onboarding"
],
"inactive": "inactive"
},
"errorCode": "401",
"errorCodePath": "errorCode",
"userResponsePath": "data"
}

Thanks in advance.

Best regards

Harish

[Sharukh_Shaikh]

Hi Harish,

Can you please check if the user API is having any limitation. I assume the API returns a specific amount of users till last month and now it requires pagination support.

Could you please check the user import in postman and check if the number of users API returns is matching with the user data in Saviynt? If yes then it require pagination support for the new users.

Thanks,

Sharukh.

[rushikeshvartak]

Please share logs

Your password might be got expired

[Harish7]

Thank you @rushikeshvartak ,

Here are the logs of import job. I see some error invalid JWS- which i highlighted in the logs. Could you please suggest me what we can do to avoid this error?

Logs:
2022-08-08 07:44:06,939 [https-jsse-nio-443-exec-86] DEBUG domain.FlatViewJobcontrolController - Inside fireJobAction params = [triggername:HRIMPORT, triggergroup:GRAILS_JOBS, jobname:UserImportJob, jobgroup:DATA, actn:6, job_name:, job_group:, controller:flatViewJobcontrol, action:fireJobAction]
2022-08-08 07:44:06,945 [https-jsse-nio-443-exec-86] DEBUG domain.FlatViewJobcontrolController - executing scheduling trigger...
2022-08-08 07:44:07,081 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - START UserImportJob
2022-08-08 07:44:07,154 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - UserImportJob ID - 402164
2022-08-08 07:44:07,154 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - Start call externalConnectionCallService
2022-08-08 07:44:07,154 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - START IMPORT USER USING EXTERNAL CONNECTION
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=tabDetail Value=daily::1__01:33:00
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=userNotInFeedAction Value=NOACTION
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=UpdateUserJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=SendOtpJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=USERSOURCE Value=2
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=zeroDayProvisioning Value=TRUE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=ENDPOINTS_FILTER Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=userOperationsAllowed Value=CREATE,UPDATE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=ImportUserJSON Value={
"connection": "userAuth",
"url": "https://api.COMPANYNAME.de/v1/company/employees",
"httpMethod": "GET",
"httpHeaders": {
"contentType": "application/x-www-form-urlencoded",
"Authorization": "${access_token}",
"Accept": "application/json"
},
"colsToPropsMap": {
"statuskey": "attributes.status.value~#~char",
"customproperty1": "attributes.status.value~#~char",
"companyname": "attributes.subcompany.value.attributes.name~#~char",
"departmentname": "attributes.department.value.attributes.name~#~char",
"email": "attributes.email.value~#~char",
"firstname": "attributes.first_name.value~#~char",
"lastname": "attributes.last_name.value~#~char",
"jobcode": "attributes.position.value~#~char",
"jobcodedesc": "attributes.position.value~#~char",
"startdate": "attributes.hire_date.value~#~char",
"termdate": "attributes.termination_date.value~#~char",
"enddate": "attributes.last_working_day.value~#~char",
"location": "attributes.office.value.attributes.name~#~char",
"employeeid": "attributes.dynamic_5737.value~#~char",
"employeetype": "attributes.dynamic_6743.value~#~char",
"systemusername": "attributes.id.value~#~char",
"manager": "attributes.supervisor.value.attributes.id.value"
},
"statusConfig": {
"active": [
"active",
"onboarding"
],
"inactive": "inactive"
},
"errorCode": "401",
"errorCodePath": "errorCode",
"userResponsePath": "data"
}
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=EnableAccountJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=PasswdPolicyJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=termUserLimit Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=TicketStatusJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=ConfigJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=AddAccessJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=RemoveFFIDAccessJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=UpdateAccountJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=userStatusConfig Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=jobnamelabel Value=User Import via a Connection (UserImportJob)
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=generateEmail Value=FALSE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=ImportAccountEntJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=AddFFIDAccessJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=CreateAccountJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=connectionname Value=Personio
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=CreateTicketJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=checkRules Value=TRUE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=buildUserMap Value=TRUE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=exporttosavcloud Value=FALSE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=generateSystemUsername Value=FALSE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=userDefaultSavRole Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=cronexpression Value=0 33 01 1/1 * ? *
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=ValidateOtpJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=org.grails.plugins.quartz.grailsJobName Value=UserImportJob
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=STATUS_THRESHOLD_CONFIG Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=importsavconnect Value=FALSE
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=connectionid Value=2
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=connectiontype Value=REST
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=endpointsToAssociateOrphanAccounts Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=RemoveAccountJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=ChangePassJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=updateUser Value=harish.gandepalli.ext@COMPANYNAME.de
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=zeroDayLimit Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=MODIFYUSERDATAJSON Value={
"ADDITIONALTABLES": {
"USERS": "SELECT username,employeeid FROM USERS"
},
"COMPUTEDCOLUMNS": [
"manager",
"username",
"leavestatus",
"displayname",
"customproperty13"
],
"PREPROCESSQUERIES": [
"UPDATE NEWUSERDATA SET username = CASE WHEN email != '' THEN email ELSE systemusername END;",
"UPDATE NEWUSERDATA SET leavestatus = CASE WHEN customproperty1 = 'leave' THEN 1 ELSE 0 END;",
"UPDATE NEWUSERDATA N1 LEFT JOIN NEWUSERDATA N2 ON N1.MANAGER = N2.SYSTEMUSERNAME SET N1.manager = N2.username;",
"UPDATE NEWUSERDATA SET displayname = CONCAT_WS(' ', firstname, lastname);",
"UPDATE NEWUSERDATA SET customproperty13 = case when employeeid != '' then 'Internal' else 'External' end;"
]
}
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=RemoveAccessJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=DisableAccountJSON Value=
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=userReconcillationField Value=systemUserName
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=jobtriggername Value=HRIMPORT
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - Key=fullorincremental Value=full
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Start Import Users
2022-08-08 07:44:07,155 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Enter initializeConnectionForUserImport
2022-08-08 07:44:07,156 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Exit initializeConnectionForUserImport
2022-08-08 07:44:07,156 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - calling getUsersData method to hit Import User API
2022-08-08 07:44:07,157 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Inside getUsersData
2022-08-08 07:44:07,157 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - connection: userAuth
2022-08-08 07:44:07,252 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Adding connectionParamMap to jsonMap for ssl :
2022-08-08 07:44:07,270 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Inside pullObjectsByRest
2022-08-08 07:44:07,270 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService -
2022-08-08 07:44:07,270 [quartzScheduler_Worker-2] DEBUG services.HttpClientUtilityService - isFipsEnabled = false
2022-08-08 07:44:07,270 [quartzScheduler_Worker-2] DEBUG services.HttpClientUtilityService - getHttpClient - sslParams : null
2022-08-08 07:44:07,270 [quartzScheduler_Worker-2] DEBUG services.HttpClientUtilityService - getHttpClient - proxyParams : null
2022-08-08 07:44:07,270 [quartzScheduler_Worker-2] DEBUG services.HttpClientUtilityService - getHttpClient - sslSocketFactory : null
2022-08-08 07:44:07,347 [quartzScheduler_Worker-2] DEBUG services.HttpClientUtilityService - getHttpClient - HttpClientBuilder.create().build() called.
2022-08-08 07:44:07,703 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService -
2022-08-08 07:44:07,703 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - pullObjectsByRest - responseStatusCode ::401
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Entered getResponseHeaders method
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - responseError : Could not decode token: The token "AiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2FwaS5wZXJzb25pby5kZS92MS9hdXRoIiwiaWF0IjoxNjQ4MTM2NjE2LCJleHAiOjE2NDgyMjMwMTYsIm5iZiI6MTY0ODEzNjYxNiwianRpIjoiODg1ZmJlOGMtNjEwOS00MjZmLTk4OTctODkyNDhiZDZhYTUzIiwic3ViIjoiT1RFek5EZ3haakprWlRoaE1tRTNOalk1TTJZM01ETXkiLCJwcnYiOiI3YTE5OTQ5OTlkMTgxZGVlYTY4ZTQzMDRiMzM0NmU3OGY4MzhlY2I3Iiwic2NvcGUiOlsiZW1wbG95ZWVzOnJlYWQiLCJlbXBsb3llZXM6d3JpdGUiLCJhdHRlbmRhbmNlczpyZWFkIiwiYXR0ZW5kYW5jZXM6d3JpdGUiLCJhYnNlbmNlczpyZWFkIiwiYWJzZW5jZXM6d3JpdGUiXX0.spZItpUd25CqAnONj1FXjQ79lnGTvuoPPSUCJiPm9EE" is an invalid JWS
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - isAuthError: false
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - pullObjectsByRest - responseMap.size : 2
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - pullObjectsByRest - objectList.size : null
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - got empty objectList
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Exit getUsersData
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - processing usersData size:: 0
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - Start Process Users
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - userFieldsMetadata: statuskey,customproperty1,companyname,departmentname,email,firstname,lastname,jobcode,jobcodedesc,startdate,termdate,enddate,location,employeeid,employeetype,systemusername,manager
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - targetSystemFieldsMetadata:: attributes.status.value,attributes.status.value,attributes.subcompany.value.attributes.name,attributes.department.value.attributes.name,attributes.email.value,attributes.first_name.value,attributes.last_name.value,attributes.position.value,attributes.position.value,attributes.hire_date.value,attributes.termination_date.value,attributes.last_working_day.value,attributes.office.value.attributes.name,attributes.dynamic_5737.value,attributes.dynamic_6743.value,attributes.id.value,attributes.supervisor.value.attributes.id.value
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - End Process Users
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestUtilService - Writing job history to import logs.
2022-08-08 07:44:07,704 [quartzScheduler_Worker-2] DEBUG rest.RestUtilService - Number of log entries to be written : 0
2022-08-08 07:44:07,721 [quartzScheduler_Worker-2] DEBUG rest.RestProvisioningService - End Import Users
2022-08-08 07:44:07,721 [quartzScheduler_Worker-2] DEBUG integration.ExternalConnectionCallService - END IMPORT USER USING EXTERNAL CONNECTION
2022-08-08 07:44:07,721 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - End call externalConnectionCallService
2022-08-08 07:44:07,721 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - Setting ecmImportJob enddate: Mon Aug 08 07:44:07 UTC 2022
2022-08-08 07:44:07,721 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - Done setting ecmImportJob enddate
2022-08-08 07:44:07,744 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - Import status :: Success
2022-08-08 07:44:07,744 [quartzScheduler_Worker-2] DEBUG services.SaviyntCommonUtilityService - Enter runNextTrigger
2022-08-08 07:44:07,744 [quartzScheduler_Worker-2] DEBUG services.SaviyntCommonUtilityService - Exit runNextTrigger
2022-08-08 07:44:07,744 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - exportDataToSavCloud= FALSE
2022-08-08 07:44:07,744 [quartzScheduler_Worker-2] DEBUG jobs.UserImportJob - END UserImportJob

Best regards

Harish

[Vedanth_BK]

Hi @Harish7 ,

Analyzing the above debug logs, I noticed a 401 error code i.e Unauthorized error code. Please verify the same API call in postman using the credentials provided in the connection JSON.