sap success factor increment connector - #1623445

[musthak_ahamad]

Hi 

we are  trying to implement the incremental connector in sap success factor,  the full import is already working fine. we are trying to import user records based on lastmodified time stamp from sap. we are able to manually update the lastmodified time stamp and bring the records via postman but setting the time dynamically from saviynt is not working . below are the json that i used in postman to manually pull records based on lastmodifieddate  from user entity, but we need help in substituting the manual time stamp into job last ran time stamp, to make it dynamically. 

postman json query:

https://********.com:443/odata/v2/User?&$format=json&$select=username,lastModified,lastModifiedDateT... ge datetimeoffset'2023-04-26T00:00:00Z'

the below recommended query is not working . and i have attached the error logs for this .

"connection": "userAuth",

"url": "https://*******.com/odata/v2/User/?format=json&paging=snapshot&customPageSize=1000&filter=((empInfo/...${(new Date()).gettime().format('yyyyMMddHHmmss')}&expand=empInfo/jobInfoNav/locationNav,empInfo/jobInfoNav,empInfo/personNav/personalInfoNav/customString3Nav,empInfo/personNav/customString6Nav,empInfo/jobInfoNav/employeeTypeNav,empInfo/jobInfoNav/employeeClassNav,empInfo/personNav/phoneNav,empInfo/personNav/emailNav,empInfo/personNav/employmentNav,empInfo/jobInfoNav/departmentNav,manager",

Saviynt working query:

https://*********.com:443/odata/v2/User?&$format=json&$select=empId,userId,firstName,lastName,country,zipCode,city,displayName,jobCode,title,addressLine1,state,mi,department,email,businessPhone,location,origHireDate,status,hireDate,isPrimaryAssignment,division,empInfo/personNav/personalInfoNav/preferredName,empInfo/jobInfoNav/emplStatusNav/localeLabel,empInfo/jobInfoNav/customString4Nav/localeLabel,empInfo/jobInfoNav/customString102Nav/localeLabel,empInfo/jobInfoNav/employmentTypeNav/localeLabel,empInfo/jobInfoNav/employmentType,empInfo/jobInfoNav/department,empInfo/jobInfoNav/division,empInfo/jobInfoNav/costCenter,empInfo/jobInfoNav/managerId,empInfo/jobInfoNav/positionEntryDate,empInfo/jobInfoNav/company,empInfo/endDate,empInfo/isContingentWorker,userIdOfEmployeeTimeNav/approvalStatus,userIdOfEmployeeTimeNav/loaActualReturnDate,userIdOfEmployeeTimeNav/startDate,empInfo/workOrderNav/vendor,empInfo/personNav/emailNav/emailAddress,empInfo/personNav/emailNav/emailType,empInfo/personNav/phoneNav/phoneNumber,empInfo/jobInfoNav/managerUserNav/empId,empInfo/jobInfoNav/eventNav/localeLabel,empInfo/jobInfoNav/customString7Nav/localeLabel,empInfo/jobInfoNav/locationNav/addressNavDEFLT/address1,empInfo/jobInfoNav/locationNav/addressNavDEFLT/city,empInfo/jobInfoNav/locationNav/addressNavDEFLT/zipCode,empInfo/jobInfoNav/locationNav/addressNavDEFLT/country,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav/localeLabel,empInfo/customString100Nav/localeLabel,userSysIdOfWorkOrderNav/cust_TerminationTimeNav/label_en_US&$filter=(status%20eq%20't'%20or%20status%20eq%20'f')%20and%20lastModifiedDateTime%20ge%20datetimeoffset'2023-04-30T00:00:00Z'&$expand=empInfo/personNav/personalInfoNav,empInfo/jobInfoNav/emplStatusNav,empInfo/jobInfoNav/customString4Nav,empInfo/jobInfoNav/customString102Nav,empInfo/jobInfoNav,empInfo,userIdOfEmployeeTimeNav,empInfo/workOrderNav,empInfo/personNav/phoneNav,empInfo/personNav/emailNav,empInfo/jobInfoNav/employmentTypeNav,empInfo/jobInfoNav/managerUserNav,empInfo/jobInfoNav/customString7Nav,empInfo/jobInfoNav/eventNav,empInfo/jobInfoNav/locationNav/addressNavDEFLT,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav,empInfo/customString100Nav,userSysIdOfWorkOrderNav/cust_TerminationTimeNav

[This post has been edited by a Moderator to mask the url.]

[timchengappa]

Hi @musthak_ahamad 

While I check the supportability for the SAP SuccessFactors incremental import and get back to you, can I ask where you got the recommendation from?

the below recommended query is not working . and i have attached the error logs for this .

"connection": "userAuth",

"url": "https://*******.com/odata/v2/User/?format=json&paging=snapshot&customPageSize=1000&filter=((empInfo/...${(new Date()).gettime().format('yyyyMMddHHmmss')}&expand=empInfo/jobInfoNav/locationNav,empInfo/jobInfoNav,empInfo/personNav/personalInfoNav/customString3Nav,empInfo/personNav/customString6Nav,empInfo/jobInfoNav/employeeTypeNav,empInfo/jobInfoNav/employeeClassNav,empInfo/personNav/phoneNav,empInfo/personNav/emailNav,empInfo/personNav/employmentNav,empInfo/jobInfoNav/departmentNav,manager",

[musthak_ahamad]

this is recommended by saviynt also i can see in saviynt forums where they use this  dynamic date format in REST connectors to provide date formats in query. but the same is not working in success factor rest connector cause not sure if saviynt can give lasttimestamp of a ran job in a  date format like this '2023-04-26T00:00:00Z' 

[rushikeshvartak]

Is the query working from postman

[musthak_ahamad]

its working if i give a manual date , but if i use above dynamic format like this it wont work

[prashantChauhan]

Hi @musthak_ahamad 

For a format like '2023-04-26T00:00:00Z' please use the dynamic format like below-

{
"connection": "acctAuth",
"url": "https://saviynt7272.zendesk.com/api/v2/users.json?${Calendar.getInstance().getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")}",


Also, please do not use (new Date()) as it has been deprecated by Saviynt.

[musthak_ahamad]

i tried this , but i couldn't find anything in logs . except null return and job failed. but this is my json ..let me know if the format and pattern is correct or not .

https://**********/odata/v2/User?&$format=json&$select=empId,userId,firstName,lastName,country,zipCode,city,displayName,jobCode,title,addressLine1,state,mi,department,email,businessPhone,location,origHireDate,status,hireDate,isPrimaryAssignment,division,empInfo/personNav/personalInfoNav/preferredName,empInfo/jobInfoNav/emplStatusNav/localeLabel,empInfo/jobInfoNav/customString4Nav/localeLabel,empInfo/jobInfoNav/customString102Nav/localeLabel,empInfo/jobInfoNav/employmentTypeNav/localeLabel,empInfo/jobInfoNav/employmentType,empInfo/jobInfoNav/department,empInfo/jobInfoNav/division,empInfo/jobInfoNav/costCenter,empInfo/jobInfoNav/managerId,empInfo/jobInfoNav/positionEntryDate,empInfo/jobInfoNav/company,empInfo/endDate,empInfo/isContingentWorker,userIdOfEmployeeTimeNav/approvalStatus,userIdOfEmployeeTimeNav/loaActualReturnDate,userIdOfEmployeeTimeNav/startDate,empInfo/workOrderNav/vendor,empInfo/personNav/emailNav/emailAddress,empInfo/personNav/emailNav/emailType,empInfo/personNav/phoneNav/phoneNumber,empInfo/jobInfoNav/managerUserNav/empId,empInfo/jobInfoNav/eventNav/localeLabel,empInfo/jobInfoNav/customString7Nav/localeLabel,empInfo/jobInfoNav/locationNav/addressNavDEFLT/address1,empInfo/jobInfoNav/locationNav/addressNavDEFLT/city,empInfo/jobInfoNav/locationNav/addressNavDEFLT/zipCode,empInfo/jobInfoNav/locationNav/addressNavDEFLT/country,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav/localeLabel,empInfo/customString100Nav/localeLabel,userSysIdOfWorkOrderNav/cust_TerminationTimeNav/label_en_US&$filter=(status%20eq%20't'%20or%20status%20eq%20'f')%20and%20isPrimaryAssignment%20eq%20'1'%20and%20lastModifiedDateTime%20ge%20${Calendar.getInstance().getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")}&$expand=empInfo/personNav/personalInfoNav,empInfo/jobInfoNav/emplStatusNav,empInfo/jobInfoNav/customString4Nav,empInfo/jobInfoNav/customString102Nav,empInfo/jobInfoNav,empInfo,userIdOfEmployeeTimeNav,empInfo/workOrderNav,empInfo/personNav/phoneNav,empInfo/personNav/emailNav,empInfo/jobInfoNav/employmentTypeNav,empInfo/jobInfoNav/managerUserNav,empInfo/jobInfoNav/customString7Nav,empInfo/jobInfoNav/eventNav,empInfo/jobInfoNav/locationNav/addressNavDEFLT,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav,empInfo/customString100Nav,userSysIdOfWorkOrderNav/cust_TerminationTimeNav

[prashantChauhan]

Hi @musthak_ahamad 

REST Connector expects $ to be URL encoded since ${} is used for its binding variables as well.

So, we need to replace $ used in the URL other than for binding variables. Please use the below JSON element-

"url": "https://**********/odata/v2/User?&%24format=json&%24select=empId,userId,firstName,lastName,country,zipCode,city,displayName,jobCode,title,addressLine1,state,mi,department,email,businessPhone,location,origHireDate,status,hireDate,isPrimaryAssignment,division,empInfo/personNav/personalInfoNav/preferredName,empInfo/jobInfoNav/emplStatusNav/localeLabel,empInfo/jobInfoNav/customString4Nav/localeLabel,empInfo/jobInfoNav/customString102Nav/localeLabel,empInfo/jobInfoNav/employmentTypeNav/localeLabel,empInfo/jobInfoNav/employmentType,empInfo/jobInfoNav/department,empInfo/jobInfoNav/division,empInfo/jobInfoNav/costCenter,empInfo/jobInfoNav/managerId,empInfo/jobInfoNav/positionEntryDate,empInfo/jobInfoNav/company,empInfo/endDate,empInfo/isContingentWorker,userIdOfEmployeeTimeNav/approvalStatus,userIdOfEmployeeTimeNav/loaActualReturnDate,userIdOfEmployeeTimeNav/startDate,empInfo/workOrderNav/vendor,empInfo/personNav/emailNav/emailAddress,empInfo/personNav/emailNav/emailType,empInfo/personNav/phoneNav/phoneNumber,empInfo/jobInfoNav/managerUserNav/empId,empInfo/jobInfoNav/eventNav/localeLabel,empInfo/jobInfoNav/customString7Nav/localeLabel,empInfo/jobInfoNav/locationNav/addressNavDEFLT/address1,empInfo/jobInfoNav/locationNav/addressNavDEFLT/city,empInfo/jobInfoNav/locationNav/addressNavDEFLT/zipCode,empInfo/jobInfoNav/locationNav/addressNavDEFLT/country,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav/localeLabel,empInfo/customString100Nav/localeLabel,userSysIdOfWorkOrderNav/cust_TerminationTimeNav/label_en_US&%24filter=(status%20eq%20't'%20or%20status%20eq%20'f')%20and%20isPrimaryAssignment%20eq%20'1'%20and%20lastModifiedDateTime%20ge%20${Calendar.getInstance().getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")}&%24expand=empInfo/personNav/personalInfoNav,empInfo/jobInfoNav/emplStatusNav,empInfo/jobInfoNav/customString4Nav,empInfo/jobInfoNav/customString102Nav,empInfo/jobInfoNav,empInfo,userIdOfEmployeeTimeNav,empInfo/workOrderNav,empInfo/personNav/phoneNav,empInfo/personNav/emailNav,empInfo/jobInfoNav/employmentTypeNav,empInfo/jobInfoNav/managerUserNav,empInfo/jobInfoNav/customString7Nav,empInfo/jobInfoNav/eventNav,empInfo/jobInfoNav/locationNav/addressNavDEFLT,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav,empInfo/customString100Nav,userSysIdOfWorkOrderNav/cust_TerminationTimeNav",

[musthak_ahamad]

Hi Prashant,

thanks for your reply , i replaced  and ran the job , the job got success but nothing got pulled from sap. if you see the log screenshot  this just prints the command we wrote in json, is there a way to know what is the output of that calender....** . cause i expect it to print in logs so i can correct it but its not , also the job doesn't give any result , the result is empty.. if we could find the format of the date then we can fix this .

[prashantChauhan]

Hi @musthak_ahamad 

You can set the showLogs config to true in the connection(ConfigJSON parameter) and it should show the date output. 

It must be taking the current date here.

Also, note that the REST Connector does not support Incremental Import. This might help you bring in the data based on the date filter you will apply in the API but there might be a data loss since the Connector is not comparing the Last job run date (saviynt) and the last update date field (target) that happens in the true Incremental Import. 

I would recommend creating an enhancement request in the Idead Portal to get the Incremental Import support in REST Connector.

[musthak_ahamad]

sure , we are looking for the true incremental support like passing the  last job ran timestamp and compare it with target , cause if its passing the current timestamp we won't get any data . i will create it in idea portal for enhancement. is there a api in saviynt to pass the last time stamp of a job ran ..?

[musthak_ahamad]

Hi @Vedanth_BK 

as per your earlier post "Calculate dates dynamically in URL: REST"  you mentioned that you are able to pass the dynamic variable randomly in sap success factor json. am am trying to do the same to create a incremental scenario,  i tried all possibilities as below. but still no success . am using 23.4 version saviynt. **attached my json for better understanding.

1. ${(new Date()).gettime().format('yyyyMMddHHmmss')}
2. ${Calendar.getInstance().getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")}
3. ${Calendar.getInstance().getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")}-2

the above specified is my requirement and but am still getting null response from target. providing it manual leads to success eventually, but is there a way to fulfil my requirment am in dead end here. or am trying here wrong ...reviewing the above formats and examples let me know if you are able to meet the requirement or success in your json.

PS : @Vedanth_BK  am looping you here cause am not able to edit or reply to you older post since its locked.

[Vedanth_BK]

Hi @musthak_ahamad 

Please try the below:

"url": "***********/odata/v2/User?&\\$format=json&\\$select=empId,userId,firstName,lastName,country,zipCode,city,displayName,jobCode,title,addressLine1,state,mi,department,email,businessPhone,location,origHireDate,status,hireDate,isPrimaryAssignment,division,empInfo/personNav/personalInfoNav/preferredName,empInfo/jobInfoNav/emplStatusNav/localeLabel,empInfo/jobInfoNav/customString4Nav/localeLabel,empInfo/jobInfoNav/customString102Nav/localeLabel,empInfo/jobInfoNav/employmentTypeNav/localeLabel,empInfo/jobInfoNav/employmentType,empInfo/jobInfoNav/department,empInfo/jobInfoNav/division,empInfo/jobInfoNav/costCenter,empInfo/jobInfoNav/managerId,empInfo/jobInfoNav/positionEntryDate,empInfo/jobInfoNav/company,empInfo/endDate,empInfo/isContingentWorker,userIdOfEmployeeTimeNav/approvalStatus,userIdOfEmployeeTimeNav/loaActualReturnDate,userIdOfEmployeeTimeNav/startDate,empInfo/workOrderNav/vendor,empInfo/personNav/emailNav/emailAddress,empInfo/personNav/emailNav/emailType,empInfo/personNav/phoneNav/phoneNumber,empInfo/jobInfoNav/managerUserNav/empId,empInfo/jobInfoNav/eventNav/localeLabel,empInfo/jobInfoNav/customString7Nav/localeLabel,empInfo/jobInfoNav/locationNav/addressNavDEFLT/address1,empInfo/jobInfoNav/locationNav/addressNavDEFLT/city,empInfo/jobInfoNav/locationNav/addressNavDEFLT/zipCode,empInfo/jobInfoNav/locationNav/addressNavDEFLT/country,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav/localeLabel,empInfo/customString100Nav/localeLabel,userSysIdOfWorkOrderNav/cust_TerminationTimeNav/label_en_US&\\$filter=(status%20eq%20't'%20or%20status%20eq%20'f')%20and%20lastModifiedDateTime%20ge%20datetimeoffset'${Calendar.getInstance().getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")}'&\\$expand=empInfo/personNav/personalInfoNav,empInfo/jobInfoNav/emplStatusNav,empInfo/jobInfoNav/customString4Nav,empInfo/jobInfoNav/customString102Nav,empInfo/jobInfoNav,empInfo,userIdOfEmployeeTimeNav,empInfo/workOrderNav,empInfo/personNav/phoneNav,empInfo/personNav/emailNav,empInfo/jobInfoNav/employmentTypeNav,empInfo/jobInfoNav/managerUserNav,empInfo/jobInfoNav/customString7Nav,empInfo/jobInfoNav/eventNav,empInfo/jobInfoNav/locationNav/addressNavDEFLT,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav,empInfo/customString100Nav,userSysIdOfWorkOrderNav/cust_TerminationTimeNav",

If you still face issues, please share the working URL from the postman.

Thank you 
Vedanth B.K

[musthak_ahamad]

Hi @Vedanth_BK 

this still giving me null response from target. i have attached the working json below from postman. your json is working from postman if i give manual date instead of calender. our requirement is to  bring  recently modified data from sap. i have attached logs for your clarification. sent the json to you in inbox.

1.) am not able to see the output of the ${Calendar.getInstance().getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")} in saviynt logs , even after i have provided showlogs in config json.

2.) is there a way to minus the calendar time and so that it can show only the past records from current timestamp.

3.) there is a possibility that the calendar time is sending the current time and thats why may be no data . but the job history should show at least 0 users count, its just simple nothing  as below.

[prashantChauhan]

Hi @musthak_ahamad 

I suspect that the API response is null due to sending the filter of the current date as there would be no record of modification after the current date.

Let me check and get back to you if there is a way to minus the time from the calendar time or send the last job time.

Thanks.

[prashantChauhan]

Hi @musthak_ahamad 

Passing the last job run time is not possible but you can minus the calendar time by using the below format. 

Calendar aa=Calendar.getInstance(); aa.add(Calendar.DATE,-2);aa.getTime().format("yyyy-MM-dd'T'HH:mm:ss'Z'")

Sample to minus 2 days-

"url": "***********/odata/v2/User?&\\$format=json&\\$select=empId,userId,firstName,lastName,country,zipCode,city,displayName,jobCode,title,addressLine1,state,mi,department,email,businessPhone,location,origHireDate,status,hireDate,isPrimaryAssignment,division,empInfo/personNav/personalInfoNav/preferredName,empInfo/jobInfoNav/emplStatusNav/localeLabel,empInfo/jobInfoNav/customString4Nav/localeLabel,empInfo/jobInfoNav/customString102Nav/localeLabel,empInfo/jobInfoNav/employmentTypeNav/localeLabel,empInfo/jobInfoNav/employmentType,empInfo/jobInfoNav/department,empInfo/jobInfoNav/division,empInfo/jobInfoNav/costCenter,empInfo/jobInfoNav/managerId,empInfo/jobInfoNav/positionEntryDate,empInfo/jobInfoNav/company,empInfo/endDate,empInfo/isContingentWorker,userIdOfEmployeeTimeNav/approvalStatus,userIdOfEmployeeTimeNav/loaActualReturnDate,userIdOfEmployeeTimeNav/startDate,empInfo/workOrderNav/vendor,empInfo/personNav/emailNav/emailAddress,empInfo/personNav/emailNav/emailType,empInfo/personNav/phoneNav/phoneNumber,empInfo/jobInfoNav/managerUserNav/empId,empInfo/jobInfoNav/eventNav/localeLabel,empInfo/jobInfoNav/customString7Nav/localeLabel,empInfo/jobInfoNav/locationNav/addressNavDEFLT/address1,empInfo/jobInfoNav/locationNav/addressNavDEFLT/city,empInfo/jobInfoNav/locationNav/addressNavDEFLT/zipCode,empInfo/jobInfoNav/locationNav/addressNavDEFLT/country,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav/localeLabel,empInfo/customString100Nav/localeLabel,userSysIdOfWorkOrderNav/cust_TerminationTimeNav/label_en_US&\\$filter=(status%20eq%20't'%20or%20status%20eq%20'f')%20and%20lastModifiedDateTime%20ge%20datetimeoffset'${Calendar aa=Calendar.getInstance(); aa.add(Calendar.DATE,-2);aa.getTime().format(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")}'&\\$expand=empInfo/personNav/personalInfoNav,empInfo/jobInfoNav/emplStatusNav,empInfo/jobInfoNav/customString4Nav,empInfo/jobInfoNav/customString102Nav,empInfo/jobInfoNav,empInfo,userIdOfEmployeeTimeNav,empInfo/workOrderNav,empInfo/personNav/phoneNav,empInfo/personNav/emailNav,empInfo/jobInfoNav/employmentTypeNav,empInfo/jobInfoNav/managerUserNav,empInfo/jobInfoNav/customString7Nav,empInfo/jobInfoNav/eventNav,empInfo/jobInfoNav/locationNav/addressNavDEFLT,empInfo/jobInfoNav/locationNav/addressNavDEFLT/stateNav,empInfo/customString100Nav,userSysIdOfWorkOrderNav/cust_TerminationTimeNav",

[reshma_sonawane]

Hi Prashant,

We tried with above however it is throwing  BadRequestException in postman and "NullResponseFromTarget" in saviynt.

Thanks,

Reshma

[prashantChauhan]

Hi @musthak_ahamad @reshma_sonawane 

Can you please share the ConnectionJSON and ImportJSON that you are using?

Also, if you can share the postman API which is working with the filter? It will help me debug the issue and provide you the correct JSONs to use.

You can mask any sensitive info that is present.

Thanks.

[musthak_ahamad]

Hi Prashant,

i have attached the sample json for you reference.

[prashantChauhan]

Hi @musthak_ahamad 

Can you also share the complete debug logs with show logs set to true in the connection?