Announcing the Saviynt Knowledge Exchange unifying the Saviynt forums, documentation, training,
and more in a single search tool across platforms. Read the announcement here.

User UpdateDate and ServiceNow App Incremental Import

BarCar
Regular Contributor
Regular Contributor

We have an authoritative employee feed from our HR system into Saviynt. Users get updated via this feed as expected. For example a manager change occurs and the Update History on the user object shows the change occurring when the import runs.

We also use the Saviynt App on ServiceNow which has a full and incremental sync jobs to update the corresponding Saviynt Users table in ServiceNow. The full job works fine but the incremental job is missing changes.

On investigation, the ServiceNow incremental import seems to be querying Saviynt for users based on the UpdateDate.

But, as can be seen in Saviynt Data Analyzer, the action of a user update via the HR import job does not change the UpdateDate property of the user in Savinyt. I'm told by support that this is "by design" and the date only gets changed for updates via the UI. Since this is our system of record for employees that seems odd.

So the incremental import job in the Saviynt App is basically useless and user data in ServiceNow is out of date until the next full import. This feels like a design flaw to me.

Has anybody else seen this issue?

12 REPLIES 12

rushikeshvartak
All-Star
All-Star

You can raise enhancement request for same


Regards,
Rushikesh Vartak
If you find the response useful, kindly consider selecting Accept As Solution and clicking on the kudos button.

I know that I can.

But fixing an incremental import job which misses changes to employee data due to an internal design choice is a defect, not an enhancement. 

You can raise support ticket if they accept it good luck else raise enhancement ticket


Regards,
Rushikesh Vartak
If you find the response useful, kindly consider selecting Accept As Solution and clicking on the kudos button.

pruthvi_t
Saviynt Employee
Saviynt Employee

Hi @BarCar ,

Hope you're doing well.

Please give the below approach a try in the lower environment and see if it works.

You can map the 'sys_updated_on' value from servicenow to the updatedate of saviynt in the user import json of your servicenow connection.

Then you can use the below query in the user preprocessor (MODIFYUSERDATAJSON)

{

"ADDITIONALTABLES": {

"USERS": "Select USERNAME,UPDATEDATE from USERS"

},

"COMPUTEDCOLUMNS": [

"UPDATEDATE"

],

"PREPROCESSQUERIES": [

"UPDATE NEWUSERDATA LEFT JOIN CURRENTUSERS ON
NEWUSERDATA.USERNAME = CURRENTUSERS.USERNAME SET
NEWUSERDATA.UPDATEDATE = CURRENTUSERS.UPDATEDATE where
NEWUSERDATA.UPDATEDATE < CURRENTUSERS.UPDATEDATE"],

"TABLEINDEXES": {

"currentusers": [

"username",

"updatedate"

],

"NEWUSERDATA": [

"username",

"updatedate"

]

},

"CUSTOMFUNCTIONS": {}

}

Once change is made try running the import in savinyt and after that you can try importing incremental user import in SNOW app and see if the import is working.

If you're using the user's updatedate column in any analytics or rules, please be aware that this might affect them.

Kindly take back up of jsons before making the change just in case.

documentation for preprocessor for your reference: https://docs.saviyntcloud.com/bundle/SSM-Admin-v55x/page/Content/Chapter03-User-Management/User-Impo...

Thanks,


Regards,
Pruthvi

BarCar
Regular Contributor
Regular Contributor

Thanks Pruthvi. I appreciate the reply and the suggestion.

We have taken a slightly different route with accomodates the fact that there are internal updates being made to our User records via Sav4Sav REST jobs. Our concern was that your suggested approach might mean that Update Dates would jump back in time after the Sav4Sav job updates the record.

We have created a Sav4Sav DB connection which uses a SQL query to:

  • Join the Users and User Update History table
  • Select the latest Update History date for each user
  • Push the latest Update History date to the User's update date if it is newer than the current value

We have chained this Sav4Sav job with our source HR import so that all updated user records get a new Update Date when imported via the feed.

This means that the ServiceNow Incremental import picks up both the HR sourced changes and the Sav4Sav sourced changes and update date always moves forward in time.

I still believe that the default behaviour should be that the Update Date should reflect the date of the last change irrespective of source.

I ahve not looked yet, but I'm curious if this issue also applies to Account imports where changes to some fields are being made directly in the source system and these are not flowing into the ServiceNow app via incremental updates.

ekorh
Regular Contributor
Regular Contributor

Thank you for the workaround! We have had the exact same issue with the Saviynt snow app and this helps a lot.

Please share SQL Query


Regards,
Rushikesh Vartak
If you find the response useful, kindly consider selecting Accept As Solution and clicking on the kudos button.

BarCar
Regular Contributor
Regular Contributor

Here is the UserImport JSON for the Sav4Sav DB connection we are using:

 

<dataMapping>
    <sql-query description="This is the Source DB Query" uniquecolumnsascommaseparated="employeeid">
      <![CDATA[SELECT u.employeeid, max(uh.updatedate) as latesthistoryupdatedate 
          from users u, usershistory uh 
          WHERE u.userkey = uh.userkey and uh.updatedate > u.updatedate
          group by u.userkey
      ]]>
    </sql-query>
    <importsettings>
        <zeroDayProvisioning>false</zeroDayProvisioning>
        <generateEmail>false</generateEmail>
        <userNotInFileAction>NOACTION</userNotInFileAction>
        <checkRules>false</checkRules>
        <buildUserMap>false</buildUserMap>
        <generateSystemUsername>false</generateSystemUsername>
        <userOperationsAllowed>UPDATE</userOperationsAllowed>
        <userReconcillationField>employeeid</userReconcillationField>
    </importsettings>
    <mapper description="This is the mapping field for Saviynt Field name">
        <mapfield saviyntproperty="employeeid" sourceproperty="employeeid" type="character"/>
		<mapfield saviyntproperty="updatedate" sourceproperty="latesthistoryupdatedate" type="date"></mapfield>
    </mapper>
</dataMapping>

 

For clarity, here is the embedded SQL:

 

SELECT u.employeeid, MAX(uh.updatedate) AS latesthistoryupdatedate 
FROM users u, usershistory uh 
WHERE u.userkey = uh.userkey
AND uh.updatedate > u.updatedate
GROUP BY u.userkey;

 

The logic is that when this runs, after the HR data import, we take the latest date in the User History table for each user where that date is later than the user update date. And we udpate those user records.

I still can get my head around why Saviynt would not do this OOTB - it's importing from the source of truth so the update date on the target record should be modified to reflect changes from that source of truth. Seems like it was not designed with incremental updates in mind.

My ideas to fix this are https://ideas.saviynt.com/ideas/EIC-I-5641 (for the stagnant UpdateDate issue) and  https://ideas.saviynt.com/ideas/EIC-I-5642. (for the ServiceNow incremental import issue).

Hope that helps.

ekorh
Regular Contributor
Regular Contributor

Hi,

I'm trying to execute this in the new version of Saviynt where we do not have DB connectors. I am able to update the history date to user update date with a runtime analytic but there are some issues.

With this approach I'm not able to pass both date and time to the user updatedate column, only the date part of the history updatedate value. I have tried to troubleshoot this multiple ways but with no luck.

I guess this is why we still experience the issue that the incremental update does not pick the users.

BarCar
Regular Contributor
Regular Contributor

What version are you running? We're on 23.10 and just asked support to set up credentials on a Sav4Sav DB connector. We created it as follows:

  • Connection Type: DB
  • URL: jdbc:mysql://<instancename>prodmysql.mysql.database.azure.com:3306/saviyntazure?serverTimezone=UTC&useUnicode=yes&characterEncoding=UTF-8
  • Username: saviyntadmin@<username>prodmysql
  • Password: <set by Saviynt support>
  • Drivername: com.mysql.jdbc.Driver
  • UserImportJSON: as per prior post

And then logged a support case for them to check the URL/username and set the password. Hopefully that will be enough to help you out.

The unchanged updatedate is a defect/feature in the Saviynt design (depending on your perspective) so, without this or a similar workaround, incremental imports in the ServiceNow app will miss changes from your authoritative source.

I'm lobbying Savinyt to fix the design flaw but, for now, they say that it's working as designed. I'd encourage you to vote for the relevant ideas:

BarCar
Regular Contributor
Regular Contributor

I'm told that Saviynt are engineering a solution to this issue.

[This post has been edited by a Moderator to remove sensitive information.]

ekorh
Regular Contributor
Regular Contributor

That is amazing news! We also got this working with the DB connection but you are right, this should be a feature that is already included in the solution. Voted for all the ideas.