User Import Preprocessor

[Community_User]

Originally posted on May 27 2020 at 16:07 UTC

Can someone explain what the user import preprocessor is doing?

I think it runs during a user import job if I've got my modifyuserdatajson configured. Will it update existing users or only run on new users?

In what context are the preprocessqueries running? The docs and samples show them using currentusers and newuserdata tables. What are those? Are those temporary? Are those used during import? What is their schema?

Why do we configure computedcolumns? It looks like the preprocessor queries are updating those columns anyway, so I don't understand why they are configured separately.

How is the dataset that is built from additionaltables referenced or used. I see users, accounts, etc built in the sample doc but I don't see those referenced in the preprocessor queries, so what is going on with those. If I did want to build a table, how would I reference it? Are these temporary tables that exist only during import?

I'd like to see a walk through of what the process is. I'm guessing during an import it builds the additional tables first, then for each user in the import it builds a newuserdata table and populates with data from the feed/schema map. Then it performs the preprocessqueries one by one to update the newuserdata. Then it inserts the newuserdata into the users table. I'm making all this up, because its not clear. Can someone confirm what is actually happening?

--Jim

[Community_User · Accepted solution]

Originally posted on May 27 2020 at 20:19 UTC

Purpose of User Inline processing is to address the use cases where certain Identity attributes needs to be calculated and saved within Saviynt Identity. Such as the Display name needs to be firstname + last name , Username should be something unique and to be generated by Saviynt, end date should be start date + 30 days by default for a contractor etc.

Technically, after you have added the ModifyUserJSON in the connection and then you run the user import, here is what happens under the hood -

a) Saviynt platform imports the data from the HR Target and stores it in a TEMP USERS TABLE, which is referred to as NEWUSERDATA table in ModifyJSON config. This NEWUSERDATA schema is exactly same as of Saviynt users schema. This table is deleted after job is completed. And next time when user import starts, new table is created afresh.

b) Also, all the tables referred in ADDITIONAL Tables config of ModifyJSON are also created by the user import process. These additional tables are referred to as CURRENT<Tablename> in PRE-PROCESSOR queries config of ModifyJSON. The purpose of these tables is to copy the data from the existing Saviynt Identity Repository and made them available in temporary tables so that while executing pre-processing queries, these temp tables can be referred/joined to implement the desired business logic. These tables are also deleted as soon as job is finished and re-created next time when a new user import starts. Important is to ensure that dataset in these Additional tables should be minimum.

c) After all the temp tables are created i.e NEWUSERDATA which has HR data and Current<tables>, pre-processing queries are executed and at the end of query execution, NEWUSERDATA table has all the data ready.

d) Now, NEWUSERDATA is updated back to actual Saviynt Users table data. You can consider this as a scenario where NEWUSERDATA is the source of user import for Saviynt users table internally.

e) COMPUTED COLUMN configuration is to capture which columns needs to be computed as part of this User Inline processing configs.

f) After data is saved in Saviynt users tables, all tables are deleted and finally we have HR data long with computed columns in Saviynt users.

Note: You can have a scenario where business logic to calculate value of a user property cannot be implemented by Pre-processing query i.e a MYSQL statement and in such cases you want to implement a JAVA code to do the logic processing. So, User Inline config does have the ability to call your JAR to update NEWUSERDATA table

[Community_User]

Originally posted on June 18 2020 at 07:45 UTC

Hi ,

Is the modifyuserdataJSON working for DB connectors as well?

Please let me know.

Thanks

[Community_User]

Originally posted on September 21 2021 at 09:07 UTC

Yes it will work for DB connector as well.

[Community_User]

Originally posted on October 26 2021 at 07:08 UTC

If user is created via create user API, and we want to normalise,  compute few user attributes. How's that feasible.