Saviynt unveils its cutting-edge Intelligence Suite products to revolutionize Identity Security!
Click HERE to see how Saviynt Intelligence is transforming the industry.
Saviynt Copilot Icon

File Based Import Encoding Problem

HichamElk
Regular Contributor
Regular Contributor

Hello,

We have a problem when we import users from a file based import.
Users with a first or last name with an accent are incorrectly displayed on Saviynt.
Example: José / Chloé

At first we had special characters instead of accents like this 

HichamElk_0-1720112003770.png

 


So we converted our file to UTF-8 and reimported it into Saviynt, but now a black diamond with a question mark appears instead of the accent.

HichamElk_1-1720112036002.png

 

Could you please help us correct this problem?

20 REPLIES 20

rushikeshvartak
All-Star
All-Star
  • Use database functions to remove accent characters 

Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

Hi, 

Could you detail how to do it please ? 

Use database functions from the Saviynt application ? What type of request to do ? 

Because I have 1800 users and I can't check manually what users are badly encoded 

Thanks

Ok, 

So we tried using this : 

{
    "ADDITIONALTABLES": 
    {
        "USERS": "SELECT username, firstname, lastname, displayName FROM USERS"
    },
	
    "COMPUTEDCOLUMNS": 
    [
        "username",
        "firstname",
        "lastname",
        "displayName"
    ],
	
    "PREPROCESSQUERIES": 
    [
        "UPDATE NEWUSERDATA SET firstname = FN_EIC_REPLACE(NEWUSERDATA.firstname, null)",
        "UPDATE NEWUSERDATA SET lastname = FN_EIC_REPLACE(NEWUSERDATA.lastname, null)",
        "UPDATE NEWUSERDATA SET displayName = FN_EIC_REPLACE(NEWUSERDATA.displayName, null)"
    ]
}

We entered this json in the part User Pre-Processor Config JSON in the page User File Based Import. Also our CSV file that contains those columns :

username,firstname,lastname,DisplayName

Selected First Row as Heading YES and then clicked on Upload and Preview
The preview still shows the special caracters. 

Are we doing it good or is there any mistake ? 

Thanks

Did you added preprocessor JSON during import? 


Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

We did it this way, 

HichamElk_0-1720185696367.png

We completed every field then clicked on upload and preview. Does it answer the question ? 

We haven't Imported as we don't see changes in the preview

HichamElk_1-1720185772647.png

 

Share logs


Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

Could you help us finding the logs of the preview import 

We dont' find anything related to it

Do you have any keyword to search or anything ? 

Thanks

Once you upload files share logs for last 3 minutes 


Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

You can find the export of the last 3 minutes here 

 

Try with below JSON

{"ADDITIONALTABLES":{"USERS":"SELECT username, firstname, lastname, displayName FROM USERS"},"COMPUTEDCOLUMNS":["username","firstname","lastname","displayName"],"PREPROCESSQUERIES":["UPDATE NEWUSERDATA SET firstname = FN_EIC_REPLACE(NEWUSERDATA.firstname, null)","UPDATE NEWUSERDATA SET lastname = FN_EIC_REPLACE(NEWUSERDATA.lastname, null)","UPDATE NEWUSERDATA SET displayName = FN_EIC_REPLACE(NEWUSERDATA.displayName, null)"]}

 


Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

Hi, 

This JSON doesn't work too, Saviynt seems to find no difference between old import and new one. Check the screenshot below 

HichamElk_0-1720422854469.png

Thanks 

 

HichamElk
Regular Contributor
Regular Contributor

Is it possible trying on your side the function and if it works, to show us what you have done to make it 

Thanks 

Are you changing data like firstname/lastname during import ? Please share logs


Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

We import the same data, nothing changes. 

Maybe there is something in the global configs ? We also have special characters like this on the attributes of accounts (schema upload)

HichamElk_0-1720536577785.png

 

  • Change some data in import file and validate.
  • Logs does not have any relevant data related to import file

Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

Yes you are right, 

I just imported a new file in UTF8 with the special characters corrected and it worked fine, no function needed. 

Thanks for the help 

👍Please click the 'Accept As Solution' button on the reply (or replies) that best answered your original question.


Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

Please confirm if provided preprocessor worked ? If not provide working solution to help others


Regards,
Rushikesh Vartak
If this helped you move forward, click 'Kudos'. If it solved your query, select 'Accept As Solution'.

Hi, 

Preprocessor solution was not needed.

What we have done is : 

  • Retrieve the import file containing special characters.
  • Copy and paste values into an excel file encoded in UTF8
  • The values will then appear unencoded and readable. (UTF8 allows you to view special characters)
  • Simply infile the new UTF8 CSV import file into Saviynt.