Integration Consideration Feedback

[Manu269]

Hello folks ,

We are trying to work on an implementation and need few suggestions for below action points :

1. We are integrating an hrms system with Saviynt IGA where we are getting the end date for the user but no status field. 

Can someone assist what would be best option to mark the status as inactive?

Sample queries etc. would be helpful.

2. As a part of integration we may also need to perform some data migration activities.

Can someone assist best practices to be followed for migration? Please note we need details around data massaging, data formatting and data load processes.

Looking forward for some recommendations.

Assist please..

[rushikeshvartak]

#1 -> How user can be consider as inactive ? based on certain attributes ?

UPDATE NEWUSERDATA NU1 INNER JOIN CURRENTUSERS CU1 ON CU1.USERNAME = NU1.USERNAME SET NU1.STATUSKEY = CASE WHEN (CU1.CUSTOMPROPERTY6 = 'L' AND CU1.customproperty10 <= curdate() AND CU1.customproperty14 <> 1) THEN '0' WHEN (CU1.CUSTOMPROPERTY6 = 'A' AND CU1.customproperty10 IS NULL AND CU1.customproperty9 <= curdate() AND CU1.customproperty14 <> 1) THEN '1' WHEN (DATEDIFF(CU1.CUSTOMPROPERTY9, curdate()) > 7 AND CU1.CUSTOMPROPERTY14 = 0) THEN '0' WHEN (CU1.CUSTOMPROPERTY9 <= curdate() AND CU1.CUSTOMPROPERTY14 = 0 AND CU1.customproperty6 = 'A') THEN '1' WHEN (DATEDIFF(CU1.CUSTOMPROPERTY9, curdate()) <= 7 AND CU1.CUSTOMPROPERTY14 = 0) THEN '1' WHEN (CU1.CUSTOMPROPERTY13 <= curdate() AND CU1.CUSTOMPROPERTY14 = 1) THEN '0' WHEN (CU1.CUSTOMPROPERTY13 > curdate() AND CU1.CUSTOMPROPERTY14 = 0) THEN '1' END

#2 elaborate what are data migration activities ?

[Manu269]

Hi Rushi,

Data migration can be for 2 aspects:

1. Migration from existing IAM system to Saviynt IGA

OR

2. Disconnect system onboarding 

It should cover areas for data massaging, data load etc

.

[rushikeshvartak · Accepted solution]

1. Data Massaging

Data Massaging involves transforming and cleaning data to ensure it is in the correct format and free from errors before migration.

• Data Profiling: Analyze the source data to understand its structure, quality, and content. Identify inconsistencies, duplicates, and incomplete records.
• Data Cleansing: Correct or remove inaccurate records. Standardize formats (e.g., date formats, capitalization). Address missing values by either imputing them or excluding those records based on the context.
• Data Enrichment: Enhance the data by adding missing attributes or enhancing existing ones (e.g., adding organizational hierarchy information if missing).
• Transformation Rules: Define and document transformation rules to map source data formats to the target Saviynt IGA formats. Ensure these rules are aligned with the business requirements.
• Validation and Testing: Implement validation checks to ensure data integrity post-massage. Use a subset of data to test these transformations before applying them to the entire dataset.

2. Data Formatting

Data Formatting ensures the data adheres to the schema and requirements of the Saviynt IGA system.

• Schema Mapping: Map source data fields to Saviynt IGA fields. Create a comprehensive mapping document outlining source to target field mappings.
• Data Types and Constraints: Ensure data types in the source align with Saviynt's requirements (e.g., strings, integers, dates). Respect any constraints such as field length, required fields, and unique keys.
• Normalization: Normalize data to eliminate redundancy and ensure efficient access. This may involve splitting fields into separate entities or tables if needed.
• File Formats: If using file-based import (e.g., CSV, XML), ensure the files are formatted correctly. Pay attention to delimiters, encodings, and escape characters.
• Consistent Formatting: Apply consistent formatting across the data. For example, use a consistent date format (YYYY-MM-DD), phone number format, and address format.

3. Data Load Processes

Data Load involves importing the transformed and formatted data into Saviynt IGA.

• Data Load Strategy: Decide on a full load versus incremental load strategy. Full load imports all data at once, whereas incremental load imports data in stages.
• Pre-Load Validation: Conduct pre-load validation checks to ensure data integrity and completeness. This can include row counts, checksums, and key integrity checks.
• Batch Processing: If the dataset is large, use batch processing to avoid performance issues. Split the data into manageable chunks.
• ETL Tools: Use ETL (Extract, Transform, Load) tools that support robust data migration processes. Tools like Talend, Informatica, or even custom scripts can help automate and manage the data load process.
• Saviynt Connectors: Leverage Saviynt’s native connectors and APIs for data import. Ensure connectors are configured correctly and tested.
• Logging and Monitoring: Implement logging to track the data load process. Capture details such as start and end times, records processed, errors, and warnings.
• Post-Load Validation: Conduct post-load validation to ensure data was imported correctly. Compare source and target data to verify consistency and completeness.
• Error Handling: Define clear error handling procedures. Ensure that any data that fails to load is logged, analyzed, and reprocessed as needed.

Additional Best Practices

• Stakeholder Involvement: Engage key stakeholders throughout the migration process. Regularly communicate progress, issues, and milestones.
• Documentation: Maintain comprehensive documentation of all processes, transformation rules, and data mappings.
• Testing: Perform extensive testing in a staging environment before migrating to production. This includes unit tests, integration tests, and user acceptance tests (UAT).
• Backup and Recovery: Ensure you have reliable backup and recovery processes in place. This is crucial for preventing data loss during migration.
• Training and Support: Provide training for end-users and administrators on the new system. Ensure support is available to address any issues post-migration.