Leveraging Intelligent Recommendations for Operational Transformation. AMS Partners click HERE | EMEA/APJ Partners click HERE |
06/23/2023 09:30 AM
Hi Team,
We are configuring coupa application, where AccountImport job runs for around 2 hours and stops without success or failure status. Also there won't be job end date. The job has imported 42k+ accounts. Though we do not have the total count of accounts from target, I could manually validate from postman, there are many accounts yet to be imported. Below are the ImportAccountEntJSON and ConfigJSON.
ImportAccountEntJSON :
{
"accountParams": {
"connection": "acctAuth",
"processingType": "SequentialAndIterative",
"statusAndThresholdConfig": {
"inactivateAccountsNotInFile": "true",
"accountThresholdValue": 70000
},
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "https://${connection.hosturl}/api/users",
"httpContentType": "application/json",
"httpMethod": "GET",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
}
},
"listField": "",
"keyField": "accountID",
"statusConfig": {
"active": "true",
"inactive": "false"
},
"colsToPropsMap": {
"accountID": "id~#~char",
"name": "login~#~char",
"displayname": "fullname~#~char",
"status": "active~#~char",
"lockedState": "#CONST#${String isActive = response.active; isActive = (isActive.equals('false') ) ? 1:2; return isActive}~#~char",
"customproperty1": "email~#~char",
"customproperty2": "firstname~#~char",
"customproperty3": "lastname~#~char",
"customproperty4": "fullname~#~char",
"customproperty5": "call1?.message?.mention-name~#~char",
"customproperty6": "company~#~char",
"customproperty7": "active~#~char",
"customproperty8": "purchasing-user~#~char",
"customproperty9": "expense-user~#~char",
"customproperty10": "sourcing-user~#~char",
"customproperty11": "inventory-user~#~char",
"customproperty12": "contracts-user~#~char",
"customproperty13": "analytics-user~#~char",
"customproperty14": "created-by.id~#~char",
"customproperty15": "sso-identifier~#~char",
"customproperty16": "employee-number~#~char",
"customproperty31": "STORE#ACC#ENT#MAPPINGINFO~#~char",
"customproperty47": "cost-center~#~char"
},
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "limit",
"batchSize": 50,
"totalCountPath": 70000
}
}
}
},
"acctEntMappings": {
"User Group": {
"importAsEntitlement": false,
"listPath": "user-groups",
"idPath": "id",
"keyField": "entitlementID"
}
}
},
"entitlementParams": {
"processingType": "SequentialAndIterative",
"entTypes": {
"User Group": {
"entTypeOrder": 0,
"entTypeLabels": {
"customproperty1": "Meta Created"
},
"call": {
"call1": {
"connection": "acctAuth",
"callOrder": 0,
"stageNumber": 0,
"http": {
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json"
},
"url": "https://${connection.hosturl}/api/user_groups",
"httpContentType": "application/json",
"httpMethod": "GET"
},
"listField": "",
"keyField": "entitlementID",
"colsToPropsMap": {
"entitlementID": "id~#~char",
"entitlement_value": "name~#~char",
"description": "description~#~char"
},
"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "limit",
"batchSize": 50,
"totalCountPath": 70000
}
}
}
}
}
}
},
"acctEntParams": {
"processingType": "entToAcctMapping"
}
}
ConfigJSON :
{
"showLogs": true,
"connectionTimeoutConfig": {
"import": {
"timeout": 300,
"retryWait": 500
},
"provisioning": {
"timeout": 300,
"retryWait": 500
}
}
}
06/26/2023 12:33 AM
Validate if "accountThresholdValue": 70000 is getting exceeded.
06/27/2023 10:20 AM - edited 06/27/2023 10:24 AM
Accounts count is not exceeding 70k. We have updated the ImportAccontEntJSON (attached) and now the job is completing. But when I manually validated, few accounts are missing in saviynt and some accounts status not updated as in target. Also seeing the below error in log. Txt file attached with complete error
2023-06-27 15:05:21,404 [quartzScheduler_Worker-1] DEBUG rest.RestProvisioningService - associated entitlement : User Group with account is not for import.
2023-06-27 15:05:21,486 [quartzScheduler_Worker-1] DEBUG rest.RestProvisioningService - associated entitlement : User Group with account is not for import.
2023-06-27 15:05:21,529 [quartzScheduler_Worker-1] DEBUG rest.RestUtilService - Exception in RestUtil.getAt : groovy.lang.MissingPropertyException: No such property: name for class: java.lang.String
2023-06-27 15:05:21,529 [quartzScheduler_Worker-1] DEBUG println.PrintlnToLogger - Println :: [1;31m| Error [22;39mgroovy.lang.MissingPropertyException: No such property: name for class: java.lang.String[m
2023-06-27 15:05:21,529 [quartzScheduler_Worker-1] DEBUG println.PrintlnToLogger - Println :: [1;31m| Error [22;39m at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:50)[m
2023-06-27 15:05:21,529 [quartzScheduler_Worker-1] DEBUG println.PrintlnToLogger - Println :: [1;31m| Error [22;39m at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:273)[m
2023-06-27 15:05:21,529 [quartzScheduler_Worker-1] DEBUG println.PrintlnToLogger - Println :: [1;31m| Error [22;39m at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)[m
2023-06-27 15:05:21,529 [quartzScheduler_Worker-1] DEBUG println.PrintlnToLogger - Println :: [1;31m| Error [22;39m at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)[m
2023-06-27 15:05:21,529 [quartzScheduler_Worker-1] DEBUG println.PrintlnToLogger - Println :: [1;31m| Error [22;39m at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:55)[m
07/04/2023 02:24 AM
Hi @RakeshMG
We identified that the accounts are 74k+ so we had set accountThresholdValue to 75000 and tried. However, we are still seeing the same error.
"ecm-worker","2023-07-03T16:04:15.480+00:00","{"log":"2023-07-03 16:04:15,232 [quartzScheduler_Worker-9] DEBUG rest.RestUtilService - Exception in RestUtil.getAt : groovy.lang.MissingPropertyException: No such property: name for class: java.lang.String\n","stream":"stdout","time":"2023-07-03T16:04:15.232434968Z"}"
"ecm-worker","2023-07-03T16:04:15.480+00:00","{"log":"2023-07-03 16:04:15,291 [quartzScheduler_Worker-9] DEBUG rest.RestProvisioningService - associated entitlement : User Group with account is not for import.\n","stream":"stdout","time":"2023-07-03T16:04:15.291828443Z"}"
07/04/2023 04:00 AM
The reported exception is encountered only if there is no value found for a primary field such as accountID, or name in the API response.
Please confirm if you getting the same error in each API call or in any particular API page. If it happens only for a single API, Call the same API page through Postman and check the response data for the login attribute to find if there is a value that can cause such issues.
07/10/2023 04:59 AM
1) Now if the job is completing, can you confirm if the account count in target is same as the one's imported to Saviynt?
2) Can you add {"showLogs": true} in the ConfigJSON and confirm in which paginated call you are receiving the error?
"ecm-worker","2023-07-03T16:04:15.480+00:00","{"log":"2023-07-03 16:04:15,232 [quartzScheduler_Worker-9] DEBUG rest.RestUtilService - Exception in RestUtil.getAt : groovy.lang.MissingPropertyException: No such property: name for class: java.lang.String\n","stream":"stdout","time":"2023-07-03T16:04:15.232434968Z"}"
"ecm-worker","2023-07-03T16:04:15.480+00:00","{"log":"2023-07-03 16:04:15,291 [quartzScheduler_Worker-9] DEBUG rest.RestProvisioningService - associated entitlement : User Group with account is not for import.\n","stream":"stdout","time":"2023-07-03T16:04:15.291828443Z"}"
3) Like @khalidakhter suggested usually this exceptions happen when the primary field is blank. So probably in Coupa's case, your login mapped to name might be blank which is causing few accounts to be not imported. Pin point the offset call that's causing the error - https://${connection.hosturl}/api/users?offset=x&limit=50 and try in postman.
4) Other suggestion to avoid this error is to map id for accountID as well as name & check the recon behaviour.
-Siva
07/10/2023 05:48 AM
Hi @Sivagami,
Thank you for your response.
We had used below parameters in JSON. After removing them I do not see that error in logs.
"customproperty7": "cost-center.name~#~char",
"customproperty17": "default-account-type.name~#~char",
"customproperty18": "default-account.name~#~char",
"customproperty19": "roles.name~#~char",
However, the job is not completing now and I do not find error in logs. I could see only 1837 accounts are updated and the updated records ids are not in order. So, facing difficulty in finding exact page or error in log.
I tried the below filters from postman to find the records with login value blank
https://<coupa_url>?middlename[blank]=true
response gives list of records with middlename value null and ""
https://<coupa_url>?login[blank]=true
response []
Please find the latest ImportAccountEntJSON in the attachment.
07/10/2023 06:05 AM
Few counter questions? & Suggestions
1) In the json why CP5 mapping is below instead of direct mapping - "customproperty5":"mention-name~#~char",
"customproperty5": "call1.message.mention-name~#~char",
2) The exception you faced before is specific to primary attributes - accountID / name attribute mappings. customproperty mapping doesn't impact & cause that specific exception.
"customproperty7": "cost-center.name~#~char",
"customproperty17": "default-account-type.name~#~char",
"customproperty18": "default-account.name~#~char",
"customproperty19": "roles.name~#~char",
3) Can you try the below mapping and run the recon? Map login to one of the customproperties to find if it had blank values?
"accountID": "id~#~char", "customproperty22": "login~#~char",
"name": "id~#~char",
-Siva
07/11/2023 04:36 AM
Hi @Sivagami
"customproperty5": "call1.message.mention-name~#~char",
its used as per document
We wont be able to map this now "name": "id~#~char" which will mark the accounts as deleted. So I have done mapping as below and ran the job.
"accountID": "id~#~char",
"name": "#CONST#${String name = response.login; name = (name==null) ? response.id:response.login; return name}~#~char",
"customproperty22": "login~#~char",
Still not all records are updated and there are no blank records found in name or name value similar to id.
07/11/2023 11:20 AM
Your keyfield for accounts mapping is accountid, so changing the mapping wouldn't make the account go to suspended status. Is it possible to try in lower environment?
"keyField": "accountID",
08/30/2023 04:46 AM
We tried this, but still not all records are getting imported and updated. And there is no Job end date. I have attached some of relevant logs. Please check and let us know.
09/04/2023 01:44 AM
This is still an issue. Could you please check and help us?