Hi, we are trying to import about 50,000 accounts but when we run the job it takes around 2 days to do the full import (we assume this is because we need to make 50,000 api calls, one for each user to get their attributes). We are just wondering if there is any way to shorten this time or if there is a way to do some sort of incremental import instead? Thanks.
This is the job we ran:
Here is our ImportAccountEntJSON:
{
"accountParams": {
"connection": "acctAuth",
"processingType": "SequentialAndIterative",
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "<url>,
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "text/json"
},
"httpContentType": "application/json",
"httpMethod": "GET",
"httpParams": {},
"successResponses": {
"statusCode": [
200
]
},
"unsuccessResponses": {
"statusCode": [
400,
401,
404,
405,
500
]
}
},
"listField": "Rsp.Result",
"keyField": "accountID",
"colsToPropsMap": {
"accountID": "Guid~#~char",
"name": "Guid~#~char"
}
},
"call2" : {
"callOrder" : 1,
"stageNumber" : 3,
"http": {
"url": "<url>",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "text/json"
},
"httpContentType": "application/json",
"httpMethod": "GET",
"httpParams": {},
"successResponses": {
"statusCode": [
200
]
},
"unsuccessResponses": {
"statusCode": [
400,
401,
404,
405,
500
]
}
},
"inputParams": {
"dependentCall": true
},
"listField": "",
"keyField": "accountID",
"nextApiKeyField" : "accountID",
"colsToPropsMap": {
"accountID": "Rsp.Result.Guid~#~char",
"name": "Rsp.Result.Guid~#~char",
"displayName": "Rsp.Result.Name~#~char",
"status": "Rsp.Result.State~#~char",
"customproperty1" : "Rsp.Result.FirstName~#~char",
"customproperty2" : "Rsp.Result.LastName~#~char",
"customproperty3" : "Rsp.Result.ExpirationDate~#~char",
"customproperty4" : "Rsp.Result.EmailAddress~#~char",
"customproperty5" : "Rsp.Result.MobilePhoneNumber~#~char",
"customproperty6" : "Rsp.Result.AccessPermissionLevel~#~char",
"customproperty7" : "Rsp.Result.CreatedOn~#~char",
"customproperty8" : "Rsp.Result.ActivationDate~#~char",
"customproperty9" : "Rsp.Result.CustomFields[0].Value~#~char",
}
}
}
}
}
Hi @JoshuaLawrence ,
If it is not mandatory to run full import every day, you can run an incremental import and you can choose this option under job type, from the drop down in the job trigger.
You can schedule full import once a week and run incremental import every day if it works in your case.
In the meantime, on why the import is taking such long times, please raise a Freshdesk ticket with appropriate details so that the team can look further into it.
Thanks,
