Signature not being included in importUserJson call - REST

[jralexander137]

It looks like the HMAC signature is not being passed correctly to my importuserjson config from connectionJson. Test connection works fine. What am I missing here?

Error is "2024-05-31T01:57:01.531+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-10-h2q6k","DEBUG","Error while getting User Import response for url- https://api-.duosecurity.com/admin/v1/users is: {"code": 40103, "message": "Invalid signature in request credentials", "stat": "FAIL"}"

Connection config is:

{
"authentications": {
"acctAuth": {
"authType": "BasicWithHmac",
"url": "api-.duosecurity.com",
"httpMethod": "POST",
"properties": {
"IKEY": "DE",
"SKEY": "rnaQ"
},
"authError": [
"InvalidAuthenticationToken",
"AuthenticationFailed",
"Authentication_MissingOrMalformed",
"Authentication_ExpiredToken"
],
"errorPath": "error.code",
"maxRefreshTryCount": 5,
"tokenResponsePath": "access_token",
"tokenType": "Basic",
"accessToken": "abc",
"testConnectionParams": {
"http": {
"basicUrl": "api-.duosecurity.com",
"hostUrl": "/admin/v1/users",
"url": "https://api-.duosecurity.com/admin/v1/users?limit=1&offset=1",
"httpHeaders": {
"Authorization": "${access_token}"
},
"httpContentType": "application/x-www-form-urlencoded",
"httpMethod": "GET",
"httpParams": "{\"limit\":\"1\",\"offset\":\"1\"}"
},
"successResponses": {
"statusCode": [
200
]
},
"errors": [
"Invalid signature in request credentials",
"Invalid integration key in request credentials"
],
"errorPath": "message"
}
}
}
}

and userImport config is 

{
"connection": "acctAuth",
"successResponses": {
"statusCode": [
200,
201,
202,
203,
204,
205
]
},
"url": "https://api-.duosecurity.com/admin/v1/users",
"httpMethod": "GET",
"httpHeaders": {
"Accept": "application/json",
"Authorization": "${access_token}"
},
"userResponsePath": "response",
"colsToPropsMap": {
"username": "notes~#~char",
"customproperty59": "phones[0].number~#~char"
}
}

I've reviewed the developers handbook and these configs look to be correct so I'm not sure what I am overlookiing here.

[NM]

Hi @jralexander137 , can you check once in logs if access token is getting populated after connection is established? And I believe you have to do a retry after first failure of connection

[jralexander137]

I have retry count set to 5. I am struggling to find any log entry that shows accesstoken being populated or not. 

[rushikeshvartak]

Share logs when are you performing test connection

[jralexander137]

Log snip attached. One is for test connection and one is for user import job. The user import job just shows the error message in OP. I am not seeing any log entries to determine if accesstoken is being populated or not. I see a nullpoint exception when it looks like the system is trying to parse the resulting data set from user import but since there is no data returned, nothing for it to parse.

[jralexander137]

The provisioning configs are working fine in the connection but user import is not.

[rushikeshvartak]

Do you mean connection is established ?

[jralexander137]

ConnectionJson works for account related import and operations. But when I try to use 

{
"connection": "acctAuth",
"successResponses": {
"statusCode": [
200,
201,
202,
203,
204,
205
]
},
"url": "https://api-removed.duosecurity.com/admin/v1/users",
"httpMethod": "GET",
"httpParams": "",
"httpHeaders": {
"Authorization": "${access_token}",
"Accept": "application/json",
"Content-Type":"application/x-www-form-urlencoded"
},
"userResponsePath": "response",
"colsToPropsMap": {
"username": "notes~#~char",
"customproperty59": "phones[0].number~#~char"
}
}

for import User config, i get the 401303 invalid or missing signature error. Its literally hitting the same endpoint as the other operations and account import. My understanding is the OOTB rest connector obsfucates the generation of the HMAC signature so I can't even see if its being generated. Nor can I see any of the outbound payload values to make sure headers, signature, etc are all being included.

This connection JSOn is working as is, only things i removed to post here are the url, ikey and skey, everything else is configured as shown.

{
"authentications" : {
"acctAuth" : {
"accessToken" : "Basic xyz",
"authError" : [
"InvalidAuthenticationToken",
"AuthenticationFailed",
"Authentication_MissingOrMalformed",
"Authentication_ExpiredToken"
],
"authType" : "BasicWithHmac",
"errorPath" : "error.code",
"httpMethod" : "POST",
"maxRefreshTryCount" : 5,
"properties" : {
"IKEY" : "",
"SKEY" : ""
},
"tokenResponsePath" : "access_token",
"tokenType" : "Basic",
"url" : "api-.duosecurity.com"
}
}
}

[rushikeshvartak]

I think you need to pass Date also

[jralexander137]

Thats what I am thinking at this point. I'm just not sure why it seems the connector adds that Date header for all the other operations and not the userImportJson.

[jralexander137]

This is my updated connectionConfig, when I run the user import job now, it just retries 5 times and fails because of 401 and therefore no data to process.

{
"authentications": {
"acctAuth": {
"authType": "BasicWithHmac",
"url": "api-.duosecurity.com",
"httpMethod": "POST",
"properties": {
"IKEY": "ikey",
"SKEY": "skey"
},
"authError": [
"InvalidAuthenticationToken",
"AuthenticationFailed",
"Authentication_MissingOrMalformed",
"Authentication_ExpiredToken",
"Invalid signature in request credentials"
],
"errorPath": "message",
"retryFailureStatusCode": [
401
],
"maxRefreshTryCount": 5,
"tokenResponsePath": "access_token",
"tokenType": "Basic",
"accessToken": "Basic xyz",
"testConnectionParams": {
"http": {
"basicUrl": "api-.duosecurity.com",
"hostUrl": "/admin/v1/users",
"url": "https://api-.duosecurity.com/admin/v1/users?limit=1&offset=1",
"httpHeaders": {
"Authorization": "${access_token}"
},
"httpContentType": "application/x-www-form-urlencoded",
"httpMethod": "GET",
"httpParams": "{\"limit\":\"1\",\"offset\":\"1\"}"
},
"successResponses": {
"statusCode": [
200
]
},
"errors": [
"Invalid signature in request credentials",
"Invalid integration key in request credentials"
],
"errorPath": "message"
}
}
}
}

[NM]

Hi @jralexander137 , can you share your token call from postaman?

[jralexander137]

There is no token call to share. The authentication 'token' is a HMAC Signature leveraging basic auth. Its calculated by Postman and sent with the outbound request call. There is no token generation endpoint.

curl --location 'https://api-.duosecurity.com/admin/v1/users' \
--header 'Date: Wed, 05 Jun 2024 09:40:26 -0400' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--header 'Authorization: Basic RElEVU5NNE1ZQ05GWUU4NVk0TEU6NDZlNTBmMTEwMjBjOTE0NmIzN2JjZGY0NzhiYTQ1Y2NjMjViZDE5Yw=='

[NM]

@jralexander137 , does any other opration works? And it gives the result ..can you share the json for one of the opration then

[jralexander137]

Yes, all other account related operation in the rest connection work as expected. That includes account import, create/update/enable/disable. Here is the account import config that is calling the same endpoint as importUser:

{
"accountParams": {
"connection": "acctAuth",
"processingType": "SequentialAndIterative",
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "https://api-.duosecurity.com:443/admin/v1/users",
"httpMethod": "GET",
"httpHeaders": {
"Accept": "application/json",
"Authorization": "${access_token}"
}
},
"listField": "response",
"keyField": "accountID",
"colsToPropsMap": {
"accountID": "user_id~#~char",
"name": "username~#~char",
"displayName": "realname~#~char",
"customproperty1": "firstname~#~char",
"customproperty2": "lastname~#~char",
"customproperty4": "email~#~char",
"customproperty5": "phones[0].number~#~char",
"comments": "email~#~char",
"customproperty7": "notes~#~char",
"customproperty9": "email~#~char",
"customproperty14": "status~#~char"
}
}
},
"statusConfig": {
"active": "active",
"inactive": "disabled"
},
"successResponses": {
"statusCode": [
200,
201,
202,
203,
204,
205
]
}
},
"entitlementParams": {
},
"acctEntParams": {
}
}

And here is create account:

{
"accountIdPath": "${(requestAccessAttributes?.get('Account Type') != null && requestAccessAttributes?.get('Account Type')?.equals('AdminUser')) ? 'call1.message.response.admin_id' :'call1.message.response.user_id'}",
"dateFormat": "ddd, DD MMM YYYY HH:mm:ss ZZ",
"responseColsToPropsMap": {
"accountsStatus": "${(requestAccessAttributes?.get('Account Type') != null && requestAccessAttributes?.get('Account Type')?.equals('AdminUser')) ? '#CONST#Active~#~char' : 'call1.message.status~#~char'}",
"displayName": "call1.message.response.realname~#~char",
"name": "${(requestAccessAttributes?.get('Account Type') != null && requestAccessAttributes?.get('Account Type')?.equals('AdminUser'))?'call1.message.response.name~#~char' : 'call1.message.response.username~#~char'}",
"accountType": "${(requestAccessAttributes?.get('Account Type') != null && requestAccessAttributes?.get('Account Type')?.equals('AdminUser')) ? '#CONST#AdminUser~#~char' :'#CONST#EndUser~#~char'}",
"accountRole": "call1.message.response.role~#~char"
},
"call": [
{
"name": "call1",
"connection": "acctAuth",
"basicUrl": "api-.duosecurity.com",
"hostUrl": "/admin/v1/users",
"url": "https://api-143f79bb.duosecurity.com/admin/v1/users",
"httpMethod": "POST",
"httpParams": "{\"username\":\"${user.systemUserName.toUpperCase()}\", \"email\": \"${user.email}\", \"status\": \"active\", \"notes\": \"${user.username}\", \"realname\": \"${user.firstname.toUpperCase()} ${user.lastname.toUpperCase()}\"}"
},
{
"name": "call2",
"connection": "acctAuth",
"basicUrl": "api-.duosecurity.com",
"hostUrl": "/admin/v1/users/enroll",
"url": "https://api-.duosecurity.com/admin/v1/users/enroll",
"httpMethod": "POST",
"httpParams": "{\"email\": \"${user.email}\", \"username\": \"${user.systemUserName}\"}",
"successResponses": {
"statusCode": [
200,
400
]
}
}
]
}

[jralexander137]

I misspoke in my reply. The AccountImport only works with the OOTB DUO connector, my config in the rest connector I am testing with does not work. HOWEVER, the account operation configs work like the createAccountJson i provided.

[rushikeshvartak]

Did you tried creating new connection ?

[jralexander137]

Yes and while test connection is successful I am now seeing the following error when trying to do a user import:

"2024-06-06T13:00:05.132+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","Start Import Users"
"2024-06-06T13:00:05.132+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","Enter initializeConnectionForUserImport"
"2024-06-06T13:00:05.133+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","Exit initializeConnectionForUserImport"
"2024-06-06T13:00:05.133+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","calling getUsersData method to hit Import User API"
"2024-06-06T13:00:05.133+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","Inside getUsersData"
"2024-06-06T13:00:05.133+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","connection: acctAuth"
"2024-06-06T13:00:05.139+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","Exception in canonRequest :"
"2024-06-06T13:00:05.853+00:00","ecm-worker","","null-2br28","","java.lang.NullPointerException: Cannot invoke method toLowerCase() on null object at com.saviynt.provisoning.rest.RestProvisioningService.canonRequest(RestProvisioningService.groovy:3912) at com.saviynt.provisoning.rest.RestProvisioningService.signRequest(RestProvisioningService.groovy:3867) at com.saviynt.provisoning.rest.RestProvisioningService.populateHttpParamsForBasicWithHmac(RestProvisioningService.groovy:3625) at com.saviynt.provisoning.rest.RestProvisioningService.populateHttpParams(RestProvisioningService.groovy:3491) at com.saviynt.provisoning.rest.RestProvisioningService.getUsersData(RestProvisioningService.groovy:11801) at com.saviynt.provisoning.rest.RestProvisioningService.getUsersData(RestProvisioningService.groovy:11781) at com.saviynt.provisoning.rest.RestProvisioningService.importUsers(RestProvisioningService.groovy:2977) at com.saviynt.ecm.integration.ExternalConnectionCallService.importUserUsingExternalConnection(ExternalConnectionCallService.groovy:1251) at UserImportJob.execute(UserImportJob.groovy:108) at org.quartz.core.JobRunShell.run(JobRunShell.java:199) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)"
"2024-06-06T13:00:05.181+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","parsing original httpParams after binding"
"2024-06-06T13:00:05.206+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","Adding connectionParamMap to jsonMap for ssl : "
"2024-06-06T13:00:05.223+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG","Inside pullObjectsByRest"
"2024-06-06T13:00:05.224+00:00","ecm-worker","rest.RestProvisioningService","quartzScheduler_Worker-8-2br28","DEBUG",""
"2024-06-06T13:00:05.224+00:00","ecm-worker","services.HttpClientUtilityService","quartzScheduler_Worker-8-2br28","DEBUG","calling executeRequestWithTimeoutConfig for api..."
"2024-06-06T13:00:05.224+00:00","ecm-worker","services.HttpClientUtilityService","quartzScheduler_Worker-8-2br28","DEBUG","Enter getTimeOutConfig"
"2024-06-06T13:00:05.225+00:00","ecm-worker","services.HttpClientUtilityService","quartzScheduler_Worker-8-2br28","DEBUG","connectionType: REST"
"2024-06-06T13:00:05.225+00:00","ecm-worker","services.HttpClientUtilityService","quartzScheduler_Worker-8-2br28","DEBUG","connectionTimeoutConfig before guardRail validation: null"

Looks like the connector is failing to calculate the HMAC Signature properly.

Here is the connection config I am using:

{
"authentications": {
"acctAuth": {
"authType": "BasicWithHmac",
"url": "api-.duosecurity.com",
"httpMethod": "GET",
"httpParams": {},
"properties": {
"IKEY": "removed",
"SKEY": "removed"
},
"authError": [
"InvalidAuthenticationToken",
"AuthenticationFailed",
"Authentication_MissingOrMalformed",
"Authentication_ExpiredToken"
],
"errorPath": "error.code",
"maxRefreshTryCount": 5,
"tokenResponsePath": "access_token",
"tokenType": "Basic",
"accessToken": "abc",
"testConnectionParams": {
"http": {
"basicUrl": "api-.duosecurity.com",
"hostUrl": "/admin/v1/users",
"url": "https://api-.duosecurity.com/admin/v1/users?limit=1&offset=1",
"httpHeaders": {
"Authorization": "${access_token}"
},
"httpContentType": "application/x-www-form-urlencoded",
"httpMethod": "GET",
"httpParams": "{\"limit\":\"1\",\"offset\":\"1\"}"
},
"successResponses": {
"statusCode": [
200
]
},
"errors": [
"Invalid signature in request credentials",
"Invalid integration key in request credentials"
],
"errorPath": "message"
}
}
}
}

and here is the userImportJson

{
"connection": "acctAuth",
"successResponses": {
"statusCode": [
200,
201,
202,
203,
204,
205
]
},
"url": "https://api-.duosecurity.com/admin/v1/users",
"httpMethod": "GET",
"httpParams": {},
"httpHeaders": {
"Authorization": "Basic xyz",
"Accept": "application/json",
"Content-Type": "application/x-www-form-urlencoded"
},
"userResponsePath": "response",
"dateFormat": "ddd, DD MMM YYYY HH:mm:ss ZZ",
"colsToPropsMap": {
"username": "notes~#~char",
"customproperty59": "phones[0].number~#~char"
}
}

I saw that this line was being included in the account operation configs that are working so I added but still failed:

"dateFormat": "ddd, DD MMM YYYY HH:mm:ss ZZ",

The API documentation says: 

• Is the Authorization header correctly formatted? If not, you may receive a 40101 error.

Which matches the error I am seeing now Failed url-https://api-.duosecurity.com/admin/v1/users with Error Message-{"code": 40101, "message": "Missing request credentials", "stat": "FAIL"}

The problem is that I can't tell what the connector is looking for and missing when trying to calculate the HMAC signature and generates the nullpointer.