We are delighted to share our new EIC Delivery Methodology for efficiently managing Saviynt Implementations and delivering quick time to value. CLICK HERE.

SuccessFactors (REST) Full Import causing ECM restart

ChrisBellobuono
New Contributor III
New Contributor III

We are experiencing the exact issue described in this now closed forum post: https://forums.saviynt.com/t5/identity-governance/indicator-of-ecm-restart-in-v23-x/m-p/33983

Unfortunately, there is no mention as to what the actual fix was as it was resolved via a private ticket.

In our case the job runs for about 30-40 minutes, stops (or is killed), and ECM is restarted.  In all cases, no other jobs were running.  This is an example of what we see in the logs.

"2023-08-11T19:07:04.642+00:00","ecm-worker","","",""," created: WEB-INF/classes/com/saviynt/webservice/"
"2023-08-11T19:07:04.642+00:00","ecm-worker","","",""," created: WEB-INF/classes/com/saviynt/workflowmgt/"
"2023-08-11T19:07:04.641+00:00","ecm-worker","","","","ecm-start: extracting ECM.war"
"2023-08-11T19:07:00.365+00:00","ecm","domain.JobcontrolController","http-nio-8080-exec-147","INFO","Auditing for list of JobControl"
"2023-08-11T19:07:00.366+00:00","ecm","domain.JobcontrolController","http-nio-8080-exec-147","INFO","Auditing process completed for JobControl"
"2023-08-11T19:07:00.366+00:00","ecm","domain.JobcontrolController","http-nio-8080-exec-147","DEBUG","order by FIRED_TIME desc"
"2023-08-11T19:06:46.779+00:00","ecm-worker","","","","/init/ecm-start.sh: line 171: 60 Killed /usr/local/tomcat/bin/catalina.sh run"
"2023-08-11T19:06:44.916+00:00","ecm-worker","rest.RestUtilService","quartzScheduler_Worker-2","DEBUG","Exception in RestUtil.getAt : java.lang.IndexOutOfBoundsException: Index: 0,

Additionally, the job's history log is left like this:

ChrisBellobuono_0-1691783855111.png

It says it was successful, but there is no end date, the stats link on the left is missing, and from what I can see, none of the users were updated.

Any help would be greatly appreciated.

5 REPLIES 5

Dave
Community Manager
Community Manager

@BrandonLucas_BF - Are you able to help with this post?  

BrandonLucas_BF
Regular Contributor III
Regular Contributor III

Thanks for the tag @Dave.

While I cannot say for certain how our issue was resolved, I'm happy to report the issue in the other thread was resolved. Our symptoms were exactly like yours but we were doing a massive user import using Workday SOAP connection.. something to the tune of 17,000 records being imported for the user table. About halfway through our services would crash and restart and the job would be left with a status similar to yours showing success but no details and it was clearly not successful.

Our ticket was escalated to engineering for a couple of months to diagnose the issue and then fixed. We experienced the issue in both of our instances running v23.6 and it persisted between version upgrades from 23.5. If you have a ticket open, you might reference our ticket number: 1623059.

We experienced the same issue in prod after we moved the job there and a second ticket was opened to address that issue - 1659536. That was resolved in less than 12 hours using the findings from the earlier ticket, with the same resolution, so if your issue is the same as ours, it should be a quick fix.

Good luck!

 

 

Dave
Community Manager
Community Manager

Thanks @BrandonLucas_BF !

Also, @ChrisBellobuono - If you do open a ticket you can just link to this post as well so they can see this dialogue. The more information they have to work from, the better! 

rushikeshvartak
All-Star
All-Star

Job End Date , usually is not shown on form when Job is execution mode in background and one of the Service / pod is restarted due to load or any other reason.


Regards,
Rushikesh Vartak
If you find the response useful, kindly consider selecting Accept As Solution and clicking on the kudos button.

ChrisBellobuono
New Contributor III
New Contributor III

Per the ticket I opened, some of my scripts were generating enough IndexOutOfBoundsExceptions, due to the number user being imported, causing the application to eventually restart.  After fixing the scripts so the errors stopped we no longer experienced any restarts.  It is still unclear at this time how the two are related.