Navigating the Pitfalls of Simulation Project Management

While simulation can mean many things to many people (see our post on we need a simulation), managing the expectations and interactions within a simulation project team often leads to issues.

The challenge is getting ahead of these issues and identifying them quickly when they do come up. This post will give you some key phrases to watch out for when organizing and conducting a simulation project.

“My manager says we need a simulation”

If you hear this, you could have a few different problems heading your way. First, the person talking to you is not the decision maker. You need to make sure you get the proper scope and purpose for a simulation. Second, the person talking to you does not seem interested in the process. The person is saying they do not want a simulation but their manager does. Sometimes you are intruding on what someone thinks is their territory and they want to make the decision without the simulation.

“I need a simulation that says…”

This statement is worse than the last one. People will often use a simulation to promote their idea. They are convinced they are correct, and they want a simulation to show everyone why they are right.

Resolution: Never provide answers to the goals of the project until the team and key players agree the model is valid. Going through that validation and getting that buy-in allows the simulation answers to be accepted by the team and it stops anyone with ulterior motives from claiming the model is invalid after they see an answer that they do not like.

Simulation project

“Our system would never do that…”

This statement is actually pretty useful. It means that people are paying attention and there may be an issue with your model that needs to be resolved. We’ll hear this most often when we are trying to validate a model. You need to work with the team to figure out if the difference matters based on the questions you are trying to answer.

Resolution: One way to reach a consensus during this phase it to replicate reports that engineers typically see from the real world. Show them the output formatted in a familiar way so they can quickly point out which results or statistics appears incorrect to them. This will allow you to quickly focus on a potential solution.

Simulation project results presentation

“Why didn’t we model … (insert anything out of scope here)”

I like to refer to these people as helicopter managers. They keep their distance early in the project and only show up for the results meeting. They try to interject new questions and often want to revisit the validation process of the model. If you get to this point in a project, it could be too late.

Resolution: Just like the “manager wants a simulation” comment earlier, this one needs to be controlled at the beginning of the project. One effective way to know if you have the decision maker in the room is to add a signature line to the specification. Have someone sign off on your scope. If you do this, at least you will have more time/money if a more senior person decides to inject themselves into the process.

“But that scenario is better than the one you recommended”

Simulations will often span a wide cross section of a company. Each person has their area of expertise, but you are asking them to review some technical aspects of your results and recommendations. As part of that, the concept of a confidence interval or margin of error may need to be addressed. We often see results that have slight differences in our target metric, but the difference is not statistically significant or it falls in the margin of error for whatever type of simulation is being conducted.

Resolution: Have prepared slides that cover the concept of variation and statistical accuracy prior to your meeting. It will be a good refresher for many people in the office and it will show people that this is a common topic that you cover.

Conclusion

There are many other pitfalls. I’d love to start a conversation about situations that you have seen. Hopefully we can all learn from these situations to make our projects run smoothly.