High Performance Computing: Why Wait?

For the majority of my almost 20-year career in FEA, I have solved my simulation models locally almost 100% of the time. Occasionally my employer would invest the 10s of thousands of dollars required to purchase a moderate HPC (High Performance Computing) system. No doubt, the solve times were reduced. But the inconvenience of large simulation result file transfers made it less than ideal for anything that takes less than a day to solve on my local workstation. Furthermore, 3-5 years after purchasing the HPC, a new $2,000 workstation would match the computation power for most of my models.

Fast-forward to today and things are a lot different. First off, I work on a much more diverse set of models (implicit & explicit with varying amounts of objects and complexity). Also, the options for HPC and the related required licensing have never been better.

Lubrication velocity contour at 3474 RPM

Lubrication velocity contour at 3474 RPM

Case 1

Gearbox lubrication simulations, using XFlow, are highly optimized on GPU solvers. One of the best GPUs on the market for physics solving is the NVIDIA A100. Historically, these have cost more than $20,000/each! But now I see that you can get one at Walmart for a little less than $10,000. This goes to show the extreme depreciation experienced with computation hardware.

Real world gearbox simulations can take days to solve. On a high-end desktop GPU (ex. NVIDA Tesla T4 – Cost: $1,000-$5,000), our client’s simulation with 6000 RPM was taking 4 days and 12 hours to solve. We teamed up with the Ohio Supercomputer Center (OSC) to run the simulation, using our license, on OSC’s A100 GPU and achieved a computation time of 1 day and 2 hours. This is a 75% reduction and a game changer in using simulation to make changes in the transmission design phase.


Case 2

OSC, and startups like Rescale, offer HPC-solving capabilities for Abaqus with reasonable rates, $0.12 and $0.11 per node, respectively. You bring your own license and can solve your critical jobs with blazing speed. Now, there is no longer any need to invest in expensive HPC hardware. We recently worked with a client to perform 48-core solving at OSC and could iterate the simulation inputs several times in one day instead of waiting for the results of an over-night run. 48-core solving costs $5.76/hr. at OSC.

Case 3

Dassault Systèmes (makers of Abaqus) have entered the HPC services game as well. They offer cloud computing with up to 192 cores. With third-party HPC service providers, you only pay for the HPC computing power when you are using it. The negative is that you must bring your own license and you are paying for that whether you are solving it or not. Because Dassault controls HPC and licensing, it is possible to get creative with licensing and only pay for what you need. There are now two ways of licensing the solver: Tokens and Credits

Tokens: These are taken up during a solve and then returned and available for subsequent solves as soon as a simulation job finishes.

Credits: These are consumed at a specific rate/hr. (as a function of number of CPUs). When they are consumed, they are gone. But you can purchase more at any time.

Credits are more economical if you infrequently require the solver. And Tokens are a better value if you are running simulations every day, non-stop 24/7. For example, the break-even point for 18 CPU solving is around 21 days (515 hours) per year. If you need less than 515 hours of 18 CPU solving, go with credits. If you need more, go with tokens. Many of our clients choose a hybrid of tokens and credits that fills their need for moderate solve times on their local hardware with the ability to have accelerated solving available for high-priority simulations.

The many HPC options can be overwhelming and hard to navigate. But we can help you find an economical solution for your specific needs. Please reach out if you want to talk about how you can utilize HPC to solve faster and save money at the same time!

Author’s Bio:

Tom Feister leads the structures simulation team at TriMech Enterprise Solutions. His background includes metal forming, materials testing, damage modeling, process optimization, structural integrity, and tire workflows. He honed his expertise in finite element analysis (FEA) in previous positions at EWI, KTH Parts, Scientific Forming Technologies Corporation (SFTC), and AutoForm Engineering. Tom specializes in helping engineers develop new skills in FEA that can be used to create a safer and more sustainable world. He is also an Ohio-certified Professional Engineer (PE) in the metallurgical and materials field.