By seeking maximum improvement with each iteration, gradient-based optimization algorithms almost always offer design improvement faster than their non-gradient-based (genetic algorithms, simulated annealing, particle swarm, etc.) counterparts for continuous design spaces. However, the strength of gradient-based algorithms also leaves them susceptible to “getting stuck” in local optima, where no minor adjustment of variables results in improvement. One powerful approach to address design spaces with many local optima is with a Multi-start technique. By initiating simultaneous optimizations from multiple points, it is much more likely that the optimizer will find the true global optimum. Because multi-start methods can use gradient-based algorithms, they can often be many times faster than a single non-gradient-based algorithm.
One flexible and concise approach to implement multi-start optimizations in ILIAD is by nesting an Optimization component inside a Design of Experiments (DOE) component. The design points provided by the DOE component serve as starting points for the optimizer. Using one of ILIAD’s many sampling algorithms allows us to quickly obtain starting points that are well distributed throughout the design space.
Parameterizing the Initial Value field of the variables in the Optimization component allows the DOE component to dictate the starting point of each optimization.
In this demonstration, the DOE component has been given five variables: two (x_initial and y_initial) for the optimization starting point, two (x_best and y_best) to record the location of the resultant optimum, and one (f_best) to record the best value obtained by each optimization subflow.
We can view and modify how the DOE and Optimization components transfer the starting points and optimization results using Data Linker. Here we see the DOE variables passing their values to the Optimization’s Initial Value parameters and the optimum point of the Optimization being passed to the responses of the DOE.
Suppose we want to find the maximum value of the surface in figure 1 but do not know where to start our search. Utilizing this framework, an example multi-start workflow is constructed to launch five gradient-based optimizations starting from points chosen by the Latin Hypercube sampling algorithm.
After executing the workflow, analysis Design Points Table of the DOE in the Post-Processing tab shows that one of the five points converged to the global maximum while the other four found various local maxima. Notice the ways the initial coordinates move to the best coordinates and how these align with the various local maxima in the figure 1 surface. In a real-world example, we would not know the true response surface or be absolutely certain that the best optimum we find is the global optimum, however we can better understand the volatility of a design space if the workflow discovers many unique local optima or the prominence of a particular optimum if several disparate starting points yield the same optimum.
This week we discussed the basics of multi-start optimization and a powerful and concise nested workflow to implement this approach in ILIAD. This is a rich topic which lends itself well to ILIAD’s visual programmatic structure and post-processing tools, so check back weekly for more tips, tricks, and deep dives into multi-start optimization in ILIAD as well as a plethora of fascinating topics from across the OmniQuest software suite!
Connect with us now for complimentary webinars and evaluation software.
Our engineering team can work with you to conduct a Test Case showing how OmniQuestTM will improve your designs, processes and your overall business