Multi-start Optimization in ILIAD – Part 1: The Nested Workflow Approach

Apr 19, 2022

By seeking maximum improvement with each iteration, gradient-based optimization algorithms almost always offer design improvement faster than their non-gradient-based (genetic algorithms, simulated annealing, particle swarm, etc.) counterparts for continuous design spaces. However, the strength of gradient-based algorithms also leaves them susceptible to “getting stuck” in local optima, where no minor adjustment of variables results in improvement. One powerful approach to address design spaces with many local optima is with a Multi-start technique. By initiating simultaneous optimizations from multiple points, it is much more likely that the optimizer will find the true global optimum. Because multi-start methods can use gradient-based algorithms, they can often be many times faster than a single non-gradient-based algorithm.

Figure 1: Example surface with several local extrema and one global maximum.

One flexible and concise approach to implement multi-start optimizations in ILIAD is by nesting an Optimization component inside a Design of Experiments (DOE) component. The design points provided by the DOE component serve as starting points for the optimizer. Using one of ILIAD’s many sampling algorithms allows us to quickly obtain starting points that are well distributed throughout the design space.

Figure 2: Workflow of a multistart optimization using the nested approach of enclosing an Optimization within a DOE.

Parameterizing the Initial Value field of the variables in the Optimization component allows the DOE component to dictate the starting point of each optimization.

Figure 3: Right-click on a parameterizable data field of a variable to reveal the ‘add’ and’ remove’ parameter options. Adding a parameter to an ‘Initial Value’ field of a variable signals to ILIAD that the value will come from an external source.

In this demonstration, the DOE component has been given five variables: two (x_initial and y_initial) for the optimization starting point, two (x_best and y_best) to record the location of the resultant optimum, and one (f_best) to record the best value obtained by each optimization subflow.

Figure 4: DOE Component Editor showing two variables (x_initial and y_initial) that dictate optimization starting points, two responses (x_best and y_best) that record the location of each optimum, and one objective (f_best) recording the value of each optimum.

We can view and modify how the DOE and Optimization components transfer the starting points and optimization results using Data Linker. Here we see the DOE variables passing their values to the Optimization’s Initial Value parameters and the optimum point of the Optimization being passed to the responses of the DOE.

Figure 5: Data Linker showing transfer of starting point from the DOE to the Optimization and optimum point from the Optimization back to the DOE.

Suppose we want to find the maximum value of the surface in figure 1 but do not know where to start our search. Utilizing this framework, an example multi-start workflow is constructed to launch five gradient-based optimizations starting from points chosen by the Latin Hypercube sampling algorithm.

Figure 6: Plot showing a five-point Latin Hypercube sampling of a two-dimensional search space. Contours are roughly indicative of a likelihood to find particular local optima from these starting locations.

After executing the workflow, analysis Design Points Table of the DOE in the Post-Processing tab shows that one of the five points converged to the global maximum while the other four found various local maxima. Notice the ways the initial coordinates move to the best coordinates and how these align with the various local maxima in the figure 1 surface. In a real-world example, we would not know the true response surface or be absolutely certain that the best optimum we find is the global optimum, however we can better understand the volatility of a design space if the workflow discovers many unique local optima or the prominence of a particular optimum if several disparate starting points yield the same optimum.

Figure 7: Design Points Table of the DOE component showing each optimization start point and the location and value of its corresponding maximum.

This week we discussed the basics of multi-start optimization and a powerful and concise nested workflow to implement this approach in ILIAD. This is a rich topic which lends itself well to ILIAD’s visual programmatic structure and post-processing tools, so check back weekly for more tips, tricks, and deep dives into multi-start optimization in ILIAD as well as a plethora of fascinating topics from across the OmniQuest software suite!

Connect with us now for complimentary webinars and evaluation software. 

Our engineering team can work with you to conduct a Test Case showing how OmniQuestTM will improve your designs, processes and your overall business

Contact Us