When designing a digital or physical experiment with several variables (or more) it can be difficult to choose parameter value combinations that faithfully capture the design space. The distribution of design points can radically alter our perception of the behavior a system’s responses. To address this challenge, ILIAD offers a wide variety of sampling algorithms or DOE designs that intelligently specify variable value permutations that are well-distributed throughout our multidimensional design-spaces.
Figure 1: Example surface used as a known substitute for a generic analysis.
To illustrate this utility, we will treat the surface shown in figure 1 as our black box analysis and will sample this surface using the various design algorithms found in the Design of Experiments component. By applying a Kriging interpolation on the resultant points, we can better understand how the distribution of data dictates the statically likely shape of the underlying response surface.
Figure 2: Sampling algorithms in ILIAD located in the ‘Designs’ category of the Design of Experiments component.
Unsurprisingly, each sampling algorithm has unique strengths and weaknesses that make them better suited to certain domains. While we do not need to be experts on each method to make use of them, a rudimentary understanding of their behavior allows us to choose an algorithm that best matches our needs and constraints.
Figure 3: Factorial sampling (left) and simplex sampling (right) of a two-dimensional search space.
Factorial and Simplex sampling are sparse methods that seek to capture general behavior of the design space with very few points. They are both appropriate when we expect the analysis to have consistent global behavior. The factorial approach distributes points at the edge of the domain, making it very accurate when the response surface is close to linear. The simplex approach samples the interior of the search space which can produce better results if the global behavior is approximately quadratic or of higher order
Figure 4: Kriging interpolation of points acquired from sampling the surface in figure 1 using the factorial (left) and simplex (right) methods.
In this instance, we see that both the factorial and simplex methods are too sparse to capture the complex sinusoidal behavior of the surface. The simplex method produces a slightly more accurate model of the interior of the space but does not have enough points to capture key local features.
Figure 5: Box-Behnken (left), Koshal (middle), and Composite (right) sampling of a two-dimensional search space.
The Box-Behnken, Koshal, and Composite methods offer some flexibility that makes them effective in dictating design points for domains with up to quadratic complexity. In the two-dimensional search space, the methods produced eight, seven, and nine points respectively with an easily discernable and systematic distribution. The Box-Behnken approach also prioritizes sampling the edge of the domain to capture global trends. The Koshal approach sacrifices direct knowledge of certain regions on the edge of the domain to check the behavior in the interior of the domain. The Composite approach samples the center of each edge and each plane (or hyperplane for higher dimensional search spaces) in the domain. Here, the Composite approach is effectively the combination of the Box-Behnkin and Koshal methods and provides slightly better accuracy though requires additional points to be evaluated.
Figure 6: Kriging interpolation of points acquired from sampling the surface in figure 1 using the Box-Behnken (left), Composite (middle), and Koshal (right) methods.
In this example, each of these algorithms were able to capture the defining features of the surface even with fairly sparse sampling. One potential drawback of these methods that can be seen in this example is that because they form such a regular grid, they may mask the importance of variable interactions. Here we see the reconstructed surfaces form sharper, more linear trends than the original surface due to the interpolation seeking the simplest fit along the implied gridlines of design points.
Figure 7: Random (left), Taguchi (middle), and Latin Hypercube (right) sampling of a two-dimensional search space.
Random, Taguchi, and Latin Hypercube sampling offer the most flexibility in terms of number of design points produced. Random sampling is perhaps the most intuitive approach, however in figure 7 we see that this can lead to a mixture of clusters of densely packed points and sparse areas. This means that some points are nearly redundant while regions of the design space may be largely unexplored. The Taguchi method produces either 3, 9, 27, or 81 systematically distributed design points based on user specification. In the case of 9 points on a two-dimensional domain, this method gives identical results as ILIAD’s Composite sampling method. The Latin Hypercube method constructs a set of points in multidimensional space that are more uniformly distributed than those generated random sampling. Latin Hypercube method can accommodate any number of desired points and is almost always preferable to random sampling when exploring a continuous design space. Latin Hypercube is a good choice when local features are expected to be present in the interior of the domain.
Figure 8: Kriging interpolation of points acquired from sampling the surface in figure 1 using the random (left), Taguchi (middle), and Latin Hypercube (right) methods.
While interpolation of points acquired by random sampling appears quite nice at a glance, in this example it failed to accurately capture any of the surfaces’ maxima, instead predicting low values or minima in their places. By contrast, the Taguchi and Latin Hypercube methods succeeding in qualitatively matching the features of the original surface while still using the same number of points as the random sampling. Though somewhat erratic, the interpolation based on the Latin Hypercube points does the best job of representing the smooth, sinusoidal quality of the surface and the true locations of the extrema.
We hope this helps illuminate ILIAD’s arsenal of DOE designs and provides practical guidance for user looking to make the most out of each analysis and trial in their experiments. Check back in weekly for more tips, tricks, and deep dives into OmniQuest’s software solutions!