Evolutionary Optimization Using Big Data from Engineering Simulations and Apache Spark

This paper presentsa novel data flow architecturethat utilizes data from engineering simulations to generate a reduced order model within Apache Spark. The reduced order model from Spark is then utilized by anevolutionary algorithm in the optimization of an industrial system component. This work is presented in the context of the shape optimization of a heat exchanger fin and demonstrates the ability of theengineering simulation, the reduced order model and the evolutionary algorithm to exchange data with each other by utilizing Spark as the common data-processing framework. In order to enable a user to monitor the input design parameter space,self-organizing maps are generated for visualization. The results of theevolutionary optimization utilizing this data flow are compared with results from invoking high-fidelity engineering simulations. This novel data flow architecture decouples the evolutionary algorithm from the reduced order model and allows improvement of the optimization results by continuously augmenting the reduced order model with data from the evolutionary algorithm.Additionally, when constraints on the optimization algorithm are modifiedthe evolutionary algorithm canadapt and evolve good solutions. Themethodology presented in this articlealso makes it feasible to simultaneously tune evolutionary optimization experiments along with engineering simulations at a relatively low computational cost.