Comparison between Deterministic and Stochastic formulations of Particle Swarm Optimization, for Multidisciplinary Design Optimization.

The heuristics Particle Swarm Optimization (PSO) has been widely adopted since 1995, for the approximate solution of unconstrained global optimization problems. The main reason which motivates the use of PSO in place of exact methods (e.g. Pattern Search methods, Derivative-free optimization methods, etc.), is the fact that it often provides an acceptable compromise between the cost involved in the computation of the solution, and the accuracy to detect the latter. Hence, those frameworks where large simulations are involved, represent the natural context where PSO is possibly required, since designers may reckon on fast computation which is also reasonably accurate. On this guideline, in the literature of evolutionary methods a large number of PSO variants was proposed. As a result, heuristic PSO-type methods have been extended to the solution of constrained and Multiobjective Optimization problems (e.g. [3]), so that they are often the methods of choice by practitioners. PSO original iteration, as well as many of its variants as the Deterministic Particle Swarm Optimization (DPSO) algorithm (see also [2]), was investigated for the solution of optimization subproblems within Multidisciplinary Design Optimization frameworks. Examples of the latter approach were analyzed in [1], where a tough multidisciplinary problem (equivalent to a Distributed Analysis Optimization) requiring heavy simulations is adopted. A numerical experience proved that the overall optimization algorithm was convergent to a satisfactory approximation of a stationary point. Moreover, a preliminary theoretical analysis has partially proved that, under suitable assumptions, PSO is always convergent to limit points which lay in a compact set. The latter result is a preliminary goal, in order to prove any convergence result. Global convergence of PSO algorithm has been also partially demonstrated in [2]. In particular, the PSO iteration has been suitably rewritten in an equivalent formulation, resulting in analogy with a classical mechanical system. In this form, two separate terms came out and can be separately studied. One is addressed as the free-response of a dynamic linear system, and is affected only by the initial position of the swarm particles, their initial speed and the PSO coefficients. The other one, namely the forced-response of a suitable dynamic linear system, is strongly influenced by the shape of the objective function. The latter part is responsible for the overall convergence of PSO, since it determines whether any particle trajectory is diverging. On the other hand, very general considerations can be traced for the free-response, in order to assess a suitable range of the PSO coefficients. In particular, necessary conditions (which are in general not sufficient) can be provided, so that the particles in PSO iteration do not have diverging trajectories. On the other hand, under suitable assumptions on the objective function, specific considerations about the forced response can also be traced, but they can hardly be fulfilled in practice. In this paper, we study some novel aspects of PSO-type methods, in order to assess criteria of convergence. As a preliminary material, we have considered the results reported in [1, 2]. Then, we have analyzed the standard results of exact methods for unconstrained global optimization, in order to provide sufficient conditions for the convergence of PSO- type methods. We found that, in order to analyze the convergence of PSO iteration, the main drawbacks are related to the following issues. First, it is unable to exploit a dense subset of points, in the design space. La latter drawback is specifically evident when the algorithm detects points nearby a solution, but eventually its progress is very poor. Second,