Decision Analytics for Complex Systems

Patrick Reed, School of Civil & Environmental Engineering


From Reed Research Group | Decision Analytics for Complex Systems



Kollat, J.B., and P. Reed

AeroVis is advanced visualization software that enables efficient exploration of complex multi-dimensional spaces associated with many-objective complex engineering design problems. AeroVis is distributed by DecisionVis, LLC, and can be obtained by registering here. The software is freely available for academic users; other interested individuals can contact Dr. Kollat ( for licensing options.


Hadka, D. and Reed, P.

The Borg Multiobjective Evolutionary Algorithm (Borg MOEA) features auto-adaptive multi-operator search, dynamic population sizing, and epsilon-dominance archiving to solve many-objective optimization problems. See our publications for details on its design as well as studies demonstrating the effectiveness and efficiency of the Borg MOEA on a number of real-world applications. The C/C++ source code is freely available for academic and non-commercial use. Visit for more information and to request access to the source code.

MOEA Diagnostic Tool

MOEA Framework

MOEA Framework is a free, open-source Java framework for experimenting with several popular MOEAs including GDE3, MOEA/D, NSGA-II, ε-MOEA, ε-NSGA-II and random search. These MOEAs can be run on over 80 test problems, including problems from the ZDT, DTLZ and WFG test problem suites, as well as all constrained and unconstrained problems from the IEEE CEC 2009 special competition. The framework and source code can be downloaded from New users connecting their own optimization problems to MOEAFramework are encouraged to read through the MOEAFramework Setup Guide.

Interested users may also explore the in-browser MOEA Diagnostic Tool (requires Java).

Interactive Parallel Axis Plots

Bernardo Trindade and Jon Herman
Interactive parallel axis plots
Built with Parcoords D3.js library. Upload a CSV data file to explore tradeoffs between conflicting objectives.


Hadka, D., Keller, K., & Reed,P.(Link to Paper)

Robust Decision Making (RDM) is an analytic framework developed by Robert Lempert and his collaborators at RAND Corporation that helps identify potential robust strategies for a particular problem, characterize the vulnerabilities of such strategies, and evaluate trade-offs among them [Lempert et al. (2006)]. Multiobjective Robust Decision Making (MORDM) is an extension of RDM to explicitly include the use of multiobjective optimization to discover robust strategies and explore the tradeoffs among multiple competing performance objectives [Kasprzyk et al. (2013)].

MOEA Benchmark Problems

Reed, P. et al. 2013 (Link to Paper)

Benchmark optimization problems published in "Evolutionary multiobjective optimization in water resources: The past, present, and future", an invited submission to the 35th Anniversary Issue of Advances in Water Resources. The problems are intended for use with MOEAFramework but may be modified for other algorithms or libraries.

Sensitivity Analysis Library (SALib)

  • SALib, a Python library containing Sobol, Morris, and FAST sensitivity analysis methods. Requires Numpy. Designed for decoupled analysis, where sampling, model evaluation, and analysis steps are all performed separately.

Open-Source Models

  • Hymod and HBV rainfall-runoff models in C/C++. Written for use with MOPEX hydrologic data, but can be modified for use with other forcing and response data. Based on work from this paper but later modified for generality.

Figure Examples

  • Plotting Examples: Simplified applications of built-in plotting functions for different types of data in Matlab, R, and Python (Matplotlib).

Technical Reports

Making Animated GIFs in AeroVis: A Beginner's Guide By JR Kasprzyk and JB Kollat. 2012001. The goal of this document is to discuss the procedure for making animated GIFs in AeroVis and importing them into Powerpoint. Video | PDF

Basic MOEA Concepts and Reading By JD Herman, JR Kasprzyk, JB Kollat, and MJ Woodruff. 2012002. This document gives basic reading and concepts important for MOEA optimization. PDF

Getting the Most Out of Academic Papers By JR Kasprzyk. 2012003. An educational report about techniques for finding academic papers, reading them, and how to summarize what you found in your reading. PDF

MOEAFramework Setup Guide By Group. 2012004. A step-by-step guide to connecting users' external optimization problems to MOEAFramework. HTML

Benchmark Data

General Aviation Aircraft Problem

The General Aviation Aircraft (GAA) problem seeks the design of three propeller-driven aircraft for general aviation use. The three planes must seat 2, 4, and 6 passengers and satisfy a number of performance and economic constraints. Furthermore, the three aircraft should share similar subsystems to reduce manufacturing costs. This requirement is captured using a product family penalty function. In total, there are 27 real-valued decision variables, 10 objectives, and 1 constraint. Source code for C/C++ and Java as well as the best-known reference set for this problem can be found in Our most recent study on this problem is cited below.

  • Hadka, D. et al. Diagnostic Assessment of the Borg MOEA on Many-Objective Product Family Design Problems. WCCI 2012 World Congress on Computational Intelligence, Congress on Evolutionary Computation, Brisbane, Australia, 10-15 June 2012, pp. 986-995. (PDF)

Knapsack Study

Here are the reference sets for the recently published knapsack study: Data
Reference sets are included for the Weighted Scalar Repair Approach (WSR) for both random and correlated 2, 3, and 4 knapsack problem instances involving 100, 250, 500, and 750 items. Item weight values are included in the "weight" files.

Shah, R., and Reed, P.M., “Comparative Analysis of Multiobjective Evolutionary Algorithms for Random and Correlated Instances of Multiobjective d-Dimensional Knapsack Problems.”, European Journal of Operational Research, v211, n3, 466-479, 2011.

Personal tools