Optimization

(mathematics) The maximizing or minimizing of a given function possibly subject to some type of constraints.
(systems engineering) Broadly, the efforts and processes of making a decision, a design, or a system as perfect, effective, or functional as possible. Narrowly, the specific methodology, techniques, and procedures used to decide on the one specific solution in a defined set of possible alternatives that will best satisfy a selected criterion. Also known as system optimization.

The design and operation of systems or processes to make them as good as possible in some defined sense. The approaches to optimizing systems are varied and depend on the type of system involved, but the goal of all optimization procedures is to obtain the best results possible (again, in some defined sense) subject to restrictions or constraints that are imposed. While a system may be optimized by treating the system itself, by adjusting various parameters of the process in an effort to obtain better results, it generally is more economical to develop a model of the process and to analyze performance changes that result from adjustments in the model. In many applications, the process to be optimized can be formulated as a mathematical model; with the advent of high-speed computers, very large and complex systems can be modeled, and optimization can yield substantially improved benefits.

Optimization is applied in virtually all areas of human endeavor, including engineering system design, optical system design, economics, power systems, water and land use, transportation systems, scheduling systems, resource allocation, personnel planning, portfolio selection, mining operations, blending of raw materials, structural design, and control systems. Optimizers or decision makers use optimization in the design of systems and processes, in the production of products, and in the operation of systems.

The first step in modern optimization is to obtain a mathematical description of the process or the system to be optimized. A mathematical model of the process or system is then formed on the basis of this description. Depending on the application, the model complexity can range from very simple to extremely complex. An example of a simple model is one that depends on only a single nonlinear algebraic function of one variable to be selected by the optimizer (the decision maker). Complex models may contain thousands of linear and nonlinear functions of many variables. As part of the procedure, the optimizer may select specific values for some of the variables, assign variables that are functions of time or other independent variables, satisfy constraints that are imposed on the variables, satisfy certain goals, and account for uncertainties or random aspects of the system.

System models used in optimization are classified in various ways, such as linear versus nonlinear, static versus dynamic, deterministic versus stochastic, or time-invariant versus time-varying. In forming a model for use with optimization, all of the important aspects of the problem should be included, so that they will be taken into account in the solution. The model can improve visualization of many interconnected aspects of the problem that cannot be grasped on the basis of the individual parts alone. A given system can have many different models that differ in detail and complexity. Certain models (for example, linear programming models) lend themselves to rapid and well-developed solution algorithms, whereas other models may not. When choosing between equally valid models, therefore, those that are cast in standard optimization forms are to be preferred. See also Model theory.

The model of a system must account for constraints that are imposed on the system. Constraints restrict the values that can be assumed by variables of a system. Constraints often are classified as being either equality or inequality constraints. The types of constraints involved in any given problem are determined by the physical nature of the problem and by the level of complexity used in forming the mathematical model.

Constraints that must be satisfied are called rigid constraints. Physical variables often are restricted to be nonnegative; for example, the amount of a given material used in a system is required to be greater than or equal to zero. Rigid constraints also may be imposed by government regulations or by customer-mandated requirements. Such constraints may be viewed as absolute goals.

In contrast to rigid constraints, soft constraints are those constraints that are negotiable to some degree. These constraints can be viewed as goals that are associated with target values. The amount that the goal deviates from its target value could be considered in evaluating trade-offs between alternative solutions to the given problem.

When constraints have been established, it is important to determine if there are any solutions to the problem that simultaneously satisfy all of the constraints. Any such solution is called a feasible solution, or a feasible point in the case of algebraic problems. The set of all feasible points constitutes the feasible region.

If no feasible solution exists for a given optimization problem, the decision maker may relax some of the soft constraints in an attempt to create one or more feasible solutions; a class of approaches to optimization under the general heading of goal programming may be employed to relax soft constraints in a systematic way to minimize some measure of maximum deviations from goals.

A key step in the formulation of any optimization problem is the assignment of performance measures (also called performance indices, cost functions, return functions, criterion functions, and performance objectives) that are to be optimized. The success of any optimization result is critically dependent on the selection of meaningful performance measures. In many cases, the actual computational solution approach is secondary. Ways in which multiple performance measures can be incorporated in the optimization process are varied.