We briefly touched on numerical optimization in the section on finding minimum values of functions. In general, optimization is a process of choosing the best element from a set of possible elements. The criteria for choosing the "best" element are quantified in the form of an objective function that is to be minimized or maximized. The elements may be discrete elements, or they may take on a continuous range of values. In general, optimization is a very difficult problem that can be approached in many different ways. We will focus on the case in which we seek to minimize a scalar-valued objective function by choosing the values of variables from a continuous set. The values of the variables may be limited by constraints. Optimization is of great importance in science and engineering, where it is used for fitting functions to data sets and finding optimal combinations of design parameters.