In this chapter, we'll address various methods for finding the best outcome in a given situation. This is calledoptimizationand usually involves either minimizing or maximizing an objective function. An objective function is a function that takes a number of parameters as arguments and returns a single scalar value that represents the cost or payoff for a given choice of parameters. The problems regarding minimizing and maximizing functions are actually equivalent to one another, so we'll only discuss minimizing object functions in this chapter. Minimizing a function, f(x), is equivalent to maximizing the function -f(x). More details on this will be provided when we discuss the first recipe.

The algorithms available to us for minimizing a given function depend on the nature of the function. For instance, a simple linear function containing one or more variables has different algorithms...