## Optimization

Optimization involves finding extreme values of functions or their roots. We have already seen the power of optimization in the curve-fitting arena, but it does not stop there. There are applications to virtually every single branch of engineering, and robust algorithms to perform these tasks are a must in every scientist's toolbox.

The `curve_fit`

routine is actually syntactic sugar for the general algorithm that performs least-squares minimization, `leastsq`

, with the imposing syntax:

leastsq(func, x0, args=(), Dfun=None, full_output=0, col_deriv=0, ftol=1.49012e-8, xtol=1.49012e-8, gtol=0.0, maxfev=0, epsfcn=0.0, factor=100, diag=None):

For instance, the `curve_fit`

routine could have been called with a `leastsq`

call instead:

leastsq(error_function,p0,args=(x,y))

Here, `error_function`

is equal to `lambda p,x,y: target_function(x,p[0],p[1],p[2])-y`

The implementation is given in the corresponding section on the IPython Notebook of this chapter. Most of the optimization routines...