The recent explosion in the number of digital devices has generated an ever-increasing amount of data. The best analogy I can find to describe the need, desire, and urgency for extracting knowledge from large datasets is the process of extracting a precious metal from a mine, and in some cases, extracting blood from a stone.
Knowledge is quite often defined as a model that can be constantly updated or tweaked as new data comes into play. Models are obviously domain-specific, ranging from credit risk assessment, face recognition, maximization of quality of service, classification of pathological symptoms of disease, optimization of computer networks, and security intrusion detection, to customers' online behavior and purchase history.
Machine learning problems are categorized as classification, prediction, optimization, and regression.
The purpose of classification is to extract knowledge from historical data. For instance, a classifier can be built to identify a disease from a set of symptoms. The scientist collects information regarding body temperature (continuous variable), congestion (discrete variables of HIGH, MEDIUM, and LOW), and the actual diagnosis (flu). This dataset is used to create a model such as IF temperature > 102 AND congestion = HIGH THEN patient has the flu (probability 0.72), which doctors can use in their diagnosis.
Once the model is trained using historical observations and validated against historical observations, it can be used to predict some outcome. A doctor collects symptoms from a patient, such as body temperature and nasal congestion, and anticipates the state of his/her health.
Some global optimization problems are intractable using traditional linear and non-linear optimization methods. Machine learning techniques improve the chances that the optimization method converges toward a solution (intelligent search). You can imagine that fighting the spread of a new virus requires optimizing a process that may evolve over time as more symptoms and cases are uncovered.
Regression is a classification technique that is particularly suitable for a continuous model. Linear (least squares), polynomial, and logistic regressions are among the most commonly used techniques to fit a parametric model or function, y= f (x), x={xi} to a dataset. Regression is sometimes regarded as a specialized case of classification for which the output variables are continuous instead of categorical.