This chapter introduces the concepts of validating methods that can be applied to the data passing through ETL processes in order to cleanse and conform it according to the defined Data Quality standards. It includes validation methods that consist of defining validation expressions with the help of validation functions and then splitting data into two data sets: valid and invalid data. Invalid data that does not pass the validation function conditions usually gets inserted into a separate target table for further investigation.
Another topic discussed in this chapter is dataflow audit. This feature of Data Services allows the collection of executional statistical information about the data processed by the dataflow and even controls the executional behavior depending on the numbers collected.
Finally, we will discuss the Data Quality transforms—the powerful set of instruments available in Data Services in order to parse, categorize, and make cleansing suggestions in order to...