Sampling Methodology
Sampling is the process of selecting data from a population. By analyzing samples, characteristics of the entire population can be identified. Sampling is performed when it is not feasible to study the entire population due to time and cost constraints. Therefore, samples are a subset of the population.
Sampling Types
This is a very important topic from a CISA exam perspective. Two or three questions can be expected from this topic. A CISA candidate should have an understanding of the following sampling techniques:
|
Sampling Types |
Description |
|
Statistical sampling |
This is an objective sampling technique. This is also known as non-judgmental sampling. It uses the laws of probability, where each unit has an equal chance of selection. In statistical sampling, the probability of error can be objectively quantified, and hence the detection risk can be reduced. |
|
Non-statistical sampling |
This is a subjective sampling technique. It’s also known as judgmental sampling. The auditor uses their experience and judgment to select the samples that are material and represent a higher risk. |
|
Attribute sampling |
Attribute sampling is the simplest kind of sampling based on certain attributes; it measures basic compliance. It answers the question, “How many?” It is expressed as a percentage—for example, “90% complied.” Attribute sampling is usually used in compliance testing. |
|
Variable sampling |
Variable sampling offers more information than attribute sampling. It answers the question, “How much?” It is expressed in monetary value, weight, height, or some other measurement—for example, “an average profit of $25,000.” Variable sampling is usually used in substantive testing. |
|
Stop-or-go sampling |
Stop-or-go sampling is used where controls are strong and very few errors are expected. It helps to prevent excess sampling by allowing the audit test to end at the earliest possible moment. |
|
Discovery sampling |
Discovery sampling is used when the objective is to detect fraud or other irregularities. If a single error is found, the entire sample is believed to be fraudulent/irregular. |
Table 2.3: Types of sampling and their descriptions
The following diagram will help you to understand the answers to specific CISA questions:
Figure 2.5: Different types of sampling
Also, remember the term AC-VS—Attribute Sampling for Compliance Testing and Variable Sampling for Substantive Testing.
Sampling Risk
Sampling risk refers to the risk that a sample is not a true representation of the population. The conclusion drawn by analyzing the sample may be different from the conclusion that would have been drawn by analyzing the entire population.
Other Sampling Terms
A CISA candidate should be aware of the following terms related to sampling.
The Confidence Coefficient
A confidence coefficient, or confidence level, is a measure of the accuracy of and confidence in the quality of a sample. The sample size and confidence correlation are directly related. A high sample size will give a high confidence coefficient.
Look at the following example:
|
Population |
Sample Size |
Confidence Correlation |
|
100 |
95 |
95% |
|
50 |
50% |
|
|
25 |
25% |
Table 2.4: Example of confidence coefficient
In the case of poor internal controls, the auditor may want to verify 95 samples (sample size) out of a total population of 100. This gives a 95% confidence correlation.
In the case of strong internal controls, the auditor may be satisfied with only 25 samples out of the total population of 100. This gives a 25% confidence correlation.
Level of Risk
The level of risk can be derived by deducting the confidence coefficient from 1. For example, if the confidence coefficient is 95%, then the level of risk is 5% (100%–95%).
Expected Error Rate
This indicates the expected percentage of errors that may exist. When the expected error rate is high, the auditor should select a higher sample size.
Tolerable Error Rate
This indicates the maximum error rate that can exist without the audit result being materially misstated.
Sample Mean
The sample mean is the average of all collected samples. It is derived by adding all the samples and dividing the sum by the number of samples.
Sample Standard Deviation
This indicates the variance of the sample value from the sample mean.
Compliance versus Substantive Testing
A CISA candidate should be able to differentiate between compliance testing and substantive testing. They should be able to determine which type of testing is to be performed under different scenarios.
The Differences between Compliance Testing and Substantive Testing
The following table differentiates between compliance and substantive testing:
|
Compliance Testing |
Substantive Testing |
|
Compliance testing involves the verification of the controls of a process. |
Substantive testing involves the verification of data or transactions. |
|
Compliance testing checks for the presence of controls. |
Substantive testing checks for the completeness, accuracy, and validity of the data. |
|
In compliance testing, attribute sampling is preferred. |
In substantive testing, variable sampling is preferred. |
Table 2.5: Differences between compliance testing and substantive testing
Essentially, verifying whether a control is present or not is compliance testing. Meanwhile, verification of the complete process by testing the data/transaction to “substantiate” that the process is working is substantive testing.
Examples of Compliance Testing and Substantive Testing
The following examples will further help you understand the different use cases of compliance testing and substantive testing:
|
Compliance Testing |
Substantive Testing |
|
To check for controls in router configuration |
To count and confirm the physical inventory |
|
To check for controls in the change management process |
To confirm the validity of inventory valuation calculations |
|
Verification of system access rights |
To count and confirm cash balance |
|
Verification of firewall settings |
Examining the trial balance |
|
Reviewing compliance with the password policy |
Examining other financial statements |
Table 2.6: Differences between the use cases of compliance testing and substantive testing
The Relationship between Compliance Testing and Substantive Testing
A CISA candidate should understand the following points about the relationship between compliance testing and substantive testing:
- Ideally, compliance testing should be performed first and should be followed by substantive testing.
- The outcome of compliance testing is used to plan for a substantive test. If the outcome of compliance testing indicates the existence of effective internal controls, then substantive testing may not be required or may be reduced. However, if the outcome of compliance testing indicates a poor internal control system, more rigorous substantive testing is required. Thus, the design of substantive tests is often dependent on the result of compliance testing.
- The attribute sampling technique (which indicates that a control is either present or absent) is useful for compliance testing, whereas variable sampling will be useful for substantive testing.
Apart from the appropriate sampling technique, another important aspect of the audit process is using appropriate evidence-gathering techniques. Audit evidence should be collected properly to establish its reliability. Details on the reliability of audit evidence and collection techniques are covered in the next section.
Key Aspects from the CISA Exam Perspective
The following table covers important aspects from the CISA exam perspective:
|
CISA Questions |
Possible Answers |
|
Which sampling technique should be used when the probability of error must be objectively quantified? |
Statistical sampling. |
|
How can sampling risk be mitigated? |
By using statistical sampling. |
|
Which sampling method is most useful when testing for compliance? |
Attribute sampling. |
|
In the case of a strong internal control, should the confidence coefficient/sample size be increased or lowered? |
The confidence coefficient/sampling size may be lowered. |
|
Which sampling method would best assist auditors when there are concerns of fraud? |
Discovery sampling. |
|
How can you differentiate between compliance testing and substantive testing? |
The objective of compliance testing is to test the presence of controls, whereas the objective of substantive testing is to test individual transactions. Take the example of asset inventory: Compliance testing verifies whether a control exists for inward/outward movement of the assets. Verifying the count of physical assets and comparing it with records is substantive testing. |
|
What are some examples of compliance testing? |
To verify the configuration of a router for controls. To verify the change management process to ensure controls are effective. Reviewing system access rights. Reviewing firewall settings. Reviewing compliance with a password policy. |
|
What are some examples of substantive testing? |
A physical inventory of the tapes at the location of offsite processing. Confirming the validity of the inventory valuation calculations. Conducting a bank confirmation to test cash balances. Examining the trial balance. Examining other financial statements. |
|
In what scenario can the substantive test procedure be reduced? |
The internal control is strong/the control risk is within acceptable limits. |
Table 2.7: Key aspects from the CISA exam perspective