If we're refactoring, there's a chance that what we want to refactor isn't currently under test. This means that if we do perform refactoring on the code, we'll have to manually test the system through the established user interfaces to verify that the code works. Realistically, this doesn't verify the code; this verifies that the external behavior hasn't changed. There could very well be a hidden problem in the code that won't manifest itself until the external behavior has been modified distancing detection of the defect from when it was created. Our goal is to not affect external behavior when refactoring, so verification through the graphical user interface doesn't fully verify our changes and is time consuming and more prone to human error.
What we really want to do is unit test the code. The term "unit test" has become overloaded over the years. MSDN describes unit testing as taking:
…the smallest piece of testable software in the application, [isolating] from the remainder...