Performance is important for many reasons, which we will delve into soon. It is very important to understand when is the right time to try and improve performance. My guiding principle is: make it work, make it right, make it fast. That is, first, just get the system to do whatever it needs to do, however slow and clunky. Then, clean up the architecture and the code. Now, you are ready to take on performance and consider refactoring, changes, and many other factors that can impact performance.
But there is a preliminary step for performance improvements, and that's profiling and benchmarking. Trying to improve performance without measuring what you try to improve is just like trying to make your code work correctly without writing any tests. Not only is it futile, but, even if you actually got lucky and improved the performance, how would you know...