Solr offers many knobs and levers for increasing performance; the biggest challenge can be figuring out which knobs and levers to use! Make sure you budget your time to try a number of approaches, and take a step-wise approach to trying different approaches out. From turning the simpler knobs for enhancing the performance of a single server, to pulling the big levers of scaling wide through using SolrCloud, performance and scalability with appropriate hardware are issues that can be solved fairly easily, and Solr provides almost linear scaling. The SolrCloud codebase has evolved quickly over the course of 4.x, so if you are on an older version, especially prior to 4.7.1, you will find many enhancements and stability fixes to encourage you to upgrade.