VMs (in our case Hotspot) and collectors have improved immensely in many performance-related aspects over the years, both in major as well as minor versions. The changes can involve internal behavior that might affect your application in both positive and negative ways. Either way, the changes and their effects are seldom possible to realize by just reading the release notes.

The changes can also quite often be externally visible. The output of various VM parameters, such as the ones for GC logging, has changed many times.

All of these changes are rarely documented in a satisfactory way and changes that can affect your entire application might turn up even in minor releases. To handle this in an orderly fashion involves, as per usual, rigorous and strategic tests.

No VM upgrade, changes in GC strategy, or adaption of VM parameters should be allowed in the production environment without proper tests. As always, change one thing, and one thing only, between each test!