At the time of writing this book, GeoServer is at its 2.5.0 release, and it's definitely a stable and mature project. It has a well-defined release schedule with major and minor versions released at regular intervals.
Note
For detailed and up-to-date information about the next releases, have a look at http://blog.geoserver.org/2014/02/26/extended-release-schedule/.
Upgrading frequently is a good thing; with any new release, you get many bugs fixed and new features included. Some experimental module, often from the community, may be included in the stable release. If you are running GeoServer in a production site, you have to choose between the benefits of upgrading it frequently and policies that your customer and/or system administrators may have enforced.
In any case, you need to fully understand what upgrading GeoServer means and its consequences on the following:
Your service and data
Establishing a standard procedure to upgrade your GeoServer installation
In this recipe...