Benchmarking is always needed in a complex environment. There are several reasons for benchmarking, for example making sure that an application is actually usable in the real world or to detect and avoid performance regressions when new code is checked in. Benchmarking can also help optimize a complex application by breaking it down into more manageable problem domains (specialized benchmarks) that are easier to optimize. Finally, benchmarking should not be underestimated as a tool for marketing purposes.
Benchmarking is relevant in software development on all levels, from OEM or Java Virtual Machine vendors to developers of standalone Java applications. It is too often the case in software development that while the functionality goals of an application are well specified, no performance goals are defined at all. Without performance goals and benchmarks in place to track the progress of those goals, the end result may be stable but completely unusable...