A map is generally composed of a set of layers. Each layer contains features of a determined type. When you ask GeoServer for a map, it has to extract features from the repository (for example, from a shapefile) and draw them according to some rules. Of course, it needs a repository for storing those rules and hence GeoServer developers need to decide a format for the storage medium containing rules.
Map rendering is not just a GeoServer problem; not surprisingly, it is common to all software-producing maps. Hence, it is not surprising that someone has defined a standard approach to styling layers. Indeed, GeoServer doesn't use a custom format for styles; instead it leverages on an OGC standard.
The standard describes the structure of the documents and which rules can be used. A document containing symbols' definitions and drawing rules is called a Styled Layer Descriptor (SLD) style and it is a text/XML file (its extension in GeoServer is .sld). SLD is...