Geographical data is usually stored in databases and programs called Geographic Information Systems (GIS). These are usually tools to design, store, manipulate, analyze, and display the cartographic data. The following figure shows how the different aspects of the world can be abstracted as layers and managed in such a system:
A typical GIS abstraction
Most of these systems provide an interface to access or download these layers separately—that is, cartographic data, statistical data, or other geographic data—in a specified GIS format. There exists a variety of different GIS pixel- and vector-based formats; however, we will look into two JSON-based vector formats: GeoJSON and TopoJSON.
If you are looking for geographic data sets, I recommend you to visit http://www.data.gov/, http://geocommons.com/ as well as the extensive list on Robin Wilson's page at http://freegisdata.rtwilson.com/.