Before digging into the wonderful world of extensions, it's good to know they come in different shapes and sizes. Basically, there are three types of extensions:
The big ones called Components. You manage them through a special Components menu in the backend. They are the most comprehensive extensions, sometimes providing lots of administration options and settings. Component output is usually displayed in the main content area. An example is the Contacts components (to manage a system of contacts, contact details, and contact forms).
The medium ones called Modules. Modules are "blocks" that contain special functionality. You've already seen examples at work, such as the menu module. You manage modules in Extensions | Module Manager. Modules usually turn up around the main content area: in the left-hand side and right-hand side column, or in the header and footer. These module positions ('left', 'right', and so on) are predefined by the template designer...