When installing Joomla!, we have the option of installing it with a minimum configuration or to add sample data. It is usually better to start with the sample data and then remove, or change the things we don't want or need, as the minimum configuration leaves many people wondering where to start with their site. At least with the sample data we will have a working site right from the start and be able to visualize the changes we want to make.
The first thing we will do with our site is remove or disable any Core extensions that we don't need on our site. This is done mainly to reduce clutter and make the site easier for us to work with. It may also potentially improve the performance of our site, but probably not enough to be noticeable.
Some Core extensions can be removed without negative effects, but others are important to the running of Joomla! or are intertwined with Core tables and so we are only able to disable these extensions, not uninstall them...