Every module that is part of core Moodle has gone through a thorough quality assurance process. The potential problem with third-party add-ons is that you don't know anything about the quality of the software.
While it is possible to uninstall modules if they don't suit your purpose, you will have to make sure that you don't put barriers in place for future updates. If an add-on is not maintained, it is unlikely to support any forthcoming versions of Moodle and you will either have to delete the module already in use or you won't be able to upgrade your system. Also, a module might cater to some required functionality but compromise the security of your system.
There are a number of criteria that indicate whether an add-on is trustworthy or not:
Popularity: Moodle keeps statistics about downloads from Moodle, which you can find at
download.moodle.org/stats.php. In the Plugins section, you will see all plugins that have been downloaded in the previous 60 days....