There are several ways to set up the browser of your choice to use the proxy server. We'll go over two of the most common ways, starting with my personal favorite, which is a browser extension.
Google Chrome and Firefox have vibrant browser plugin ecosystems that allow you to extend the capabilities of your browser with each plugin that you choose. For setting up a proxy, I really like FoxyProxy (http://getfoxyproxy.org/). It is a neat add-on to the browser that allows you to set up various proxy settings and toggle between them on the fly, without having to mess around with setting systems on the machine. It really makes the work hassle-free. Thankfully, FoxyProxy has a plugin for Internet Explorer, Chrome, and Firefox. If you are using any of these, you are lucky! Go ahead and grab it!