To configure custom modules, recompile nginx with specific flags or copy new files into specific locations. You can run multiple instances of nginx side by side, as long as the virtual hosts do not overlap.

Running two or more nginx side by side can be beneficial as it reduces overhead and has a high level of isolation. The standard nginx can serve up your PHP application, while another version of nginx will be streaming a live video and a third could be handing Node.JS. You could run them all in one instance of nginx. But if you need to monitor performance, then separating them like this gives you greater control while still being more efficient and faster than Apache.

The drawback is that you have configuration files for each instance of nginx, but that seems like a small price to pay for such flexibility.

You can find a list of common modules at http://wiki.nginx.org/Modules.