We dipped our toes into the design side of things, looked at some great examples, and tried to learn how they work. As a non-designer, that's how I approach any new visualization I want to create—look at a bunch of examples and figure out what works on my dataset.
Then it's experimentation, experimentation, and playing around until I find something that works.
For a deeper look at designing visualizations, I suggest reading books devoted to the subject. Data Visualization: a successful design process, Andy Kirk and his blog Visualizing Data are a good start. I haven't read the book, but the blog helped me a lot during this chapter.
Another good resource is Visualize This, Nathan Yau. The first few chapters are about design; the rest are about using R—a language for statistical analysis. It would also be good to read Interactive Data Visualization for the Web, Scott Murray.