At least a third of your phone screening time should be devoted to technical questions or challenges. If your engineering team is on the small side, consider holding technical phone screenings with two or three engineering team members. Although this compounds the man hours necessary for a phone screening, it can be very useful. Your employees can ask candidates how they would handle a challenge that the employee has personally run into.

Even if your expertise has nothing to do with engineering, you can certainly expect to get an accurate picture of how technically skilled a candidate is. One type of question I favor is the "explain it like I'm five" variety—ask the candidate to explain a technical concept to you while assuming that you have no background knowledge of technology at all. Listen carefully and ask the candidate to clarify any technical terms they use or assumptions they make. Good candidates will leave you with a clear understanding of even very abstract...