If you're looking for a skilled developer, but knows nothing in programming - you're just aware of the existence of languages like ruby on rails, java script, python, etc..
If you're not technical yourself, then you might not be able to gauge the efficiency of a candidate's algorithms or critique her code.
But there are still some higher-level, more behavioral things that a non-technical interviewer should be looking for in a strong development candidate:
1. What are some tech blogs that you follow? Explain an interesting article to me that you read from one of them.
The software development world changes all the time. Best practices are constantly evolving and new libraries are regularly released which make developers more productive. If a candidate doesn't keep up with the latest software news, that might be a red flag that they're not curious or trying to improve themselves.
Also, having them explain a technical concept to someone who's non-technical is a great way to gauge their communication skills. Do they seem like someone you could work with and understand easily? Do they care about pausing to make sure you understand, or do they just drone on with jargon? If you feel overwhelmed while they're explaining this answer, imagine how you'll feel when they're telling you why the product has bugs or isn't going to be done on schedule.
2. Tell me about a time you ran into a big roadblock with something you were building. How did you get past it?
It's inevitable that a software developer will get tripped up or have to solve some Gordian Knot. Everyone has to bang their head against the wall from time to time. Maybe an API didn't have the data they needed or some function was running too slow and they weren't sure how to speed it up.
You're looking to see how they are as a problem solver. Did they come up with a clever but hacky solution? Were they methodical or did they fly by the seat of their pants? Did they go back to the stakeholders and see if the feature's requirements were flexible? Did they work on it for hours and hours trying new things? Did they ask for help from colleagues or on the internet?
No right or wrong answers here, but you want to get the sense that this isn't someone who throws up their hands when they hit some friction.
3. Tell me about your favorite project that you worked on. What work are you most proud of?
By asking them about the project they're most proud of, you'll get to see what it is that they value most. Maybe one candidate is most proud of a side project they built, even if it wasn't that technically complex, while another candidate is proud of their esoteric PhD project or some specific algorithm they improved.
Again, no right or wrong answers, it really depends what type of candidate you're looking for. But it lets you see into their mind a bit, and get at some of the aspects that can make someone a strong development candidate.
If you want to talk more specifically about hiring for your team, I'd be happy to do a call!
It's hard to evaluate a technical candidate as a non-technical person. Probably the best thing you can do is find another technical person you trust to do the evaluation.
That said, there are some tertiary skills you can definitely verify. If somebody is going to be reporting to nontechnical personnel (which is probably true if you're trying to evaluate them, being a nontechnical person yourself) then things like clear communication, expectations management, and raising concerns before they become problems are all really important.
When evaluating a technical person who's going to report to non-technical management, I would ask things like this:
1. Show me a report you've written to advise upper management.
Somebody who has clear communication skills with management will have a track record where they've demonstrated those skills. Depending on confidentiality issues, they should be able to pull up an email, a document, a presentation, where they advised a nontechnical person. Maybe they were approached to evaluate a technical direction for a product, a team, the whole company, etc.
2. Tell me what you did when a project had serious problems.
All projects have problems sooner or later. The question is how you handle them. A good engineer should raise concerns, accurately, promptly, and clearly, and be ready to discuss possible changes in course. However some organizations shoot the messenger, so be careful not to judge a person for a whole organization's dysfunction.
3. Show me a project you're proud of.
A good engineer will have a demo of something to show you. Don't get caught up too much in how closely it matches what you're hiring them for--it's hard for a nontechnical person to judge whether software is alike or different under the hood. Instead focus on how they communicate. Do they explain the problem well? Do they explain the solution? Does it work well?
Keep in mind the candidate may have had factors outside of their control (customers, management, technology) that explain some deficiencies in what they've built. But how do they respond when challenged? Are they hostile? Are they empathetic? Do they start to problem-solve? Those behaviors are probably ones that will carry forward if they're hired.
It's hard to judge a programmer at programming if you're not one yourself. That's why you should go to another programmer you trust to get an opinion. But you can definitely judge his/her communication skills. And if your developer is reporting directly to you, that's going to be equally, if not more important.
#1 Ask their opinion about language X or framework Y
Programmers should be opinionated. There should be a language or tool that they believe is the best (until something better eventually convinces them otherwise). This is because they've gone through growing pains and faced challenges that they were able to solve with some specific language or tool.
I always tell people that a red flag is when a developer has no preference. I've used many languages and currently work with 3-4 throughout any given day, but there is one that I'm most excited about.
Remember to ask "why" a lot, but you don't need to understand how the languages, frameworks, or tools work. You're just trying to dig for more and maybe even add a little pressure. You're looking for interest, motivation, and enthusiasm.
You can also, sometimes, determine here if the person is good cultural fit...Sometimes being over opinionated can be a bad thing. You certainly don't want someone who is really negative on a team.
This is a really good entry into the next two questions.
#2 Ask about challenges they may have solved.
This should be very natural coming off the last discussion because, again, they likely use what they use because it helped them solve a difficult problem in the past.
Nothing is easy about programming (especially for the web or mobile). You should never hear from someone that they didn't face any challenges. If you do, then they likely aren't experienced enough.
This question is important to see how they understand business problems too. Look for that. Ensure that they understand business goals and how sometimes the code isn't exactly how they'd like...But it helped the company make money or made the client happy.
There's a give and take to web programming that many programmers don't get. This leads to a lot of frustration because something wasn't done "properly" or up to some standard. Believe me, I've warned people many a time and I'm not always happy with the code I have to work with...But I've also seen "bad" code make millions of dollars.
#3 Ask what communities and projects they belong to and contribute to.
Most web developers should be into open-source projects and tools. They help us get the job done faster, plain and simple. They also help us learn.
To see a web developer unaware of any, not using any, or not engaging with people (even to ask a question or file an issue on GitHub)...This is a huge red flag in my book.
You want a developer who is active in the community because they likely then can reach out to others to solve problems that they have a tough time with.
Contributing to open-source projects also means they understand process and how to work with a team (usually) because these projects have guidelines and a process for contributors.
This can, yet again, be a signal for cultural fit. Does the developer go off and argue with people in the community? Or are they constructive and friendly? Do they not even bother to talk to others in their industry? Why?
These are all high level questions that pretty much naturally fit together and are what I've been using for years...And I'm a developer! I could ask more technical questions, but at the end of the day it's the cultural fit, motivation, and ability to learn and problem solve that interest me more.
I don't want someone who knows only X language and is even the best at it. I want someone who can learn X, Y, and Z should they need to.
I've been a developer, consultant, manager. I've conducted hundreds of phone screens and interviews. Even now, I can tell you that it's difficult to gauge someone's experience, particularly in a short timeframe. I doubt I could accurately gauge someone's experience in three questions.
Technical assessment is hard. There's no simple solution, so I can't give you something totally reliable. The two most reliable ways I know to assess someone are:
- Referral: If you trust someone's judgement and they've worked with that person before and they strongly believe that person can help you, that's usually been pretty reliable for me.
- Actual Work: If you have a very small list of candidates, preferably one that you're seriously considering, start doing some actual work with that person, but keep your options open. Start building trust with regular delivery of working software. Get some early deliverables defined, have that person deliver you an early work product -- maybe a prototype. Continue to build on that incrementally. Either the candidate will show you with each incremental delivery that they can get the job done, or they won't, and you'll be able to make ongoing decisions about whether or not you want to trust the rest of your project to that candidate.
But, back to the meat of your real question -- how would you quickly assess a potential candidate to see if they might be a good fit? To be honest, I think trying to assess detailed technical skills if you don't have them isn't really going to work. Any question I could give you, they could fake easily enough if they're more technical than you are. You'd be better off trying to relate to them in the way that you expect to -- as someone who doesn't know programming and will rely on them to bring that skill-set. Ask them questions you want answers to about how easy and hard parts of your project will be. Ask them to help you understand how they will accomplish parts of the project. You won't necessarily be able to assess the detailed technology they're describing, but you will get a sense for whether or not they seem like they know how to go about your work, and whether they'll be able to communicate well with you and give you the information you need.
Once you find the right person who can communicate with you, then go back to my earlier response, and work on building trust through continuous delivery of software.
I've interviewed hundreds of folk for coding roles and indeed, I still have to do it in my own startup.
Although I myself am a coder, I still use a lot of non-coding techniques to sift through other coders. There are a few options nowadays, all of them you have to go fish a bit and there are a few pitfalls with this which can result in false positives/negatives, so don't rely on any one option as your only source of truth. Combine the results and make sure to get a decent sized historical record where they exist.
1. Go to sites like Stack Overflow and see if they are on it and what their ratings are.
2. Check if they have a GitHub account. You won't be able to verify their code, but look at how often they've done stuff recently, what they're contributing to, check their icket contributions and discussion, how many commits they've made in the codebase etc.
3. Read their blogs. Again, useful as a historical record.
4. When speaking to coders, make sure they're talking about the value they deliver to you, not just the code. Code isn't written for code's sake, it's got some value to it.
As others above have said, there is definitely mileage in evaluating the way they communicate as well. Since you'll be working with them and if they can't communicate with you, then you'll both be a bit stuck.
I've missed out a lot of detail here. Give me a prod if you need me to goo into more detail for you.