The best way is to lean how to ask great questions and to physically spend time with your users watching them work.
I don't have all the killer questions, but my favourite to help understand product roadmap opportunities is "What do you do 3 minutes before, and after you use our product?". That helps me understand where the opportunity lies to build a solid solution end-to-end.
Also, finding a set of users that feel comfortable having you hangout and work with them, taking notes and asking questions is invaluable. Just to understand their environment, the way they organize their work, and how they might communicate around the problems your solution is/could help with is incredible.
Every time I'm at an event and someone ask me what I do, I always ask them if I can "show them" and if they say yes, I have them find / download / signup and explore our app. Then I ask them questions about how they might see someone using it, who they think might be the best customer, etc.
It's not about not knowing, it's about continuously learning and testing different perspectives.
As for tools I use:
www.usertesting.com / especially new mobile tests!
I love what Dan said about asking what users do 3 mins before and after using your product.
I think the best product people know how to extrapolate better than others. Its one thing to gather feedback and observe users, but its much more challenging deciding what to do with that information. Simply asking users what they want wont always give you the best outcome. Someone once said, although Henry Ford is often credited, "if i had asked, people would have wanted faster horses". Theres some truth there. Its about the ability to observe customers and extrapolate the features.
Trying to understand what job your product is being hired by the user to do is another framework to uncover users unstated desires.
Observing your user while they are using your product while going about their daily routine with the above question in mind is one way to get to this.