Short answer: Super important.

Here is why:
1. Google explicitly penalizes non-mobile enables websites
2. People have a poor experience on non-mobile enabled websites from mobile phones, which further kick-in other google algorithms to lower your website.

Example: One of the signals google uses to rank a page is the time spent on a website. If someone using a mobile phone comes to your website from Google and jump off it because it doesn't render well. The time spent on website will be low, and google will lower your rank.
3. Mobile internet browsing has officially overtaken desktop browsing - http://bgr.com/2016/11/02/internet-usage-desktop-vs-mobile/

I would highly recommend not ignoring mobile users when thinking of building a website.

Hope this helps.