~ Is it really viable?
Distributed teams are very common. If you are startup is not having to deal with shipping/receiving or if onsite events are not critical; then there are plenty of proven examples where a remote team is viable.
~ Will it be harder to bounce ideas and be innovative?
As long as all stakeholders communicate regularly and a system like Slack or Discord is adopted, it shouldn't be.
~ Will this kill momentum?
Only time this might is around an actual product launch where timing is critical for success. Otherwise it shouldn't impact anything.
~ What are your immediate concerns?
My only concern would be that adoption is key. If you both are unfamiliar with or haven't adopted agile methods for remote work, I think there will be some growing pains. But if you both commit fully to the process and the project, being remote shouldn't slow you down.