I suppose the answer varies depending on the growth stage of the startup.
I was just speaking with a colleague about this as we are both starting new startups and were reflecting on our previous projects together. In hindsight we recognize that one of the main challenges was proper expectation setting around adoption and engagement. If someone else is setting those expectations for you, it can be challenging to live up to them unless that person is familiar tech startup analytics. Not every business is a unicorn. You have to learn how to survive and grow when things don't go viral. Once you set expectations accordingly you're able to focus on what really matters and hone in on the user experience to identify what is sticky. You may have a long roadmap of features on your wish list, but it may not be what people want or need. Be open to pivoting and listen to your users.
One of my favorite quotes is actually by LinkedIn Founder Reid Hoffman, "If you are not embarrassed by the first version of your product, you’ve launched too late." Being in tech B2B and product development for almost 9 years now, I completely agree with this statement. There is so much you can learn from getting your product out into the world and in people's hands; invaluable lessons that you can't learn coding in a silo.
Here's a great book for user engagement analytics: Lean Analytics. Also, happy to share my experiences on this topic.
In my experience the hardest thing facing a CTO is the very nature of the ever-changing world of web applications. In other words, stuff changes real fast!
We started prototyping an MVP back in November 2015, and now since we've been looking to pivot, we've found that we're approaching our stack very differently. Frameworks have been updated, some drastically, some open source software we used is now deprecated and non-supported, other software we built our app upon has shown to not be scalable enough, and so on.
One has to just make a decision to commit to their stack and work with it, or (if absolutely necessary) make sure that the application you are building is built in such a way that you can still take it apart and put it back together again on a different stack – which we did actually.
For me personally, I like to make sure that the application is well documented from the beginning and designed first on paper (or online sketches), with a database structure clearly defined, routes documented, and the flow of the user experienced explained in human-readable text. If that is done well, then changes to the stack will have a lesser impact on the path to your MVP.
Also varies depending on product, which might be a...
1) Front facing Website
2) Membership Website
3) API
Each have their own unique characteristics.
In general though, all systems bottleneck around disk i/o, so if you have a database heavy application, ensure you have someone who understands how to design tables for extreme speed.
And if you still require more speed, run a memory resident database, which must...
1) Sync from disk to memory when database server starts.
2) Periodically sync from memory to disk during runtime.
3) Do final sync, when ever database server stops, to bring disk backing store consistent.
Memory resident databases are a bit tricky + produce blazing fast throughput.
Trying to build technology around an ever-shifting business model is super tricky.
Finding reliable help is also a common challenge.
