*Hosting*
When building software for our customers, we always use Heroku (http://heroku.com). We refuse to use EC2 etc in the beginning, since the name-of-the-game is to focus all time and money on the actual software. I really hate the hosting question since this doesn't matter in the beginning. For most start-ups today: heroku-free option is just fine.

*Deployment*
This is taken care of when using Heroku, 'cause all you need to do push your code to the repo and it's deployed. Time-saved.

*Chat*
Today, use Slack (http://slackhq.com). It has a ton of integrations.

*Error Monitoring*
Airbrake or something hosted.

Smartest CTO's use whatever get the job done that uses the least-friction for the team.