If you want full control for the checkout, Magento is probably going to be the best option. You can also do it with woocommerce. The woo checkout has gotten better and you can tweek it to improve conversions, but you will get more flexibility (if you have a developer) with Magento.

Saying which system has the best conversion rates between those two, BigCommerce and Shopify is hard to say because there are too many factors. To get the best conversions in your cart, the tips would be to ensure you QA the full experience thoroughly on every browser and every device and then iterate consistently to find ways to push more people to complete the sale. You can also use Qualaroo to find out why people do not convert.