First, you need to make sure you have the conversion funnel set up properly. Enter the urls for all of the checkout steps (cart, login/register, checkout page, etc) and then look at the conversion funnel. This will help you to see if there are any bottlenecks or major problems causing a huge portion of your customers to drop-off at any point in the checkout process. Ideally a checkout should be one page, the more clicks, the higher the drop-off.

Also look at all of your individual pages, see which ones have the highest abandonment and the highest conversion. That will help you see what you are doing right/wrong.

I'd also highly recommend doing extensive A/B testing. You'd be surprised how much difference the wording of a button or even the color can make on whether or not a user clicks on it. Same goes for navigation elements, position of items on a page, promotions, etc. You can never A/B test enough!