Hello,
a 95% significance threshold simply means that you have a 5% chance that the results you're seeing are wrong given the sample you're analyzing. In other words it means that 95% of the time the results you see is better (or worst) then the one you see on the originale page you're testing.

If you want to run a test that really has an impact you should:
1. Compute the sample size (here a tool: http://www.evanmiller.org/ab-testing/sample-size.html )
2. Run the test for at least 7 days
3. Record at least 300 hundred conversion per variation
4. Check significance here ( http://www.evanmiller.org/ab-testing/chi-squared.html )

The more you want to be sure about the result the higher the threshold. So 95% is an accepted standard, 99.5% of significance means that you are pretty sure that the result is not due to chance.

I run hundreds of test for my customers, what I can tell you is that the best test are the one that provide a bigger difference between the two tested pages (from 20% increase or more). That you should run the test for at least one week. And that you should be sure that you didn't make the sample "dirty" (like: changing traffic sources, sending a special offer newsletter etc), and that you should run your test for at least 300 conversions for each variation.

If you happen to run a good test, what you will see from the chart in your ab testing tool is that the line representing the winning variation will always be "over" the one of the loosing variation.

This is the signal that you run a very good test!

And BTW .. Don't forget that each test you run teach you something about your customers ..

Hope this helped.

Luca