When determining your niche the first question one has to ask themselves is what am I passionate about? Your niche is a focused and targeted area that your organization serves particularly well so you'll be spending all your time and money learning about and working throughout that industry. If you're not passionate about an industry it'll be a lot easier for you to quit or put things off so ensuring that you're passionate helps to keep you motivated to run your business.
Once you know what you're passionate about ask yourself what problems can your interest could solve? For example, if you're passionate about the beauty industry maybe you could develop a new shampoo that promotes rapid hair growth with a special formula made just for people suffering from alopecia. Your niche here would be the shampoo product that serves a whole community of people with alopecia, ultimately creating a results yielding solution for this group of individuals.
Next, check out your competition. Things to look out for is the strength of their brand identities, their marketing content and their transparency with patrons. If the competition is weak in any of these areas there's a good chance that you can jump into this market and make a pretty good return on investment. You always want to be as transparent with patrons as possible where as being more corporate can cause trust issues for a company.
After you're sure there's space for you in this industry check to be sure that the actual product or service you're offering is profitable. You can use websites like clickbank.com to search through the products in your niche's category. When doing so you're looking to see if there are similar products to yours. If there are that means there's already a steady market for your niche and it's time to bring your product to the table. If you do not see many products in your niche's category this isn't a good sign and could very well mean that the product you're offering hasn't yet been successfully monetized in that industry. You always want to be sure any investment is worthwhile based on the return on said investment.
Once you've explored this checklist go ahead and test the waters. Put your product out there by creating a website and linking to it through paid advertising. Paid ads will bring in customers and spread awareness of your brand or product/service. Being consistent with ads is the key to sustaining an online business so be sure to have enough room in your budget for marketing your organization.

Good Luck!