Hi! Every state will be a bit different and I don't want to confuse with license numbers in case I have them wrong.

But here are some basic steps you will probably take or should take as you legitimize and grow your business:
1. Check with local jurisdiction for tax licensing. Depending on the 'final product' or service and the state you might not require a tax license. (you may or may not need an llc)
2. Get an EIN, usually need this before getting a Tax license and sometimes available during the same process. Free and acts as a social security for your business.
4. Share on your Facebook that your launching a food business. {Get marketing effort going that way...}
5. Get your food handling card. Usually a registration fee applies and you get a study booklet and can take it up to 3 times.
6. Share on social media that you just got your FH card so your all clean and approved to go! {...marketing again...}
7. Start looking into a nicely designed and mobile friendly website that makes you look professional. {I can help you with that at www.unthink.me or www.arizonawebstudio.com }
8. Apply for an inspection of your home kitchen. Some states require this, some don't and some do only for certain foods.
{Your city's office of business development should be able to guide you to all needed places, phone numbers of links to complete these steps.}
9. Familiarize yourself with the FDA regulations. I have provided some links below.
10. Open a bank account. Look for one that has no minimum balance required and has a ton of ATMs or at least give you a refund of the ATM fees you might pay for. {you never know}
11. Share on social media about where you are... get that anticipation going.
12. Look for possible local events where you can cheaply or free put up a tent and treats for people to try. {important for seasonal foodies}
13. Strategize a good marketing plan for your business as you don't want to waste time with the learning curve and miss the season with a ton of lost potential.
14. Continue being a rockstar.

Best of luck!
The FDA has a great list of things to consider:
https://www.sba.gov/blogs/starting-home-based-food-production-business-making-your-culinary-hobby-your-job
Familiarize yourself with this website and its contents:
http://www.fsis.usda.gov/wps/portal/fsis/topics/regulatory-compliance

Also look into this link:
http://www.fda.gov/Food/default.htm