M.J.
I have been a freelance writer for nine years now. I highly recommend meeting with a CPA to get your business set up properly. You will need to pay quarterly taxes to the state and IRS, so you want to get that set up right to reduce your chances of being audited.
Also, you will need a business license. Where you get this from will vary depending on where you live. It may be your city offices, it may be the county offices. In my case, it was the city. Just go to the government websites and look for the licensing department for more information. The fees are usually pretty low, based on your anticipated earnings.
Do you have a business plan? If not, you'll want to get that in place before you get started. Your local library should have some great resources to help with the process if you need it. You can also find information online, although be sure you're using reputable sources so you do this properly.
Good luck!