To start a business, all you need to do is hang up a shingle. You can operate by yourself as a sole proprietor, with others as a partnership, or in a formal entity such as a corporation or Limited Liability Company (LLC). As a sole proprietor, there are few legal formalities - if operating under an assumed name, you may need to register (usually at the county level.)
Partnerships are also easy to create. You need to draft a partnership agreement which is the agreement (basically your own private law) which governs the relationship between the parties - who gets what percentage, who makes the decisions, etc. Again, there are few legal formalities. General partnerships usually do not need to register anywhere except maybe at the county level. Limited partnerships are more like corporations and usually require registration at the state level and the services of a registered agent. Click here for more information about the role of a registered agent.
OK, so corporations and LLCs require more work and are a little more expensive - why should I bother?
Personal liability! When a corporation enters into a transaction, it is the corporation and not the shareholders who is responsible. When starting up, a bank may require a personal guaranty since your company doesn't yet have a credit history but, as a rule, a shareholder's liability is limited to the amount invested into the company. Creditors cannot reach beyond the assets of the company in normal circumstances. The same is true with lawsuits. It is the corporation not you who will be sued (assuming the lack of unusual circumstances such as fraudulent undercapitalization.)
Tax Considerations! It's true that the potential exists for double taxation (see What is "pass-through taxation?) However, you can avoid this with the use of a LLC or by electing to be treated as an S Corporation. With the corporate form of ownership, you can usually elect to implement various tax-free benefits such as life and health insurance and retirement plans.
Transferring ownership and raising capital are usually easier through the use of stock. Corporations usually have a perpetual life as well, distinct from that of the shareholders.