There are some businesses where all you need is a bill-board style website. It’s just a place for you to put up a sign that says “Hey! I’m a business. I have real-world brick-and-mortar shop that you should visit. Here’s my contact information.”
If you want any sort of customer interactivity or online ordering, you’re going to need a lot more. There are various free and cheap solutions, but ultimately you’re going to get what you pay for. You could spend a lot of time online researching a web host, and maybe finally make a choice. More than likely you’ll find tech support limited or you don’t have enough space, or they don’t run the correct scripting language that you need, or they run a Windows server but you need a Linux server.
Why not save some time, head over to a comparison site like http://www.webhostingchoice.com, see what’s available that matches your needs and desires for your online presence the best, and compare pricing? That way, you’re getting the most bang for your buck and not wasting your time. You’ve got better things to do, like starting your business and figuring out how to make this site turn you a profit.