I don't remember where I heard this quote, but it went something like,
Teaching is 10% preparation and 90% performance
I live by that quote. I believe that preparation is key to anything you do involving talking or teaching other people. And while I believe the way you teach and talk to your students is the most important, I also believe that if that first 10 percent of preparation isn't done the other 90% doesn't matter. If you have ever gone into a presentation or classroom and you could tell that the teacher wasn't prepared or the presentation/lesson was all over the place, the teacher/person loses a little bit of credibility. Students will pick up on it really quick. No matter how good an actor is, the movie will suck if the script he is following is terrible.
I always prepare what I will speak or teach, whether that preparation is in a simple note form or if it is a structured to the minute plan. And during my talks or classes it almost always changes. Depending on what your students/audience find interesting or boring, you should be able to adjust your plan accordingly. I always plan more than I think I will use, because I hate having extra time at the end of class.
Planning in general normally helps in the flow of your class, and makes things go smoother. Also, planning helps you spot potential problems that might arise before they come up in the classroom. Those other teachers you have heard from might not plan, and maybe their lessons are great, but I think for the most of us planning is in invaluable resource.