Good post -
You brought up some very true points. Web design definitely has a lot of things needs in it to be successful. What I usually do is try to find out as much as possible from our web clients as to what they are looking for, the point of what they are hoping to achieve with their website, and also if they have any website examples that they know of that they are hoping for something similar.
Unfortunately clients are clients, and not many of them know exactly what they want. Most of the clients that come to my company seek consulting and recommendations on what their website should and could have. That makes it a little easier on the designers as it might give us more freedom to do things as we would like, but also we have to be careful and make sure it doesn't fail.
Design is yes, something that you want to let yourself free and go with all your creative impulses and such and do something that is new and different and never done before, but we have to remember that it isn't always the case when it comes to who we are designing for. There are a number of designs and work that I have put out there that I am not proud of but the client loved it, it hurts but sometimes thats the way it goes.
I definitely agree with the redesign your site every 2 years thing though - I am actually in the middle of redesigning our company website and am hoping to give it a look that is fresh and keeping up with the times.
Again good post and it's good to remind people out there that there is so much that goes into Web Design.