Brainstorming needs to be an exclusively inclusive process.  (Read it again if you don't get it).  

Let's be honest, not all ideas are good ideas and many people are not great at the brainstorming process.  I am blessed to have worked with some of the most creative people in my industry.  I have also worked with people that are not only uncreative, but have the ability to stop the creative process dead in its tracks.  Sadly, if you lack the ability to be creative, you may never gain that skill.  You either got it or you don't.  

When putting together a productive brainstorming session, you have to look at it much like picking an all-star kickball team.  The fat kid deserves to be picked last, and if you can help it, you do your best to keep him off the team.  When there is a lot on the line and you are responsible for creating game changing ideas; now is not the time to worry about people getting their feelings hurt because they were not invited into the group. The fat kid can probably find something else to do while the meeting is happening.   

Sometimes people are placed into roles that stifle them creatively.  There are barriers that occur, sometimes physical or sometimes mental, that prevent people from reaching their creative potential.  These situations often come to light because they are playing the wrong role on the team or they are dwarfed by others who are far more skilled. Even though the fat kid can kick or outrun the thrown ball, there are others that are just better. Not everyone is an all-star in their environment and that's ok.  

Years ago I worked with someone who was in a creative position, but was not a very creative person.  By being willing (or desperate) enough to take on the role, they were placed into this position.  Everyone around them, including the person who assigned them this role, was a reflection of this person's ability or lack of ability.  They were a group of like minded and uncreative people that looked to this person to lead their creative efforts.  The environment was dictated from the top down and the group moved slowly and within their comfort level.  

I was hired as part of a regime change and was given the task of putting together a new team.  When we started to assemble a more creative group of people, it was quickly recognized that this person was not right for the role. Unfortunately, I believe they realized this as well.  Instead of embracing a role that still allowed them to contribute in their own unique way, this person decided the best thing to do was to attempt to derail brainstorming sessions and bring things closer to their comfort level.  When the meetings started to head in a direction that they were uncomfortable with (contrary to their idea) they applied the brakes and brought everything and everyone to a grinding halt.  

George S Patton once said, "If everyone is thinking alike, then somebody isn't thinking".  I completely agree with this statement, but there is a difference between the meaning of that quote and someone who takes a meeting in the wrong direction. Most of us can identify the difference between the two.  After several frustrating brainstorming sessions, we began to stop inviting this person to the meetings.  I assure you this was not a case of a mob of like minded people bullying a fellow employee. The meetings needed to move forward and be productive.  By the definition of their title and the requirements of that role, there was a need to be creative and contribute.  This forced us to make a change that allowed the group to move smarter and faster.  It wasn't personal, but at the end of the day, there is a reason why the fat kid gets picked last. 

This isn't a heartless act.  You can certainly allow people to be a part of the meeting, learn from the creative process and grow from the ideas and teachings of others.  Yet, when they start to hinder creativity and prevent the team from moving forwarded, you have to clear the way and allow the ideas to breathe.  Those who are not contributing to the positive momentum of the session must be cut from the group.  As you know, you are only as good as your weakest link.  When that link prevents the group from achieving the goals of the meeting, you have to move on.  Even if you like the weakest link, you have to have the courage to do what it takes to be successful. 

To quote the "great" Bob Sugar...it's not "show friends", it's "show business".