This blog comes in light of some renewed Star Wars discussion due to Disney's acquisition of Lucasfilm. Before I get into the actual topic, I'd just like to put down a quick word on my opinion of this turn of events.
This is a good thing. George Lucas has a solid creative mind, but he's a world-builder. He's not a writer or a director. His talent is coming up with the scope and outline. He drew on his influences and ideas to create the greater Star Wars universe, and others filled in the smaller details. That's why the original trilogy turned out as good as it did. George Lucas' excessive micromanagement of the prequel trilogy made for poorer movies, despite the fact it featured some incredible visuals, concepts, planets, and people.
The Disney acquisition brings the opportunity to breathe some fresh life into the franchise, keeping George Lucas as a creative director (where he should be) but turning the reins for the writing, direction, and production over to someone else. Disney is a huge organization, so chances are pretty high that they can come up with a team that will make the next Star Wars movie a true blockbuster hit. Remember, Disney is more than just kid-oriented animation, goofy villains, and G-ratings. If you don't believe me, go watch Finding Nemo and The Lion King again and tell me that those movies are strictly for children. Or think back to the old (sadly closed) Extra-terror-estrial Alien Encounter attraction. Disney has capable creative minds.
Disclaimer: Expanded Universe is well and good, but ultimately leads to conflicting conclusions. EU material is, after all, based on the interpretations and opinions of various creators and authors. Here I will specifically discuss my own "head-canon" based on my observations and experiences with Star Wars canon. I don't follow, for example, the storyline wherein the Emperor is only "temporarily" dead because he cloned himself, as this inherently undermines the entire purpose of Anakin Skywalker defeating him in Return of the Jedi. What follows is my interpretation of the Star Wars series and its themes, specifically with regards to Balancing the Force.
"Remember, abilities are not inherently good or evil, it's how you use them." -Kyle Katarn
Darth Vader is well-known for his use of the Force to choke victims. Emperor Palpatine notably never raised a weapon against Luke Skywalker. Instead he demonstrated his power through his words, proving to be an effective manipulator, and his strength in the Force by conjuring lightning to finish Luke off. "Force choke" and "Force lightning" thus became the archetypal "Dark Side" powers. That lends to an incorrect notion of the Force and what "Light Side" and "Dark Side" really mean.
My basic understanding of the Force is that it's just the Force. It doesn't have a Light Side or a Dark Side. It just is. The few times that the phrase "Dark side of the Force" are even used in the movies are to describe not aspects of the Force, but aspects of a person. Yoda says "anger, fear, aggression; the dark side of the Force they are." These are emotions that can be channeled and focused to produce certain effects, such as Vader's grip and Palpatine's lightning. And according to Yoda, they come quickly and easily during trying times, tempting one to submit to them for an immediate gain.
With that in mind, Anakin Skywalker was the "chosen one" who would bring balance to the Force. The Jedi traditionally interpreted this to mean destroying the Sith. According to George Lucas, the original Star Wars trilogy centers around Anakin's children redeeming him and his fulfillment of the prophecy by the end of Return of the Jedi, confirming that the Jedi interpretation was more-or-less correct. Despite George Lucas' inconsistencies and shortcomings, I still believe he is the most credible source for general conceptual canon; therefore, I accept it as fact that Anakin Skywalker "brought balance to the Force" when he turned from the Dark side and killed Emperor Palpatine.
I reject the idea that the balancing of the Force took place during Revenge of the Sith. By the end of the movie, two Jedi and two Sith remained; Expanded Universe says more survived beyond the events of Episode III, but we can at least conclude that by the time of A New Hope, there were two of each remaining. Some fans consider this to be a balance, and it is in strict numerical terms. I like the thematic effect this promotes of a misread prophecy, because it creates dramatic tension and leads to character development for Obi-Wan and Yoda. But this is not the balance to which the prophecy referred. It's a twisted sort of balance, the kind that a man like Palpatine would embrace as right. It's the kind of thing that would cast doubt and instill fear in the last of the Jedi, which are two things that feed the Dark side.
I also reject the idea that "balancing the Force" means literally destroying the Sith. As I said above, the Jedi interpretation was more-or-less correct. It wasn't completely correct. The prequel trilogy gives a look at the Jedi Order and the Old Republic, and what those movies show is very important to the true meaning of balance. The Jedi had become corrupt during a time of lengthy peace, as had the Republic. The Jedi fell into the assumption that their ways were always right, their code was flawless, and their teachings promoted the best characters in pupils. On the other side of their identity coin were the Sith, who were the evil in the universe. The Jedi thought the Sith were extinct. When they discovered that Sith still existed, the Jedi immediately latched on to the idea that the prophecy would be fulfilled by their defeat.
Prophecies are tricky things. They almost always manifest in unexpected way that make perfect sense retrospectively. Aside from the generic end of bringing balance to the Force, there was absolutely no mechanism suggested by which this would occur other than by the hand of a particularly strong individual. And the Jedi failed to realize the true scope of the imbalance in the Force.
Remember, the Force is an energy field that connects all life. Jedi and Sith in particular are adept at tapping into it, but it nonetheless connects every living being. The Jedi had a narrow-minded view of balance that dealt specifically with Force-sensitive people only. They failed to regard the billions of beings in the galaxy who were not Force-sensitive and focused only on the struggle between Jedi and Sith. Unfortunately, most of those billions were just as unaware of the imbalance and the corruption that characterized the fall of the Old Republic. Bringing balance to the Force was a much bigger task than anyone realized.
The means by which Anakin Skywalker began the process of balancing the Force were unfortunate but necessary. He started by throwing things into a much bigger, more marked imbalance. If he had truly fulfilled the prophecy by wiping out the Jedi, we would expect to see harmony and peace. Instead we see the rise of a fascist, evidently human supremacist empire headed by a master manipulator with a disregard for the lives of those beneath him. The Jedi and the Republic were corrupt, but nowhere near as corrupt as the Empire and its Sith master. It was that level of corruption that couldn't be missed or ignored that caused the Rebellion to be born.
In the end, Anakin completed his prophetic task by turning from the Dark side (redeemed by his love for his son), defeated Palpatine, and brought an end to the Empire. In so doing, he defeated the man that embodied the Dark side, but also defeated the greater impacts and influences of the Dark side. What was important was not eliminating the persons of the Sith, as thought by the Jedi, but removing the corruption. He didn't win a duel with a Sith Lord, he won an ideological struggle with the Sith teachings. In the wake of the Battle of Endor, the Rebellion was able to found a New Republic and a New Jedi Order to replace the old, in a newly-balanced galaxy.
So ultimately, the imbalance wasn't the over-abundance of Jedi compared to Sith, it was the influence of the Dark side. Not just in the Sith, but in the Jedi and in all of the citizens of the Republic. That influence temporarily gained complete control, but when it was eventually expunged the galaxy went back into order.