That was Obi-Wan Kenobi.
Firstly, fans get confused by the Sith 'rule of two'. This does not necessarily mean that there are only two Sith in the galaxy - indeed we know for a fact that this can't be true. Palpatine must have been apprenticing Dooku at the same time as Maul for the timelines to add up, and there are several Sith in the same time period in the Clone Wars cartoon series. Palpatine himself may believe in the 'rule of two' but this does not mean there can only be two Sith at any one time. If this was the case it would make extinguishing the Sith very easy!
Palpatine is also considered the chief Sith because of the scale of his evil plans, and his overt strikes against the Jedi. But Palpatine's plans are concerned almost entirely with material gain, not spiritual. Simply because he is the Sith Lord most out in the open doesn't mean he was the master villain that the Jedi were looking for.
Obi-Wan Kenobi does not act as a Jedi should. For a start, his mentor was the mysterious Qui-Gon Jinn, a Jedi who displayed many unjedi-like traits, constantly employing lies and deceit to get his way. He attempts to use Jedi trickery to steal from Watto, actively pursues a war that he admits the Jedi have no place in fighting and constantly defies the Jedi council. Don't forget that Jinn himself was trained by the evil Darth Tyranus. Could Jinn have similarly mentored Obi-Wan in the ways of the Sith?
Jedi's should not be emotional, yet Obi-Wan is quick to anger. He is furious during his battle with Maul, and when guarding Amidala in 'Attack of the Clones' he leaps out of a window after the droid flying outside, caught up in a whirlwind of emotion. He brutally hacks off the arm of a bar patron in Tatooine. During his fight with Maul, Maul has the high ground and yet Obi-Wan is able to still beat him, despite Obi-Wan repeating time after time that if a Jedi has the low ground he cannot win a fight. Could Obi-Wan have been channelling his Sith powers to beat a rival?
After all, Obi-Wan is in no hurry to save Qui-Gon, gently jogging through the Theed power room instead of using his incredible Jedi running abilities that were shown at the start of the film. Perhaps Kenobi was simply using Maul to kill his Sith master in order to become more powerful himself.
Anakin Skywalker was Obi-Wan's apprentice, and he quickly fell to the powers of evil. Whilst Palpatine is generally blamed for this, we know that Anakin's descent happened much sooner. Under the tutelage of Obi-Wan, Anakin murdered Sand People women and children, as well as the helpless captive Count Dooku. It is impossible that someone as powerfully force sensitive as Obi-Wan would not notice his apprentice had turned to evil, especially when Anakin began wearing black and talking about his woes.
Obi-Wan even admits his Sith nature during his final fight with Anakin. He tells Anakin that "Only the Sith deal in absolutes", a statement that itself is an absolute. This is very similar to the way in which Palpatine reveals himself to Anakin, a clear Sith trait.
Obi-Wan's murder of Anakin is cruel, as befits a Sith. After hacking off Anakin's arms and legs, he leaves his young apprentice in tortured agony, burning alive. A true Jedi would have put him out of his misery, not simply watched with pleasure before kidnapping his newly born babies.
Tellingly, when Obi-Wan reminisces to Luke about Anakin using lies (another Sith tactic) he speaks of Anakin as "a good friend" as he leans back happily, eyes closed in memory. How can Obi-Wan possibly think of the child-murdering Anakin as "a good friend" unless Obi-Wan himself took pleasure in the murder of the Jedi children. He has no reason to lie to Luke at this point of time, so we can take his happiness at the memories as genuine.
The Sith Lord that the Jedi Council had been searching for had been clouding the minds of the Jedi for many years. Palpatine was often in his office or on important political work, whereas Obi-Wan was with the Jedi at all times, giving him far more opportunity for this evil deed.
What was Obi-Wan's plan then? We know he was not directly in league with Palpatine (and of course it would be racist to suggest that all Sith were somehow in cahoots in a 'grand conspiracy') but Palpatine's plans lined up with Obi-Wan's more grander ambitions.
Obi-Wan was a master of chaos, and so let the war take him where it would. As the destruction of the Jedi is constantly on his mind, he seizes the opportunity to help Palpatine win, convincing Yoda to go into exile rather than fight Palpatine again. He was happy to let Palpatine's Empire carry on until he was ready to put the final phase of his plan into operation and destroy it.
Remember when Palpatine talks of an ancient Sith Lord. "To cheat death is a power only one has achieved" he says. But who is the first character we see to cheat death?
It is Obi-Wan Kenobi. Could he be the evil Darth Plagueis? Remember, the concept of a 'Force ghost' is something unknown to the Jedi. Yoda only learns the technique after speaking to the 'ghost' of Qui-Gon Jinn, but we never see this ghost. Could this 'ghost' in fact be Obi-Wan using the same mind-clouding powers to trick Yoda? After all, he obviously showed Anakin the same technique, indicating he was the originator.
At the end of the films, Obi-Wan has captured the spirits of Yoda and Anakin, tricking them into relinquishing their powerful spirits into an afterlife of his own creation. He has caused the destruction of both the Jedi order and his Sith rivals. He has installed a faithful apprentice as Luke, outwardly a Jedi but inwardly a Sith. Luke dresses in black, has cyborg parts and uses dark powers such as mind control and furious emotional fighting.
As the dust from the Star Wars films settles, we see that Obi-Wan played the long game. His patience was rewarded in his ultimate victory - not one of mere possessions as the lesser Sith Palpatine achieved, but a deeper victory over the spiritual world and nature of the Force itself.