You mean in Hollow Bastion? That was a matter of double crossing and betrayal. At least, if I remember it correctly. Riku was possessed by Xehanort's Heartless, so it was actually Xehanort's Heartless who stabbed Maleficent.
It could also be to transform Maleficent into her dragon form to stop Sora, Donald and Goofy for good. I'm not sure though, sorry!
Riku was possesed by Ansem and he's a guy that really overloads with the whole power thing.
Simply, he did it so he could unleash her evil power and hatred, turning her into a viscous dragon, so Ansem could stop Sora, Donald and Goofy for good.
First off, Xehanort's Heartless was controling Riku. Second, I'd say the reason he used the keyblade on Malifacent was cause he needed a test subject for it. I mean if you think about it, any bad guy would use their own allies to get what they want.