Here's my theory on Michael's actions: He killed Ana and Libby, shot himself, and is going to frame Henry for it. That way it'll make Henry look even worse. It will get all of the Losties to go out to the Others' camp in a fit of rage because Henry killed more of their people.
I think he had just planned on killing AL. I don't believe he had it planned for a long time, it was probably spur-of-the-moment. But I think killing Libby was an accident. He shot at her out of fear that someone was seeing him do the dirty deed when he was attempting to frame HH for it.
That's just what I think...

__________________
"To be a genius is to be misunderstood..." -Ralph Waldo Emerson