He "lost" Walt's trust and respect. Walt was very connected to the island and the people there. He would be crushed to know his father killed and betrayed his own people.