In addition to radiation, they mentioned the strong magnetism and we all know the island has/had the magnetic properties (right before the hatch imploded we saw everything with metal properties flying through the air). Desmond spent a lot of time down there as well as was blown clean naked after the implosion so perhaps that's what kicked his time travel into high gear.