He has known Harbaugh for a long time and I don't think what Harbaugh is doing is any different than Harbaugh has always done; pretty sure the main driver for leaving was that he wanted to be a head coach and was finally given the opportunity:
A few days before Michigan won the Big Ten title game, Poggi got serious consideration from UConn to be its head coach. Jim Mora got the job. Poggi would be lying if he said he wouldn’t love a crack at being a college head coach; he loves projects, and there are certainly some programs that need fixing. One of his proteges, Macdonald, said Poggi’s age and experience might elicit skepticism, but “for some creative athletic director who is willing to think outside the box, it’d be a **** of an investment.”
In a high school coach who built a national powerhouse from scratch, the Wolverines found the man who helped right the ship in Ann Arbor.
theathletic.com