I'd say talent differential matters, and I'd argue that even though Toledo consistently finishes with the #1 class, the degree of difference in talent level between Toledo at #1 and some MAC team at #6 in the recruiting rankings is not that different. There'll be a bigger talent differential here.
I just looked at 2021 using 247 numbers:
Toledo finished with the #1 ranked recruiting class. 82.5 player average. 163.18 total points.
Ball State finished with the #6 ranked recruiting class. 81.00 player average 140.84 total points.
Miami finished #2 in the ACC. 90.51 player average. 266.08 total points.
Virginia finished #6 in the ACC. 86.03 player average. 206.33 total points.
Even though he was hauling in the "best" recruiting classes at Toledo, they were pretety indistinguishable from the median recruiting class in the conference. The fall off is much bigger in the ACC. We outrecruit the **** out of the median recruiting class in the conference. We have a much more talented roster than our coastal competitors whereas Candle had a slightly more talented roster than most of his MAC competitors.