@The Dude Perlman injected himself into the conversation when he publicly reprimanded Bo. Osborne as the AD would normally handle any discipline of a head coach, but Perlman went over Osborne's head in that case, and it probably wasn't the only time he did so. Perlman also shut out Osborne from having any input in who his successor as AD should be (even Bo has said that he felt that Osborne had earned that right).
Perlman meddled in the athletic department and football program, and therefore he's perfectly fair game for criticism in those areas.