I don't think people see a current need to set aside athletic scholarships based on race. Athletic programs have the leeway to take students based on skill. This is different than say something like an academic scholarship where white caucasians receive a disproportionately high number of scholarships in comparison to minorities. I don't think minorities feel they're being unfairly treated in regards to athletic opportunities, but I may be mistaken.
As for your second part, "judging people by the content of their character" is a worthy albeit naive viewpoint. If we judged people that way, and only that way, the world be a much better place. The problem is we don't and that's why things like affirmative action were created. We still have a lot of cultural and socio economic issues in the country that have not been corrected. Hell, some states had Jim Crow era laws on the books as recently as 2013, and some still might.
The only way you can judge people by the content of their character is when all other variables are equal.
My point about awarding athletic scholarships by race was not a serious one, but rather intended to demonstrate how ridiculous awarding athletic scholarships by gender is. Even if you want to make the argument that is was necessary 50 years ago, it is far more difficult to argue that it is necessary now. The whole logic behind affirmative action programs is to provide a temporary remedy for a historical injustice. By definition then, those programs need to eventually be phased out and end.