I was also against the hire at first, but in year 3 Riley has really grown on me and I not only like what he is doing, but I like where he is taking the program.
If you think Riley is a .500 guy and don't think he is much more then that, then just take a look at what is happening at Oregon State under Gary Anderson and without Riley. I think Anderson is a good coach (see Utah State and Wisconsin) and he is flopping in Corvallis. I think this shows just how amazing Riley actually was at Oregon State.
But like others have said, if he doesn't win, he doesn't stay much longer. It is the nature of the job and if that is the case I think when it would be time, Scott Frost will be very well ready as he is doing good things at UCF. Also Riley will have stacked up the roster and left the place better then when he found it.