I think the truth is probably somewhere in the middle? I agree, prime time will likely rate better but I don't think casual viewers make that much of a difference.
For Nine, I think they like the fact they can start the day with a Footy Show earlier in the day, roll right though prime time with the footy and then basically straight into late night who gives a f**k programming. An arvo grand final puts the early games on before most people are settling in for the GF day, and leaves them with prime time to fill too.