It's not that bad.
I'm only 180cm tall and I can only remember being hammered bad enough to end up going off the field twice (in the same game).
I played from 10yo to 18yo in Victoria against mostly Maori and Islander opposition and I certainly never struggled. Again, that's just another excuse.
If the clubs marketed themselves better, they would pick up a lot more kids. Sadly, most are happy to just exist with ex-pats playing for them, be it Kiwi's Islanders or Northerners rather than go out an actively promote the game.