No, one test wouldn't be fair. But it would be fair to judge him on his last 6 tests, say, where he has averaged 20. Or his last 24 matches where he has averaged 27.
It wouldn't be fair to judge him just on his second last game though. Where he got some cheap runs while the field was spread while the no 11 batsman was tonking at the other end. It would be fairly ridiculous to keep someone in the team on the basis of that one innings amongst all the other failures.
3 of his last 4 tests would be classed as successes by any reasonable person.
Firstly going back any further than his latest stint in the team is unfair because he has already been dropped for those results. His last 8 tests since his comeback should be the only ones he gets judged on.
In that we have 4 half centuries. In the middle of it we have an ugly bit which are his first two ever tests in India. He clearly and obviously improved as that series went on. In his final 2 tests he did very well getting two good scores and being amongst our best batsmen for both tests.
Since then he was our best top 6 bat in the 1st test and he had a bad 2nd test where almost every batsman failed.
Scoring 40+ in 3 of our last 4 tests isn't much to write home about but based on that alone half our top order deserves to go before him.