I misunderstood because normally content does redirect after it's published.
But actually I just realised that's only when you click "Edit" in the front-end.
Unfortunately it only redirects to "returnUrl" from the querystring and as far as I can see there's no way to override the controller to go somewhere else (without implementing your own controller which seems a bit excessive).
It's worth considering that this might be good default behaviour (with an option to disable it). Anyone have any thoughts?