Can I legally post junk mail back to Royal Mail by simply putting it in a postbox?
I can't give legal advice but, as far as I'm aware, it's perfectly legal to put junk mail in a postbox.
That said, it would be pointless to simply put addressed junk mail in a postbox – it would go through the system and end up in your letterbox again. In a way this would be an interesting experiment (how many times will Royal Mail deliver the item?) but it would be more effective to cross out your name and address and write 'Return to sender' on the envelope.
I'm also not too sure how much sense it makes to deposit unaddressed mail in a postbox. It might make some sense to dump leaflets distributed by Royal Mail in the post box if you got a 'No Junk Mail' sign on your door. The fact that Royal Mail ignores such signs and instead runs an obscure and ineffective opt-out scheme is their problem, not yours. However, it would be more effective to return such items to the senders in an unstamped envelope – if anything, it makes the senders aware that their junk mail didn't have the intended effect.
Finally, many people will consider putting any junk mail in a postbox littering. That's certainly a fair point, but also raises an interesting question: why should putting unsolicited junk mail in a postbox be considered "littering" and putting the same material in a letterbox be considered "advertising"? It seems to me that it's simply a cultural convention that will only stand for as long as people are willing to accept that pushing unsolicited adverts through doors is normal.