Reading this, I was trying to imagine some kind of "fight-or-flight" engagement system that could preserve the freedom of open-world PvP without the headache of completely unfair ganks.
I'm imagining the scene as if it were in a movie: you're traveling the landscape and you're set upon by a superior foe. There's a stand-off as you assess each other's abilities. You could try to fight, but you know you're better off running. He lifts his sword and charges, but you dodge and make a break for it instead...
This feels more like the dynamic we want when a max level enemy player wants to fight out in the wild. I can't think of any MMORPGs I've played that have a combat system that supports this. Most have some kind of engage move that you use on players and non-players alike and then you just duke it out and CC until one of you wins. Players sometimes have the ability to disengage and run, but you're never given that choice in a ganking situation. I know when I want to get the drop on people, I do it when they're low on heath and/or in the middle of fighting some NPC enemies.
So lets have some kind of Standoff mode that engages when enemy players come into range of each other. Cameras focus on the foe, your characters slowly approach each other, weapons are drawn. At this point, you choose to fight or to run. If both players choose to fight the battle continues as normal, if one chooses to run, then they use some sort of disengage that allows them to flee with reasonable certainty that they'll escape. This could be a faster run, a big dodge roll, a teleportation, something that gives them an escape advantage. the trade-off here is that fleeing players lose their ability to re-engage effectively until they've fully escaped. Fighting players have engaged too fully to give proper chase.
Now this eliminates the gank as we know it. Both players must choose to engage for the fight to continue and surprising someone isn't the same sort of experience. You can still make a weaker player cower in your presence, but you won't be able to beat them mercilessly unless they foolishly choose to fight. Some considerations:
Enemy NPCs will need to managed. If a player is fighting monsters when a standoff begins, those monsters will either need to disengage or be otherwise managed so the fight between players is given proper attention. I like the idea of them cowering or wandering off until the fight is finished, but the implications of that need to be considered (players gaming the system to handle monster fights, logical/thematic appropriateness, etc).
Stealth is a popular role, and such characters wouldn't approach a fight the same as a warrior or paladin. I think some concession could be made for an initial backstab that applies damage, poison, or some other debuff, after which the Standoff proceeds as normal. In this way the satisfaction of getting the drop on a player from stealth is maintained while still giving those players the choice to fight or flee.
There's obviously other concerns, but I think this could ease one of the major pain points for players on open-world PvP.
