How to challenge high level characters, piloted by players I've been playing with for a decade who know me and my tactics intimately? By level 20, they were handily taking down monsters with 500+ hitpoints and extravagant amounts of armor class and damage potential. In the Temple of Demogorgon, I put in a hechatonchieres which was meant to be too scary for them to fight. They focused fire and took it down without a casualty.
At the same time I've been playing a lot of Slay the Spire this year. The game has a lot of great monster mechanics that play with your powers in really interesting ways, and makes even normal combats scary enough to kill you under the right circumstances. So many good powers which really fuck with your tactics and interact in so many interesting ways with your cards and powers, allowing an infinity of tactics, and an infinity of ways to fuck up. So I made a list of similar powers of my own. Most of these did not allow saving throws or other soft mitigation, and instead became hard boundaries that players were forced to navigate.
1. Gain Strength each round -- damage increases linearly at the start of each monster turn. Even a weak monster can quickly become a serious threat.
2. Gain an extra attack each round -- very similar to the above, though it interacts differently with player abilities.
3. Gain an extra attack for each action taken by an enemy in its vicinity on the preceding round -- players who shrug and hope for the best quickly pay the price
4. Deal additional damage as the target's hitpoints decrease
5. When it deals damage, it is attached to you and moves with you. If you detach from the monster by any means, take additional damage
6. When an attacker deals damage to the monster, the attacker takes retribution damage
7. For every two points of damage dealt the target, each ally takes 1 psychic damage -- a really rough deterrent that interacts in interesting ways with persistent hazards
8. When the monster deals damage, an equal amount is dealt to the character with the lowest hitpoints -- a very rude power
9. Every time a magic item is used near the creature, it gains hitpoints
10. When the monster deals damage, the target loses a spell slot of a random level
11. When the monster deals damage, it dispels a magical effect on the target
12. Every time a spell is cast near the creature, another spell slot of a lower level must be expended
13. Whenever an attacker misses the target by a certain amount, it can counter-attack for free
14. When an attacker kills a monster, that attacker's next source of damage is reduced to 0
15. At half health, the monster splits into two monsters of equal health
16. The monster ignores the first instance of damage dealt to it each round
17. AOE effect causes targets to deal half damage and move at half speed
18. Immune to all magic and magical weaponry
19. Turn incorporeal and teleport long distances for free
20. High regeneration
Some examples of stat-blocks for monsters they've killed:
GUARDIAN APE
Giant white ape 30 feet tall, stained with blood down his mouth and chest, with bones littering the ground around him. He's sleeping peacefully and makes perception checks at disadvantage until woken.
HP 600, AC 24, Move 90, and an additional 90 as a bonus action. He can jump, bound, and leap the full 90 feet.
Magic absorption: Any spell cast in the ape's presence causes the caster to lose another spell-slot of a lower level of the target's choice.
Berserk: The ape attacks with advantage, and all attacks against it are made with advantage. At the end of his turn he gains a +4 bonus to damage with all attacks. This stacks and lasts for ten minutes.
When reduced to 300 HP, divine light rains from the sky. First story ceilings collapse, buildings are ruined. All magic effects on the ape are dispelled. All enemies have one magic effect removed from them. Also, all enemies must make a Con save DC 24 or take 6d6 radiant damage. Furthermore, the guardian ape is surrounded with divine light, which persists until the end of the player's NEXT turn. For every 2 points of damage dealt it, every ally of that character within 300' takes 1 psychic damage, even if they are out of line of effect. After this effect ceases, the rain will repeat at the beginning of every other turn.
ATTACKS:
3 punches +16 for 7d8+16 damage. This can punch through walls and buildings.
Grab, bite, and throw +16 for 21d8+54 damage and get tossed up to 100' in a direction of the ape's choosing (off a cliff)
2 CRIMSON SCORPIONS
They are the size of mack trucks and sing an anarchic, disharmonious, wordless duet.
HP 300, AC 26, Move 40
Each time a scorpion is dealt damage, its attacker takes 2d8 psychic damage
It makes Strength based checks at +20
Attacks:
Claw/Claw/Tail +16 : 4d12+10 and grab/4d12+10 and grab/2d8+10 and Con DC 23 or 10d10 poison damage and forget 1d4 prepared spells, chosen at random
Spray bubbling green poison in a 90' cone (Recharge 5/6): Dex DC 23 or 14d8 poison damage and have all magical effects on you dispelled, on success take half damage and no dispel
KATANA-WIELDING MONKEYS
There are 3d20 of them.
HP 45, AC 17, Move 60
2 attacks +12 for 1d8+6 damage
They may use an action to pray for strength, gaining a stacking +4 damage bonus that lasts for ten minutes.
SER YHARNAM THE RED
A knight in armor made of tempered cardinal feathers, wielding a greatsword made of bluejay and blackbird wings, and wielding a greatshield emblazoned with a mouse and seven stars. His longbow has a bright red bowstring.
HP 300 AC 26, or 30 when using his shield
Undead and no longer sentient, but implacable and fights intelligently
All saves vs magic are made with +18
Every time a magic item is used within 300' of him, he heals 100 hitpoints (no maximum)
If an attacker misses him by 5 or more, he may immediately counter-attack. He can do this any number of times per round, but he can't counter melee attacks with his bow, or long-ranged attacks with his sword.
Attacks 4 times per round with greatsword or longbow: +15 2d8+10
When he hits a target with his longbow, they must make an Int DC 23 saving throw or lose a random spell slot
FLAGELLANT MONKS
Dead monks hanging from the walls and ceilings, they carry whips with broken stone shards tied to them, their robes hang in tatters, the skin on their backs is hanging off them in bloody strips and the congealed blood which runs out their backs is sticking to the walls and ceilings like a slime and sending down tendrils to puppet their heads and hands like sick marionettes.
5 of them
HP 120 AC 21
Move 30, can walk on walls and ceilings
Undead
When reduced to 60 or fewer hitpoints, they fall from the blood slime that holds them, and are no longer able to walk on walls and ceilings. The blood slime becomes an independent monster with as many hitpoints as the flagellant monk was reduced to. i.e. If an attack reduces a monk to 61 or above to 60 hitpoints, a 60 hp blood slime spawns. If an attack reduces a monk from 61 or above to 1 hitpoint, a 1 hp blood slime spawns.
Flagellant whip +16, 3d8+10 and take +5 damage from subsequent attacks from flagellant whips
or spit congealed blood up to 60': All in a 15' radius must make a Dex save DC 24 or be slowed as the spell, and may continue to save at the end of each turn. Monks and blood slimes are immune to this.
BLOOD SLIME
HP = whatever hitpoints the flagellant monk was reduced to. AC 21
Move 30, can walk on walls and ceilings
When a blood slime is killed, it explodes congealed blood every, slowing targets in a 15' radius just like the blood spit above. Monks and blood slimes are immune to this.
Bludgeon +10, 8d8+15 damage