EndermanGrief is a server-sided mod (that should also work on Singleplayer) that allows you to tweak various (annoying) aspects of Endermen via a simple config file which can be found in \config\endermangrief.conf

Currently, EndermanGrief allows you to:

• Disable Endermen block placing & picking up
• Disable Endermen from randomly teleporting
• (since 0.3) Disable Enderman's AI
• Stop Endermen from being hurt while in lava & fire
• Stop Endermen from being hurt while being in water
• (since 0.3) Stop Endermen from being hurt from freezing
• (since 0.3) Stop Endermen from being hurt from explosions
• (since 0.3) Stop Endermen from making obnoxious noises
• Stop Endermen from despawning
• Make Endermen be always angry at player
• (since 0.3) Make Endermen be invisible

It's a relatively simple mod I threw up together while still learning Java but I'll still try to fix issues that may arise, but generally anything that modifies Endermen as a entity might be incompatible with this. (Also it only modifies vanilla Endermen, not modded ones)