You might not notice it, every time you check something different other than what you are working on, it takes about 15 minutes for you to get focused to get back to the task at hand. If you get a notification for every email then you be wasting away hours trying to focus then getting distracted again.
I have turned off all email notifications since nothing is an emergency and can wait. I still have my email program running, and checking for emails, and I can reminders from my calendar (I know I should use a separate program for my calendar, I haven't found anything that works for me yet). I now only check emails when I want (or I'm waiting for a reply), if it's an emergency then there are other ways to talk.
Stop getting distracted by small things like email notifications and get back to your task at hand.