I’m assuming you’ve read this 100 times already but it’s worth bolding what Mozilla says installs this in the first place:
Let me note something as well: Windows Live Essentials shows up after any recent Microsoft Office install as well as an “optional” update after Office is installed. Skip it, never install it, and uninstall it if it’s there. If you really, really need Windows Live Essentials, then you should probably get used to that Bing bar.
To add to what Rei is saying, I think he’s right, you’re installing optional updates either by choice or inadvertently with your Windows Update settings.