To Host or Not to Host

To host ir not to host is the core decision that shapes your website. When should you host your website and why?