I vote for adding it to the core.
I try to use the least number of plugins possible to achieve what my clients need as the more plugins the more possibility of conflicts, failures, and bogging down the site.
And, as others have said here, some widgets have a lot of configurations that are time consuming to setup. For instance, "Special Recent Posts FREE Edition" which I use often. It's so frustrating to have to go through each instance of the widget and setup all the areas just to tweak one setting.