Rack rigidity and load securing are absolutely critical to bike stability and predictable handling. Far more than where a load is carried, a stiff rack with zero flex and a completely secure load are the biggest contributors to stable and controlled handling when carrying cargo.
A number of people have complained to me about loaded bikes handling poorly. In almost every case, the culprit was a weak, flexible rack, or a loosely secured load, both of which allow the load to sway, taking control away from the rider. This problem is particularly acute when loads are carried on the front fork. I know quite a few people who don’t like to carry a load on the front of the bike. I’m guessing that in a majority of cases, the underlying issue is a sub-par rack or mounting method, and that a sufficiently stiff front rack with a high quality bag would change their minds.
On my Surly Long Haul Trucker (which I use for commuting and cargo hauling), I run a Tubus Cargo in the rear, and a Pass & Stow up front. Both are made from tubular steel and act as completely rigid extensions of the frame. I can comfortably carry 75-100 lbs. on that bike with no issues whatsoever. I attribute this excellent performance much more to the racks than to the bike itself.