How can I create more realistic expectations about the trajectory of my job?

I've really only worked for two companies in my 15 years of work experience and I am presently on the hunt for a new job. In both cases, my job hunt has been initiated by the simple fact that I'm long past my breaking point in terms of being unhappy at work. The unhappiness comes as the result of not being challenged, not being given opportunities to advance, bad managers, and other routine afflictions of corporate working life. I think I should have started hunting for a new job much, much sooner but every time I considered doing that I would think to myself, "Oh, maybe it will get better." Or "This problem isn't that bad and it could go away in a month or two." And so I stick it out because there are some advantages to having a long tenure. But by the time I actually kick off my job search I am burned out, apathetic, frustrated, and unproductive. Surely there has to be a better way to pick up on those trends before it reaches this level of misery. How do I know when the problem lies just with my specific circumstances that I might change by moving to a new team, as opposed to a systemic problem within the company?