Largely depends on what you mean by "framework." People generally mean one or both of two things: a class library, and a set of classes for defining where the code that handles a given request should be.
Over the years I've rolled my own "framework" by collecting code I ended up writing more than once in to a single place. It's in a namespace and contains a whole bunch of utility classes for accessing databases, memcached, handling cookies, dealing with authentication, a generic user class, and much more. It also has classes such as Router, Controller, Template, and View, which form the basis of a system for defining what code handles requests for any given URL. This well-defined structure, with a single entry point, makes it easy to quickly get up and running with a new project.
As you rightly say they're are lots of these things around, and each one has its own idiosyncrasies. Your options are to either pick one and run with it, or roll your own. Rolling your own will increase development time for your next few projects, but what you end up with will be to your requirements and not somebody else's. If that's worth the extra time I'd say it's well-worth doing, but I'd also recommend studying and learning from other systems because they've definitely seen and resolved a huge number of issues you'll come across when developing your own.
My advice for implementation would be:
Keep it logically and physically separate from your application code. Your "framework" should be able to be updated from a central repository without modifying your application.
Treat is as a third-party library. Define your interfaces, and stick to them. When you decide a change is needed in the framework, make it in the framework ensuring that existing interfaces (I think of them as contracts) are not broken. From your application's point of view the framework is a black box. If your application needs to know any of the inner workings of the framework then you've done something wrong and need to fix it.
Apply a threshold of common value to anything you consider adding to the framework. The best frameworks tend to be small and simple. If what you're about to add is specific to the project you're current working on, put it in the application code until you get another project that needs the same functionality. Obviously this doesn't apply to obviously shared stuff like cookie access, but littering your framework with code that's used in only one project with the reason that it "might be useful to something else one day" is equally as bad.
Or pick one and run with it. Then decide you don't like it, pick another for your next project and run with that. Repeat as necessary until you stop switching.
I never did, so that's why I built my own. I hate it with a passion, but it's mine and that means I can only blame myself when it doesn't get better over time.
