Realistically, they’re SOL for bank accounts and jobs. Some jobs provide employee photo IDs, but for the most part if they aren’t willing to use parts of their government identity (Birth certificate/immigration papers, social security number or Visas) they won’t be hired. One job I had got my social security number wrong of file, so I can only assume they didn’t look closely at the card copy I sent, or maybe they just copied my chicken scratch from W-4 without verifying at all… A lot of people in the United States, not sure about elsewhere, don’t have bank accounts. There are check cashing places if by miracle they can get around the requirements of identification from the I-9 form, they take a decently sized percentage from your check but that’s the price of not using banks or credit unions. Working under the table is also an option, as can nontraditional job options from online, like Etsy, entertainment with funding via whatever service won’t require an identity, assuming you hide the income well enough.
I don’t know how hard people will be looking for your character, but I think that going down the courthouse to have a name changed, moving far away, and being unlisted in the phone book/no phone number attached to a name would be enough. Sites like Spokeo have options to remove information from their site. Spokeo | People Search | Opt Out It depends on who is looking for them. If high priority criminals like Whitey Bulger can stay hidden for decades, so can they. If they’re being stalked it could be the same way, but I’ve heard some crazy ex stories that makes me think the government should hire some of them as investigators.
Using no aspects of the original identity, not being a witness needing protection, and not stealing an identity (which is a horrible thing to do to anyone, so only do this if your protagonist is an asshole with no regard for others whatsoever) they will be living similarly to undocumented immigrants. Could even get a change of pace by legitimately travelling abroad, “losing” all the paperwork, and being a real undocumented immigrant. Or just be homeless, a lot of people do not care about homeless people, and even those who do only require identifying information if they apply for public assistance.
Whatever they decide to do, life will be much different, and it will lack the stability that being linked to a unique identity has. So hopefully they can make that 25k last, but on the other hand having to struggle will make for an engrossing plot. Maybe you’ll win a Pulitzer. 
Found this on changing SSNs. https://faq.ssa.gov/link/portal/34011/34019/Article/3789/Can-I-change-my-Social-Security-number Probably not super helpful for the protagonist, but as an author it can help you write some frustrating exposition in chapter 1. I saw an opportunity to overuse SWIM style and I took it shamelessly. Hope this was helpful.