Learning how to develop/improvise piano accompaniment

I’m a mediocre pianist - meaning my technique/skill on classical music, for example, is atrocious, but I can sight-read most popular songs and 32-bar standards with little difficulty and make them sound pretty good. I usually just play what’s written, but now I’ve run into a problem: I’m working up some songs with a vocalist friend and her bass-player husband, and realized I have no idea how to improvise/embellish the sheet music. I don’t want to double either the vocal line or the bass, much, which leaves me lamely playing filler chords, though I know I should be developing some interesting counterpoint or riffs.

But how?? Are there tips and tricks that can be related via message board? Is there a standard reference or instructional book I should check out? Does this only come with creativity and experience? Help!

If it helps, so far we’re working on standards from the 30’s through the early 60’s…think Gershwin, Porter, Louis Armstrong, etc.