For the OP, tipping even when they have a small delivery charge is always nice to tip, imho.
There is a similar question I’ve been wondering about:
At work, one of the nearby sandwich shop’s contracts a delivery service. The delivery service charges a $15 flat fee for the delivery.
I pay the sandwich shop by credit card - they pay the delivery service. I can’t sign the credit card statement with an extra tip for the delivery person (as I usually do) when they arrive as it’s already been finalized, at the restaurant itself - NOT the delivery service.
I’ve been wondering what to do about that. My instinct is to tip, but they’ve made it hard to do it the normal way and this is a work lunch, not a personal lunch so tipping in cash without a receipt will not get me reimbursed by the company.