The Long and the Short and the Tall is kinda relevant.
It’s set in a war - Malaya in WW2 - and all characters are soldiers.
There is no direct combat - it’s just a bunch of bored soldiers sitting around in a jungle hut - so in that sense it could take place anywhere.
The themes discussed are related to war - so war is integral to the story - but ultimately it goes much deeper into the human condition and (IMO) goes beyond what we’d usually understand as a “war movie”.
Does the final series of Blackadder count?
Mister Roberts, a service comedy, is set on a Navy ship during WWII that never sees any battle action. Part of the plot involves Roberts’s efforts to get transferred into a battle area.
To finish the trifecta, * Dr. Zhivago*
Pan’s Labyrinth
maybe?
Belle Epoque and Lengua de las mariposas (I think the English title might have just been “Butterfly”) are set during the Civil War in Spain, but I think that Pan’s Labyrinth is actually set just after the conclusion of the live war, but during the aftermath when they are hunting down the remaining resistance.
There are several good foreign films that fit this category. Louis Malle’s Au revoir, les enfants is a fantastic example of a film set during the second World War. Schindler’s List and Life is Beautiful would be examples of the subgenre of Holocaust films. The Diary of Anne Frank would be another obvious example.
Since it popped into my head while reading this, I have to throw in “A League of Their Own.”
It’s a baseball movie, yet the story line can only exist because of WW2.
Would you count a movie like Bridge on the River Kwai or other prison-camp movies?
I’ve seen both of those and the Oscar for best picture that year went to The Last Emperor.
OH and A Very Long Engagement is a great war movie, but from the point of view of the women and the aftermath of WW I.
Paths of Glory
For a funnier movie try Father Goose, which is one of favorite films and not just because it has Leslie Caron or another great Cary Grant film Operation Petticoat.
Tigerland is one of the best movies I’ve seen about the Vietnam war, despite the fact that it never leaves American soil.
Please disregard the fact that Colin Farrel and Joel Schumacher were involved with the project. It’s a good movie, honest! 
I still think Kelly’s Heroes fits the OP’s revised definition. It’s also one of my favorite movies period so I wanted to bring it up again.
Oh shit, that reminds me of The Boys of C Company which is some movie I saw on TV a long time ago about soldiers in Vietnam who form a soccer team. This is a perfect example of what I’m talking about.
Cadence with Charlie and Martin Sheen. All the characters are soldiers during Viet Nam but it takes place in a stockade in Germany.
Command Decision with Clark Gable is not exactly what the OP is asking for. It is a war movie set in WWII but there is not combat. It takes place in England and deals with a general sending aircrews off to die.
This is possibly my all-time favorite film, as my user name signals, but I don’t think it meets the OP’s definition. Most of the first half is about the attack on the Anthill, so it’s directly concerned with combat.
The Good, the Bad, and the Ugly uses the Civil War (oops, the American Civil War) as a backdrop.
Cold Mountain takes place during the American Civil War, and the male protagonist is sent off to fight for the South, but after, as I recall, one battle scene, the rest of the movie is about him trying to make his way home, and how his sweetheart gets along without him.
How about Hollywood Canteen and Stage Door Canteen?
On the lighter side, there’s Steven Spielberg’s 1941.
I guess I’ve gotta disagree with the general premise of the thread, because I’d say most of the films listed here are War Movies. I think it’s specious to say a war movie is only a war movie if it’s a Combat Movie (that’s like saying a romantic comedy is only a romantic comedy if it has heterosexuals as the lovebirds). Most of the films here deal with the themes, trappings, and repercussions of War. Sometime, the war can be peripheral to the action (Casablanca, GWTW), sometimes the themes are not war-exclusive (Apocalypse Now), and sometimes there is a comingling of genres (Three Kings, Buster Keaton’s The General), but to say that any of these are somehow not War movies is to assert a narrow-to-the-point-of-arbitrary definition. That is not to say that anything with a soldier or a tank is automatically a war movie, but just because it doesn’t have an epic frontline battleground doesn’t mean it isn’t, either.
The Thin Red Line is set on a battlefield, during a war, and the characters are soldiers; but what’s going on in the movie has little to do with war, storytelling, plausible human interaction, or indeed even any understanding of cause-and-effect or physics itself.
It’s not a war movie in any intelligible sense. If pressed, I’d guess it’s a shock piece about the effects of cocaine on Hollywood.
Sailboat