A word of warning about mace and dogs. I work for USPS and we get dog training, repeatedly, every year. We all have the option of carrying “dog spray”, basically dog mace. Some of us do, some don’t. We have been warned however, that a truly vicious dog, one that is attacking, especially those of a more aggressive breed, will be infuriated by the spray instead of repelled. Carriers get bit all the time and USPS has extensive records on dog bites, so I tend to believe them. (At least on this subject!) So, consider your options, and carry a big stick AND dog spray!
I carry a taser with me when I walk the dogs. I don’t know how much good it would do, but it’s probably better than nothing. I imagine I could pick the littlest dog up and kick the attacker in the face. The middle dog (actually a large breed, but smaller than the Big Dog) would probably run away crying if he were attacked; the Big Dog would probably beat the attacking dog down and eat its face off. I have no worries for him.
There’s no one answer to the question.
If you trust your ability to take on the belligerent dog, then go for it. If you can catch it early enough, just giving a firm no (like in the OP case) will get its attention and then it’s just a question of whether you have taken the stance of the dominant dog and it recognizes that it’s the smaller dog.
If you have to intervene in the fight, it’s best if the respective owners go for their respective dogs. If one dog actually has the other within its bite, that’s a bit of a problem, but probably your best bet is to get it to release the bite by distracting it, and grabbing your dog up with one arm and keeping your body between them. If they’re still fighting, then pretty much you want to not so much fight the belligerent dog as grab it. You want to secure its head and neck and pull it up against your chest.
If you haven’t wrestled with a lot of dogs in your time, ultimately I’d say you’re screwed. If the dog is anything near your weight, again you’re probably screwed (unless you have a taser or other weapon).
I broke up a few fights when I had my whippet, at the dog park. I was generally more worried about the owner getting mad about me messing with their dog than about getting hurt. However, these were all dogs that were small enough (sub-German Shepherd) that I felt safe dealing with, and that were considered trained well enough to be safe to take to a dog park. I wouldn’t try fighting down a dog that was protecting its territory or that the owner doesn’t let out.
I often carry a 357 magnum; I’m licensed to do so. Reason being that I live across the street from a large apartment complex that seems to be filled with people who keep very large dogs in very small apartments. I might add that I often see those dogs off leash and I’ve never noticed a collar or tags on any of them. If our little dog were to be attacked, I wouldn’t hesitate to kill the attacking dog and the consequences be damned.
When police use their guns, their policy is not to fire warning shots or to try to shoot so as to “disable;” they shoot to kill.
If a dangerous animal is attacking me and I have a knife at my disposal, I’m not going to cut it once and then see if it stops attacking before I decide whether to cut it again; I will stab and slash until it’s abundantly clear that no hazard remains.
re: tasers, they work fine on dogs. Also works on much larger quadrupeds. But keep in mind that with a taser, you only get one shot before you have to load a new cartridge; so if you miss with that first shot, or if the barbed darts don’t hook in, you may have a problem. You’ll still get stun-gun action if you make direct contact with the animal, but you won’t enjoy any sort of standoff distance like you had with that first, single shot.
I read this previously on the dope a couple months ago so I tried it out on Trogdor my pit bull. I walked up behind him and picked up his back legs. Trog just kinda stood there and looked at me with a confused expression. The second time I got him good and excited. When we wrestle he is allowed to “mouth” but if he applies any pressure the game is over. In the middle of the match I grabbed his back legs and did the wheelbarrow trick. He had absolutely no issue with turning around and getting my wrist/forearm area in his mouth. In addition with him twisting and turning it was a lot harder to hold to his back legs then you would think.
Whenever we had other dogs approach I would try and put my body between the dog and Trogdor. Never had a serious attack usually just posturing. It was ridiculous the number of people in my old town that would put their dogs outside without leashes. I had more issue with small ones in general. People with big dogs would generally realize the danger of having their dog running loose.
I have a similar but yet opposite problem. I have a big dog so if another dog attacks, as they have, I just let the leash go because I don’t want to be in the middle. There has never been blood drawn in at least a half dozen encounters with big dogs.
The problem comes from people like the idiots down the street who let their little poodle mixes run free. They chase and even nip at my dog and won’t go away. I would like nothing better than to let my dog handle the situation and he would probably do it with no damage but then there would be the crying and lawsuits. One day some little fluff-ball followed us for a block with its owner following us, yelling at us to bring his dog back! Those little dogs think nothing of actually bitting and their owners think it doesn’t count if they have the smaller dog.
My impression is that you are supposed to grab the dog’s hind legs to break up a fight SO THAT he lets go of the dog he has his mouth on. The reason he lets go might be to turn around toward you (with intent to bite).
I totally believe you that he can turn all the way around – mine can reach their own backsides turning around.
Dog experts say that pit bulls are less willing to bite humans than other dogs are, and (so far) that’s been my experience. But they are sometimes inclined to bite other dogs. It’s an awful thing to talk about, but pit bulls have been selectively bred to be handled while they are fighting other dogs; the handlers are supposed to be able to reach into the pit and separate them without being harmed, because they are disinclined to redirect onto a human. I’ve never been to a dog fight, but when mine (one’s an American Pit Bull Terrier and the other an American Staffordshire mix) are playing with each other, they’re instantly submissive when I lay a hand on them, no matter how excited they are.
But I’m not advising anyone to try that with a strange pit bull. 
Just emphasizing that the dog is almost certainly making trouble for your dog, not for you, and (as turned out to be the case in your encounter) he or she will likely be overawed by, and even obey you, the human.
And to address another point:
Neighborhood beagle attacks little girl
Beagle attacks several people, is shot by police for being dangerous
There are more things in heaven and earth, Horatio,
Than are dreamt of in your philosophy.
Can you check your local ordinances? I’ll bet there’s a leash law, whether people follow it or not, it puts you in the right if your dog teaches the little beasties a lesson. If a leash law is indeed in place, you would also be in the right to pick the mutts up and take them to animal control. Your neighbors would probably have to pay a fine to get them back and maybe start thinking about that leash law and keeping control of their animals. As least that’s the case where I live. It’s worth checking into.