Or lighter when I forget?
No.
INAN (I’m not a neuroscientist) but my opinion would be that things in your brain just ***change ***when you learn new things, but no mass is added. Knowledge is the *rewiring *of existing brain cells, not the addition of them.
Hence the term, “big head” and “air head”. 
D&R
It is not the addition of cells, but it probably involves the addition of synapses, and certainly involves protein synthesis. On the other hand it may well involve the removal of some synapses too, and, in general, any changes in overall mass due to learning is going to be negligible compared to natural fluctuations arising from other causes.
Of course it does. Albert Einstein had tremendously overdeveloped neck muscles from carrying that head around. Also, remember Sherlock Holmes’s advice: You have only so much room in your head to learn stuff with, so make sure you don’t fill it up with a bunch of rubbish. 
If only. Instead of all this standardized testing, we could just weigh kids’ brains to find out how much they’ve learned!
But you’d have to subtract the weight of “boobies” from boy’s brains…
No good. Too much danger there’d be a mixup when it was time to put them back into the kids’ heads.
Studies show brain growth in adults due to learning (taxi drivers, learning to juggle, names of colors, etc.) in specific areas related to the task being learned. But there could be losses in other areas at the same time.
The answers are Yes and Yes.
Learning causes new synapses (connections between neurons) to be formed.
Over time, unused synapses will tend to wither (be re-absorbed) or get repurposed, so as you permanently forget something (without it getting replaced by learning something new).
Of course, the trimming a single fingernail will affect your weight more than several years of learning.
New synapses are using material that was in the brain already, though, and, in any case, it is probably usually more a matter of strengthening pre-existing synapses than growing whole new ones (although the stronger ones are probably a bit bigger and heavier than they were before).
Old, unused synapses can’t really be re-purposed, although the material that composed them may be if they do wither away.
Anyway, it is not really right to say that synapse formation and strengthening is learning/remembering and synapse withering is forgetting; you can’t really separate the two processes. Removing or weakening of some synapses is just as much a part of the learning process as is adding or strengthening of other ones. It is not impossible that laying down some particular memory might tend to (very slightly) reduce the mass of the brain, weakening more synapses than it strengthens, although, as I said before, any such changes in mass due to learning are going to be negligible compared to other quite unrelated (and still small, but perfectly normal) fluctuations in its mass.
You grow new neurons all the time in places like the hippocampus and olfactory system. However, this is a constant process, so over the long term I don’t think the average weight would change much.
Doesn’t quantum theory state that information has mass? So learning something should increase the mass of your brain.
I actually thought about that, but realized that the material used has to be replenished, which ultimately means more material coming in (in addition to the usual amount for routine maintenance).
Sure there is stuff coming in, but there is stuff going out too, same as any other bodily organ, or the body as a whole. In the short term, synapses are created from what is already in the brain, and in the longer term the general ebb and flow of material through the brain dwarfs any changes due to learning.
Anyway, there is no reason to think that learning leads to more synapses over all: it just leads to them being arranged a bit differently. (Actually, if anything it may well tend to lead to fewer synapses over all. Pruning of unused synapses is at least as important a mechanism of learning as is forming new ones, and very possibly is more important.)
Oh yes, our brains get much heavier as we absorb knowledge. Why, over my lifetime, I’ve probably learned 10 to 15 pounds of stuff, mostly useless. Fortunately or not, I’ve forgotten about 20 pounds worth of information, mostly important.
Although there are many studies that measure the increase in volume of gray and white matter, there didn’t seem to be as many that actually weighed brains, but I found a couple.
While they didn’t weigh human brains, for rats there are studies that show increase in matter and studies that show an increase in brain weight for rats in complex environments.
“Strengthening” 
There are multiple studies that show an increase in synapse density leading to a conclusion of net overall increase.