A complicating factor is that calories on a label are determined through averaging the components of food.
In the old days, calories were calculated simply by burning the food in a bomb calorimeter. This was a useful measure of the easily available chemical energy, but of course humans don’t “burn” (as in burn with a flame) food to extract chemical energy. So this was an overestimate.
I suggest reading Catching Fire: How Cooking Made Us Human for some idea of why this is a lousy measure.
These days, I think the Atwater System is used.
Probably a better estimate, but still a lot of assumptions.
For weight loss, as has been mentioned, the absolute best way to tell if you’re properly losing weight is controlled measurement. And this isn’t hard.
Weigh yourself every day at the same time. A pretty basic bathroom scale and Microsoft Excel are all you need. Record this data.
After a month of this, start plotting it in Excel. I really like the exponentially smoothed moving average advocated by The Hacker’s Diet.
Basically, your weight can and will swing several pounds up and down day to day simply from variations in water retention. No big deal. The moving average, after a month or two of keeping data, eliminates this noise.
If you keep the data every day and plot a good moving average you can tell with very good confidence whether you’re losing or weight over a reasonable period. Then you can start adjusting your diet accordingly.