Is the second law of thermodynamics routinely violated?

[Max_S]

@Half_Man_Half_Wit

It’s not? How so? What I set out to demonstrate was how somebody observing a system could come away with a law of the form ‘the entropy increases or stays constant’, even though it’s possible for it to also decrease. My claim was that for this, it’s enough to realize the difference in number regarding high(er)- vs low(er)-entropy states.

So let’s think about how one might come away with that law. The experimenter may set the system up in either a low, intermediate, or high entropy state. If they set it up in a hight entropy state, they’ll always find it to stay constant. If they set it up in a low entropy state, they’ll always find it to stay constant, or, with some sizeable probability that increases sharply once we get to larger system sizes, to increase.

If, now, they set it up in an intermediate entropy state, they will find that, with overwhelming likelihood, the entropy will stay the same. Indeed, for some sufficiently large system, they likely never will observe it to decrease.

Consequently, they’ll formulate a law, generalizing from their observations, that entropy always increases, or stays constant.

We have more information, however. We know that this quantity, entropy, is realized by a system deterministically evolving among microstates. We can thus conclude that once in a (very large) while, entropy must also decrease. The law that was formulated thus is not an exact, but merely a statistical one.

For any sufficiently large system, that we’ll ever observe such a violation is vanishingly unlikely (think a tornado rushing through a junkyard spontaneously assembling a jumbo jet unlikely). But for small enough systems, these violations may be observable.

Moreover, we can predict just how likely such violations are going to be. That’s the fluctuation theorem. And, indeed, if we do the experiment, this prediction is confirmed.

I already admitted that such an observer subject to the “experiment” could formulate such a law. But if there is an equiprobability of initial microstates, it is not given that the system starts out in an intermediary state and you must use the probabilities for the system as a whole - 1/27, 25/27, and 1/27.

You cannot simply assume that the system starts out in an intermediary state, or you are rigging your experiment. By doing so you have reduced 1/27 25/27 1/27 to 1/18 17/18 and 0/18.

Your contention, if I recall, is that the probability of observing an increase is astronomically higher than observing a decrease.

~Max

[DPRK]

@Max_S

Your contention, if I recall, is that the probability of observing an increase is astronomically higher than observing a decrease.

The entropy of your gas (or whatever) is proportional to the logarithm of the number of accessible states. Therefore ending up in a smaller region becomes exponentially unlikely.

As a relevant mathematical exercise, flip a fair coin a bunch of times (N times) and compute the average value, where heads count as 0 and tails as 1. Now compute the probability that the average is greater than x (eg x could be 0.6). This average value is your “macrostate”. You will find that the probability decreases exponentially as a function of minus the “entropy”. You are more likely to observe a higher-entropy state.

[Max_S]

@DPRK

The entropy of your gas (or whatever) is proportional to the logarithm of the number of accessible states. Therefore ending up in a smaller region becomes exponentially unlikely.

As a relevant mathematical exercise, flip a fair coin a bunch of times (N times) and compute the average value, where heads count as 0 and tails as 1. Now compute the probability that the average is greater than x (eg x could be 0.6). This average value is your “macrostate”. You will find that the probability decreases exponentially as a function of minus the “entropy”. You are more likely to observe a higher-entropy state.

I don’t understand. Flipping a coin over n times represents a system with two states, either state 0 for heads and state 1 for tails. The initial state is already random - 50% probability of being heads or tails. The number of trials is also random - 50% probability of being even or odd. I should not have to point out that the probability of a coin showing heads after n flips is exactly 50%. To ask whether random variable 0 < x < 1 is greater than 50% is equally trivial - the probability approaches 50% depending on how precisely you define x.

~Max

[Max_S]

@Half_Man_Half_Wit

You can’t keep shaking it forever. Unless you’re a very different entity from what I take you to be, you (I’m sorry to say) won’t be around forever. So let’s just say you shake it good until you get bored, or hungry, or need a bathroom break.

This is, by the way, completely irrelevant to the hypothetical, because of course, if you shake it, it will get more mixed. I mean, you can’t really reasonably think otherwise; you’re just trying special pleading to get out of accepting an unwelcome conclusion.

If you are asking whether the marbles would be more mixed after shaking the box than when I started, I would take that bet unless the marbles filled the box to the brim (no room to mix). I thought you were asking about the degree of mixed-ness after the fact, which I would not bet on.

~Max

[Max_S]

@Half_Man_Half_Wit

(One awkwardness of the current setup, I should point out before we bump into it, is that the absolute number of high entropy states is smaller than the number of intermediate entropy states. Thus, you could have all high entropy states transition to intermediate entropy states; this is no longer the case for larger systems, where the high entropy states will vastly dominate.)

I think larger systems also have the same number of high and low entropy states if all you are counting is the ensemble of possible atomic configurations. However, many of those configurations will be inaccessible from each other.

Within a specific evolutionary cycle there needs not be an equal number of relatively high and low entropy states. But the cycle is periodic and by definition any increases in entropy must be matched with decreases of equal total magnitude. With only two accessible levels of entropy the probability of randomly observing an increase in entropy must be equal to the probability of observing a decrease. With more than two accessible levels of entropy it is possible for a random observation to favor an increase over a decrease in entropy - for example if entropy was plotted as a sawtooth wave. But without defining the evolution cycle this is not a given, entropy could just as well be plotted as a sine wave.

~Max

[DPRK]

@Max_S

I don’t understand. Flipping a coin over n times represents a system with two states, either state 0 for heads and state 1 for tails. The initial state is already random - 50% probability of being heads or tails. The number of trials is also random - 50% probability of being even or odd. I should not have to point out that the probability of a coin showing heads after n flips is exactly 50%. To ask whether random variable 0 < x < 1 is greater than 50% is equally trivial - the probability approaches 50% depending on how precisely you define x.

The trials are independent, and the state consists of the results of N coin flips. The number of states equals 2[sup]N[/sup] and the probability of each one is 2[sup]-N[/sup]. Now x is a fixed parameter and you ask what is the probability of observing a mean value M greater than x after N trials. For instance, if x = 0.5, then the limit of this probability as N→∞ is 50%, but if x = 0.55 then it is zero. But let’s say x = 0.55 and N starts to grow. What is the probability as a function of N?

[Max_S]

@DPRK

The trials are independent, and the state consists of the results of N coin flips. The number of states equals 2[sup]N[/sup] and the probability of each one is 2[sup]-N[/sup].

Do you mean to say that flipping a coin is absolutely random? I thought you meant the deterministic process of flipping a coin over. If a random process determines whether a coin lands on heads or tails, I would agree by all means that for any given n trials there is a probability distribution over 2[SUP]n[/SUP] states.

~Max

[DPRK]

For our purposes it does not matter if the flips are “absolutely random”; you may assume so. Or instead of coins think of them as non-interacting atoms that may be in one of two spin states. That is all beside the point, which is not that there is a “probability distribution”, but to calculate what it is and verify the exponential falloff of the probability of any deviation.

[Max_S]

@DPRK

For our purposes it does not matter if the flips are “absolutely random”; you may assume so. Or instead of coins think of them as non-interacting atoms that may be in one of two spin states. That is all beside the point, which is not that there is a “probability distribution”, but to calculate what it is and verify the exponential falloff of the probability of any deviation.

I’m becoming confused, and I likely misunderstand your argument. You were saying the entropy of a system is proportional to the logarithm of the number of accessible states. I take it this is because you define entropy as S = k[SUB]B[/SUB]lnΩ, where k[SUB]B[/SUB] is Boltzmann’s constant and Ω is the number of accessible microstates. That’s fine, there’s clearly a logarithm in that formula so everything checks out.

For a fully deterministic system where the initial state and number of steps and all other variables are given, there is only one accessible state at the end of the trial. That gives me S = k[SUB]B[/SUB] * ln(1) = 0.

So let’s say everything is defined except the initial state. In the coin flipping-over example that gives us two accessible states at the end of the trial, depending on which of two initial states was chosen. For a system with two equiprobable states, that gives me system entropy of S = k[SUB]B[/SUB] * ln(2) or about 9.56993x10[SUP]-24[/SUP] joules per kelvin. But this definition of entropy is macroscopic - it says nothing about which microstate is “high entropy” and which state is “low entropy”. As soon as we apply the actual microstate the macroscopic entropy drops to zero.

Could you explain what you mean by “ending up in a smaller region becomes exponentially unlikely”?. I’m not making any connection between the probability of any end state in a coin flipping exercise and that conclusion.

~Max

[Half_Man_Half_Wit]

@Max_S

I already admitted that such an observer subject to the “experiment” could formulate such a law. But if there is an equiprobability of initial microstates, it is not given that the system starts out in an intermediary state and you must use the probabilities for the system as a whole - 1/27, 25/27, and 1/27.

Equiprobability of microstates means microstates consistent with a given macrostate. That is, if the system is in the macrostate (A1B1C1), then it is with equal probability in either of the states (A:1,B:2,C:3), (A:3,B:1,C:2), (A:2,B:3,C:1), (A:1,B:3,C:2), (A:2,B:1,C:3) or (A:3,B:2,C:1). (Where the numbers denote which of the three balls is in what box.)

The states as I have given them initially are what we have macroscopic control over; they’re how we set up the system in the experiment. We don’t have control over which of the possible microstates realizes that particular macrostate; that’s where the probabilistic part comes in.

Your description would correspond to an ‘experiment’ where we just set up the system randomly, and see what happens. In that case, we’d most likely (again, in the sense of ‘with virtual certainty’) just always see systems at nearly maximum entropy not doing much at all.

But real experiments will typically involve setting up a low-entropy state, and then looking to see what happens. Think about the historic experiments with steam engines and the like. In that case, the probabilities as I have given them are the only appropriate ones.

@Max_S

If you are asking whether the marbles would be more mixed after shaking the box than when I started, I would take that bet unless the marbles filled the box to the brim (no room to mix). I thought you were asking about the degree of mixed-ness after the fact, which I would not bet on.

~Max

I don’t understand the difference between the two settings you propose. You would bet that the box is more mixed after the shaking, but you wouldn’t bet on whether it’s more or less mixed…?

@Max_S

I think larger systems also have the same number of high and low entropy states if all you are counting is the ensemble of possible atomic configurations.

I never know where you take these things from. That’s patently false: think about a system of 100 coins. The minimum entropy states are ‘all coins showing heads’ and ‘all coins showing tails’. Each of these can be realized in exactly one way.

The maximum entropy states is ‘half the coins showing heads, half showing tails’. This state can be realized in (100 choose 50) ~ 10[sup]29[/sup] ways. there are thus 10[sup]29[/sup] microstates corresponding to the maximum entropy state, and 1 corresponding to the minimum entropy state. Simple continuity shows that each state with an in-between amount of entropy must have an in-between number of microscopic realizations, as well. Hence, vastly more microstates correspond to high-entropy states than to low-entropy states.

Or, being even more explicit, look at the fraction of microstates of the form ‘x coins out of a 100 showing heads’ compared to the maximum-entropy ‘50 showing heads’. You can easily calculate that 96.5% of all microstates lie in the range between 40 and 60 coins flipped. If we double the number, with 200 coins, 99.6% of all microstates lie in the interval between 80 and 120 coins showing heads.

For any remotely macroscopic system, thus, with on the order of 10[sup]23[/sup] atoms (instead of 200), each of which can be in a huge number of states (rather than a coin’s two), if you just randomly look at the system, you’re basically guaranteed to find it in a state extremely close to the maximum entropy state.

So this idea, that the real-world experiment is modeled by just observing the system in a random state, simply doesn’t work—because then, with virtual certainty, you just won’t ever observe it doing anything.

Rather, a real-world experiment involves setting up a low-entropy state. Upon doing so, you will, with virtual certainty, as I’ve shown, observe it evolving into a higher-entropy state, simply by virtue of the relative number of accessible microstates.

From these observations, the second law may be—and historically, was—abstracted. With increased understanding of the microscopic basis of matter, it became clear that the second law can only apply statistically. And so it does.

[DPRK]

@Max_S

I’m becoming confused, and I likely misunderstand your argument. You were saying the entropy of a system is proportional to the logarithm of the number of accessible states. I take it this is because you define entropy as S = k[SUB]B[/SUB]lnΩ, where k[SUB]B[/SUB] is Boltzmann’s constant and Ω is the number of accessible microstates. That’s fine, there’s clearly a logarithm in that formula so everything checks out.

For a fully deterministic system where the initial state and number of steps and all other variables are given, there is only one accessible state at the end of the trial. That gives me S = k[SUB]B[/SUB] * ln(1) = 0.

There is definitely a misunderstanding here. Deterministic or not, the final microstate does not have an “entropy”; the entropy is associated to the MACRO-state in question or, more generally, to a probability distribution.

So let’s say everything is defined except the initial state. In the coin flipping-over example that gives us two accessible states at the end of the trial, depending on which of two initial states was chosen. For a system with two equiprobable states, that gives me system entropy of S = k[SUB]B[/SUB] * ln(2) or about 9.56993x10[SUP]-24[/SUP] joules per kelvin. But this definition of entropy is macroscopic - it says nothing about which microstate is “high entropy” and which state is “low entropy”. As soon as we apply the actual microstate the macroscopic entropy drops to zero.

Could you explain what you mean by “ending up in a smaller region becomes exponentially unlikely”?. I’m not making any connection between the probability of any end state in a coin flipping exercise and that conclusion.

~Max

In our toy example, the “system entropy” is just proportional to N log(2) like you say. Individual microstates don’t have entropy associated to them; they are just points in a phase space.

In our example the macrostate is the observed “mean energy” x. What I was getting at is, some of these states are more probable than others, the most probable state being x = 0.5. How improbable are the other states? Well, the probability of observing a mean greater than or equal to x is bounded by [let’s assume 1.0 > x > 0.5 here to get the correct signs] exp(-N(log 2 + x log x + (1-x) log(1-x))); by symmetry there is the same probability of a state with mean less than 1-x. You can see that for a fixed deviation size, the probability of being off by that much or more decays exponentially fast as N grows; an overwhelming number of states are very close to the most probable state; this is the big, i.e., maximal entropy, region.

[Max_S]

@Half_Man_Half_Wit

Equiprobability of microstates means microstates consistent with a given macrostate. That is, if the system is in the macrostate (A1B1C1), then it is with equal probability in either of the states (A:1,B:2,C:3), (A:3,B:1,C:2), (A:2,B:3,C:1), (A:1,B:3,C:2), (A:2,B:1,C:3) or (A:3,B:2,C:1). (Where the numbers denote which of the three balls is in what box.)

The states as I have given them initially are what we have macroscopic control over; they’re how we set up the system in the experiment. We don’t have control over which of the possible microstates realizes that particular macrostate; that’s where the probabilistic part comes in.

Your description would correspond to an ‘experiment’ where we just set up the system randomly, and see what happens. In that case, we’d most likely (again, in the sense of ‘with virtual certainty’) just always see systems at nearly maximum entropy not doing much at all.

But real experiments will typically involve setting up a low-entropy state, and then looking to see what happens. Think about the historic experiments with steam engines and the like. In that case, the probabilities as I have given them are the only appropriate ones.

Very well, I think (hope) I understand you. I was still confusing macrostate with thermodynamic state, but it is clear to me now that these are not the same thing. If an isolated thermodynamic system starts out in an intermediate-entropy macrostate, and all microstates consistent with that macrostate are equiprobable, then a single observation over one time step might show the system evolving into a higher-entropy or lower-entropy state depending on the internal dynamics. Without knowing the rules of evolution, the probability of observing such a change is not given, but depending on how the rules of evolution are specified it is possible for the probabilities of observing an increase versus a decrease in entropy to be unequal, or even one-sided.

@Half_Man_Half_Wit

I don’t understand the difference between the two settings you propose. You would bet that the box is more mixed after the shaking, but you wouldn’t bet on whether it’s more or less mixed…?

Right, without many more details I would not bet, for example, on whether the ratio of white to black on each side is within 10%.

@Half_Man_Half_Wit

I never know where you take these things from. That’s patently false: think about a system of 100 coins. The minimum entropy states are ‘all coins showing heads’ and ‘all coins showing tails’. Each of these can be realized in exactly one way.

The maximum entropy states is ‘half the coins showing heads, half showing tails’. This state can be realized in (100 choose 50) ~ 10[sup]29[/sup] ways. there are thus 10[sup]29[/sup] microstates corresponding to the maximum entropy state, and 1 corresponding to the minimum entropy state. Simple continuity shows that each state with an in-between amount of entropy must have an in-between number of microscopic realizations, as well. Hence, vastly more microstates correspond to high-entropy states than to low-entropy states.

Or, being even more explicit, look at the fraction of microstates of the form ‘x coins out of a 100 showing heads’ compared to the maximum-entropy ‘50 showing heads’. You can easily calculate that 96.5% of all microstates lie in the range between 40 and 60 coins flipped. If we double the number, with 200 coins, 99.6% of all microstates lie in the interval between 80 and 120 coins showing heads.

For any remotely macroscopic system, thus, with on the order of 10[sup]23[/sup] atoms (instead of 200), each of which can be in a huge number of states (rather than a coin’s two), if you just randomly look at the system, you’re basically guaranteed to find it in a state extremely close to the maximum entropy state.

So this idea, that the real-world experiment is modeled by just observing the system in a random state, simply doesn’t work—because then, with virtual certainty, you just won’t ever observe it doing anything.

Rather, a real-world experiment involves setting up a low-entropy state. Upon doing so, you will, with virtual certainty, as I’ve shown, observe it evolving into a higher-entropy state, simply by virtue of the relative number of accessible microstates.

From these observations, the second law may be—and historically, was—abstracted. With increased understanding of the microscopic basis of matter, it became clear that the second law can only apply statistically. And so it does.

I’m not sure what I was thinking there.

~Max

[Max_S]

@Half_Man_Half_Wit

Rather, a real-world experiment involves setting up a low-entropy state. Upon doing so, you will, with virtual certainty, as I’ve shown, observe it evolving into a higher-entropy state, simply by virtue of the relative number of accessible microstates.

I don’t yet understand this conclusion. If a thermodynamic system starts out in a low-entropy macrostate, nothing I am aware of dictates that the system must evolve into a higher-entropy macrostate. Even if there are more higher-entropy macrostates, it is not given that those macrostates are accessible. Without giving microscopic details (including microscopic evolution), it is impossible to even assign a probability as to whether the multitudes of higher-entropy macrostates are accessible from any given lower-entropy macrostate.

@Half_Man_Half_Wit

From these observations, the second law may be—and historically, was—abstracted. With increased understanding of the microscopic basis of matter, it became clear that the second law can only apply statistically. And so it does.

I’m not sure if I should respond to this paragraph while the above issue is still outstanding. Nevertheless I still do not equate your definition of entropy with my definition of entropy, therefore your definition of the second law is materially different from my definition and a violation of your law does not imply a violation of mine.

~Max

[Half_Man_Half_Wit]

@Max_S

Very well, I think (hope) I understand you. I was still confusing macrostate with thermodynamic state, but it is clear to me now that these are not the same thing.

A thermodynamic state—the way I would use the term—is a kind of macrostate, described in terms of thermodynamic variables (temperature, pressure) which are aggregates of microscopic variables (kinetic energy, momentum).

If an isolated thermodynamic system starts out in an intermediate-entropy macrostate, and all microstates consistent with that macrostate are equiprobable, then a single observation over one time step might show the system evolving into a higher-entropy or lower-entropy state depending on the internal dynamics. Without knowing the rules of evolution, the probability of observing such a change is not given, but depending on how the rules of evolution are specified it is possible for the probabilities of observing an increase versus a decrease in entropy to be unequal, or even one-sided.

But still, as I’ve demonstrated, no matter the evolution, we can say that observing an entropy increase is more likely than observing a decrease. Just take the extreme case: there’s 100 intermediate-entropy states (by which I mean, ten microstates corresponding to intermediate-entropy macrostates), 1 low-entropy state (same), and 10000 high-entropy states. Only one of the intermediate-entropy states can, under any evolution whatever, evolve to a low-entropy one; so for any given dynamics, the chances of observing an increase (or entropy staying constant), given that the system starts out in an intermediate-entropy state, is 99%, given only that information.

Right, without many more details I would not bet, for example, on whether the ratio of white to black on each side is within 10%.

OK, but you do now agree to the general principle of the thing, right? In other words, that 8 indeed follows from 7 in the list I gave?

@Half_Man_Half_Wit

[ol]
[li]Suppose you have two boxes, A and B.[/li][li]Box A is filled with white marbles, and box B is filled with black ones.[/li][li]Both boxes are placed on a vibrating plate, such that the marbles in them bounce around.[/li][li]The walls of the boxes are removable.[/li][li]Suppose you put both boxes next to one another, and remove the now adjacent walls, creating one big box.[/li][li]Marbles from the white box A will bounce into the black box, and marbles from the black box B will bounce into the white box.[/li][li]There are more ways of realizing a state that’s pretty uniformly grey, than there are to realize a state that’s (say) all white in box A, and all black in box B.[/li][li]Consequently, there are more ways to go from a state that’s slightly inhomogeneous to one that’s more homogeneous, than there are ways to go to a state that’s even more inhomogeneous.[/li][li]To any observer who, as before, is only capable of seeing gross colors, the formerly black-and-white separated box will gradually tend to a shade of even gray.[/li][li]That observer might formulate a law, stating that black and white, if brought into contact, eventually even out to a uniform gray.[/li][li]Knowing the microscopic description, we know that this is just, again, a law of averages: there is nothing that prohibits, say, all or a sizable fraction of the white marbles from bouncing back into box A.[/li][li]Given a long enough timescale, the even grey will, eventually, separate into black and white patches.[/li][/ol]

@Max_S

I don’t yet understand this conclusion. If a thermodynamic system starts out in a low-entropy macrostate, nothing I am aware of dictates that the system must evolve into a higher-entropy macrostate.

Well, I never said so—virtual certainty, i. e. with a probability so ridiculously high that it’s not really worth distinguishing from absolute certainty, but still, in principle, merely statistically.

Perhaps it helps if you think about all of the states of the system. Say, there’s k states. Now, thus, at any given time t, the system is going to be in one of those states. At time t + 1, it likewise will be in one of those states; any evolution thus is specified by specifying what state each of the system’s states evolves to. (This is the idea behind the evolution matrices I introduced.)

Start with the trivial evolution, which takes every state to itself. You can represent it like this:

+ ----> +
+ ----> +
+ ----> +
+ ----> +
+ ----> +
* ----> *
* ----> *
* ----> *
x ----> x

Here, the + are high-entropy states, the * are intermediate-entropy states, and the x is a low-entropy state.

Now, introduce any variation to that diagram. Say, have one of the high-entropy states evolve to a low-entropy state, like so:

+ ----> +
+ ----> +
+ ----> +
+ ----> +
+ ----> *
* ----> +
* ----> *
* ----> *
x ----> x

What has to happen, as a consequence, is that one of the high-entropy states must evolve to an intermediate-entropy state. But then, we’re done: a higher fraction of the intermediate-entropy states evolves to a high-entropy state, than high-entropy states evolve to low-entropy states. This must be the case; nothing else is possible.

I would encourage you to play around with this a little. See if you can find an evolution such that entropy increase won’t, overall, occur more often than entropy decrease.

If that doesn’t make things clearer, let’s just go to the most stupendously simply case, a system with two high-entropy states, and one low-entropy state. (Or more accurately, a system with one macrostate realized by two microstates, and another one realized by a single microstate.)

This yields either:

+ ----> +
+ ----> +
x ----> x

or:

+ ----> +
+ ----> x
x ----> +

That is, a system where either nothing changes, or, if you’re in the low-entropy state, entropy increases with certainty, while in the high-entropy state, in one out of two cases, entropy decreases.

Even if there are more higher-entropy macrostates, it is not given that those macrostates are accessible. Without giving microscopic details (including microscopic evolution), it is impossible to even assign a probability as to whether the multitudes of higher-entropy macrostates are accessible from any given lower-entropy macrostate.

I have no idea what you mean by accessibility here. If a state isn’t accessible, then it’s not really a valid state of the system.

Sure. For a single evolution, starting in some microstate, it may well be that certain states are never visited. But that means those states, then, aren’t accessible anymore for evolutions starting in another microstate, which consequently will have to visit others; and carrying that through, we arrive at the general consequence that observing entropy increase is always more likely than observing decrease.

I’m not sure if I should respond to this paragraph while the above issue is still outstanding. Nevertheless I still do not equate your definition of entropy with my definition of entropy, therefore your definition of the second law is materially different from my definition and a violation of your law does not imply a violation of mine.

~Max

Don’t worry, I haven’t forgotten. We’ll get to that in due course. The argument is, of course, very simple—basically amounting to showing that there are more ways in a system for heat to be evenly distributed, and that thus, whenever ‘my’ entropy increases, so does ‘yours’, and hence, whenever ‘my’ second law is violated, so is ‘yours’—but no, I don’t expect you to follow that for now.

[Max_S]

@DPRK

There is definitely a misunderstanding here. Deterministic or not, the final microstate does not have an “entropy”; the entropy is associated to the MACRO-state in question or, more generally, to a probability distribution.

Thank you for clearing that up, my posts must read like an ignorant fool. I do appreciate both you and Half Man Half Wit (and others) for putting up with me.

@DPRK

In our toy example, the “system entropy” is just proportional to N log(2) like you say. Individual microstates don’t have entropy associated to them; they are just points in a phase space.

In our example the macrostate is the observed “mean energy” x. What I was getting at is, some of these states are more probable than others, the most probable state being x = 0.5. How improbable are the other states? Well, the probability of observing a mean greater than or equal to x is bounded by [let’s assume 1.0 > x > 0.5 here to get the correct signs] exp(-N(log 2 + x log x + (1-x) log(1-x))); by symmetry there is the same probability of a state with mean less than 1-x. You can see that for a fixed deviation size, the probability of being off by that much or more decays exponentially fast as N grows; an overwhelming number of states are very close to the most probable state; this is the big, i.e., maximal entropy, region.

This part flew over my head. I don’t see how N fits into the entropy formula from statistical mechanics. In a system of one coin being flipped over N times, N determines the number of steps, not independent trials. There are two possible macrostates and two accessible microstates. One macrostate is given if the coin starts on heads (macrostate[SUB]H[/SUB]), the other if the coin starts on tails (macrostate[SUB]T[/SUB]). If N = 0, the microstates and macrostates are the same: State[SUB]H[/SUB] { H }, State[SUB]T[/SUB] { T }.

If N = 0,
macrostate[SUB]H[/SUB] = { H }
macrostate[SUB]T[/SUB] = { T }
If N = 1,
macrostate[SUB]H[/SUB] = { H, T }
macrostate[SUB]T[/SUB] = { T, H }
If N = 2,
macrostate[SUB]H[/SUB] = { H, T, H }
macrostate[SUB]T[/SUB] = { T, H, T }
If N = 3,
macrostate[SUB]H[/SUB] = { H, T, H, T }
macrostate[SUB]T[/SUB] = { T, H, T, H }
If N = 4,
macrostate[SUB]H[/SUB] = { H, T, H, T, H }
macrostate[SUB]T[/SUB] = { T, H, T, H, T }
If N = 5,
macrostate[SUB]H[/SUB] = { H, T, H, T, H, T }
macrostate[SUB]T[/SUB] = { T, H, T, H, T, H }

Let us next define a function to calculate the average of heads given a particular macrostate.

Let f(x) be the function calculating the ratio of heads to tails in set x.
Therefore:
f(macrostate[SUB]H[/SUB]) = (floor(N/2) + 1) / (N + 1)
f(macrostate[SUB]T[/SUB]) = floor(N/2) / (N + 1)

Now you ask, what is the probability that the ratio of heads to tails is greater than 0.6? This is simply asking whether f(macrostate) > 0.6. If we pass in macrostate[SUB]H[/SUB], we get the inequality (floor(N/2) + 1) / (N + 1) > 0.6, with two positive integer solutions of N=0 and N=2.

If we pass in macrostate[SUB]T[/SUB], this gives me floor(N/2) / (N + 1) > 0.6 which has no solutions.

I take as a premise that the initial macrostate of the system is random, that is, the probability of the initial state being macrostate[SUB]H[/SUB] vs macrostate[SUB]T[/SUB] is 50%/50%.

So to answer that question, if N∈{ 0, 2 } then the probability that the ratio of heads to tails is greater than 0.6 is 50%. For any other N, the probability is 0%. What does this have to do with entropy?

~Max

[Max_S]

@Half_Man_Half_Wit

If that doesn’t make things clearer, let’s just go to the most stupendously simply case, a system with two high-entropy states, and one low-entropy state. (Or more accurately, a system with one macrostate realized by two microstates, and another one realized by a single microstate.)

This yields either:

+ ----> +
+ ----> +
x ----> x

or:

+ ----> +
+ ----> x
x ----> +

That is, a system where either nothing changes, or, if you’re in the low-entropy state, entropy increases with certainty, while in the high-entropy state, in one out of two cases, entropy decreases.

Half Man Half Wit’s Three State System

[SPOILER]The tiny system has two possible macrostates: high-entropy macrostate[SUB]H[/SUB] and low-entropy macrostate[SUB]L[/SUB]

There are two microstates consistent with macrostate[SUB]H[/SUB]: microstate A and microstate B.

There is one microstate consistent with macrostate[SUB]L[/SUB]: microstate C.

Consider now all of the possible evolutions of this system. There are twenty-four possible evolutions, many of which are redundant so I have reduced that to six. Each evolution has between one and three evolutionary “cycles”. Here is a list of all six different evolutions, with their probability in parenthesis (assuming a random distribution, which has not yet been established):

[ul][li]evolution_1 (3/24)[/li][LIST][li]cycle_1: A->B->C->[/ul][/li][li]evolution_2 (3/24)[/li][ul][li]cycle_1: A->C->B->[/ul][/li][li]evolution_3 (4/24)[/li][ul][li]cycle_1: A->B->[/li][li]cycle_2: C->[/ul][/li][li]evolution_4 (4/24)[/li][ul][li]cycle_1: A->[/li][li]cycle_2: B->C->[/ul][/li][li]evolution_5 (4/24)[/li][ul][li]cycle_1: A->C->[/li][li]cycle_2: B->[/ul][/li][li]evolution_6 (6/24)[/li][ul][li]cycle_1: A->[/li][li]cycle_2: B->[/li][li]cycle_3: C->[/ul][/LIST][/li]
Or consider the pictoral cycles in the spoiler.

         evolution_1 (3/24)
           cycle_1
           **A**──>──**B**
           │     │
           ^     v
           │     │
           └──<──**C**

         evolution_2 (3/24)
           cycle_1
           **A**──>──**C**
           │     │
           ^     v
           │     │
           └──<──**B**

         evolution_3 (4/24)
     cycle_1    cycle_2
     **A**──>──┐    **C**──>──┐
     │     │    │     │
     ^     v    ^     v
     │     │    │     │
     └──<──**B**    └──<──┘

         evolution_4 (4/24)
     cycle_1    cycle_2
     **A**──>──┐    **B**──>──┐
     │     │    │     │
     ^     v    ^     v
     │     │    │     │
     └──<──┘    └──<──**C**

         evolution_5 (4/24)
     cycle_1    cycle_2
     **A**──>──┐    **B**──>──┐
     │     │    │     │
     ^     v    ^     v
     │     │    │     │
     └──<──**C**    └──<──┘

         evolution_6 (6/24)
cycle_1    cycle_2    cycle_3
**A**──>──┐    **B**──>──┐    **C**──>──┐
│     │    │     │    │     │
^     v    ^     v    ^     v
│     │    │     │    │     │
└──<──┘    └──<──┘    └──<──┘

For the raw table I made of all 24 evolutions, see this spoiler.

[SPOILER]

# mapping cycle_1 cycle_2 cycle_3
 1 1 abc
 2 2 acb
 3 2 bac
 4 1 bca
 5 1 cab
 6 2 cba

 7 3 ab c
 8 5 ac b
 9 3 ba c
10 4 bc a
11 5 ca b
12 4 cb a
13 4 a bc
14 4 a cb
15 5 b ac
16 5 b ca
17 3 c ab
18 3 c ba

19 6 a b c
20 6 a c b
21 6 b a c
22 6 b c a
23 6 c a b
24 6 c b a

[/SPOILER][/SPOILER]

You claim that, given the above system with an initial macrostate of macrostate[SUB]L[/SUB], in the next time step entropy will either remain constant or increase with certainty. That is correct and I agree.

Then you claim that, given the above system with an initial macrostate of macrostate[SUB]H[/SUB], in the next time step entropy will either stay the same or decrease. That is also correct and I agree, although I would not use the verbage “in one out of two cases” because that could be misinterpreted as a 50%/50% probability which does not follow. In evolution_3 and evolution_6, for example, the probability of consistent entropy/decreasing entropy is 100%/0%.

Then you claim that a system with an initial macrostate of macrostate[SUB]L[/SUB] is “virtually certain” to evolve into a higher-entropy state, simply by virtue of the relative number of accessible microstates.

You have not assigned any sort of probability to the different possible evolutions of a system so it doesn’t make sense to assert that an observer is likely to observe any particular macrostate after one time step. Try as you may, unless you flesh out the evolution matrix (which means you know the microscopic dynamics) or assume the microscopic evolution is as random as the initial microstate, you cannot make that conclusion.

Once you assume a random evolution, it is easy to show that a macrostate[SUB]L[/SUB] has a 10/24 probability of keeping the same macrostate after one timestep compared to a 14/24 probability of changing to macrostate[SUB]H[/SUB]; that a macrostate[SUB]H[/SUB] has a 7/24 probability of changing to a macrostate[SUB]L[/SUB] while the probability of staying macrostate[SUB]H[/SUB] is 17/24.

~Max

[DPRK]

@Max_S

I don’t see how N fits into the entropy formula from statistical mechanics.

It should not come as too much of a surprise that the entropy will be proportional to N, the number of particles. For instance, consider the entropy of the entire system. There are 2[sup]N[/sup] equally probable states. The logarithm of this number is N log 2.

In a system of one coin being flipped over N times, N determines the number of steps, not independent trials.

The system consists of N atoms, not one atom observed at N discrete times. You may regard the state of each atom as determined by an independent coin flip because I stipulated that there was no interaction among different atoms. In our example they are just randomly flipping around; the details are irrelevant.

There are two possible macrostates and two accessible microstates.

The possible macrostates are the possible number of heads: 0, 1, 2, …, N. The state with k heads consists of N! / k! (N-k)! micro-states; the logarithm of this number is the entropy of being in that particular state (obtained by counting all of them, because in this case they are equally probably, then taking the logarithm), up to Boltzmann’s constant.

Let us next define a function to calculate the average of heads given a particular macrostate.

If N is not too small, you may approximate the number of heads as normally distributed with mean N/2 and variance N/4.

Now you ask, what is the probability that the ratio of heads to tails is greater than 0.6?

0.6 was supposed to be more simply the proportion of heads. I did actually do some calculations, which you should check; my conclusion was that this is less than exp(-0.02 N). For example, if N = 3, the probability of at least 2 heads is 0.5 < exp(-0.06) = 0.9418

What does this have to do with entropy?

It goes to demonstrate that, even in a wildly fluctuating system like this one, as soon as you have more than a few atoms the physical properties will be dominated by the most probable (= maximal entropy) state, and also that the probability of the system being in a state with entropy S will be e[sup]S[/sup]/total # of microstates (here I have again suppressed Boltzmann’s constant), which will be indistinguishable from zero if S is not the maximum possible value (and N is at least some reasonable number). Run a computer simulation if you don’t believe it…

[Half_Man_Half_Wit]

@Max_S

You have not assigned any sort of probability to the different possible evolutions of a system so it doesn’t make sense to assert that an observer is likely to observe any particular macrostate after one time step. Try as you may, unless you flesh out the evolution matrix (which means you know the microscopic dynamics) or assume the microscopic evolution is as random as the initial microstate, you cannot make that conclusion.

The exact opposite is the case. The evolution is just the laws of physics for the given system; I trust you will grant me that they don’t change between experiments. We may not know those laws, we may not even have any idea that there are such further laws (after all, we only see the macrostate), but they’re there, and they are given by one of the possible evolutions; one of the six matrices:

     (1 0 0)          (1 0 0)          (0 1 0)
M1 = (0 1 0)     M2 = (0 0 1)     M3 = (1 0 0)
     (0 0 1)          (0 1 0)          (0 0 1)

     (0 1 0)          (0 0 1)          (0 0 1)
M4 = (0 0 1)     M5 = (1 0 0)     M6 = (0 1 0)
     (1 0 0)          (0 1 0)          (1 0 0)

Whatever way we set up the system, its underlying dynamics will always be given by one—and just one—of those matrices, just like the underlying dynamics of the molecules of a classical gas are always given by the same Newtonian physics.

The relevance of the preceding exercise is just to establish that no matter which of those describes the correct microscopic evolution law, the conclusion holds that entropy will more often increase (or stay constant) than decrease.

There’s no sense to assigning probabilities to these evolutions; the laws of physics don’t get chosen anew upon each experiment (if they did, this whole science business would be right out of the window). We don’t know which one is true, and can’t tell based on our macroscopic observations (remember, we only know the macrostate, not that it’s, for example, realized by two versus two hundred or two million microstates). But no matter which one it is, we’ll come away describing the macroscopic world that’s accessible to our investigations by means of the second law (with overwhelming likelihood).

[Max_S]

@DPRK

It should not come as too much of a surprise that the entropy will be proportional to N, the number of particles. For instance, consider the entropy of the entire system. There are 2N equally probable states. The logarithm of this number is N log 2.

The system consists of N atoms, not one atom observed at N discrete times. You may regard the state of each atom as determined by an independent coin flip because I stipulated that there was no interaction among different atoms. In our example they are just randomly flipping around; the details are irrelevant.

The possible macrostates are the possible number of heads: 0, 1, 2, …, N. The state with k heads consists of N! / k! (N-k)! micro-states; the logarithm of this number is the entropy of being in that particular state (obtained by counting all of them, because in this case they are equally probably, then taking the logarithm), up to Boltzmann’s constant.

OK, so it is a system of N coins and each coin can be heads or tails. I agree that makes 2[SUP]N[/SUP] equally probable states; 2[SUP]N[/SUP] = Ω. I can derive log[SUB]2/SUB = N. How are you getting N log 2 and what does that signify? It can’t be the entropy: S = k[SUB]B[/SUB] log[SUB]e/SUB = k[SUB]B[/SUB] log[SUB]e/SUB

If N is not too small, you may approximate the number of heads as normally distributed with mean N/2 and variance N/4.

0.6 was supposed to be more simply the proportion of heads. I did actually do some calculations, which you should check; my conclusion was that this is less than exp(-0.02 N). For example, if N = 3, the probability of at least 2 heads is 0.5 < exp(-0.06) = 0.9418

Bear with me as I relearn basic statistics…

Let N = 3 and let X = { 0, 1, 1, 1, 2, 2, 2, 3 }

*i* │ state │ total heads (X[SUB]*i*[/SUB])
──┼───────┼─────────────────
1 │ 0 0 0 │ 0
2 │ 1 0 0 │ 1
3 │ 0 1 0 │ 1
4 │ 0 0 1 │ 1
5 │ 0 1 1 │ 2
6 │ 1 0 1 │ 2
7 │ 1 1 0 │ 2
8 │ 1 1 1 │ 3

Therefore the mean µ = 3/2

(0+1+1+1+2+2+2+3)/8 = 12/8 = 3/2

Therefore the standard deviation σ = sqrt(3)/2

[SPOILER]Unicode

       ________________ 
      / 1    Ω          
σ =  /  ─ ·  **Σ** (X[SUB]*i*[/SUB] - µ)²
    √   Ω   *i*=1         
       _________________
      / 1    2[SUP]3[/SUP]⎛     3⎞²
σ =  /  ─ ·  **Σ** ⎜X[SUB]*i*[/SUB] - ─⎟ 
    √   2[SUP]3[/SUP]  *i*=1⎝     2⎠ 
       ____________________________________________________________________________________________
      / 1   ⎛⎛    3⎞²   ⎛    3⎞²   ⎛    3⎞²   ⎛    3⎞²   ⎛    3⎞²   ⎛    3⎞²   ⎛    3⎞²   ⎛    3⎞²⎞
σ =  /  ─ · ⎜⎜0 - ─⎟  + ⎜1 - ─⎟  + ⎜1 - ─⎟  + ⎜1 - ─⎟  + ⎜2 - ─⎟  + ⎜2 - ─⎟  + ⎜2 - ─⎟  + ⎜3 - ─⎟ ⎟
    √   8   ⎝⎝    2⎠    ⎝    2⎠    ⎝    2⎠    ⎝    2⎠    ⎝    2⎠    ⎝    2⎠    ⎝    2⎠    ⎝    2⎠ ⎠
       ____________________________________________________________________
      / 1   ⎛⎛  3⎞²   ⎛  1⎞²   ⎛  1⎞²   ⎛  1⎞²   ⎛1⎞²   ⎛1⎞²   ⎛1⎞²   ⎛3⎞²⎞
σ =  /  ─ · ⎜⎜- ─⎟  + ⎜- ─⎟  + ⎜- ─⎟  + ⎜- ─⎟  + ⎜─⎟  + ⎜─⎟  + ⎜─⎟  + ⎜─⎟ ⎟
    √   8   ⎝⎝  2⎠    ⎝  2⎠    ⎝  2⎠    ⎝  2⎠    ⎝2⎠    ⎝2⎠    ⎝2⎠    ⎝2⎠ ⎠
       ____________________________________
      / 1   ⎛9   1   1   1   1   1   1   9⎞
σ =  /  ─ · ⎜─ + ─ + ─ + ─ + ─ + ─ + ─ + ─⎟
    √   8   ⎝4   4   4   4   4   4   4   4⎠
       _______
      / 1   24
σ =  /  ─ · ──
    √   8    4
       _
      /3
σ =  / ─
    √  4
     _
    √3
σ = ──
     2

ASCII

[SPOILER]

       ________________ 
      / 1    Ω          
σ =  /  ─ ·  **Σ** (X[SUB]*i*[/SUB] - µ)²
    √   Ω   *i*=1         
       _________________
      / 1    2[SUP]3[/SUP]      3  
σ =  /  ─ ·  **Σ** (X[SUB]*i*[/SUB] - ─)²
    √   2[SUP]3[/SUP]  *i*=1      2  
       _____________________________________________________________________________________________
      / 1         3          3          3          3          3          3          3           3   
σ =  /  ─ · ((0 - ─)² + (1 - ─)² + (1 - ─)² + (1 - ─)² + (2 - ─)² + (2 - ─)² + (2 - ─)²  + (3 - ─)²)
    √   8         2          2          2          2          2          2          2           2   
       ____________________________________________________________________
      / 1       3        1        1        1      1      1      1      3   
σ =  /  ─ · ((- ─)² + (- ─)² + (- ─)² + (- ─)² + (─)² + (─)² + (─)² + (─)²)
    √   8       2        2        2        2      2      2      2      2   
       ____________________________________
      / 1    9   1   1   1   1   1   1   9 
σ =  /  ─ · (─ + ─ + ─ + ─ + ─ + ─ + ─ + ─)
    √   8    4   4   4   4   4   4   4   4 
       _______
      / 1   24
σ =  /  ─ · ──
    √   8    4
       _
      /3
σ =  / ─
    √  4
     _
    √3
σ = ──
     2

[/SPOILER][/SPOILER]
Therefore variance σ² = 3/4

So far everything checks out. Now what is the probability that the number of heads for a given state X[SUB]i[/SUB] >= 2? This is just a binomial distribution so it would be the binomial coefficients divided by the number of states, 2[SUP]N[/SUP].

The probability that a random state has 2 heads is 3/8.

[SPOILER]Unicode

⎛N⎞            *N*!     
⎜ ⎟ ÷ Ω = ────────────
⎝k⎠       Ω · *k*!(*N*-*k*)!

⎛3⎞            *N*!               3!             1 · 2 · 3       6   3
⎜ ⎟ ÷ 8 = ──────────── = ─────────────── = ──────────────── = ── = ─
⎝2⎠       8 · *k*!(*N*-*k*)!   8 · 2! · (3-2)!   8  · 1 · 2 · (1)   16   8

ASCII

[SPOILER]

 *N*             *N*!     
( ) ÷ Ω = ────────────
 *k*        Ω · *k*!(*N*-*k*)!

 3             *N*!               3!             1 · 2 · 3       6   3
( ) ÷ 8 = ──────────── = ─────────────── = ──────────────── = ── = ─
 2        8 · *k*!(*N*-*k*)!   8 · 2! · (3-2)!   8  · 1 · 2 · (1)   16   8

[/SPOILER][/SPOILER]
The probability that a random state has 3 heads is 1/8.

[SPOILER]Unicode

⎛N⎞            *N*!     
⎜ ⎟ ÷ Ω = ────────────
⎝*k*⎠       Ω · *k*!(*N*-*k*)!

⎛3⎞            *N*!               3!               1 · 2 · 3            6     1
⎜ ⎟ ÷ 8 = ──────────── = ─────────────── = ───────────────────── = ────── = ─
⎝3⎠       8 · *k*!(*N*-*k*)!   8 · 3! · (3-3)!   8 · 1 · 2 · 3 · (0)!    48 · 1   8

ASCII

[SPOILER]

 *N*             *N*!     
( ) ÷ Ω = ────────────
 *k*        Ω · *k*!(*N*-*k*)!

 3             *N*!               3!               1 · 2 · 3            6     1
( ) ÷ 8 = ──────────── = ─────────────── = ───────────────────── = ────── = ─
 3        8 · *k*!(*N*-*k*)!   8 · 3! · (3-3)!   8 · 1 · 2 · 3 · (0)!    48 · 1   8

[/SPOILER][/SPOILER]
Therefore the probability that a random state has at least two heads is 1/2.

[SPOILER]

Unicode
 *N*  ⎛N⎞         ⎛3⎞         ⎛3⎞        3   1   4   1
 **Σ** (⎜ ⎟ ÷ Ω) = (⎜ ⎟ ÷ Ω) + (⎜ ⎟ ÷ Ω) = ─ + ─ = ─ = ─
*k*=2 ⎝*k*⎠         ⎝2⎠         ⎝3⎠        8   8   8   2

ASCII

[SPOILER]

 *N*   *N*           3           3         3   1   4   1
 **Σ** (( ) ÷ Ω) = (( ) ÷ Ω) + (( ) ÷ Ω) = ─ + ─ = ─ = ─
*k*=2  *k*           2           3         8   8   8   2

[/SPOILER][/SPOILER]

Alright, so the math checks out. But I’m not sure how you derived “exp(-0.02 N)”, which seems to be an arbitrary number. I have yet to fit in entropy or the second law of thermodynamics.

~Max

[Max_S]

@Half_Man_Half_Wit

The relevance of the preceding exercise is just to establish that no matter which of those describes the correct microscopic evolution law, the conclusion holds that entropy will more often increase (or stay constant) than decrease.

I will concede this general rule for the first time step in systems where there are more microstates corresponding to relatively high-entropy macrostates than there are corresponding to relatively low-entropy macrostates.

~Max