# Information and Thermodynamic Entropy or, Waiting for ...

No-Go Result for the Thermodynamics of Computation John D. Norton Department of History and Philosophy of Science Center for Philosophy of Science University of Pittsburgh 1 Warm Up Computers Generate Lots of Heat 2 Cray 2 computer (1985)

From Cray sales brochure. 3 Cooling chips fan heat pipe vanes 4 What is the Minimum Heat Generation?

5 To minimize heat generation Use the smallest systems possible = molecular scale devices 6 Landauers Principle 0 or 1

Erasure RESTORE TO ONE 1 Logically irreversible operations like erasure pass heat/ entropy to the environment. Erase one bit Pass kT ln 2 heat k ln 2 entropy Logically reversible operations can be carried out non-dissipatively.

Rolf Landauer (1961) "Irreversibility and heat generation in the computing process," IBM Journal of Research and Development, 5, pp. 183191. (p.265) 7 The Standard Erasure Procedure Model of binary memory. One molecule gas in a divided chamber. Plausibility of Landauers Principle 1 Credible erasure procedures generate kT ln 2 of heat.

2 Erasure compresses the space by a factor of 2. Corresponds to entropy change of k ln 2. Heat kT ln 2 Entropy k ln 2 passes to environment. 3 Information theoretic entropy of ln 2 is associated with distribution P(L) = P(H) = 1/2

Sketchy but someone has worked out the details..?? 8 Bennetts 2003 statement of Landauers Principle Logically irreversible Must operation pass entropy to environment Logically Can be

reversible operation thermodynamically reversible Landauers principle, often regarded as the basic principle of the thermodynamics of information processing, holds that any logically irreversible manipulation of information, such as the erasure of a bit or the merging of two computation paths, must be accompanied by a corresponding entropy increase in non-information-bearing degrees of freedom of the information-processing apparatus or its environment. Conversely, it is generally accepted that any logically reversible transformation of information can in principle be accomplished by

an appropriate physical mechanism operating in a thermodynamically reversible fashion. Bennett, Charles H. (2003). Notes on Landauers Principle, Reversible Computation, and Maxwells Demon, Studies in History and Philosophy of Modern Physics, 34, pp. 501-10. 9 Bennetts 2003 statement of Landauers Principle Why not Logically irreversible operation Must pass entropy to

environment Thermodynamically irreversible process create entropy in a thermodynamically irrreversible process? Logically reversible operation Can be thermodynamically reversible Thermodynamically reversible process

10 Erasing Random versus Nonrandom/Known Data When truly random data (e.g., a bit equally likely to be 0 or 1) is erased, the entropy increase of the surroundings is compensated by an entropy decrease of the data, so the operation as a whole is thermodynamically reversible. [I]n computations, logically irreversible operations are usually applied to nonrandom data deterministically generated by the computation. When erasure is applied to such data, the entropy increase of the environment is not compensated by an entropy decrease of the data, and the operation is thermodynamically irreversible.

Bennett, Charles H. (1988). Notes on the History of Reversible Computation, IBM Journal of Research and Development, 32 (No. 1), pp. 16-23 Random data erase Known data NO entropy created Entropy created ? ?? ? Erasure procedure on known data DOES use

what is known. Erasure is just rearrangement. No entropy created. does NOT use what is known. No difference in erasing known and random data. 11 To Come 12 This Presentation

No-Go Result Demonstrations of Landauers Principle fail. 1. Thermalization 2. Compression of phase space 3. Information entropy (4. New indirect proof) The standard inventory of processes in the thermodynamics of computation neglects fluctuations. All process must create entropy to overcome them and the quantities created swamp those tracked by Landauers principle. 13

Failed proofs of Landauers Principle 14 1. 15 1. Thermalization Initial data L or R Reversible isothermal compression passes heat kT ln 2 to heat bath.

Irreversible expansion thermalization !!! Entropy created in this illadvised, dissipative step. Data reset to L Entropy k ln 2 created in heat bath !!! Proof shows only that an Mustnt we thermalize so the

inefficiently designed erasure procedure creates entropy. No demonstration that all must. procedure works with arbitrary data? No demonstration that thermalization is the only way to make procedure robust. 16 Dissipationless Erasure or First method. Second method. 1. Dissipationlessly detect memory state.

1. Dissipationlessly detect memory state. 2. If R, shift to L. 2. If R, remove and reinsert partition and go to 1. Else, halt. 17 2. 18 2. Phase Volume Compression aka many to one argument thermodynamic entropy

= k ln (accessible phase volume) if entropy is to connect to heat via Clausius dS = dqrev/T random data occupies twice the phase volume of reset data Erasure halves phase volume. Erasure reduces entropy of memory by k ln 2.

Entropy k ln 2 must be created in surroundings to conserve phase volume. 19 2. Phase Volume Compression FAILS aka many to one argument random data DOES NOT occupy twice the phase volume of reset data It occupies the

same phase volume. Confusion with thermalized data 20 A Ruinous Sense of Reversible Hence confusion over random and known data. Random data and insertion of the partition

removal of the partition thermalized data have the same entropy because they are connected by a reversible, adiabatic process??? S = 0 random data thermalized data No. Under this sense of reversible, entropy ceases to be a state function.

S = k ln 2 21 3. 22 3. Information-theoretic Entropy p ln p Information entropy S = - k i inf Pi ln Pi random data PL = PR = 1/2

Sinf = k ln 2 reset data PL = 1; PR = 0 Sinf = 0 Hence erasure reduces the entropy of the memory by k ln 2, which must appear in surroundings. But in this case, Information entropy does NOT

equal Thermodynamic entropy Thermodynamic entropy is attached to a probability only in special cases. Not this one. 23 What it takes Information entropy p ln p DOES

equal Thermodynamic entropy Clausius dS = dQrev/T IF A system is distributed canonically over its phase space AND p(x) = exp( -E(x)/kT) / Z Z normalizes All regions of phase space of non-zero E(x)

are accessible to the system over time. For details of the proof and the importance of the accessibility condition, see Norton, Eaters of the Lotus, 2005. Accessibility condition FAILS for random data since only half of phase space is accessible. 24 4. 25 4. Indirect Proof: General Strategy Process known to

reduce entropy coupled to Arbitrary erasure process Ladyman et al., The connection between logical and thermodynamic irreversibility, 2007. Entropy reduces. Assume second law of thermodynamics holds on average.

Entropy must increase on average. 26 The Debate is Ongoing 27 The Debate is Ongoing My concerns: The inventory of processes assumed as possible is inconsistent with the statistical form of the second law. survives only with

artificial restrictions on what is possible. (controlled operations) selectively ignores fluctuations present in all molecular scale processes. No-go result Ladyman, James; Presnell, Stuart; Short, Anthony J. and Groisman, Berry (2007). The connection between logical and thermodynamic irreversibility, Studies in the History and Philosophy of Modern Physics, 38, pp. 5879. Ladyman, James; Presnell, Stuart and Short, Anthony J. (2008). The Use of the InformationTheoretic Entropy in Thermodynamics, Studies in History and Philosophy of Modern Physics, 39, pp. 315-324. Ladyman, James and Robertson, Katie (forthcoming). Landauer Defended: Reply to Norton, Studies in History and Philosophy of Modern Physics. Norton, John D. (2011). Waiting for Landauer. Studies in History and Philosophy of Modern Physics, 42, pp. 184198. Norton, John D. (2013). Author's Reply to Landauer Defended Studies in History and

Philosophy of Modern Physics. Available online, May 24, 2013 28 the same bit cannot be both the control and the target of a controlled operation The Most Beautiful Machine 2003 Trunk, prosthesis, compressor, pneumatic cylinder 13,4 x 35,4 x 35,2 in. the observers are supposed to push the ON button. After a while the lid of the trunk opens, a hand comes out and turns off the machine. The trunk closes - that's it! http://www.kugelbahn.ch/sesam_e.htm Every negative feedback control device acts on its own control bit. (Thermostat, regulator.)

29 Failed proofs of Converse Landauers Principle 30 Brownian Computation is the Constructive Proof Conversely, it is generally accepted that any logically reversible transformation of information can in principle be accomplished by an appropriate physical mechanism operating in a thermodynamically reversible fashion. (Bennett, 2003) Closer analysis Brownian computation is

thermodynamically irreversible, the thermodynamic analog of the uncontrolled expansion of a one molecule gas. Norton, John D. (Manuscript on website). Brownian Computation is Thermodynamically Irreversible. 31 No-Go Result Illustrated 32 Fluctuations disrupt Reversible

Expansion and Compression 33 The Intended Process Infinitely slow expansion converts heat to work in the raising of the mass. Mass M of piston continually adjusted so its weight remains in perfect balance with the mean gas pressure P= kT/V. Equilibrium height is heq = kT/Mg Heat kT ln 2 = 0.69kT passed in tiny increments

from surrounding to gas. 34 The massive piston .is very light since it must be supported by collisions with a single molecule. It has mean thermal energy kT/2 and will fluctuate in position. Probability density for the piston at height h p(h) = (Mg/kT) exp ( -Mgh/kT) Mean height = kT/Mg = heq Standard deviation = kT/Mg = heq 35

What Happens. Fluctuations obliterate the infinitely slow expansion intended Heat kT ln 2 = 0.69kT passed in tiny increments from surrounding to gas. A better analysis does not need external adjustment of weight during expansion. It replaces the gravitational field with piston = 2kT ln (height) energy

Mean energy of gas 3kT/2 Standard deviation (3/2)1/2kT = 1.225kT 36 Fluctuations disrupt Measurement and Detection 37 Bennetts Machine for Dissipationless Measurement FAILS Measurement apparatus, designed by the author to fit the Szilard engine, determines which half of the cylinder the molecule is trapped in without doing

appreciable work. A slightly modified Szilard engine sits near the top of the apparatus (1) within a boatshaped frame; a second pair of pistons has replaced part of the cylinder wall. Below the frame is a key, whose position on a locking pin indicates the state of the machine's memory. At the start of the measurement the memory is in a neutral state, and the partition has been lowered so that the molecule is trapped in one side of the apparatus. To begin the measurement (2) the key is moved up so that it disengages from the locking pin and engages a "keel" at the bottom of the frame. Then the frame is pressed down (3). The piston in the half of the cylinder containing no molecule is able to desend completely, but the piston in the other half cannot, because of the pressure of the molecule. As a result the frame tilts and the keel pushes the key to one side. The key, in its new position. is moved down to engage the locking pin (4), and the frame is allowed to move back up (5). undoing any work that was done in compressing the

molecule when the frame was pressed down. The key's position indicates which half of the cylinder the molecule is in, but the work required for the operation can be made negligible To reverse the operation one would do the steps in reverse order. is fatally disrupted by fluctuations that leave the keel rocking wildly. Charles H. Bennett, Demons, Engines and the Second Law, Scientific American 257(5):108-116 (November, 1987). 38 A Measurement Scheme Using Ferromagnets Charles H. Bennett, The Thermodynamics of Computation A Review, In. J. Theor. Phys. 21, (1982), pp. 905-40,

39 A Measurement Scheme Using Ferromagnets Charles H. Bennett, The Thermodynamics of Computation A Review, In. J. Theor. Phys. 21, (1982), pp. 905-40, 40 A General Model of Detection First step: the detector is coupled with the target system. The process intended: The process is isothermal, thermodynamically reversible: It proceeds infinitely slowly.

The driver is in equilibrium with the detector. The coupling is an isothermal, reversible compression of the detector phase space. 41 Fluctuations Obliterate Reversible Detection What we expected: What happens: 42 No-Go

Result 43 Preparatory notions Isothermal, thermodynamically reversible process Self-contained isothermal, thermodynamically reversible process. Computing fluctuations: How to do it. 44 Thermodynamically Reversible Processes For Two systems interacting isothermally in thermal contact with constant temperature surrounding at T Condition for All thermodynamic forces thermodynamic are in perfect balance (or

reversibility minutely removed from it). Process is a sequence of equilibrium states. T 1 2 env internal energy change heat transferred generalized

generalized force displacement dU = dq X dx X = -F/ for process parameter Total entropy of universe is constant. Total generalized forces vanish. X1+X2=0 Total free energy F=U-TS is constant.

F1+F2=constant 45 Thermodynamically reversible processes are NOT merely very slow processes. merely processes that can go easily in either way. capacitor discharges very slowly through resistor balloon

deflates slowly through a pinhole one molecule gas released 46 Self-contained thermodynamically reversible processes No interventions from non-thermal or farfrom-equilibrium systems. External hand removes shot one at a time to allow piston to rise slowly.

Slow compression by slowly moving, very massive body. Mass is far from thermal equilibrium of a one-dimensional Maxwell velocity distribution. 47 Computing Fluctuations probability P that system is in nonequilibrium state with phase volume V Isolated, microcanonically distributed system

phase volume V give equilibrium, macroscopic description of non-equilibrium state S = k ln V S = k ln P + constant P exp(S/k) 48 Computing Fluctuations

probability system at point with energy E probability P that system is in nonequilibrium state with V phase volume Canonically distributed system in heat bath at T. E exp kT Z(V)

E(x) Z(V ) = V exp dx kT give equilibrium, macroscopic description of non-equilibrium state F = -kT ln Z(V) F = -kT ln P + constant P exp(-F/kT) 49 No-Go Result It, at last. 50

Combine 1. and 2. any isothermal, reversible process final middle initial stages 1. Process is thermodynamically reversible

Finit = Fmid = Ffin 2. Fluctuations carry the system from one stage to another Pinit exp(-Finit/kT) Pmid exp(-Fmid/kT) Pfin exp(-Ffin/kT) Pinit = Pmiddle = Pfin No-Go result 51 Fluctuation Disrupt All Reversible, Isothermal Processes at Molecular Scales Intended

process =1 =2 =1 =2 Actual process 52

Beating Fluctuations 53 What it takes to overcome fluctuations Downward gradient in free energy final initial release from here ..but system can also be found in

undesired intermediate states. Process moves from high free energy state to low free energy state. Fsys recapture in most likely state Net creation of thermodynamic entropy. Stot = -Fsys/T 54 What it takes to overcome fluctuations Least

dissipative case free energy final initial release from here High free energy mountain makes it unlikely that system is in intermediate stage. Ffin Finit Pfin Stot = exp

= exp k Pinit kT Pinit = probability that fluctuation throws the system back to the initial state. recapture in most likely state Pfin Stot = k ln = k lnO fin Pinit odds of final state 55 Doing the sums

Molecular Scale Odds of completion Ofin = 20 Pfin = 0.95 Stot = k ln 20 = 3k compare Landauers principle k ln2 = 0.69 k Macroscopic Scale Odds of completion Ofin = 7.2x1010 Stot = k ln (7.2x1010) = 25k 25kT is the mean thermal energy of ten nitrogen molecules.

56 Bead on a Wire 57 Each position is an equilibrium position Macroscopically Effect of thermal fluctuation s Slow motion of bead over wire is a thermodynamically reversible process. (Tilt wire minutely.)

Molecular scale For 5g bead and T=25C For 100 amu mass (n-heptane molecule) and T=25C vrms = 9.071 x 10-10 m/s vrms = 157 m/s 58 Overcome fluctuations by tilting wire Macroscopically For Pfin = 0.999 T=25C stages 1/10 length

th For 5g bead = 5.8x10-18 radians Depress by ~10-7 Bohr radius H atom per meter. Molecular scale For 100 amu mass (n-heptane molecule), turning the wire vertically has negligible effect! n-heptane is volatile! 59 Least dissipative case 60

More complicated cases 61 Electric field moves a charge through a channel. Two state dipole measures sign of target charge. Computed in All Shook Up 62 Conclusion

63 Abstract and Concrete Information ideas and concepts = Entropy heat, work, thermodynamics And why not? Mass = Energy Particles = Waves Geometry = Gravity . Time = Money

64 The End 65

## Recently Viewed Presentations

• How the Labor Market Works. Disequilibrium and Nonmarket Influences. The labor market is subject to forces that impede the adjustment of both wages and employment to changes in supply or demand: Changing jobs often requires an employee to invest in...
• Activating Paymate Acclaim Activating Paymate Activation Online Activation Fax Activation Review and Verify Activation and License Terms Updating Activation Key About Activation Key Customer Support After you have installed Paymate you will need to activate the product. For users of...
• Combination or mixed circuits contain both series and parallel connections. Applying Kirchhoff's Voltage law to a mixed circuit. To analyze a mixed circuit, start by separating the circuit into sections that are connected in parallel and sections that are connected...
• Solutions to Problems Hydrogen transport vehicles Fuel Cell Infrastructure New Thermodynamically efficient Technologies US DOE Hydrogen, Fuel Cells & Intrastructure Technologies Program Wikipedia article on hydrogen economy Effects on the Environment Emissions of CO2 Electrolysis of water Fossil fuel reforming...
• *Remember, the Tú comes after the verb because this says it's a command and you're not making a mistake matching subject and verb! Los MandatosNegativos. 1) Regular?? Start in the Yo. Take off the "o" Use the opposite ending. No...
• Body Paragraphs. Thesis: George Orwell, author of . Animal Farm, uses satire- specifically an absurd tone and ironic narration- to create his social commentary about power and corruption; this demonstrates Lord Acton's idea "Power tends to corrupt and absolute power...
• Safety Culture Policy StatementTemplate. The following presentation slides and talking points are intended for Agreement States to use in communicating about safety culture to their licensees and can be tailored based on the State's specific needs.
• Rationales (1), (3), (4) = taxes/transfers generate Pareto improvements and correspond to failures of the first welfare theorem Rationale (2) = taxes/transfers shift the economy to another (second-best) Pareto optimum (illusory lump-sum payments of the second welfare theorem)