Lecture 6 · Part I: Quantum mechanics you need
The Hartree–Fock method
What is the best you can do with one Slater determinant, and what does it miss?
You will be able to explain
- where the exchange term comes from, and what the Fock operator contains
- why exchange cancels the self-interaction that a Hartree mean field contains
- how the Hartree–Fock equations become a matrix problem that must be solved self-consistently
- what orbital energies mean: Koopmans' theorem, and why their sum is not the total energy
- what the correlation energy is, and how Hartree–Fock relates to Kohn–Sham DFT
By Osman Goni Ridwan · Last reviewed
DFT & VASP Course · Lecture 61 / 20
Lecture 5 ended with the Slater determinant, the simplest many-electron wavefunction that obeys the Pauli principle. This lecture asks which determinant is the best one, and answers with the Hartree–Fock method.
A DFT user needs Hartree–Fock for three reasons: it introduces the self-consistent-field machinery, supplies the orbital-dependent exchange used in hybrids, and exposes the correlation missing from one determinant. Kohn–Sham DFT accounts for exchange and correlation with a different reference construction, explained at the end; its correlation energy is not simply a fixed correction added to an HF result. Helium is the worked example: two opposite-spin electrons sharing one spatial orbital, a closed-shell or restricted description. Energies are in hartree, 1 Ha ≈ 27.211 eV.
Press P to present one slide at a time, S for speaker notes.
Where does VASP's SCF loop come from?
From Hartree–Fock: the orbitals must produce the very field they move in.
DFT & VASP Course · Lecture 62 / 20
Every SCF loop solves the same chicken-and-egg problem. An electron's orbital depends on the field it moves in. That field depends on where the other electrons are, that is, on their orbitals. So you cannot compute the orbitals before you know the field, or the field before you know the orbitals.
The way out is the loop on the slide. Guess the orbitals; a common start is the orbitals of the bare nuclei, with the repulsion between electrons switched off. Build the field that the guess produces. Solve for the orbitals in that field. If the new orbitals differ from the guess, use them as the next guess and go around again. When input and output agree, the orbitals are self-consistent: they produce exactly the field in which they are the solutions.
Douglas Hartree introduced this self-consistent field in 1928 to compute atoms. In 1930 Vladimir Fock and John Slater made the wavefunction a Slater determinant, which adds the exchange term derived in this lecture; the result is Hartree–Fock theory. Kohn–Sham DFT keeps the loop and changes what goes into the second box. In VASP, each pass around the loop is one line of the OSZICAR file.
What is the best single Slater determinant?
The one with the lowest energy: Hartree–Fock applies the variational principle to the orbitals themselves.
- One-electron energy of spin-orbital a: its kinetic energy plus its attraction to the nuclei.
- Coulomb integral: the classical repulsion between the charge clouds of orbitals a and b.
- Exchange integral: a consequence of antisymmetry; nonzero only if a and b have the same spin.
DFT & VASP Course · Lecture 63 / 20
Lecture 5 built the many-electron wavefunction as one Slater determinant of spin-orbitals χa(x)=ϕa(r)σa, a spatial orbital times a spin function. With orthonormal occupied spin-orbitals and the Slater normalization factor, the determinant is antisymmetric and normalized, so it is a legitimate trial wavefunction. By the variational principle of Lecture 3 its energy therefore lies at or above the exact ground-state energy, and the best determinant is the one with the lowest energy. What is new compared with Lecture 3 is what you vary: not a few numbers, but the orbitals themselves.
The indices a and b run over occupied spin-orbitals, and a < b counts each distinct pair once. A two-electron integral adds contributions over both electron positions (and sums over spin where indicated). The energy of one determinant has the form on the slide. (The repulsion between the clamped nuclei is a constant and is left out.) Its three ingredients are
ha=∫ϕa∗(r)[−21∇2−A∑∣r−RA∣ZA]ϕa(r)dr, Jab=∬∣r1−r2∣∣ϕa(r1)∣2∣ϕb(r2)∣2dr1dr2,Kab=δσaσb∬∣r1−r2∣ϕa∗(r1)ϕb(r1)ϕb∗(r2)ϕa(r2)dr1dr2.Jab is what you would write down classically: the charge density ∣ϕa∣2 repelling the charge density ∣ϕb∣2. Kab has no classical counterpart; the next slide shows where it comes from. Both are nonnegative; for a fixed set of occupied orbitals the subtracted exchange term lowers the classical Hartree estimate. It does not establish which independently optimized spin state is lowest. The factor δσaσb is 1 for equal spins and 0 otherwise, so only pairs of the same spin have an exchange term.
Minimizing this energy shapes the orbitals. Because each electron now feels the repulsion of the others, the best orbitals are more spread out than those of the bare nucleus. The shielding of Lecture 4 comes out of the minimization rather than being put in by hand, as the helium example will show.
Where does the exchange term come from?
Antisymmetry adds cross terms between orbital assignments; their spin factors vanish for opposite spins.
- Direct term J12: each electron stays in its own orbital on both sides.
- Exchange term K12: the electrons have traded orbitals on the right. The spin sum gives 1 for equal spins, 0 for opposite.
DFT & VASP Course · Lecture 64 / 20
Take two electrons in orthonormal spin-orbitals χ1 and χ2, with the Hamiltonian h^(1)+h^(2)+1/r12. Here h^ is the one-electron part (kinetic energy and nuclear attraction) and r12=∣r1−r2∣. Write
⟨ab∣cd⟩=∬χa∗(x1)χb∗(x2)r121χc(x1)χd(x2)dx1dx2,where ∫dx means integrating over position and summing over spin.
The repulsion. Inserting the determinant gives four terms, with a prefactor 21 from the normalization:
⟨Φr121Φ⟩=21[⟨12∣12⟩+⟨21∣21⟩−⟨12∣21⟩−⟨21∣12⟩].Renaming the integration variables x1↔x2 shows that ⟨21∣21⟩=⟨12∣12⟩ and ⟨21∣12⟩=⟨12∣21⟩. The bracket is therefore twice ⟨12∣12⟩−⟨12∣21⟩, and the 21 cancels. In the first integral each electron is in the same orbital on both sides: that is the Coulomb integral J12. In the second, the two electrons have swapped orbitals between the left and the right side. Such a term exists only because the wavefunction contains both orderings, and it is the exchange integral K12.
The spin. Write χa=ϕa(r)σa. In ⟨12∣21⟩, electron 1 carries spin σ1 on the left and σ2 on the right, so its spin sum gives ⟨σ1∣σ2⟩; the same holds for electron 2. That product is 1 when the spins are equal and 0 when they are opposite. Two electrons of opposite spin therefore have no exchange term, and their repulsion is the classical J12 alone.
The rest. The one-electron part gives ⟨Φ∣h^(1)+h^(2)∣Φ⟩=h1+h2, because its cross terms contain the overlap ⟨χ1∣χ2⟩=0. The total is E=h1+h2+J12−K12. For N electrons the same algebra applies to each pair separately, which gives the sum over pairs on the previous slide.
The minus sign has a physical meaning. Two electrons of the same spin repel each other less than the classical estimate, because antisymmetry makes their joint wavefunction vanish wherever they meet (the exchange hole of Lecture 5). On average they stay farther apart.
What field does each electron move in?
The nuclei plus the average Coulomb and exchange interaction with all the electrons: a mean field.
- One-electron part: kinetic energy and attraction to the nuclei.
- Coulomb operator: multiplies by the electrostatic potential of orbital b's charge cloud.
- Exchange operator: nonlocal—its output at one point uses the orbital at other points. Only equal-spin terms survive.
- Orbital energy: a Fock eigenvalue, including interactions with other occupied orbitals.
DFT & VASP Course · Lecture 65 / 20
Minimizing the determinant energy over the orbitals, while keeping them orthonormal, leads to one-electron equations: the Hartree–Fock equations on the slide. The effective operator f^ is the Fock operator. Its two new pieces act on an orbital χ as follows:
J^bχ(x1)=[∫r12∣χb(x2)∣2dx2]χ(x1),K^bχ(x1)=[∫r12χb∗(x2)χ(x2)dx2]χb(x1).J^b is an ordinary potential: it multiplies χ by the electrostatic potential that the charge cloud of orbital b creates at r1. K^b is different in two ways. To evaluate K^bχ at one point you need χ at every point, because χ sits inside the integral; an operator like this is called nonlocal. And the result is proportional to χb, not to χ. Nonlocality is what makes exact exchange expensive in a plane-wave code (Lecture 27).
The picture is a mean field. Each electron feels the nuclei and the average charge distribution of the electrons, plus exchange, but not where the other electrons happen to be at a given instant. The orbital energy
εa=⟨χa∣f^∣χa⟩=ha+b∑(Jab−Kab)is a Fock eigenvalue containing mean-field interactions with the other occupied orbitals. It is not an additive share of the total energy; the double-counting slide will show why.
The sum runs over all occupied orbitals, including b=a. Acting on χa itself, J^aχa=K^aχa, so the self-term contributes nothing (next slide). Including it makes f^ the same operator for every orbital. Distinct-eigenvalue orbitals of this Hermitian operator are orthogonal, and degenerate ones can be chosen orthonormal, as required by the determinant construction.
How minimizing the energy gives the Hartree–Fock equations
Write the energy as E=∑a⟨a∣h^∣a⟩+21∑a,b(⟨ab∣ab⟩−⟨ab∣ba⟩); the terms with a=b cancel, so this is the pair sum of the determinant energy. Change one orbital by a small amount, χa→χa+δχa. Orbital a appears in both slots of the pair sum, which cancels the 21, and to first order
δE=⟨δχa∣f^∣χa⟩+complex conjugate.The orbitals must stay orthonormal, so introduce Lagrange multipliers, adjustable numbers that enforce those constraints. Call them λba, and minimize E−∑a,bλba(⟨χa∣χb⟩−δab). Setting the change to zero for every δχa gives f^χa=∑bλbaχb, with a Hermitian matrix λ.
Now use a freedom of the determinant. Replacing the occupied orbitals by any unitary combination of themselves (a mixing that preserves all overlaps) multiplies the determinant by a phase and leaves the density, f^ and E unchanged. Choose the combination that diagonalizes λ. In that basis f^χa=εaχa: the canonical Hartree–Fock equations on the slide, with the eigenvalues of λ as orbital energies.
Does an electron repel itself in Hartree–Fock?
No: for every electron, the exchange term cancels its repulsion with its own charge exactly.
DFT & VASP Course · Lecture 66 / 20
There are two ways to book-keep the repulsion between electrons, and comparing them shows what exchange does.
The pair sum in the determinant energy, ∑a<b(Jab−Kab), counts each pair of different electrons once and never an electron with itself. DFT books it differently. It starts from the classical electrostatic energy of the total density n(r)=∑a∣ϕa(r)∣2, the Hartree energy
U=21∬r12n(r1)n(r2)dr1dr2=21a,b∑Jab.This double sum includes the terms a=b: every electron repelling its own charge cloud. The Hartree–Fock exchange energy is Ex=−21∑a,bKab, again including a=b. Set b=a in the definitions of Jab and Kab, and they become the same integral, Kaa=Jaa. The self-terms cancel one by one, and U+Ex equals the pair sum exactly.
Helium makes this concrete. Both electrons occupy the same spatial orbital ϕ1s, with opposite spins, and their repulsion is J=J1s,1s=1.026 Ha (27.9 eV) in the converged calculation of the coming slides. The density is n=2∣ϕ1s∣2, so the Hartree energy is U=21⋅2⋅2⋅J=2J. But helium has a single pair of electrons, whose repulsion is J. The extra J is self-interaction. The exchange energy is −21(K1s↑,1s↑+K1s↓,1s↓)=−J, which removes it exactly. The two electrons have no exchange term with each other, while each occupied spin-orbital still has a diagonal self-exchange term. These are different terms: the diagonal terms cancel the artificial self-repulsion included in U. Here U denotes the Hartree electrostatic energy, not the on-site Hubbard parameter mentioned in Lecture 4.
Approximate exchange–correlation functionals replace Ex by an approximation that cancels the self-repulsion only partly. What remains is the self-interaction error, one of the weaknesses of DFT listed in Lecture 0 and treated in Lecture 10.
What does Hartree–Fock give for helium, by hand?
One orbital and two formulas: E = 2h + J and ε = h + J. Each freedom given to the orbital lowers the energy.
- One-electron energy of the 1s orbital: kinetic energy plus attraction to the nucleus.
- Repulsion between the two 1s charge clouds. No exchange between the opposite-spin pair; self-terms have already canceled.
- Bare-nucleus orbital e−2rE=−2.750 Ha
- Best e−ζr: ζ=27/16E=−2.848 Ha
- Best orbital of any shapeE=−2.862 Ha, the Hartree–Fock limit
DFT & VASP Course · Lecture 67 / 20
Helium's ground state puts both electrons in one spatial orbital ϕ, one with spin up and one with spin down. The determinant energy then has two equal one-electron terms and a single pair, of opposite spins and so without exchange: E=2h+J, with h=⟨ϕ∣−21∇2−2/r∣ϕ⟩ and J=J1s,1s. The Fock operator acting on ϕ contains the repulsion with the other electron only, since the self-terms cancel, so ε=h+J.
Now apply the variational principle by hand. Try the hydrogen-like orbital ϕ∝e−ζr with an adjustable exponent ζ. For this function h=ζ2/2−Zζ and J=5ζ/8, so
E(ζ)=ζ2−2Zζ+85ζ,Z=2.- With ζ=Z=2, the orbital of the bare nucleus, E=4−8+1.25=−2.750 Ha. This is Lecture 5's estimate, with the repulsion added as an average.
- Setting dE/dζ=0 gives ζ=Z−5/16=27/16=1.6875 and E=−(27/16)2=−2.848 Ha. Each electron behaves as if the nuclear charge were 1.69 instead of 2, because the other electron shields part of it. This is the shielding of Lecture 4, now derived rather than assumed.
- Allowing the orbital any shape lowers the energy further, to the Hartree–Fock limit, −2.862 Ha. There h=−1.944 Ha and J=1.026 Ha.
Each trial contains the previous one as a special case, so the energy can only go down; and none can go below the exact energy, −2.904 Ha. The next two slides show how a computer finds the best shape.
How does a computer solve the Hartree–Fock equations?
Expand the orbital in basis functions: the equation becomes a matrix problem, FC = SCε, in which F depends on C.
- Basis functions, here Gaussians e−αjr2, whose integrals are known in closed form.
- Fock matrix Fij=⟨gi∣f^∣gj⟩, built from the very coefficients it is meant to produce.
- Overlap matrix Sij=⟨gi∣gj⟩, not the identity, because Gaussians overlap.
DFT & VASP Course · Lecture 68 / 20
A computer cannot store a function, only numbers. As in Lecture 3, expand the orbital in M fixed basis functions gj and let the coefficients cj carry the shape. Insert the expansion into f^ϕ=εϕ, multiply by gi∗ and integrate, once for each i. The result is a matrix equation, the Roothaan–Hall equations, derived independently by Clemens Roothaan and George Hall in 1951.
It differs from an ordinary eigenvalue problem in two ways.
For many orbitals, C collects their coefficient vectors as columns, and ε is the diagonal matrix of orbital energies: FC = SCε abbreviates one generalized eigenvalue equation per orbital. Helium's occupied spatial orbital needs only the single coefficient vector c shown in the equation. A virtual orbital is an unoccupied solution of this same one-electron equation.
- The overlap matrix. These Gaussians are not orthogonal, so S records their overlaps. Changing to orthonormal coordinates converts the generalized problem into an ordinary one. One choice is X=S−1/2: diagonalize S, replace each positive eigenvalue s by 1/s, and transform back. Then F′=XTFX is symmetric and c=Xc′. The transpose T swaps rows and columns; the present example uses real coefficients. In VASP the plane waves are orthogonal, while PAW supplies a separate overlap operator. The algebraic form is shared; the origin of S differs.
- The dependence on c. F contains the Coulomb and exchange operators of the occupied orbital, so it depends on c. That is why the next slide iterates.
For helium, with one doubly occupied orbital, the Fock matrix is the formula on the slide. In it, hij=⟨gi∣−21∇2−2/r∣gj⟩, and
(ij∣kl)=∬r12gi(r1)gj(r1)gk(r2)gl(r2)dr1dr2is a two-electron integral: the repulsion between the charge distributions gigj and gkgl. The term 2(ij∣kl) is the Coulomb repulsion of both electrons. The term −(ik∣jl) is exchange; acting on the occupied orbital it cancels half of the Coulomb term, the self-repulsion, which leaves the ε=h+J of the previous slide. The energy is E=∑ijcicj(hij+Fij), which equals 2h+J.
Diagonalizing an M×M matrix gives M orbitals. The lowest is occupied by both electrons; the others are empty, or virtual, orbitals. The bottleneck is the two-electron integrals: there are about M4/8 distinct ones. That is why the cost of Hartree–Fock is usually said to grow as the fourth power of the basis size.
Every formula behind the helium figure
The figure uses M unnormalized s-type Gaussians gj=e−αjr2, with exponents from the even-tempered sequence αk=0.1×2.6k, k=0,…,11. With p=αi+αj and q=αk+αl, all integrals have closed forms:
Sij=(pπ)3/2,Tij=p3αiαj(pπ)3/2,Vij=−p2πZ,(ij∣kl)=pqp+q2π5/2,and hij=Tij+Vij, the kinetic and nuclear-attraction parts. One iteration then has four steps:
- Build F from the current c. The first iteration has no orbital yet and uses F=h, the bare nucleus.
- Solve Fc=εSc through S−1/2, as above.
- Keep the lowest eigenvector; it comes out normalized, cTSc=1.
- Evaluate E, and stop when it no longer changes.
That is the whole method; the solver is a few dozen lines of JavaScript, in the figure's source file.
Why must the equations be iterated?
The Fock matrix depends on its own solutions, so iterate until the orbitals going in and coming out agree.
DFT & VASP Course · Lecture 69 / 20
If F did not depend on c, one diagonalization would finish the job. Because it does, the eigenvector that comes out differs from the one used to build F, and you have to iterate. Press One iteration repeatedly, with the default M = 10, and watch the table.
- Iteration 1 uses F=h, so its orbital is that of the bare nucleus, with ε=−2.000 Ha (−Z2/2). Its energy is −2.750 Ha, the ζ=2 value of the previous slide but one.
- Iteration 2 builds the field of that orbital. The orbital expands in response, and the energy drops to −2.8602 Ha, already close to the final value.
- From then on each change is about 20 to 40 times smaller than the one before: −1.3 × 10⁻³, −3.3 × 10⁻⁵, −1.2 × 10⁻⁶ Ha, and so on. By iteration 7 the energy changes by less than 10⁻⁸ Ha. The orbital now reproduces the field that produced it, and ε has settled at −0.918 Ha.
A steady shrinking of the changes by a roughly constant factor is the signature of a healthy SCF loop, and it is what you want to see in VASP's OSZICAR too. Helium is an easy case. In metals and large cells, a naive loop can overshoot and oscillate, with charge sloshing back and forth between iterations. Codes then mix the old and new densities instead of taking the new one outright (Lecture 12).
The slider changes the basis size M. More Gaussians give a lower converged energy, approaching the Hartree–Fock limit, −2.86168 Ha, from above, as the variational principle requires: −2.791 Ha with 4 functions, −2.8604 Ha with 8, −2.86167 Ha with 12. Keep the two kinds of convergence apart. Iterating to self-consistency removes the error of stopping too early; only a bigger basis removes the error of the basis.
How good is Hartree–Fock for helium?
It gets 98.6% of the energy. The missing 1.14 eV, the correlation energy, is about a quarter of a covalent bond.
- Hartree–Fock limit−2.8617 Ha−77.87 eV: the best single determinant
- Exact, same Hamiltonian−2.9037 Ha−79.01 eV; measurement agrees to 0.01%
- Correlation energy−0.0420 Ha−1.14 eV: beyond any single determinant
DFT & VASP Course · Lecture 610 / 20
The Hartree–Fock limit for helium, the energy of the best single determinant in a complete basis, is −2.861680 Ha. The exact ground-state energy of the same Hamiltonian (two electrons around a fixed nucleus, without relativity) is −2.903724 Ha. It is known to many more digits from calculations with wavefunctions that depend explicitly on the distance between the two electrons.
The measured value is −2.903386 Ha. It is the sum of the measured first ionization energy, 24.587 eV, and the second, 54.418 eV, which for the one-electron ion He⁺ is calculated more accurately than it can be measured. It differs from the exact value by 0.01%, because of physics outside our Hamiltonian: mainly the finite mass of the nucleus, and small relativistic effects.
Hartree–Fock therefore recovers 98.6% of the energy and misses 0.042 Ha, or 1.14 eV. As a fraction that is small. But chemistry is about energy differences, and a typical covalent bond is worth only a few eV (about 5 eV in Lecture 5's estimate). Fluorine is a striking example. In the restricted closed-shell Hartree–Fock treatment, the F₂ molecule lies higher in energy than two separate fluorine atoms, by more than 1 eV, so it should fall apart. The real molecule is bound by about 1.6 eV. All of its binding comes from correlation.
Do Hartree–Fock orbital energies mean anything?
Yes, approximately: minus an occupied orbital energy is the energy to remove that electron if no other orbital changes (Koopmans' theorem).
- Frozen ion: one electron removed, the other left in the unchanged orbital ϕ. Its energy is just h.
- Koopmans: −ε1s24.98 eV
- Ion recomputed, orbital relaxed23.45 eV
- Exact, and measured24.59 eV
DFT & VASP Course · Lecture 611 / 20
Orbital energies are by-products of the calculation, but in Hartree–Fock they have a physical meaning. Take one electron out of helium and leave the remaining orbital exactly as it was. The ion's energy is then just the one-electron energy h, so the energy needed to remove the electron is h−(2h+J)=−(h+J)=−ε. For finite atoms and molecules with a vacuum energy reference, the same frozen-orbital algebra shows that −εa is the ionization energy from orbital a when all other orbitals are frozen. Periodic solids additionally require a consistent energy reference and charged-system treatment. This is Koopmans' theorem, published in 1934 by Tjalling Koopmans, who later shared the 1975 Nobel Memorial Prize in economics.
For helium, −ε1s=0.918 Ha = 24.98 eV, against the exact 24.59 eV: 0.39 eV too high. That small error hides two larger ones.
- Freezing the orbital overestimates the cost. In the real ion the remaining electron no longer feels the other's repulsion and contracts towards the nucleus, lowering the ion's energy. Recomputing the ion with a relaxed orbital (He⁺ has one electron, so its energy is exactly −2 Ha) gives −2 − (−2.8617) = 0.8617 Ha = 23.45 eV, 1.53 eV lower.
- Missing correlation works the other way. Neutral helium has a correlation energy of −1.14 eV that the one-electron ion lacks, so the relaxed value is 1.14 eV too low.
Koopmans' estimate is good because these two errors, +1.53 eV and −1.14 eV, mostly cancel. The same idea gives electron affinities from empty orbitals, but relaxation and correlation errors can be large and need not cancel, so the estimate is often poor. Kohn–Sham orbital energies are different again. Apart from the highest occupied one, which equals minus the ionization energy for the exact functional, they have no such general meaning (Lecture 2).
Is the total energy the sum of the orbital energies?
No. Each orbital energy contains the repulsion with every other electron, so the sum counts each pair twice.
- Double counting: each pair's repulsion is in both εa and εb; subtract it once.
- Helium: 2ε1s−1.836 Hasum of orbital energies
- Counted twice1.026 HaJ, subtract it once
- Total energy−2.862 HaE=2ε−J
- In VASP's OUTCARBand-energy sum EBANDS, then the corrections DENC and XCENC
DFT & VASP Course · Lecture 612 / 20
For electrons that do not interact, the total energy is the sum of the orbital energies. It is tempting to add them up here too, but it does not work once the electrons interact. The orbital energy εa=ha+∑b(Jab−Kab) contains the repulsion of electron a with every other electron. Summing over a counts the repulsion of each pair twice, once from each side:
a∑εa=a∑ha+2a<b∑(Jab−Kab),while the determinant energy contains it once. Subtracting the excess gives the formula on the slide. For helium, 2ε=−1.836 Ha, and subtracting J=1.026 Ha gives E=−2.862 Ha. The naive sum is off by 28 eV.
For two electrons, ε₁ contains pair (1,2), and ε₂ contains the same pair (2,1). The physical interaction has only one unordered pair. This is the bookkeeping meaning of double counting; the band-energy sum is not the total energy.
Kohn–Sham DFT has the same structure, and VASP shows it to you. Each energy block of the OUTCAR, under "Free energy of the ion-electron system", has a line EBANDS, the sum of the occupied band energies. It is followed by correction lines, DENC for the Hartree energy and XCENC for exchange and correlation, that remove what the band-energy sum counts twice. Find them in your Lab 0 OUTCAR. The DFT corrections follow its own energy functional; the full OUTCAR total also includes ionic, PAW and smearing-related terms. It is not obtained by applying the helium HF formula directly to a band-energy sum.
What is the correlation energy?
The exact energy minus the Hartree–Fock energy: what a mean field cannot capture.
- Never positive for the same Hamiltonian; zero for one electron, generally negative for interacting many-electron systems.
- Physical originElectrons avoid each other at every instant, not just on average.
- SizeAbout 1.4% of helium's binding energy here; no universal percentage applies to other systems.
- Correlated wavefunction methodsAdd more determinants: accurate, but their cost climbs steeply with size.
DFT & VASP Course · Lecture 613 / 20
The correlation energy is the difference between the exact energy and the Hartree–Fock energy for the same Hamiltonian. The Hartree–Fock determinant is a trial wavefunction, so EHF≥Eexact, and Ecorr is never positive. For helium it is −0.042 Ha.
What does a single determinant miss? Helium's Hartree–Fock wavefunction is ϕ(r1)ϕ(r2) times a spin function. The probability of finding electron 2 at a point does not depend on where electron 1 is: the two move independently, each in the average field of the other. Real electrons do not. When one is close to the nucleus, the other tends to be farther out (radial correlation), and the two prefer opposite sides of the nucleus (angular correlation). Exchange already keeps electrons of the same spin apart. Correlation is the further avoidance, for both spins, driven by the repulsion itself. Because they avoid each other better, real electrons repel each other less, and the energy is lower.
Wavefunction methods recover correlation by going beyond one determinant. They mix in determinants in which electrons are promoted to empty orbitals, which lets the wavefunction depend on where the other electrons are. The exact answer in a given basis, full configuration interaction, uses all such determinants. Their number grows exponentially with the number of electrons: the exponential wall of Lecture 0. Practical methods truncate the expansion, at a cost that still grows steeply with system size, as the fifth power for MP2 and the seventh for CCSD(T), the "gold standard" of quantum chemistry. That limits them to molecules and small cells. DFT instead approximates correlation by a functional of the density, at the cost of a mean-field calculation.
How is Kohn–Sham DFT related to Hartree–Fock?
Same machinery, orbitals in a determinant solved self-consistently, but with exchange and correlation from a density functional.
| Hartree–Fock | Kohn–Sham DFT | |
|---|---|---|
| Orbitals | One determinant, the one with the lowest energy | One determinant of an auxiliary system with the true density |
| Exchange | Exact, nonlocal | Approximated in Exc[n]; hybrids mix in a fraction of exact exchange |
| Correlation | None | Approximated in Exc[n] |
| Self-interaction | Cancels exactly | Cancels only partly with common approximations |
| Solved by | SCF iterations | SCF iterations |
DFT & VASP Course · Lecture 614 / 20
From the outside, Hartree–Fock and Kohn–Sham DFT look alike. Both describe the electrons by orbitals in a single determinant, both solve one-electron equations with an effective operator, and both iterate to self-consistency. They differ in what goes into the effective operator, and in what the determinant means.
In Hartree–Fock the determinant is the approximation to the true wavefunction; exchange is treated exactly, and correlation is missing. In Kohn–Sham theory the determinant belongs to an auxiliary system of non-interacting electrons, constructed to have the same density as the real ones; it is not meant to approximate the true wavefunction. Exchange and correlation are combined into one functional of the density, Exc[n]. Its correlation part includes the interaction and kinetic-energy corrections relative to the Kohn–Sham noninteracting reference. The conventional HF correlation energy is instead measured from the optimized HF determinant. The two correlation definitions are not numerically interchangeable; Part II builds the density-functional definition. Practical functionals approximate this required contribution.
Hybrid functionals sit in between. They take a fraction of exact Hartree–Fock exchange, 25% in PBE0, and the rest of exchange and all of correlation from a density functional. Part II builds Kohn–Sham theory from the ground up, starting with Lecture 7.
Where does Hartree–Fock appear in VASP?
In the SCF loop every run performs, and as exact exchange in hybrid functionals.
| Idea from this lecture | In VASP |
|---|---|
| SCF iterations | One OSZICAR line per electronic step; stop at EDIFF or after NELM steps |
| A starting guess before self-consistency | NELMDL: the first steps keep the initial Hamiltonian fixed (5 by default for a fresh run) |
| Orbital energies and double counting | OUTCAR: band-energy sum EBANDS, corrected by DENC and XCENC |
| Exact exchange | LHFCALC = .TRUE. turns on a hybrid functional, PBE0 by default |
| How much exact exchange | AEXX: 0.25 by default (PBE0); 1.0 gives HF with the ordinary unscreened default settings |
DFT & VASP Course · Lecture 615 / 20
Every VASP run performs an SCF loop of the kind on the first slide, with the Kohn–Sham Hamiltonian in place of the Fock matrix (Lecture 12). Each line in the OSZICAR file is one pass. The loop stops when the energy changes by less than EDIFF from one step to the next, or after NELM steps, whichever comes first.
A fresh run starts from random orbitals, far from any solution. So VASP first holds the Hamiltonian fixed for a few steps, built from a superposition of atomic charge densities, while the orbitals settle; only then does it start rebuilding the Hamiltonian from the new density. NELMDL sets the number of these steps: 5 for a fresh run with the default algorithm, 12 with ALGO = VeryFast, and none when the run restarts from a WAVECAR. Its default is written with a minus sign, −5, which means the delay applies only to the first ionic step.
LHFCALC = .TRUE. adds exact exchange; without further tags VASP uses the PBE0 hybrid. AEXX sets the fraction of exact exchange, 0.25 by default. With AEXX = 1.0, VASP switches correlation off by default and gives full Hartree–Fock for the ordinary unscreened setup. Explicit exchange/correlation scaling tags or screening settings can change that result, so inspect inherited INCAR settings as well. Hartree–Fock for a solid is therefore a two-line change to the INCAR, but an expensive one, because of the nonlocal exchange operator.
What has Part I built?
Why electronic structure is a many-body problem, and what “self-consistent” means.
- L1 Born–Oppenheimer
- L2 Ĥψ = Eψ
- L3 Variational principle, bases
- L4 Atoms, core, valence
- L5 Pauli, determinants
- L6 Hartree–Fock, SCF
- The problemElectrons in the field of clamped nuclei; their repulsion couples them all, and Ψ is too big to store.
- The first answerA variationally optimized single determinant. Its missing correlation is small as a fraction of helium's energy, but important for energy differences.
DFT & VASP Course · Lecture 616 / 20
Part I set out to explain why electronic structure is a many-body problem, and what "self-consistent" means. The chain of ideas:
- Lecture 1 clamps the nuclei and leaves an electronic problem.
- Lecture 2 gives the language of states, operators and eigenvalues.
- Lecture 3 turns them into matrices a computer can solve, with a variational guarantee.
- Lecture 4 builds atoms from orbitals and separates core from valence.
- Lecture 5 adds the Pauli principle, and shows that the full wavefunction cannot be stored.
- This lecture finds the best orbital-based answer, self-consistently, and names what it misses.
Part II replaces the wavefunction by the density and keeps the self-consistent machinery.
Lab 6: how did your Lab 0 run become self-consistent?
Read Lab 0's OSZICAR as an SCF history, then predict what a tighter EDIFF does.
Read (no run)
- Count the electronic steps, the lines before
F=. - The first five keep the starting Hamiltonian fixed (NELMDL).
- From step six on, check that
dEandd epsshrink, by a roughly steady factor once the loop settles, until both are below EDIFF = 10⁻⁶ eV. - In the OUTCAR, find EBANDS, DENC and XCENC in the last energy block.
Predict, then run (optional)
- Rerun with
EDIFF = 1E-8. - Predict: only a few more steps; E0 changes only in the last digits.
DFT & VASP Course · Lecture 617 / 20
Lab 0 ran from scratch, without a WAVECAR and with the default algorithm. Its first five electronic steps therefore kept the initial Hamiltonian, built from atomic charge densities, fixed while the random starting orbitals settled. From step six on, each step rebuilt the Hamiltonian from the new density: an SCF iteration like those of the helium figure.
If the energy changes shrink by a roughly constant factor per step, the loop converges steadily, and tightening EDIFF by two orders of magnitude costs a predictable handful of extra steps: with a factor of 10 per step, two more; with a factor of 3, about four. Read the factor off your OSZICAR before you rerun, and turn the prediction into a number. Lecture 12 explains density mixing, which damps unstable density updates and helps convergence in solids.
Which misconceptions should you drop now?
Six beliefs about Hartree–Fock and the SCF loop. Click a card to turn it over.
“Hartree–Fock ignores electron repulsion.”
It includes the average repulsion and exact exchange. It misses only correlation.
“Self-consistency means the answer is exact.”
It means the orbitals reproduce their own field. The basis and the method can still be in error.
“1% of the energy is negligible.”
For helium it is 1.14 eV, and without it F₂ is unbound. Chemistry lives in such differences.
“The total energy is the sum of the orbital energies.”
That sum counts each pair's repulsion twice. Subtract it once (VASP: DENC, XCENC).
“HF and Kohn–Sham orbital energies mean the same.”
HF ones estimate ionization energies (Koopmans); Kohn–Sham ones in general do not.
“DFT is Hartree–Fock with a correction.”
It shares the SCF machinery, but its determinant describes an auxiliary system, and one functional gives exchange and correlation.
DFT & VASP Course · Lecture 618 / 20
Check yourself
One question per outcome. Answer aloud, then click to compare.
Where does exchange come from, and what is in the Fock operator?
From the swapped term of the determinant, for equal spins only. f: kinetic energy, nuclear attraction, average Coulomb and exchange.
Why is there no self-interaction in Hartree–Fock?
For an orbital with itself, K_aa = J_aa, so exchange cancels each self-repulsion term in the Hartree energy.
Why must FC = SCε be iterated?
F is built from the coefficients it is meant to produce. Iterate until they agree: 7 steps for helium.
Helium: −ε₁ₛ = 24.98 eV, 2ε₁ₛ = −1.836 Ha. What do they mean?
−ε is Koopmans' ionization energy (exact 24.59 eV). 2ε double-counts J; E = 2ε − J = −2.862 Ha.
Define the correlation energy and give helium's.
E_corr = E_exact − E_HF, never positive: −0.042 Ha, or −1.14 eV, for helium.
DFT & VASP Course · Lecture 619 / 20
What should you remember?
The best single determinant, found self-consistently, misses only correlation, and that still matters.
- Hartree–Fock minimizes the energy of one determinant. Antisymmetry adds exchange between equal spins; each electron moves in the mean field of the Fock operator.
- Exchange cancels each electron's self-repulsion exactly; approximate DFT does not quite.
- In a basis, FC = SCε with F built from C: iterate to self-consistency, the ancestor of VASP's electronic loop.
- −ε of an occupied orbital estimates an ionization energy (Koopmans); the sum of the ε counts the repulsion twice.
- E_corr = E_exact − E_HF: −1.14 eV for helium, about 1%, yet enough to decide whether F₂ is bound. DFT's E_xc must supply it.
Next: Part II, starting with Lecture 7, replaces the wavefunction by the electron density.
DFT & VASP Course · Lecture 620 / 20
Key terms
- Self-consistent field (SCF)
- The iterative loop in which the density determines the potential, the potential determines the orbitals, and the orbitals give a new density, repeated until input and output agree.
- Spin-orbital
- A one-electron function of position and spin, χ(x)=ϕ(r)σ(s): a spatial orbital times a spin function, α (up) or β (down).
- Variational principle
- For any allowed trial function ϕ, ⟨ϕ∣H^∣ϕ⟩/⟨ϕ∣ϕ⟩≥E0, the exact ground-state energy; equality only for the ground state. Lowering the energy over a trial family therefore improves the estimate of the ground-state energy.
- Nonlocal operator
- An operator whose output at one position depends on the input function at other positions, commonly through an integral kernel. Hartree–Fock exchange is nonlocal; multiplication by a potential V(r) is local.
- Fock operator
- The effective one-electron operator of Hartree–Fock theory: kinetic energy, attraction to the nuclei, and the Coulomb and exchange operators built from the occupied orbitals. In a basis it is the Fock matrix, and the equations read FC = SCε.
- Mean field
- An approximation in which each electron feels the average field of all the others rather than their instantaneous positions. Hartree–Fock and Kohn–Sham theory are both mean-field methods.
- Self-interaction error
- The spurious interaction of an electron with its own charge that remains in approximate functionals, because their exchange–correlation energy does not exactly cancel the electron's Coulomb energy with itself.
- Basis set
- A set of known functions ϕj in which an unknown state is expanded, ψ=∑jcjϕj. It turns H^ψ=Eψ into a matrix problem for the coefficients; a finite basis gives an approximate answer. VASP's basis is plane waves.
- Koopmans' theorem
- For a finite system with a vacuum energy reference, minus an occupied Hartree–Fock orbital energy equals its frozen-orbital removal energy. Neglecting relaxation and correlation makes it an approximation to a measured ionization energy.
- Correlation energy
- Ecorr=Eexact−EHF≤0 for the same Hamiltonian and complete-basis HF reference. Zero for one electron; generally negative for interacting many-electron systems. Its fraction of total energy is system-dependent. Kohn–Sham DFT defines correlation relative to a different noninteracting reference.
- Electron correlation
- The way electrons of either spin avoid each other because they repel, beyond what a single Slater determinant describes. The correlation energy is the difference between the exact and the Hartree–Fock energy.
VASP tags in this lecture
EDIFF- Stopping criterion of the electronic self-consistency loop, in eV. The loop stops when both the change of the free energy and the change of the band-structure energy between two steps fall below EDIFF.Default: 10−4 eV · VASP Wiki: EDIFF
NELM- Maximum number of electronic self-consistency steps.Default: 60 · VASP Wiki: NELM
NELMDL- Number of non-self-consistent steps at the start, during which the Hamiltonian built from the initial density is kept fixed while the orbitals settle. A negative value applies the delay only in the first ionic step.Default: −12 if ISTART = 0, INIWAV = 1 and IALGO = 48 or 50; 0 if a WAVECAR is present; −5 otherwise · VASP Wiki: NELMDL
LHFCALC- Switches on a Hartree–Fock/DFT hybrid-functional calculation with exact exchange; without further tags VASP uses the PBE0 hybrid.Default: .FALSE. · VASP Wiki: LHFCALC
AEXX- Fraction of exact (Hartree–Fock) exchange in a hybrid-functional calculation. With AEXX = 1.0 VASP switches off correlation and runs a full Hartree–Fock calculation.Default: 0.25 with LHFCALC = .TRUE.; 0 otherwise · VASP Wiki: AEXX
Further reading
- 5.61 Physical Chemistry, Lecture 29: “Modern electronic structure theory: electron correlation” (Fall 2017, instructor R. W. Field; notes by T. Van Voorhis, used with permission), MIT OpenCourseWare, CC BY-NC-SA. · MIT OCW
- VASP Wiki, “LHFCALC”. · vasp.at/wiki