PWscf

about PWscf

methodological background

user's guide

download PWscf

tests and examples

pseudopotentials

scientific literature

USER'S GUIDE 1.1> Performance Issues

Performance Issues

Subsections

CPU time requirements
Memory requirements
Parallelization issues

	CPU time requirements

The following holds for code pw.x and for non-US PPs. For US PPs there are additional terms to be calculated. For phonon calculations, each of the 3*N_at mode requires a CPU time of the same order of that required by a self-consistent calculation in the same system.

The computer time required for the self-consistent solution at fixed ionic positions, T_scf, is:

T_scf = N_iter*T_iter + T_init

where N_iter=niter=number of self-consistency iterations, T_iter=CPU time for a single iteration, T_sub=initialization time for a single iteration. Usually T_init < < N_iter*T_iter.

The time required for a single self-consistency iteration T_iter is:

T_iter = N_k*T_diag + T_rho + T_scf

(1)

here N_k=number of k-points, T_diag=CPU time per hamiltonian iterative diagonalization, T_rho=cpu time for charge density calculation. T_scf=CPU time for Hartree and exchange-correlation potential calculation.

The time for a hamiltonian iterative diagonalization T_diag is:

T_diag = N_h*T_h + T_orth + T_sub

(2)

here N_h=number of H $\psi$ products needed by iterative diagonalization, T_h=CPU time per H $\psi$ product, T_orth=CPU time for orthonormalization, T_sub=CPU time for subspace diagonalization.

The time T_h required for a H $\psi$ product is

Th = a₁*M*N + a₂*M*N₁*N₂*N₃*log(N₁*N₂*N₃) + a₃*M*P*N.

(3)

he first term comes from the kinetic term and is usually much smaller than the others. The second and third terms come respectively from local and nonlocal potential. a₁, a₂, a₃ are prefactors, M=number of valence bands, N=number of plane waves (basis set dimension), N₁, N₂, N₃=dimensions of the FFT grid for wavefunctions ( N₁*N₂*N₃ $\sim$ 8N), P=number of projectors for PPs (summed on all atoms, on all values of the angular momentum l, and m=1,..,2l+1)

The time T_orth required by orthonormalization is

T_orth = b₁*M_x²*N

(4)

and the time T_sub required by subspace diagonalization is

T_sub = b₂*M_x³

(5)

here b₁ and b₂ are prefactors, M_x=number of trial wavefunctions (this will vary between M and a few times M, depending on the algorithm)

The time T_rho for the calculation of charge density from wavefunctions is

T_rho = c₁*M*Nr₁*Nr₂*Nr₃*log(Nr₁*Nr₂*Nr₃) + c₂*M*Nr₁*Nr₂*Nr₃ + T_us

(6)

here c₁, c₂, c₃ are prefactors, Nr₁, Nr₂, Nr₃=dimensions of the FFT grid for charge density ( Nr₁*Nr₂*Nr₃ $\sim$ 8N_g, where N_g=number of G-vectors for the charge density), and T_us=CPU time required by ultrasoft contribution (if any).

The time T_scf for calculation of potential from charge density is

T_scf = d₂*Nr₁*Nr₂*Nr₃ + d₃*Nr₁*Nr₂*Nr₃*log(Nr₁*Nr₂*Nr₃)

(7)

here d₁, d₂ are prefactors.

TOP

	Memory requirements

A typical pw.x run will require a maximum memory in the order of O double precision complex numbers, where

O = m*M*N + M*P + p*N₁*N₂*N₃ + q*Nr₁*Nr₂*Nr₃

(8)

ith m, p, q=small factors, all other variables have the same meaning as above.

This holds for the phonon code as well, with larger factors m, p, q.

TOP

	Parallelization issues

The program can run in principle on any number of processors (up to maxproc, presently fixed at 128 in para/para.inc). The N_p processors can be divided into N_pk pools of N_pr processors, N_p = N_pk*N_pr. The k-points are divided across N_pk pools ("k-point parallelization"), while both R- and G-space grids are divided across the N_pr processors of each pool ("PW parallelization"). At present there is no parallelization on the number of bands.

Note that if you restart or read data from a preceding run you must restart with exactly the same number of processors and the same number of pools. The only exception is when a SCF calculation is followed by a band-structure calculation. In this case the only link between the two is the potential file that does not depend on the number of pools and processors.

The effectiveness of parallelization depends on the size and type of the system and on a judicious choice of the N_pk and N_pr:

k-point parallelization is very effective if N_pk is a divisor of the number of k-points (linear speedup guaranteed), BUT it does not reduce the amount of memory per processor taken by the calculation. As a consequence large systems may not fit into memory.
PW parallelization works well if N_pr is a divisor of both dimensions along the z axis of the FFT grids, N₃ and Nr₃ (which may coincide). It does not scale so well as k-point parallelization, but it reduces both cpu time AND memory (the latter almost linearly).
Optimal scalar performances are achieved when the data are as much as possible kept into the cache. This is very important for SGI Origin, less so for IBM machines. As a side effect, one can achieve better than linear scaling with the number of processors, thanks to the increase in scalar speed coming from the reduction of data size (making it is easier for the machine to keep data in the cache).

TOP