# CS 5220 ## Parallelism and locality in simulation ### Particle systems ## 15 Sep 2015
### Particle simulation Particles move via Newton ($F = ma$), with - External forces: ambient gravity, currents, etc. - Local forces: collisions, Van der Waals ($1/r^6$), etc. - Far-field forces: gravity and electrostatics ($1/r^2$), etc. - Simple approximations often apply (Saint-Venant)

### A forced example

Example force: $$f_i = \sum_j Gm_i m_j \frac{(x_j-x_i)}{r_{ij}^3} \left(1 - \left(\frac{a}{r_{ij}}\right)^{4} \right), \qquad r_{ij} = \|x_i-x_j\|$$

• Long-range attractive force ($r^{-2}$)
• Short-range repulsive force ($r^{-6}$)
• Go from attraction to repulsion at radius $a$
### A simple serial simulation In Matlab, we can write npts = 100; t = linspace(0, tfinal, npts); [tout, xyv] = ode113(@fnbody, ... t, [x; v], [], m, g); xout = xyv(:,1:length(x))'; ... but I can’t call ode113 in C in parallel (or can I?)
### A simple serial simulation Maybe a fixed step leapfrog will do? npts = 100; steps_per_pt = 10; dt = tfinal/(steps_per_pt*(npts-1)); xout = zeros(2*n, npts); xout(:,1) = x; for i = 1:npts-1 for ii = 1:steps_per_pt x = x + v*dt; a = fnbody(x, m, g); v = v + a*dt; end xout(:,i+1) = x; end
### Pondering particles - Where do particles “live” (esp. in distributed memory)? - Decompose in space? By particle number? - What about clumping? - How are long-range force computations organized? - How are short-range force computations organized? - How is force computation load balanced? - What are the boundary conditions? - How are potential singularities handled? - What integrator is used? What step control?
### External forces Simplest case: no particle interactions. - Embarrassingly parallel (like Monte Carlo)! - Could just split particles evenly across processors - Is it that easy? - Maybe some trajectories need short time steps? - Even with MC, load balance may not be entirely trivial.

### Local forces

• Simplest all-pairs check is $O(n^2)$ (expensive)
• Or only check close pairs (via binning, quadtrees?)
• Communication required for pairs checked
• Usual model: domain decomposition

### Local forces: Communication

Minimize communication:

• Send particles that might affect a neighbor soon
• Trade extra computation against communication
• Want low surface area-to-volume ratios on domains

### Local forces: Load balance

• Are particles evenly distributed?
• Do particles remain evenly distributed?
• Can divide space unevenly (e.g. quadtree/octtree)

### Far-field forces

• Every particle affects every other particle
• All-to-all communication required
• Overlap communication with computation
• Poor memory scaling if everyone keeps everything!
• Idea: pass particles in a round-robin manner

### Passing particles for far-field forces


copy local particles to current buf
for phase = 1:p
send current buf to rank+1 (mod p)
recv next buf from rank-1 (mod p)
interact local particles with current buf
swap current buf with next buf
end


### Passing particles for far-field forces

Suppose $n = N/p$ particles in buffer. At each phase \begin{align} t_{\mathrm{comm}} &\approx \alpha + \beta n \\ t_{\mathrm{comp}} &\approx \gamma n^2 \end{align} So we can mask communication with computation if $$n \geq \frac{1}{2\gamma} \left( \beta + \sqrt{\beta^2 + 4 \alpha \gamma} \right) > \frac{\beta}{\gamma}$$

More efficient serial code
$\implies$ larger $n$ needed to mask communication!
$\implies$ worse speed-up as $p$ gets larger (fixed $N$)
but scaled speed-up ($n$ fixed) remains unchanged.

### Far-field forces: particle-mesh methods

Consider $r^{-2}$ electrostatic potential interaction

• Enough charges looks like a continuum!
• Poisson equation maps charge distribution to potential
• Use fast Poisson solvers for regular grids (FFT, multigrid)
• Approximation depends on mesh and particle density
• Can clean up leading part of approximation error

### Far-field forces: particle-mesh methods

• Map particles to mesh points (multiple strategies)
• Solve potential PDE on mesh
• Interpolate potential to particles
• Add correction term – acts like local force

### Far-field forces: tree methods

• Distance simplifies things
• Andromeda looks like a point mass from here?
• Build a tree, approximating descendants at each node
• Several variants: Barnes-Hut, FMM, Anderson’s method
• More on this later in the semester
### Summary of particle example - Model: Continuous motion of particles - Could be electrons, cars, whatever... - Step through discretized time - Local interactions - Relatively cheap - Load balance a pain - All-pairs interactions - Obvious algorithm is expensive ($O(n^2)$) - Particle-mesh and tree-based algorithms help An important special case of lumped/ODE models.