In addition to expressing pure maps, you can use fast collective communication operations between devices: from functools import partial from jax import lax @partial(pmap, axis_name='i') def normalize(x): return x / lax.psum(x, 'i') print(normalize(jnp.arange(4.))) # prints [0. 0.16...
In addition to expressing pure maps, you can use fastcollective communication operationsbetween devices: fromfunctoolsimportpartialfromjaximportlax@partial(pmap,axis_name='i')defnormalize(x):returnx/lax.psum(x,'i')print(normalize(jnp.arange(4.)))# prints [0. 0.16666667 0.33333334 0.5 ] ...