Distributed computing with Julia
Julia provides built-in language functionality to run a program across many processes that can run locally, across a distributed network, or in a computational cluster. In the Multiprocessing with Julia and Sending parameters to remote Julia processes recipes, we saw how can you run data and exchange data between Julia processes.
A typical scenario for distributed computing is running a parameter sweep over a significantly large set of computations. In this recipe, we will show how to create a distributed cluster that performs a parameter sweep over a numerical simulation model.
We explain how to use the--machine-file
Julia options to run Julia workers across many nodes. However, the computational example can also be run on a single machine using the multiprocessing mode (for example, in Julia launched with thejulia -p 4
command).
Getting ready
In this example, we will run a distributed cluster in Julia. You can run the cluster on a single laptop or across a cluster...