Load Test Output
In this tutorial, we will show you how to use Neocortix Cloud Services Scalable Compute to run a distributed batch Python job, in a framework where you can easily substitute your own batch Python code.
First, please follow the steps in the tutorial Setting Up For Batch Jobs. After completion of the initial setup, you will have a directory with examples,
~/ncsexamples
.
In the subdirectory
~/ncsexamples/batchMode
you will find the runBatchPython.py command. This script creates a set of instances running on mobile devices, one instance for each frame to be computed (10 in the default example provided).
startFrame = 1, endFrame = 10,
It will command the instances to install some software, then command each instance to run the simple example Python code supplied in file
plotFrame.py
, which will generate simple plots, passing in a frame number to allow each instance to plot a specific frame.
workerScriptPath = 'plotFrame.py' def frameCmd( self, frameNum ): pythonFileName = os.path.basename( self.workerScriptPath ) cmd = 'python3 %s %d' % \ (pythonFileName, frameNum) return cmd
The output of each instance will be a .png file
sine_<n>.png
containing a simple plot. These files will be sent back to the master host, and the master will then terminate the instances.

Example Command

Simply run
python3 ./runBatchPython.py
When the program is done, the output plot files will be put in a directory
./data/python_<datestamp>

Substituting Your Own Python Batch Code

With this simple framework, you can easily substitute your own Python Batch Code to process the frames of the Batch job. Just edit the
runBatchPython.py
file to change the
workerScriptPath
to replace
plot_sine_001.py
with the name of your Python file, which runs on an instance and processes a single frame. You may need to install different dependencies in the
installerCmd
line.
Pro Tip #1: When you are experimenting with getting your code running on a Neocortix Scalable Compute instance, it may be easiest to spin up a single instance and get it installed and running there, before running it at scale as a part of a Batch job.
Pro Tip #2: If the code you want to run on each instance is not a simple Python program, as in our example, but it is a binary executable from C or C++ sources, for example, then it will be necessary to build the executable for Arm aarch64 architecture. Please see our tutorial Running a Batch Job with Binary Executable (C) for a simple example of how to do this.