Neocortix Tutorial

In this tutorial, we will show you how to use Neocortix Cloud Services Scalable Compute to run a distributed batch job with a Binary Executable, in a framework where you can easily build and substitute your own Binary Executable. We will use a simple Go program to create the Binary Executable in our example.

First, please follow the steps in the tutorial Setting Up For Batch Jobs. After completion of the initial setup, you will have a directory with examples,

~/ncsexamples

Building the Binary Executable for Arm (aarch64)

In that repository in the subdirectory

~/ncsexamples/batchMode

you will find the Go source file

helloFrameGo.go

. To build the Binary Executable

helloFrameGo

, please follow these steps. First, please install the Go compiler tools for your machine, from https://golang.org/doc/install.

Next, build the aarch64 executable

helloFrameGo_aarch64

with this command:

GOARCH=arm64 GOOS=linux go build -o helloFrameGo_aarch64 helloFrameGo.go

Now you will have the binary executable file

helloFrameGo_aarch64

In the subdirectory

~/ncsexamples/batchMode

you will find the runBatchBinaryGo.py command. This script will create a set of instances running on mobile devices, one instance for each frame to be computed (3 in the default example provided).

startFrame = 1,
endFrame = 3,

It will command the instances to receive the binary executable file

helloFrameGo_aarch64

, then run the executable:

workerBinFilePath = 'helloFrameGo_aarch64'  # binary to copy to instance
commonInFilePath = binaryFrameProcessor.workerBinFilePath,
def frameCmd( self, frameNum ):
    workerBinFileName = os.path.basename( self.workerBinFilePath )
    cmd = './%s %d > %s' % \
        (workerBinFileName, frameNum, self.frameOutFileName(frameNum))
    return cmd

The output is a text file

frame_<n>.out

containing

"Hello World! From frameNum = n"

These output files will be sent back to the master host, and the master will then terminate the instances.

Simply run

python3 ./runBatchBinaryGo.py

When the program is done, the output files

frame_<n>.out

will be put in a directory

./data/binary_<datestamp>

Each file will contain the text output

Hello World! From frameNum = n

Building and Substituting Your Own Binary Executable Code

With this simple framework, you can easily build and substitute your own Binary Executable to process the frames of the Batch job. Starting with your own Go source file

yourFile.go

, just follow the instructions above to build the binary executable

yourFile_aarch64

go build -o yourFile_aarch64 yourFile.go

Now you will have the binary executable file

yourFile_aarch64

Just edit the

runBatchBinaryGo.py

file to change the

workerBinFilePath

to replace

helloFrameGo_aarch64

with the name of your binary executable

yourFile_aarch64

, which runs on an instance and processes a single frame. You may need to install different dependencies in the

installerCmd

line.

Pro Tip: When you are experimenting with getting your code running on a Neocortix Scalable Compute instance, it may be easiest to spin up a single instance and get it installed and running there, before running it at scale as a part of a Batch job.

Building the Binary Executable for Arm (aarch64)

Example Command

Building and Substituting Your Own Binary Executable Code