Neocortix Tutorial

In this tutorial, we will show you how to use Neocortix Cloud Services Scalable Compute to run a distributed batch job with a Java Jar-file.

First, please follow the steps in the tutorial Setting Up For Batch Jobs. After completion of the initial setup, you will have a directory with examples,

~/ncsexamples

Building the Java Jar-file

In the subdirectory

~/ncsexamples/batchMode

you will find the Java source file

helloFrame.java

. To build the Java Jar-file

helloFrame.jar

, please follow these steps. First, please install the headless JDK tools:

./installJdk11.sh

Then build the Java Jar-file

helloFrame.jar

with these commands:

javac -d helloFrame-bin helloFrame.java
jar -cvfe helloFrame.jar com.example.helloFrame -C helloFrame-bin/ com/example

Now you will have the Java Jar-file

helloFrame.jar

In the subdirectory

~/ncsexamples/batchMode

you will find the runBatchJava.py command. This script will create a set of instances running on mobile devices, one instance for each frame to be computed (3 in the default example provided).

startFrame = 1,
endFrame = 3,

It will command the instances to receive the Java Jar-file

helloFrame.jar

, then run it:

workerBinFilePath = 'helloFrame.jar'  # Jar-file to copy to instance
commonInFilePath = binaryFrameProcessor.workerBinFilePath,
def frameCmd( self, frameNum ):
    workerBinFileName = os.path.basename( self.workerBinFilePath )
    cmd = 'java -jar ./%s %d > %s' % \
        (workerBinFileName, frameNum, self.frameOutFileName(frameNum))
    return cmd

The output is a text file

frame_<n>.out

containing

"Hello World! From frameNum = n"

These output files will be sent back to the master host, and the master will then terminate the instances.

Simply run

python3 ./runBatchJava.py

When the program is done, the output files

frame_<n>.out

will be put in a directory

./data/java_<datestamp>

Each file will contain the text output

Hello World! From frameNum = n

Building and Substituting Your Own Java Jar-file

With this simple framework, you can easily build and substitute your own Java Jar-file to process the frames of the Batch job. Starting with your own Java source file

yourFile.java

, just follow the instructions above to build the Java Jar-file

yourFile.jar

javac -d yourFile-bin yourFile.java
jar -cvfe yourFile.jar com.example.yourFile -C yourFile-bin/ com/example

Now you will have the Java Jar-file

yourFile.jar

Just edit the

runBatchJava.py

file to change the

workerBinFilePath

to replace

helloFrame.jar

with the name of your Java Jar-file

yourFile.jar

, which runs on an instance and processes a single frame. You may need to install different dependencies in the

installerCmd

line.

Pro Tip: When you are experimenting with getting your code running on a Neocortix Scalable Compute instance, it may be easiest to spin up a single instance and get it installed and running there, before running it at scale as a part of a Batch job.

Building the Java Jar-file

Example Command

Building and Substituting Your Own Java Jar-file