Hacking the Dream Cheeky Thunder

A couple of weeks ago, Atmel tweeted about some people that hacked the Dream Cheeky Thunder Missile Launcher by soldering on a Ardunino to the circuit board.

image

A quick Google search shows there are lots of people who done something similar, including this post.  Since I was not interested in messing around wit the circuit board, I decided to go the software hack route.  When the missile launcher arrived, I downloaded software and installed it on my Windows 7 machine.  I then used Telerik’s JustDecompile software to look at the source code.

image

Fortunately, the main executable is a .NET 2.0 Windows Form application.  Unfortunately, the code is a mess and it relies on user controls.  Specifically, the library that the .NET .exe consumes is called USBLib.dll which is a 23-bit Com component that creates a windows control that the main .exe uses.

When I took the code from JustDecompile and stuck it into Visual Studio (no F# option so I went with C#), it took about 2-3 hours to get all of the references set up, the resources set up, and the embedded code put into the right location, but I did manage to get a working Visual Studio solution

image

I then decided to build a brand new solution that controls the missile launcher without the graphical components that are baked into the app.  I added a reference to the USBLib.dll and then tried to make method calls.  No luck, it looks like the application uses Windows Event hooks to call and respond:

protected override void WndProc(ref Message m) { this.USB.ParseMessages(ref m); if (m.Msg == SingleProgramInstance.WakeupMessage) { if (base.WindowState == FormWindowState.Minimized) { base.Visible = true; base.WindowState = FormWindowState.Normal; } base.Activate(); } base.WndProc(ref m); }

Yuck!  I then did a quick search on Google to find a .NETUsbDriver that I could use because I already have the byte array values that the missile launcher is expected:

I found this but the suggestions did not compile and/or did not work.  I then found this site which in the right direction.  I added the code into my project like so:

image

I then wired up the main form like this:

image

With the code behind like this:

MissileLauncher _launcher = new MissileLauncher(); public Form1() { InitializeComponent(); _launcher.command_reset(); _launcher.command_switchLED(true); } private void upButton_Click(object sender, EventArgs e) { _launcher.command_Up(2000); } private void fireButton_Click(object sender, EventArgs e) { _launcher.command_Fire(); } private void downButton_Click(object sender, EventArgs e) { _launcher.command_Down(1000); } private void rightButton_Click(object sender, EventArgs e) { _launcher.command_Right(3000); } private void leftButton_Click(object sender, EventArgs e) { _launcher.command_Left(3000); }

Then, when I run it, it works like a champ.  I now just need to translate the values passed into the Thread.Sleep() and have it correspond to the angles.  The author of the code was on the right track because s/he named the parameter “degree”. 

In the meantime, I ported the code to FSharp.  You can see it here and you can see the missle launcher in action here.  The major difference is that the C# code had 182 lines and the F# code has 83.

Advertisements

Consuming and Analyzing Census Data Using F#

As part of my Nerd Dinner refactoring, I wanted to add the ability to guess a person’s age and gender based on their name.  I did a quick search on the internet and the only place that I found that has an API is here and it doesn’t have everything I am looking for.  Fortunately, the US Census website has some flat files with the kind of data I am looking for here.

I grabbed the data and  pumped it into Azure Blob Storage here.  You can swap out the state code to get each dataset.  I then loaded in a list of State Codes found here that match to the file names.

I then fired up Visual Studio and created a new FSharp project.  I added FSharp.Data to use a Type Provider to access the data.  I don’t need to install the Azure Storage .dlls b/c the blobs are public and I just have to read the file

image

Once Nuget was done with its magic, I opened up the script file, pointed to the newly-installed FSharp.Data, and added a reference to the datasets on blob storage:

#r "../packages/FSharp.Data.2.0.9/lib/portable-net40+sl5+wp8+win8/FSharp.Data.dll" open FSharp.Data type censusDataContext = CsvProvider<"https://portalvhdspgzl51prtcpfj.blob.core.windows.net/censuschicken/AK.TXT"> type stateCodeContext = CsvProvider<"https://portalvhdspgzl51prtcpfj.blob.core.windows.net/censuschicken/states.csv">

(Note that I am going add FSharp as a language to my Live Writer code snippet add-in at a later date)

In any event, I then printed out all of the codes to see what it looks like:

let stateCodes = stateCodeContext.Load("https://portalvhdspgzl51prtcpfj.blob.core.windows.net/censuschicken/states.csv"); stateCodes.Rows |> Seq.iter(fun r -> printfn "%A" r)

image

And by changing the lambda slightly like so,

stateCodes.Rows |> Seq.iter(fun r -> printfn "%A" r.Abbreviation)

I get all of the state codes

image

I then tested the census data with code and results are expected

let arkansasData = censusDataContext.Load("https://portalvhdspgzl51prtcpfj.blob.core.windows.net/censuschicken/AK.TXT"); arkansasData.Rows |> Seq.iter(fun r -> printfn "%A" r)

image

So then I created a method to load all of the state census data and giving me the length of the total:

let stateCodes = stateCodeContext.Load("https://portalvhdspgzl51prtcpfj.blob.core.windows.net/censuschicken/states.csv"); let usaData = stateCodes.Rows |> Seq.collect(fun r -> censusDataContext.Load(System.String.Format("https://portalvhdspgzl51prtcpfj.blob.core.windows.net/censuschicken/{0}.TXT",r.Abbreviation)).Rows) |> Seq.length

image

Since this is a I/O bound operation, it made sense to load the data asynchronously, which speeded things up considerably.  You can see my question over on Stack Overflow here and the resulting code takes about 50% of the time on a my dual-processor machine:

stopwatch.Start() let fetchStateDataAsync(stateCode:string)= async{ let uri = System.String.Format("https://portalvhdspgzl51prtcpfj.blob.core.windows.net/censuschicken/{0}.TXT",stateCode) let! stateData = censusDataContext.AsyncLoad(uri) return stateData.Rows } let usaData' = stateCodes.Rows |> Seq.map(fun r -> fetchStateDataAsync(r.Abbreviation)) |> Async.Parallel |> Async.RunSynchronously |> Seq.collect id |> Seq.length stopwatch.Stop() printfn "Parallel: %A" stopwatch.Elapsed.Seconds

image

With the data in hand, it was time to analyze the data to see if there is anything we can do.   Since 23 seconds is a bit too long to wait for a page load (Smile), I will need to put the 5.5 million records into a format that can be easily searched.  Thinking what we want is:

Given a name, what is the gender?

Given a name, what is the age?

Given a name, what is their state of birth?

Also, since we have their current location, we can also input the name and location and answer those questions.  If we make the assumption that their location is the same as their birth state, we can narrow down the list even further.

In any event, I first added a GroupBy to the name:

let nameSum = usaData' |> Seq.groupBy(fun r -> r.Mary) |> Seq.toArray

image

And then I summed up the counts of the names

let nameSum = usaData' |> Seq.groupBy(fun r -> r.Mary) |> Seq.map(fun (n,a) -> n,a |> Seq.sumBy(fun (r) -> r.``14``)) |> Seq.toArray

image

And then the total in the set:

let totalNames = nameSum |> Seq.sumBy(fun (n,c) -> c)

image

And then applied a simple average and sorted it descending

let nameAverage = nameSum |> Seq.map(fun (n,c) -> n,c,float c/ float totalNames) |> Seq.sortBy(fun (n,c,a) -> -a - 1.) |> Seq.toArray

image

So I feel really special that my parents gave me the most popular name in the US ever…

And focusing back to the task on hand, I want to determine the probability that a person is male or female based on their name:

let nameSearch = usaData' |> Seq.filter(fun r -> r.Mary = "James") |> Seq.groupBy(fun r -> r.F) |> Seq.map(fun (n,a) -> n,a |> Seq.sumBy(fun (r) -> r.``14``)) |> Seq.toArray

image

So 18196 parents thought is would be a good idea to name their daughter ‘James’.  I created a quick function like so:

let nameSearch' name = let nameFilter = usaData' |> Seq.filter(fun r -> r.Mary = name) |> Seq.groupBy(fun r -> r.F) |> Seq.map(fun (n,a) -> n,a |> Seq.sumBy(fun (r) -> r.``14``)) let nameSum = nameFilter |> Seq.sumBy(fun (n,c) -> c) nameFilter |> Seq.map(fun (n,c) -> n, c, float c/float nameSum) |> Seq.toArray nameSearch' "James"

image

So if I see the name “James”, there is a 99% chance it is a male.  This can lead to a whole host of questions like variance of names, names that are closest to gender neutral, etc….  Leaving those questions to another day, I now have something I can put into Nerd Dinner.  Now, if there was only a way to handle nicknames and friendly names….

You can see the full code here.

 

 

 

 

 

 

Controlling Servos Using Netdunio and Phidgets

As part of the Terminator program I am creating, I need a way of controlling servos to point the laser (and then gun) and different targets.  I decided to create a POC project and evaluate two different ways of controlling the servos.  As step one, I purchased a pan and tilt chassis from here

image

After playing with the servos from the kit, I decided to use my old stand-by servos that had a much higher quality and whose PWM signals I already know how to use.  With the chassis done, I needed a laser pointer so I figured why not get a shark with fricken laser?

I found one here.

image

So with the servos and laser ready to go, it was time to code.  I started with Netduninos:

public class Program { private const uint TILT_SERVO_STRAIGHT = 1500; private const uint TILT_SERVO_MAX_UP = 2000; private const uint TILT_SERVO_MAX_DOWN = 1000; private const uint PAN_SERVO_STRAIGHT = 1500; private const uint PAN_SERVO_MAX_LEFT = 1000; private const uint PAN_SERVO_MAX_RIGHT = 2000; private static PWM _tiltServo = null; private static PWM _panServo = null; private static uint _tiltServoCurrentPosition = 0; private static uint _panServoCurrentPosition = 0; public static void Main() { SetUpServos(); InputPort button = new InputPort(Pins.ONBOARD_BTN, false, Port.ResistorMode.Disabled); while (true) { if (button.Read()) { MoveServo(); } } } private static void SetUpServos() { uint period = 20000; _tiltServoCurrentPosition = TILT_SERVO_STRAIGHT; _panServoCurrentPosition = PAN_SERVO_STRAIGHT; _tiltServo = new PWM(PWMChannels.PWM_PIN_D3, period, _tiltServoCurrentPosition, PWM.ScaleFactor.Microseconds, false); _tiltServo.Start(); _panServo = new PWM(PWMChannels.PWM_PIN_D5, period, _panServoCurrentPosition, PWM.ScaleFactor.Microseconds, false); _panServo.Start(); } private static void MoveServo() { _panServo.Duration = PAN_SERVO_MAX_LEFT; Thread.Sleep(2000); _panServo.Duration = PAN_SERVO_MAX_RIGHT; Thread.Sleep(2000); _panServo.Duration = PAN_SERVO_STRAIGHT; Thread.Sleep(2000); _tiltServo.Duration = TILT_SERVO_MAX_UP; Thread.Sleep(2000); _tiltServo.Duration = TILT_SERVO_MAX_DOWN; Thread.Sleep(2000); _tiltServo.Duration = TILT_SERVO_STRAIGHT; } }

And sure enough the servos are behaving as expected

I then implemented a similar app using Phidgets.  Because the code is being executed on the PC, I could use F# to code (It does not look like the Netdunino/Microframework supports F#?)

open System open Phidgets let _servoController = new AdvancedServo() let mutable _isServoControllerReady = false let servoController_Attached(args:Events.AttachEventArgs) = let servoController = args.Device :?> AdvancedServo servoController.servos.[0].Engaged <- true servoController.servos.[7].Engaged <- true _isServoControllerReady <- true [<EntryPoint>] let main argv = _servoController.Attach.Add(servoController_Attached) _servoController.``open``() while true do if _isServoControllerReady = true then _servoController.servos.[0].Position<- 100. _servoController.servos.[7].Position<- 100. Console.ReadKey() |> ignore printfn "%A" argv 0

 

The choice then becomes using the Netduino or the Phidgets with my Kinect program.  I decided to defer the decision and use an interface for now.

type IWeaponsSystem = abstract member Activate: unit -> unit abstract member AquireTarget : float*float -> bool abstract member Fire: int -> bool

My decision about using Phidgets or Netduino is a series of trade-offs.  I can code Phidgets in C# or F# but I have to code Netduino in C#.  I would prefer to do this in F# so that makes me learn towards Phidgets.  I can put the Netduino anywhere and have it communicate via an Ethernet signal but I have to have the Phidgets wired to the PC.  Since the targeting system needs to be near the Kinect and the Kinect has to be tethered to the PC also, there is no real advantage of using the mobile Netduino.  Finally, the Phidgets API handles all communication to the servo control board for me, with the Netduino I would have to hook up a router to the Netduino and write the Ethernet communication code.  So I am leaning towards Phidgets, but since I am not sure, the interface allows me to swap in the Netduino at a later point without changing any code.  Love me some O in SOLID…

Up next, integrating the targeting system into the Terminator program.

 

 

Neural Network Part 3: Perceptrons

I went back to my code for building a Perceptron and I made some changes.  I realized that although McCaffrey combines the code together, there are actually two actions for the perceptron: training and predicting. I created a diagram to help me keep the functions that I need for each in mind:

image

I also skeletoned out some data structures that I think I need:

image

With the base diagrams out of the way, I created different data structures that were tailored to each action.   These are a bit different than the diagrams –> I didn’t go back and update the diagrams because the code is where you would look to see how the system works:

type observation = {xValues:float List} type weightedObservation = {xws:(float*float) List} type confirmedObservation = {observation:observation;yExpected:float} type weightedConfirmedObservation = {weightedObservation:weightedObservation;yExpected:float} type neuronInput = {weightedObservation:weightedObservation;bias:float} type cycleTrainingInput = {weightedConfirmedObservation:weightedConfirmedObservation;bias:float;alpha:float} type adjustmentInput = {weightedConfirmedObservation:weightedConfirmedObservation;bias:float;alpha:float;yActual:float} type adjustmentOutput = {weights:float List; bias:float} type rotationTrainingInput = {confirmedObservations:confirmedObservation List;weights:float List;bias:float;alpha:float} type trainInput = {confirmedObservations:confirmedObservation List; weightSeedValue:float;biasSeedValue:float;alpha:float; maxEpoches:int} type cyclePredictionInput = {weightedObservation:weightedObservation;bias:float} type rotationPredictionInput = {observations:observation List;weights:float List;bias:float} type predictInput = {observations:observation List;weights:float List;bias:float}

Note that I am composing data structures with the base being an observation.  The observation is a list of different xValues for a given, well, observation.  The weighted observation is the XValue paired with the perceptron weights.  The confirmedObservation is for training –> given an observation, what was the actual output? 

With the data structures out of the way, I went to the Perceptron and added in the basic functions for creating seed values:

member this.initializeWeights(xValues, randomSeedValue) = let lo = -0.01 let hi = 0.01 let xWeight = (hi-lo) * randomSeedValue + lo xValues |> Seq.map(fun w -> xWeight) member this.initializeBias(randomSeedValue) = let lo = -0.01 let hi = 0.01 (hi-lo) * randomSeedValue + lo

Since I was doing TDD, here are the unit tests I used for these functions:

[TestMethod] public void initializeWeightsUsingHalfSeedValue_ReturnsExpected() { var weights = _perceptron.initializeWeights(_observation.xValues, .5); var weightsList = new List<double>(weights); var expected = 0.0; var actual = weightsList[0]; Assert.AreEqual(expected, actual); } [TestMethod] public void initializeWeightsUsingLessThanHalfSeedValue_ReturnsExpected() { var weights = _perceptron.initializeWeights(_observation.xValues, .4699021627); var weightsList = new List<double>(weights); var expected = -0.00060; var actual = Math.Round(weightsList[0],5); Assert.AreEqual(expected, actual); } [TestMethod] public void initializeBiasesUsingHalfSeedValue_ReturnsExpected() { var expected = 0.0; var actual = _perceptron.initializeBias(.5); Assert.AreEqual(expected, actual); } [TestMethod] public void initializeBiasesUsingLessThanHalfSeedValue_ReturnsExpected() { var expected = -0.00060; var bias = _perceptron.initializeBias(.4699021627); var actual = Math.Round(bias, 5); Assert.AreEqual(expected, actual); } [TestMethod] public void initializeBiasesUsingGreaterThanHalfSeedValue_ReturnsExpected() { var expected = 0.00364; var bias = _perceptron.initializeBias(.6820621978); var actual = Math.Round(bias,5); Assert.AreEqual(expected, actual); }

I then created a base neuron and activation function that would work for both training and predicting:

member this.runNeuron(input:neuronInput) = let xws = input.weightedObservation.xws let output = xws |> Seq.map(fun (xValue,xWeight) -> xValue*xWeight) |> Seq.sumBy(fun x -> x) output + input.bias member this.runActivation(input) = if input < 0.0 then -1.0 else 1.0

[TestMethod] public void runNeuronUsingNormalInput_ReturnsExpected() { var expected = -0.0219; var perceptronOutput = _perceptron.runNeuron(_neuronInput); var actual = Math.Round(perceptronOutput, 4); Assert.AreEqual(expected, actual); } [TestMethod] public void runActivationUsingNormalInput_ReturnsExpected() { var expected = -1; var actual = _perceptron.runActivation(-0.0219); Assert.AreEqual(expected, actual); }

I then created the functions for training –> specifically to return adjusted weights and biases based on the result of the activation  function

member this.calculateWeightAdjustment(xValue, xWeight, alpha, delta) = match delta > 0.0, xValue >= 0.0 with | true,true -> xWeight - (alpha * abs(delta) * xValue) | false,true -> xWeight + (alpha * abs(delta) * xValue) | true,false -> xWeight - (alpha * abs(delta) * xValue) | false,false -> xWeight + (alpha * abs(delta) * xValue) member this.calculateBiasAdjustment(bias, alpha, delta) = match delta > 0.0 with | true -> bias - (alpha * abs(delta)) | false -> bias + (alpha * abs(delta)) member this.runAdjustment (input:adjustmentInput) = match input.weightedConfirmedObservation.yExpected = input.yActual with | true -> let weights = input.weightedConfirmedObservation.weightedObservation.xws |> Seq.map(fun (x,w) -> w) let weights' = new List<float>(weights) {adjustmentOutput.weights=weights';adjustmentOutput.bias=input.bias} | false -> let delta = input.yActual - input.weightedConfirmedObservation.yExpected let weights' = input.weightedConfirmedObservation.weightedObservation.xws |> Seq.map(fun (xValue, xWeight) -> this.calculateWeightAdjustment(xValue,xWeight,input.alpha,delta)) |> Seq.toList let weights'' = new List<float>(weights') let bias' = this.calculateBiasAdjustment(input.bias,input.alpha,delta) {adjustmentOutput.weights=weights'';adjustmentOutput.bias=bias'}

[TestMethod] public void calculateWeightAdjustmentUsingPositiveDelta_ReturnsExpected() { var xValue = 1.5; var xWeight = .00060; var delta = 2; var weightAdjustment = _perceptron.calculateWeightAdjustment(xValue, xWeight, _alpha, delta); var actual = Math.Round(weightAdjustment, 4); var expected = -.0024; Assert.AreEqual(expected, actual); } [TestMethod] public void calculateWeightAdjustmentUsingNegativeDelta_ReturnsExpected() { var xValue = 1.5; var xWeight = .00060; var delta = -2; var weightAdjustment = _perceptron.calculateWeightAdjustment(xValue, xWeight, _alpha, delta); var actual = Math.Round(weightAdjustment, 5); var expected = .0036; Assert.AreEqual(expected, actual); } [TestMethod] public void calculateBiasAdjustmentUsingPositiveDelta_ReturnsExpected() { var bias = 0.00364; var delta = 2; var expected = .00164; var actual = _perceptron.calculateBiasAdjustment(bias, _alpha, delta); Assert.AreEqual(expected, actual); } [TestMethod] public void calculateBiasAdjustmentUsingNegativeDelta_ReturnsExpected() { var bias = 0.00364; var delta = -2; var expected = .00564; var actual = _perceptron.calculateBiasAdjustment(bias, _alpha, delta); Assert.AreEqual(expected, actual); } [TestMethod] public void runAdjustmentUsingMatchingData_ReturnsExpected() { var adjustmentInput = new adjustmentInput(_weightedConfirmedObservation, _bias, _alpha, -1.0); var adjustedWeights = _perceptron.runAdjustment(adjustmentInput); var expected = .0065; var actual = Math.Round(adjustedWeights.weights[0],4); Assert.AreEqual(expected, actual); } [TestMethod] public void runAdjustmentUsingNegativeData_ReturnsExpected() { weightedConfirmedObservation weightedConfirmedObservation = new NeuralNetworks.weightedConfirmedObservation(_weightedObservation, 1.0); var adjustmentInput = new adjustmentInput(weightedConfirmedObservation, _bias, _alpha, -1.0); var adjustedWeights = _perceptron.runAdjustment(adjustmentInput); var expected = .0125; var actual = Math.Round(adjustedWeights.weights[0], 4); Assert.AreEqual(expected, actual); } [TestMethod] public void runAdjustmentUsingPositiveData_ReturnsExpected() { var adjustmentInput = new adjustmentInput(_weightedConfirmedObservation, _bias, _alpha, 1.0); var adjustedWeights = _perceptron.runAdjustment(adjustmentInput); var expected = .0005; var actual = Math.Round(adjustedWeights.weights[0], 4); Assert.AreEqual(expected, actual); }

With these functions ready, I could run a training cycle for a given observation

member this.runTrainingCycle (cycleTrainingInput:cycleTrainingInput) = let neuronTrainingInput = {neuronInput.weightedObservation=cycleTrainingInput.weightedConfirmedObservation.weightedObservation; neuronInput.bias=cycleTrainingInput.bias} let neuronResult = this.runNeuron(neuronTrainingInput) let activationResult = this.runActivation(neuronResult) let adjustmentInput = {weightedConfirmedObservation=cycleTrainingInput.weightedConfirmedObservation; bias=cycleTrainingInput.bias;alpha=cycleTrainingInput.alpha; yActual=activationResult} this.runAdjustment(adjustmentInput)

[TestMethod] public void runTrainingCycleUsingNegativeData_ReturnsExpected() { var cycleTrainingInput = new cycleTrainingInput(_weightedConfirmedObservation, _bias, _alpha); var adjustmentOutput = _perceptron.runTrainingCycle(cycleTrainingInput); var expected = .0125; var actual = Math.Round(adjustmentOutput.weights[0], 4); Assert.AreEqual(expected, actual); } [TestMethod] public void runTrainingCycleUsingPositiveData_ReturnsExpected() { var cycleTrainingInput = new cycleTrainingInput(_weightedConfirmedObservation, _bias, _alpha); var adjustmentOutput = _perceptron.runTrainingCycle(cycleTrainingInput); var expected = .0065; var actual = Math.Round(adjustmentOutput.weights[0], 4); Assert.AreEqual(expected, actual); }

And then I could run a cycle for each of the observations in the training set, a rotation.  I am not happy that I am mutating the weights and biases here, though I am not sure how to fix that.  I looked for a Seq.Scan function where the results of a function applied to the 1st element of a Seq is used in the input of the next –> all I could see were examples of threading a collector of int (like Seq.mapi).  This will be something I will ask the functional ninjas when I see them again.

member this.runTrainingRotation(rotationTrainingInput: rotationTrainingInput)= let mutable weights = rotationTrainingInput.weights let mutable bias = rotationTrainingInput.bias let alpha = rotationTrainingInput.alpha for i=0 to rotationTrainingInput.confirmedObservations.Count-1 do let currentConfirmedObservation = rotationTrainingInput.confirmedObservations.[i] let xws = Seq.zip currentConfirmedObservation.observation.xValues weights let xws' = new List<(float*float)>(xws) let weightedObservation = {xws=xws'} let weightedTrainingObservation = {weightedObservation=weightedObservation;yExpected=currentConfirmedObservation.yExpected} let cycleTrainingInput = { cycleTrainingInput.weightedConfirmedObservation=weightedTrainingObservation; cycleTrainingInput.bias=bias; cycleTrainingInput.alpha=alpha} let cycleOutput = this.runTrainingCycle(cycleTrainingInput) weights <- cycleOutput.weights bias <- cycleOutput.bias {adjustmentOutput.weights=weights; adjustmentOutput.bias=bias}

[TestMethod] public void runTrainingRotationUsingNegativeData_ReturnsExpected() { var xValues = new List<double>(); xValues.Add(3.0); xValues.Add(4.0); var observation = new observation(xValues); var yExpected = -1.0; var confirmedObservation0 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(1.5); xValues.Add(2.0); yExpected = -1.0; var confirmedObservation1 = new confirmedObservation(observation, yExpected); var trainingObservations = new List<confirmedObservation>(); trainingObservations.Add(confirmedObservation0); trainingObservations.Add(confirmedObservation1); var weights = new List<double>(); weights.Add(.0065); weights.Add(.0123); var rotationTrainingInput = new rotationTrainingInput(trainingObservations, weights, _bias, _alpha); var trainingRotationOutput = _perceptron.runTrainingRotation(rotationTrainingInput); var expected = -0.09606; var actual = Math.Round(trainingRotationOutput.bias, 5); Assert.AreEqual(expected, actual); } [TestMethod] public void runTrainingRotationUsingPositiveData_ReturnsExpected() { var xValues = new List<double>(); xValues.Add(3.0); xValues.Add(4.0); var observation = new observation(xValues); var yExpected = 1.0; var confirmedObservation0 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(1.5); xValues.Add(2.0); yExpected = 1.0; var confirmedObservation1 = new confirmedObservation(observation, yExpected); var trainingObservations = new List<confirmedObservation>(); trainingObservations.Add(confirmedObservation0); trainingObservations.Add(confirmedObservation1); var weights = new List<double>(); weights.Add(.0065); weights.Add(.0123); var rotationTrainingInput = new rotationTrainingInput(trainingObservations, weights, _bias, _alpha); var trainingRotationOutput = _perceptron.runTrainingRotation(rotationTrainingInput); var expected = -.09206; var actual = Math.Round(trainingRotationOutput.bias, 5); Assert.AreEqual(expected, actual); }

With the rotation done, I could write the train function which runs rotations for N number of times to tune the weights and biases:

member this.train(trainInput:trainInput) = let currentObservation = trainInput.confirmedObservations.[0].observation let weights = this.initializeWeights(currentObservation.xValues,trainInput.weightSeedValue) let weights' = new List<float>(weights) let mutable bias = this.initializeBias(trainInput.biasSeedValue) let alpha = trainInput.alpha for i=0 to trainInput.maxEpoches do let rotationTrainingInput={rotationTrainingInput.confirmedObservations=trainInput.confirmedObservations; rotationTrainingInput.weights = weights'; rotationTrainingInput.bias=bias; rotationTrainingInput.alpha=trainInput.alpha} this.runTrainingRotation(rotationTrainingInput) |> ignore {adjustmentOutput.weights=weights'; adjustmentOutput.bias=bias}

[TestMethod] public void trainUsingTestData_RetunsExpected() { var xValues = new List<double>(); xValues.Add(1.5); xValues.Add(2.0); var observation = new observation(xValues); var yExpected = -1.0; var confirmedObservation0 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(2.0); xValues.Add(3.5); observation = new observation(xValues); yExpected = -1.0; var confirmedObservation1 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(3.0); xValues.Add(5.0); observation = new observation(xValues); yExpected = -1.0; var confirmedObservation2 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(3.5); xValues.Add(2.5); observation = new observation(xValues); yExpected = -1.0; var confirmedObservation3 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(4.5); xValues.Add(5.0); observation = new observation(xValues); yExpected = 1.0; var confirmedObservation4 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(5.0); xValues.Add(7.5); observation = new observation(xValues); yExpected = 1.0; var confirmedObservation5 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(5.5); xValues.Add(8.0); observation = new observation(xValues); yExpected = 1.0; var confirmedObservation6 = new confirmedObservation(observation, yExpected); xValues = new List<double>(); xValues.Add(6.0); xValues.Add(6.0); observation = new observation(xValues); yExpected = 1.0; var confirmedObservation7 = new confirmedObservation(observation, yExpected); var trainingObservations = new List<confirmedObservation>(); trainingObservations.Add(confirmedObservation0); trainingObservations.Add(confirmedObservation1); trainingObservations.Add(confirmedObservation2); trainingObservations.Add(confirmedObservation3); trainingObservations.Add(confirmedObservation4); trainingObservations.Add(confirmedObservation5); trainingObservations.Add(confirmedObservation6); trainingObservations.Add(confirmedObservation7); var random = new Random(); var weightSeedValue = random.NextDouble(); var biasSeedValue = random.NextDouble(); var alpha = .001; var maxEpoches = 100; var trainInput = new trainInput(trainingObservations, weightSeedValue, biasSeedValue, alpha, maxEpoches); var trainOutput = _perceptron.train(trainInput); Assert.IsNotNull(trainOutput); }

With the training out of the way, I could concentrate on the prediction.  The prediction was much easier because there are no adjustments and the rotation is run once.  The data structures are also simpler because I don’t have to pass in the knownY values.  I also only have 1 covering (all be it long) unit test that looks that the results of the prediction.

member this.runPredictionCycle (cyclePredictionInput:cyclePredictionInput) = let neuronInput = {neuronInput.weightedObservation=cyclePredictionInput.weightedObservation; neuronInput.bias=cyclePredictionInput.bias} let neuronResult = this.runNeuron(neuronInput) this.runActivation(neuronResult) member this.runPredictionRotation (rotationPredictionInput:rotationPredictionInput) = let output = new List<List<float>*float>(); let weights = rotationPredictionInput.weights for i=0 to rotationPredictionInput.observations.Count-1 do let currentObservation = rotationPredictionInput.observations.[i]; let xws = Seq.zip currentObservation.xValues weights let xws' = new List<(float*float)>(xws) let weightedObservation = {xws=xws'} let cyclePredictionInput = { cyclePredictionInput.weightedObservation = weightedObservation; cyclePredictionInput.bias = rotationPredictionInput.bias} let cycleOutput = this.runPredictionCycle(cyclePredictionInput) output.Add(currentObservation.xValues, cycleOutput) output member this.predict(predictInput:predictInput) = let rotationPredictionInput = { rotationPredictionInput.observations = predictInput.observations; rotationPredictionInput.weights = predictInput.weights; rotationPredictionInput.bias = predictInput.bias } this.runPredictionRotation(rotationPredictionInput)

[TestMethod] public void predictUsingTestData_ReturnsExpected() { var xValues = new List<double>(); xValues.Add(3.0); xValues.Add(4.0); var observation0 = new observation(xValues); xValues = new List<double>(); xValues.Add(0.0); xValues.Add(1.0); var observation1 = new observation(xValues); xValues = new List<double>(); xValues.Add(2.0); xValues.Add(5.0); var observation2 = new observation(xValues); xValues = new List<double>(); xValues.Add(5.0); xValues.Add(6.0); var observation3 = new observation(xValues); xValues = new List<double>(); xValues.Add(9.0); xValues.Add(9.0); var observation4 = new observation(xValues); xValues = new List<double>(); xValues.Add(4.0); xValues.Add(6.0); var observation5 = new observation(xValues); var observations = new List<observation>(); observations.Add(observation0); observations.Add(observation1); observations.Add(observation2); observations.Add(observation3); observations.Add(observation4); observations.Add(observation5); var weights = new List<double>(); weights.Add(.0065); weights.Add(.0123); var bias = -0.0906; var predictInput = new predictInput(observations, weights, bias); var predictOutput = _perceptron.predict(predictInput); Assert.IsNotNull(predictOutput); }

When I run all of the unit tests the all run green:

image

With the Perceptron created, I can now go back and change the code and figure out:

1) Why my weights across the XValues are the same (wrong!)

2) How to implement a more idomatic/recursive way of running rotations so I can remove the mutation

With my unit tests running green, I know I am covered in case I make a mistake