Consuming Sky Biometry’s Image Recognition API

I was looking at Sky Biometry a couple of days ago to do some facial recognition.  I am not sure how many other companies out there do this, but working with their API from F# was a great experience. 

Sky Biometry uses both Json and Xml for API calls.  They also have an example covering library in C#.  I decided to use F# and use the json type provider to explore their api and the quality of their recognition algorithms.  They have a good documentation page and it didn’t take very long to get my account up and running and then making API calls.

I thought it would be fun to use an example from the first Terminator movie when Ah-nold went around looking for Sarah Connor.  I picked two images from the movie series.  The first is this image of a photograph of Sarah Connor has taken at the end of the first movie.  I know that the terminator didn’t have this photograph in the movie, but working off-script, pretend that the Terminator has that photograph.  This second image is from the first movie when she is in a bar.  So if the Terminator was powered by Sky Biometry and found Sarah in the bar, how close would it match her to the first photo?


The first thing I did was to fire up a FSharp project in Visual Studio and start scripting the API calls that I would need to do facial recognition. 


With all of the calls working in the REPL, I then moved the code into my module.  I declared the types at the top and then crated a class that could be consumed by external projects.


You will notice that the type providers are using a local copy of the json to infer the type in the module.  I did run into the problem where the type provider using the web call was not capturing the full graph in the type definition, so I took the json and made it local.  This led to an interesting problem because a FSharp project out of the box in Visual Studio does not support adding folders.  To get around that, I went to my file system and added a folder


I then created files in the folder that correspond to the type providers


Next I unloaded the FS project and edited it


and in the project, I added those files to the ItemGroup that brings in all of the files from disk

  1. <ItemGroup>
  2.   <Compile Include="SkyBiometryImageComparer.fs" />
  3.   <None Include="Script.fsx" />
  4.   <None Include="packages.config" />
  5.   <None Include="app.config" />
  6.   <None Include="SkyBiometryImageJson/AddTags.json" />
  7.   <None Include="SkyBiometryImageJson/FaceDetection.json" />
  8.   <None Include="SkyBiometryImageJson/FaceRecognition.json" />
  9.   <None Include="SkyBiometryImageJson/FaceTraining.json" />
  10.   <None Include="SkyBiometryImageJson/RemoveTags.json" />
  11. </ItemGroup>

Once the project is re-loaded, the folder and the files show up in Solution Explorer.



In each of the .json files, I pasted in the results from the service call that I did in the REPL.  For example,


So then I swapped them out in the module and now I get the full graph


With that out of the way, I implemented a one to one match of supporting methods to the API calls:

  1. member this.DetectFace(imageUri:string)=
  2.     let stringBuilder = new StringBuilder()
  3.     stringBuilder.Append(skybiometryUri) |> ignore
  4.     stringBuilder.Append("/fc/faces/detect.json?urls=") |> ignore
  5.     stringBuilder.Append(imageUri) |> ignore
  6.     stringBuilder.Append("&api_key=") |> ignore
  7.     stringBuilder.Append(apiKey) |> ignore
  8.     stringBuilder.Append("&api_secret=") |> ignore
  9.     stringBuilder.Append(apiSecret) |> ignore
  10.     let faceDetection =  skybiometryFaceDetection.Load(stringBuilder.ToString())
  11.     faceDetection.Photos.[0].Tags.[0].Tid
  13. member this.SaveTag(tid:string)=
  14.     let stringBuilder = new StringBuilder()
  15.     stringBuilder.Append(skybiometryUri) |> ignore
  16.     stringBuilder.Append("/fc/tags/save.json?uid=") |> ignore
  17.     stringBuilder.Append(uid) |> ignore
  18.     stringBuilder.Append("&tids=") |> ignore
  19.     stringBuilder.Append(tid) |> ignore
  20.     stringBuilder.Append("&api_key=") |> ignore
  21.     stringBuilder.Append(apiKey) |> ignore
  22.     stringBuilder.Append("&api_secret=") |> ignore
  23.     stringBuilder.Append(apiSecret) |> ignore
  24.     let tags = skybiometryAddTags.Load(stringBuilder.ToString())
  25.     tags.Status
  27. member this.TrainFace()=
  28.     let stringBuilder = new StringBuilder()
  29.     stringBuilder.Append(skybiometryUri) |> ignore
  30.     stringBuilder.Append("/fc/faces/train.json?uids=") |> ignore
  31.     stringBuilder.Append(uid) |> ignore
  32.     stringBuilder.Append("&api_key=") |> ignore
  33.     stringBuilder.Append(apiKey) |> ignore
  34.     stringBuilder.Append("&api_secret=") |> ignore
  35.     stringBuilder.Append(apiSecret) |> ignore
  36.     let training = skybiometryFaceTraining.Load(stringBuilder.ToString())
  37.     training.Status
  39. member this.RecognizeFace(imageUri:string)=
  40.     let stringBuilder = new StringBuilder()
  41.     stringBuilder.Append(skybiometryUri) |> ignore
  42.     stringBuilder.Append("/fc/faces/recognize.json?uids=") |> ignore
  43.     stringBuilder.Append(uid) |> ignore
  44.     stringBuilder.Append("&urls=") |> ignore
  45.     stringBuilder.Append(imageUri) |> ignore
  46.     stringBuilder.Append("&api_key=") |> ignore
  47.     stringBuilder.Append(apiKey) |> ignore
  48.     stringBuilder.Append("&api_secret=") |> ignore
  49.     stringBuilder.Append(apiSecret) |> ignore
  50.     let recognition = skybiometryFaceRecognition.Load(stringBuilder.ToString())
  51.     if recognition.Photos.[0].Tags |> Seq.length > 0 then
  52.         recognition.Photos.[0].Tags.[0].Attributes.Face.Confidence
  53.     else
  54.         0
  56. member this.RemoveTag(tid:string) =
  57.     let stringBuilder = new StringBuilder()
  58.     stringBuilder.Append(skybiometryUri) |> ignore
  59.     stringBuilder.Append("/fc/tags/remove.json?tids=") |> ignore
  60.     stringBuilder.Append(tid) |> ignore
  61.     stringBuilder.Append("&api_key=") |> ignore
  62.     stringBuilder.Append(apiKey) |> ignore
  63.     stringBuilder.Append("&api_secret=") |> ignore
  64.     stringBuilder.Append(apiSecret) |> ignore
  65.     let tagRemoval = skybiometryRemoveTags.Load(stringBuilder.ToString())
  66.     tagRemoval.Status

The StringBuilder makes the code pretty verbose, but I understand that it is the most efficient way to aggregate strings so I went with it. Also note that this is happy path programming with no error checking and I assume that the json coming back is well formed.

In any event, with the supporting methods done, it was just an exercise of calling each one in turn:

  1. member this.CalculateFacialRecognitionConfidence(baseUri:string, comparisionUri:string) =
  2.     let tid = this.DetectFace(baseUri)
  3.     if this.SaveTag(tid) = "success" then
  4.         if this.TrainFace() = "success" then
  5.             let confidence = this.RecognizeFace(comparisionUri)
  6.             this.RemoveTag(tid) |> ignore
  7.             confidence
  8.         else
  9.             0
  10.     else
  11.         0

And hopping over to a C# unit test project, I can call the FSharp and run some tests.  I created a test for each of the supporting methods and then three happy path tests for the CalculateFacialRecognitionConfidence: comparing the same image should be 100, comparing 2 completely unrelated images should be 0, and then our exercise of identifying Sarah Connor should be better than 50/50.  Here is an example of the 100% match use case:


And here is the actual test of finding Sarah Connor.:

  1.     var imageComparer = new SkyBiometryImageComparer(skyBiometryUri, uid, apiKey, apiSecret);
  2.     String basePhotoUri = ";;
  3.     String targetPhotoUri = ";;
  4.     Int32 confidence = imageComparer.CalculateFacialRecognitionConfidence(basePhotoUri, targetPhotoUri);
  6.     bool expected = true;
  7.     bool actual = confidence > 50;
  8.     Assert.AreEqual(expected, actual);
  9. }

It runs green (the actual value is 58% for identifying Sarah Connor). 


My guess is that that Terminator would start shooting… first at Sarah Connor and then at C#.  The C# project that Sky Biometry provides has 2 classes of 1600 and 1420 lines of code to get functional equivalence of the 93 lines of F# code that I wrote (and a vast majority of that code is dealing with the string builder).


About these ads

2 Responses to Consuming Sky Biometry’s Image Recognition API

  1. Pingback: F# Weekly #26, 2014 | Sergey Tihon's Blog

  2. Well, I am enthusiastic about F# too, but I surely dislike this comparison approach that you have chosen. I checked their C# client code. First, they have lots of comments, at least 30% of all line count. Second, they have ifdefs because they target multiple platforms. Third, for some reason they decided to create REST client themselves. If they have used something like Refit, the code would be significantly smaller and this would be more identical to the type provider usage.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

%d bloggers like this: