2 months ago we introduced Bender: a framework for real-time Neural Networks inference. And we wanted to talk a bit about it :)

Nuts & Bolts of Bender

You may have a Machine Learning model you want to run smoothly on an iPhone. Either style transfer or image recognition (or many many others) are common tasks that would be nice to run on mobile to open the doors to several ideas.

Bender takes advantage of Metal and Metal Performance Shaders (MPS) to use the GPU in iOS, it runs as fast as lightning. And it supports models created in TensorFlow and in Caffe.

Here's a sample code:

import MetalBender

let url = Bundle.main.url(forResource: "myModel", withExtension: "pb")! // A TensorFlow model.
let network = Network.load(url: url, inputSize: LayerSize(h: 256, w: 256, f: 3))

network.run(input: /* … */) { output in
    // …
}

And that's pretty much it!

In the example, myModel.pb is a frozen TensorFlow graph that was imported using Benderthon, which is a Python library to ease the task of freezing models. It also supports loading just a graph definition (without the variables values) and passing the parameter values using the Network parameter parameterLoader. And there's also a beautiful API to define the network:

let network = Network(inputSize: LayerSize(h: 256, w: 256, f: 3))

network.start
    ->> Convolution(convSize: ConvSize(outputChannels: 16, kernelSize: 3, stride: 2))
    ->> InstanceNorm()
    ->> Convolution(convSize: ConvSize(outputChannels: 32, kernelSize: 3, stride: 2), neuronType: .relu)
    ->> InstanceNorm()
    ->> FullyConnected(neurons: 128)
    ->> Neuron(type: .tanh)
    ->> FullyConnected(neurons: 10)
    ->> Softmax()

// …

The repo includes some examples that contain a real-time digit classifier based on the MNIST dataset and a real-time style transfer camera. You should take a look at them.

Soon we will be writing about Bender motivation and a comparison against other frameworks. So stay tuned!