# HOW TO¶

In this HOW-TO page, available blocks are described.

TODO: finished this section

## Block¶

akid builds another layer of abstraction on top of Tensor: Block. Tensor can be taken as the media/formalism signal propagates in digital world, while Block is the data processing entity that processes inputs and emits outputs.

It coincides with a branch of “ideology” called dataism that takes everything in this world is a data processing entity. An interesting one that may come from A Brief History of Tomorrow by Yuval Noah Harari.

Best designs mimic nature. akid tries to reproduce how signals in nature propagates. Information flow can be abstracted as data propagating through inter-connected blocks, each of which processes inputs and emits outputs. For example, a vision classification system is a block that takes image inputs and gives classification results. Everything is a Block in akid.

A block could be as simple as a convonlutional neural network layer that merely does convolution on the input data and outputs the results; it also be as complex as an acyclic graph that inter-connects blocks to build a neural network, or sequentially linked block system that does data augmentation.

Compared with pure symbol computation approach, like the one in tensorflow, a block is able to contain states associated with this processing unit. Signals are passed between blocks in form of tensors or list of tensors. Many heavy lifting has been done in the block (Block and its sub-classes), e.g. pre-condition setup, name scope maintenance, copy functionality for validation and copy functionality for distributed replicas, setting up and gathering visualization summaries, centralization of variable allocation, attaching debugging ops now and then etc.

## Source¶

Signals propagated in nature are all abstracted as a source. For instance, an light source (which could be an image source or video source), an audio source, etc.

As an example, saying in supervised setting, a source is a block that takes no inputs (since it is a source), and outputs data. A concrete example could be the source for the MNIST dataset:

source = MNISTFeedSource(name="MNIST",
url='http://yann.lecun.com/exdb/mnist/',
work_dir=AKID_DATA_PATH + '/mnist',
center=True,
scale=True,
num_train=50000,
num_val=10000)


The above code creates a source for MNIST. It is supposed to provide data for placeholders of tensorflow through method get_batch. Say:

source.get_batch(100, get_val=False)


would return a tuple of numpy array of (images, labels).

It could be used standalone, or passed to a Sensor.

### Developer Note¶

A top level abstract class Source implements basic semantics of a natural source. Other abstract classes keep implementing more concrete sources. Abstract Source s need to be inherited and abstract methods implemented before it could be used. To create a concrete Source, you could use multiple inheritance to compose the Source you needs. Available sources are kept under module sources.

## Sensor¶

The interface between nature and an (artificial) signal processing system, saying a brain, is a Sensor. It does data augmentation (if needed), and batches datum from Source.

Strictly speaking, the functional role of a sensor is to convert the signal in the natural form to a form the data processing engine, which is the brain in this case, could process. It is a Analog/Digital converter. However, the input from Source is already in digital form, so this function is not there anymore. But the data batching, augmentation and so on could still be put in preprocessing. Thus we still use the name sensor for concept reuse.

Mathematically, it is a system made up with a series of linked blocks that do data augmentation.

As an example, again saying in supervised setting, a sensor is a block that takes a data source and output sensed (batched and augmented) data. A sensor needs to be used along with a source. A concrete example could be the sensor for the MNIST dataset. Taking a Source, we could make a sensor:

sensor = FeedSensor(name='data', source_in=source)


The type of a sensor must match that of a source.

For IntegratedSensor, it is supported to add Joker to augment data. The way to augment data is similar with building blocks using Brain, but simpler, since data augmentation is added sequentially, shown in the following:

sensor = IntegratedSensor(source_in=cifar_source,
batch_size=128,
name='data')
sensor.attach(FlipJoker(flip_left_right=True, name="left_right_flip"))
sensor.attach(CropJoker(height=32, width=32, name="crop"))


The end computational graph is shown as following.

## Brain¶

A brain is the data processing engine to process data supplied by Sensor to fulfill certain tasks. More specifically,

• it builds up blocks to form an arbitrary network
• offers sub-graphs for inference, loss, evaluation, summaries

To use a brain, feed in data as a list, as how it is done in any other blocks. Some pre-specified brains are available under akid.models.brains. An example that sets up a brain using existing brains is:

# ... first get a feed sensor
sensor.setup()
brain = OneLayerBrain(name="brain")
input = [sensor.data(), sensor.labels()]
brain.setup(input)


Note in this case, data() and labels() of sensor returns tensors. It is not always the case. If it does not, saying return a list of tensors, you need do things like:

input = [sensor.data()]
input.extend(sensor.labels())


Act accordingly.

Similarly, all blocks work this way.

A brain provides easy ways to connect blocks. For example, a one layer brain can be built through the following:

class OneLayerBrain(Brain):
def __init__(self, **kwargs):
super(OneLayerBrain, self).__init__(**kwargs)
self.attach(
ConvolutionLayer(ksize=[5, 5],
strides=[1, 1, 1, 1],
out_channel_num=32,
name="conv1")
)
self.attach(ReLULayer(name="relu1"))
self.attach(
PoolingLayer(ksize=[1, 5, 5, 1],
strides=[1, 5, 5, 1],
name="pool1")
)

self.attach(InnerProductLayer(out_channel_num=10, name="ip1"))
self.attach(SoftmaxWithLossLayer(
class_num=10,
inputs=[
{"name": "ip1", "idxs": [0]},
{"name": "system_in", "idxs": [1]}],
name="loss"))


It assembles a convolution layer, a ReLU Layer, a pooling layer, an inner product layer and a loss layer. To attach a block (layer) that directly takes the outputs of the previous attached layer as inputs, just directly attach the block. If inputs exists, the brain will fetch corresponding tensors by name of the block attached and indices of the outputs of that layer. See the loss layer above for an example. Note that even though there are multiple inputs for the brain, the first attached layer of the brain will take the first of these input by default, given the convention that the first tensor is the data, and the remaining tensors are normally labels, which is not used till very late.

## System¶

This module provides systems of different topology to compose Blocks to create more complex blocks. A system does not concern which type of block it holds, but only concerns the mathematical topology how they connect.