Does Tensorflow support unique neural network topologies


Author: Valerio Maggio

PostDoc Data Scientist @ FBK/MPBA

Contacts:

git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git

  • Part I: Introduction

    • Intro to Artificial Neural Networks

      • Perceptron and MLP
      • naive pure-Python implementation
      • fast forward, sgd, backprop
    • Introduction to Deep Learning Frameworks

      • Intro to Theano
      • Intro to Tensorflow
      • Intro to Keras
        • Overview and main features
        • Overview of the layers
        • Multi-Layer Perceptron and Fully Connected
          • Examples with and
        • Keras Backend
  • Part II: **Supervised Learning **

    • Fully Connected Networks and Embeddings

      • Intro to MNIST Dataset
      • Hidden Leayer Representation and Embeddings
    • Convolutional Neural Networks

      • meaning of convolutional filters

      • Visualising ConvNets

      • Advanced CNN

        • Dropout
        • MaxPooling
        • Batch Normalisation
      • HandsOn: MNIST Dataset

        • FC and MNIST
        • CNN and MNIST
      • Deep Convolutiona Neural Networks with Keras (ref: )

    • Transfer Learning and FineTuning

    • Hyperparameters Optimisation

  • Part III: Unsupervised Learning

    • AutoEncoders and Embeddings
    • AutoEncoders and MNIST
      • word2vec and doc2vec (gensim) with
      • word2vec and CNN
  • Part IV: Recurrent Neural Networks

    • Recurrent Neural Network in Keras
    • LSTM for Sentence Generation
  • PartV: Additional Materials:

    • Custom Layers in Keras
    • Multi modal Network Topologies with Keras

Requirements

This tutorial requires the following packages:

(Optional but recommended):

The easiest way to get (most) these is to use an all-in-one installer such as Anaconda from Continuum. These are available for multiple architectures.


Python Version

I'm currently running this tutorial with Python 3 on Anaconda


Setting the Environment

In this repository, files to re-create virtual env with are provided for Linux and OSX systems, namely and , respectively.

To re-create the virtual environments (on Linux, for example):

conda env create -f deep-learning.yml

For OSX, just change the filename, accordingly.

Notes about Installing Theano with GPU support

NOTE: Read this section only if after pip installing, it raises error in enabling the GPU support!

Since version Theano introduced the in the stable release (it was previously only available in the development version).

The goal of is (from the documentation) make a common GPU ndarray (n dimensions array) that can be reused by all projects that is as future proof as possible, while keeping it easy to use for simple need/quick test.

Here are some useful tips (hopefully) I came up with to properly install and configure on (Ubuntu) Linux with GPU support:

  1. [If you're using Anaconda] should be just fine!

Sometimes it is suggested to install using the channel:

  1. [Works with both Anaconda Python or Official CPython]

After Theano is installed:

Installing Tensorflow

To date comes in two different packages, namely and , whether you want to install the framework with CPU-only or GPU support, respectively.

For this reason, has not been included in the conda envs and has to be installed separately.

Tensorflow for CPU only: