The content of this post is mostly copied from here. The reason I do this is to ensure that that really helpful post will be accessible and to add few modifications.

1. Update and dependencies

# Update apt-get
sudo apt-get update
sudo apt-get upgrade

# Kernel header
sudo apt-get install linux-headers-$(uname -r)

2. Install NVIDIA CUDA Toolkit

Go to https://developer.nvidia.com/cuda-downloads and download CUDA Toolkit 9.0 (Legacy) for Ubuntu 16.04. Download deb (local) which is 1.2 GB.

sudo dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64.deb
sudo apt-key add /var/cuda-repo-9-0-local/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda-9.0
# Reboot
sudo reboot

vi ~/.bashrc
# add those 2 lines at the end of the file then save it
export PATH=/usr/local/cuda-9.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64:$LD_LIBRARY_PATH

Then, execute the following commands and check that nvidia-smi works.

source ~/.bashrc
sudo ldconfig
nvidia-smi

3. Install cuDNN 7.2.1

Go to https://developer.nvidia.com/cudnn, login/register, go to cuDNN Download, Archived cuDNN releases and download cuDNN v7.2.1 (August 7, 2018), for CUDA 9.0. Now, for some reason, it seems that that version of cuDNN is only available for CUDA 9.2, but we can just change the download links manually from the CUDA 9.2

cuDNN v7.2.1 Runtime Library for Ubuntu16.04 (Deb): https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v7.2.1/prod/9.0_20180806/Ubuntu16_04-x64/libcudnn7_7.2.1.38-1_cuda9.0_amd64
cuDNN v7.2.1 Developer Library for Ubuntu16.04 (Deb): https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v7.2.1/prod/9.0_20180806/Ubuntu16_04-x64/libcudnn7-dev_7.2.1.38-1_cuda9.0_amd64
cuDNN v7.2.1 Code Samples and User Guide for Ubuntu16.04 (Deb): https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v7.2.1/prod/9.0_20180806/Ubuntu16_04-x64/libcudnn7-doc_7.2.1.38-1_cuda9.0_amd64

Install them in the right order:

sudo dpkg -i libcudnn7_7.2.1.38–1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.2.1.38–1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-doc_7.2.1.38–1+cuda9.0_amd64.deb

Verify the installation

cp -r /usr/src/cudnn_samples_v7/ $HOME
cd $HOME/cudnn_samples_v7/mnistCUDNN
make clean && make
./mnistCUDNN

You should expect a “Test passed!”.

CUPTI ships with the CUDA Toolkit, but you also need to append its path to the LD_LIBRARY_PATH environment variable

vi ~/.bashrc
export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH
source ~/.bashrc

4. Install TF and verify it works

pip install tensorflow-gpu

Run

import tensorflow as tf
with tf.device('/gpu:0'):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)

with tf.Session() as sess:
print(sess.run(c))

If this script runs without giving any error you are using gpu version.

My computer:
Windows 10, Python 3.4, OpenCV 3.0.0

I had uncountable desperate attempts to install OpenCV 3.0 in my machine, including the always last plan when installing software: building it from the source. There is an apparently nice tutorial from the official OpenCV webpage which didn’t work for me because I needed to do a couple of steps, so please, try to follow this tutorial (section Building OpenCV from source) with this addition tips:

• 7.2: Use the path you want, but remember that you will not be able to delete it (because you need to refer to that path) so choose wisely.
• 7.4: You can use any Visual Studio. In fact, I used Visual Studio 13. Just remember to specify it when configuring CMake
• 8,9,10,11: When you are checking and unchecking all those checkboxes you will realize that many of the options are not listed on the provided pictures. What to do in that case? Easy: just leave it there as it is.
• 16: Some of the projects will not be built and will be skipped. Don’t worry.

Now, it’s supposed to be installed in your computer, and if you try to import cv2, it should work.
However, at least for me, it didn’t work. The final step to get rid of that annoying message is to add the appropiate path to the PATH system variable. This is the path that you have to add: X\bin\Release where X is the folder where you compiled it.

I hope this is helpful. This could have saved me many hours…

Since Machine Learning and, in general, Artificial Intelligence became my favorite subject, I have spent a lot of time learning about it by myself. I consider that a good, readable and pleasant book can be key in learning about such difficult topics. I have to say that I started reading which probably is the most famous book in this issue: Machine Learning and Pattern Recognition (Bishop), but I failed at finishing it. Plenty of maths, abstract concepts and no real examples make this book very tough, in my opinion. I think this book collects and admirable amount of work and may serve as a great reference, but in order to fully comprehend and learn from an undergraduate background, it can be hard.

Because of the lack of examples, I tried to find a book with examples in some programming language, as Machine Learning: An algorithmic Perspective does. However, I didn’t like it very much neither, because it uses Python classes that you cannot see them from inside. A quite understandable argument is that you don’t really need to understand how it compeltely works from inside in order to understand the general idea of some algorithms, but I think that the only way to understand how an algorithm works is by programming it.

I tried to read each chapter of this book carefully and spending a lot of time researching, reading papers, watching Youtube videos to help me to understand concepts and of course writting down some notes. I have decided to publish all the stuff I’m doing, from personal written notes to source code because it may help more people who are reading this book or who simply try to understand specific concepts.

As I don’t want to write a very long post about it, this entry will serve as an index and for each chapter I consider interesting, I will write a post uploading my own stuff and giving some explanations.

Machine Learning: An Algorithmic Perspective
1.-Introduction
2.-Linear discriminants
3.-The Multi-Layer Perceptron
4.-Radial Basis Functions and Splines
5.-Support Vector Machines
6.-Learning with Trees
7.-Decision by Commitee: Ensemble Learning

2.-Linear discriminants

Downloads: perceptron.pdf, activation.m, NN2outputs.m, NNand.m
Contents: Transfer function, Why bias is needed?, Learning rate, examples, Matlab code.

3.-The Multi-Layer Perceptron

Downloads: MLP.pdf, NN231.m
Contents: Backpropagation, Multi-Layer Perceptron
Further study: Create a general algorithm to use a NN with N inputs, M layers, P nodes, Q outputs.

4.-Radial Basis Functions and Splines

Downloads: kmeans.pdf, kmeans.m
Contents: K-Means algorithm
Further study: Go back to this section in a future to better understand it. Study more about splines.

5.-Support Vector Machines

Downloads: SVM.pdf
Contents: SVM algorithm explained, Non-linear SVM, several examples
Further study: Implement this efficiently in Matlab, learn more about Non-linear SVM, learn how to find support vectors among all others automatically

6.-Learning with Trees

Downloads: Trees.pdf
Contents: Example of Decision Tree (classification)
Further study: Learn/implement C5.0 algorithm, and CART (for regression).

7.-Decision by Commitee: Ensemble Learning

Downloads: Adaboost.pdf
Content: Adaboost formulae and 2 examples
Further study: implement Adaboost in Matlab, example of bagging and Mixture of experts