Despite what the bad media are saying, computers haven’t understood human language (yet). We need to turn sentences and words into a format that can be effectively manipulated by a Machine Learning or Deep Learning algorithm. This is called language modeling. Here I will explain several methods that can turn words into a meaningful representation.

Integer encoding

This approach is the simplest. Once we have a list of the tokens composing the vocabulary, we associate each one with an integer. For example, if the vocabulary is “Roses, are, red, Violets, blue”, we can create a mapping: Roses : 0, are: 1, red: 2, Violets: 3, blue: 4.

This procedure has been tested on Fedora 29, on a HP laptop with this graphical card: NVIDIA Corporation GP107M GeForce GTX 1050 Mobile (rev a1)

The commands have to be run as the root user. This tutorial assumes the nvidia driver is already working.

Install pip

dnf install python3-pip

Install Cuda 10.0

Download the installer from the Nvidia website and run it. Make sure to install the Perl module Term::ReadLine::Gnu beforehand because the cuda installer relies on it.

With its recent gain in popularity, a lot of things have been called “Artificial Intelligence”. But what is it anyway? According to Wikipedia, it’s “intelligence demonstrated by machines”, but does such a thing exist? At time of writing, they are 4 main types of AI development algorithms.

• Expert systems defines a category of computer programs that are specifically designed to do a task using prior human knowledge. Software engineers work closely with a domain expert to build the program, that will act in a predicable way, like the domain expert would have done if he or she had the same processing power. For example, Deep Blue is an expert system designed by IBM to play chess that won against the former world champion Kasparov.

The drawback of the expert system is that it’s tough to maintain (because it requires an expert on the domain at hand) and it’s costly.

Stochastic Gradient Descent (SGD) is used in many Deep Learning models as an algorithm to optimize the parameters (the weights of each layer). Here is how it works:

At each step in the training process, the goal is to update the weights towards the optimal value. For this, SGD uses the equation:

As Deep learning is becoming more and more popular, there is an ongoing debate on whether it’s possible to create Deep Learning applications with a Free Software license. See for example this discussion on the debian-devel mailing list.

The argument we often see is that:

• It’s impossible to study the inner workings of a Deep Learning software (for example, an image classifier or a text generator) or improve it, because one cannot understand how it’s going to make predictions only by looking at the weights of the Deep Learning model
• Training a Deep Learning model requires a specialized and expensive hardware that runs non-Free software

But the first statement misses the point of Deep Learning programs. We should not treat deep learning programs as the “regular” ones. A regular program contains a set of tasks the computer has to do. The human has the knowledge of how the tasks that should be completed. But this is not true for Deep Learning. The software is not the set of actions that solve the problem, it is the set of instructions used to learn how to solve it. So the Deep Learning program is not the knowledge (the weights) used to perform the mission, it’s how to guide computers to that knowledge. In a way, this is similar to the compilation of a large program to assembly. The compilation output is hardly readable and editable, but the program can easily be studied and analyzed. The same goes for Deep Learning if we consider the model weights as the compilation output. They are not meant to be edited by hand.

This procedure has been tested on Fedora 28, on a HP laptop with this graphical card: NVIDIA Corporation GP107M GeForce GTX 1050 Mobile (rev a1).

The commands have to be run as the root user. This tutorial assumes the nvidia driver is already working.

Install pip

dnf install python3-pip

Install Cuda

We install cuda 9.0 as it is the latest version supported by tensorflow at the time of writing.

wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda_9.0.176_384.81_linux-run
sh cuda_9.0.176_384.81_linux-run

You will have a few choices. Select “install cuda”, don’t install samples or drivers.Select “create a symbolic link”. You can set the TMPDIR en variable before executing the script to select the location of the temporary folder used during the installation. Can be used if the default, /tmp, is not large enoguh.