Stochastic Gradient Descent and its variants

Stochastic Gradient Descent (SGD) is used in many Deep Learning models as an algorithm to optimize the parameters (the weights of each layer). Here is how it works:

At each step in the training process, the goal is to update the weights towards the optimal value. For this, SGD uses the equation:

Read More

On Deep Learning and Free Software

As Deep learning is becoming more and more popular, there is an ongoing debate on whether it’s possible to create Deep Learning applications with a Free Software license. See for example this discussion on the debian-devel mailing list. The argument we often see is that: It’s impossible to study the inner workings of a Deep Learning software (for example, an image classifier or a text generator) or improve it, because one cannot understand how it’s going to make predictions only by looking at the weights of the Deep Learning model Training a Deep Learning model requires a specialized and expensive hardware that runs non-Free software But the first statement misses the point of Deep Learning programs.
Read More

How to install cuda 9.0, cudnn, tensorflow, pytorch on Fedora 28

This procedure has been tested on Fedora 28, on a HP laptop with this graphical card: NVIDIA Corporation GP107M GeForce GTX 1050 Mobile (rev a1). The commands have to be run as the root user. This tutorial assumes the nvidia driver is already working. Install pip dnf install python3-pip Install Cuda We install cuda 9.0 as it is the latest version supported by tensorflow at the time of writing. wget https://developer.
Read More

On trading security for convenience

If we have to choose between a convenient system and a secure one, we often pick the former rather than the latter. The reason is mainly psychological. Several scientific studies have shown that we prefer instant gratification over delayed gratification, because that’s how our brains are wired. We are surrounded by instant gratification, our day-to-day actions like our hobbies, usage of social media, got us hooked on having a quick feedback.
Read More

As developpers we are doing the easy work

We should not think that programming is complicated. It is often the easiest part of an IT project, because one simply needs to communicate with a computer. The communication between human beings is far more complicated. Computers are the most predictable things in the universe. Humans are unpredictable by nature. They can lie, change mood or theirs opinions multiple times, decline an offer because they had a bad day or due to the weather, or the horoscope, you name it.
Read More