TensorFlow And Deep Learning: Starting With The Basics
Being able to identify patterns and predict events in your data allows for much more effective decision making. However, manually doing this is practically impossible, especially if you have a substantial amount of data to go through. It’s why many businesses have begun to implement deep learning models. Deep learning is a form of machine learning (which are models that use algorithms to analyze data and to learn from it) that consists of networks that can learn from large sets of unstructured and unlabeled data without human input. While there are numerous libraries for deep learning that you can implement, the most famous and commonly used is Google’s TensorFlow.
What Is TensorFlow?
TensorFlow is an open-source software library built by Google for the purpose of implementing deep learning systems. It’s a library for dataflow programming and uses optimization techniques to make it easier and more effective to calculate mathematical expressions.
Before Google created TensorFlow, it had been using machine learning in almost all of its products. Machine learning was used to improve its search engine, recommendations, translations, and image capturing. Take for example when you type a query into their search bar — their machine learning function allows Google to guess what you’re typing in as you’re typing. However, they soon realized that deep learning would outperform their traditional machine learning models due to their massive datasets and that it would greatly improve their search engine as well as their email application, Gmail.
They built the TensorFlow framework with the goal of allowing researchers and developers to work together on an artificial intelligence (AI) model. Once it was built and scaled, it was made available for free. Google released it in 2015, followed by a more stable version in 2017. Because it’s an open-source framework under the Apache Open Source license, anyone can use it as well as modify it and even redistribute modified versions for a fee without having to worry about paying a licensing fee to Google.
Components Of Tensorflow
TensorFlow consists of three main components: the dataflow graph, sessions, and the tensors themselves. Here’s a brief rundown of each component and how to execute them:
Dataflow Graph
TensorFlow uses a graph framework as its spine. Every computation, operation, and variable resides in the graph. Using dataflow graphs, TensorFlow can easily run on multiple CPUs, GPUs, and mobile operating systems. Graphs are also portable, making it easier to preserve computations for later use since graphs can be saved and executed at a later date.
Tensorflow Environment Or “Sessions”
Although TensorFlow uses graphs to define operations, operations can only be run within a session. Graphs and sessions are created separately. The graph helps to define the computations. Think of the graph as a blueprint of sorts. Variables and values do not exist within the graph until the graph is run within a session. A session can be created by:
After opening a session, you’ll need to close it afterwards. You can also use with block. Using with block will automatically close your session at the end of the with block.
You can use with block by:
Tensors
Data is held by TensorFlow using Tensors. Tensors are similar to numPy multi-dimensional arrays. There are three main types of tensors, which include constants, variables, and placeholders.
Data Types:
Constants
Constants are tensors whose value cannot be changed. A constant can be declared by:
Variables
Unlike constant tensors, variable tensors can hold different values. In order to hold different values, however, variable tensors need to be initialized separately by an initial operation. As you can imagine, attempting to initialize all of the variables individually would be a time-consuming task. Fortunately, TensorFlow provides a function that lets you initialize all the variables at once.
Placeholders
Placeholder tensors are used for training data that is initialized when the code is executed inside a session. Essentially, placeholders are just tensors that are waiting to be initialized.
Neural Network In Tensorflow
The neural network, which is also called the artificial neural network, is a set of algorithms modeled on the biological neural networks that make up the human brain. Like the brain, the artificial neural network is capable of storing massive amounts of data and can “learn” to perform tasks by considering the data it has access to and without being specifically programmed to perform those tasks. Neural networks help to cluster and classify data based on similarities to example inputs.
There are two main types of neural network architectures: encoders and decoders. However, some neural network models can be a combination of both. Neural networks that are encoders identify patterns in raw data and translate that data into more organized and useful representations. Decoder neural networks use the representations created by encoders to generate high-resolution data.
Installation – Tensorflow
Now that you have a basic idea of how TensorFlow works and what its components are, it’s time to install TensorFlow so that you can begin building your deep learning architecture. Before doing anything, the first thing you should keep in mind is that TensorFlow is tested and supported on these 64-bit systems:
- macOS 10.12.6 or later (no GPU support)
- Raspbian 9.0 or later
- Ubuntu 16.04 or later
- Windows 7 or later
1.Download a Package
There are several ways to install TensorFlow; however, arguably the best way is to use Python’s pip package manager, which is available for macOS, Raspberry Pi, Ubuntu, and Windows systems. In order to install TensorFlow, you will first need to install the Conda package dependency management system, which will allow you to separate multiple environments on a single system.
Once you’ve installed Conda, you can create the environment you need to install TensorFlow. To install the TensorFlow Python API in the Condo environment, you will need to use Python 2.7. You will then use the pip installation tool, which is a standard part of the Conda environment, to install the TensorFlow library. Just make sure you update the pip installation tool to the latest version first.
2. Run a Tensorflow Container
Containers are used by Docker to create virtual environments. This isolates the TensorFlow installation from the rest of your system. As a result, you will be able to access directories, connect to the Internet, use the GPU, and more since you will be able to share resources with the host system. To run a TensorFlow container, you will need to install Docker onto your local host machine.
Create A Neural Network
Once you’ve installed TensorFlow, you can begin building your neural network architecture. Building a basic neural network architecture will require these six steps:
Build
- Define your placeholders – Because you don’t have any real data yet, you will need to define placeholders for inputs and labels. Once you run the session, your placeholders will obtain the correct values from your dataset.
- Flatten the input – Once you’ve defined your placeholders, you will need to use the flatten() function to flatten the input. Doing this will provide you with an array of shapes instead of the shape of your grayscale images.
- Construct a fully connected layer – You will need to build a fully connected layer that generates logits of size, which uses the previous layers’ unscaled output to operate. In order for logits to understand that the units are linear, it must use the relative scale.
- Define the loss function – Loss function represents the cost associated with the event that it maps out. You won’t be able to define the loss function until you’ve built out the multi-layer perceptron. Choosing a loss function will depend on the task at hand. One thing to keep in mind is that even though regression is used to predict continuous values, you’ll need to use classification to predict classes of data points or discrete values.
- Define a training optimizer – Defining a training optimizer offers you several options. ADAM, RMSprop, and Stochastic Gradient Descent are considered some of the more popular optimization algorithms. Certain parameters will need to be tuned based on the optimization algorithm you choose.
- Initialize operations – Before you can run the neural network you’ve just built, you will need to initialize operations to execute.
Run
- Once your neural network architecture is in place, run a session. Follow these steps to successfully run your new neural network:
- Initialize a session – Initialize a session by using Session(). You can pass the graph that you defined while building your neural network to Session().
- Run a session – Once initialized, use run() to the session. You can then pass the initialized operations using the init variable defined while building your neural network.
- Begin training loops – Begin training loops or epochs by choosing 201 so that you can register the last loss_value.
- Use the training optimizer – Use the training optimizer when running your session through the training loop. Use the loss metric that you defined before while building your neural network as well.
- Pass a feed_dict argument – While in the training loop, begin feeding data to the model by passing the feed_dict argument. You should receive a log that provides insight into the loss or cost of your model after every ten training loops (or epochs).
Evaluate
Even though you have successfully run your neural network, you will still need to test it. One way to evaluate how it is performing is by randomly selecting a handful of images and comparing their predicted labels with the real labels. This will give you a good general idea; however, you’ll want to do some further testing for more accurate insight. This can be done by loading in your test data using the load_data() function. Be sure to keep track of any training and testing errors that come up. If both training and testing errors go down and then go back up, then you should stop training. If this happens, it means that your neural network is likely overfitting the training data.
What Is Deep Learning?
Whereas a basic neural network consists of three layers, a deep neural network is a neural network that uses more than three layers. These additional, hidden layers are the foundation for a deep learning system, which allows a computer to train itself to process and learn from available data and execute tasks based on what it has learned without being taught or instructed to do so.
Architectural Paradigms Of Deep Learning
There are three categories of machine learning — supervised learning, unsupervised learning, and reinforcement learning. Deep learning models can be built using any of these three types of algorithms. However, there are seven main types of deep learning. The first four types are supervised learning models, the fifth and sixth are unsupervised, and the last one uses reinforcement learning.
Feed Forward Neural Networks (FFNNs)
A feed forward neural network is one that doesn’t make use of cycles. Instead of data cycling through, it passes from input to output without any state memory of what came before. You could argue that all neural networks are feed forward; however, the term “feed forward neural network” actually refers to its densely-connected MLP (multilayer perceptron).
Convolutional Neural Networks (CNNs)
A convolutional neural network, also commonly referred to as a ConvNet, is a type of feed forward neural network that can learn local patterns in images through the use of a spatial-invariance technique. ConvNets are able to do this efficiently by sharing weights across space. A ConvNet uses convolutional layers to detect, classify, and recognize data that has spatial invariances within its structure; for example, in images, video, and speech audio.
Recurrent Neural Networks (RNNs)
Unlike feed forward neural networks, recurrent neural networks do use cycles. This means that they also have a state memory. Whereas a ConvNet is able to share weights across space, a Recurrent Neural Network shares weights across time, allowing it to process and represent patterns in sequential data. A Recurrent Neural Network can also become a feed forward network where weights are shared if it is unrolled in time. Common applications of Recurrent Neural Networks include speech recognition, speech generation, and natural language modeling.
Encoder-Decoder Architectures
Feed Forward Neural Networks, Convolutional Neural Networks, and Recurrent Neural Networks all make predictions using either a dense encoder, convolutional encoder, or recurrent encoder. Depending on the raw data that you’re creating a representation of, these encoders can be switched. Encoder-Decoder architecture is a bit different. Instead of making a prediction, it builds on the encoding step to generate a high-dimensional output. It does this through a decoding step. For instance, an image capturing network using a convolutional encoder for image input may use a recurrent decoder for a natural language output. Encoder-Decoder architectures are commonly used for machine translation and semantic segmentation among other applications.
Autoencoders
When it comes to unsupervised learning, Autoencoders are one of the basic forms. Autoencoders use encoder-decoder architecture, which allows them to generate an exact copy of the input data. Because the ground truth data is obtained via the input data, it does not require human instructions. Some of the common Autoencoder applications include image denoising and unsupervised embedding.
Generative Adversarial Network (GANs)
A Generative Adversarial Network is a framework for a training process that involves two networks: the Generator and the Discriminator. The purpose of the General Adversarial Network is essentially to generate new samples that are realistic from a specific representation. The Generator Network generates new data instances in an attempt to fool the Discriminator Network, which determines whether images are either real or fake.
Deep Reinforcement Learn (Deep RL)
Deep Reinforcement Learn is a framework that uses reinforcement learning to determine how to act in the real world in order to maximize rewards. Deep Reinforcement Learn frameworks allow you to apply neural networks in either real-world or simulated environments where a number of decisions need to be made. There are three types of Deep Reinforcement Learn frameworks — model-based, policy-based, and value-based. Common applications include neural architecture search, game playing, and robotics.
Additional Resources and Educational Sources
Although this overview should give you a good idea what TensorFlow is and how you can use it to your benefit, to get a better a better understanding of how to effectively implement and use TensorFlow, consider taking an online course. Fortunately, there are a number of worthwhile tutorials and courses available online. The following are a few highly recommended TensorFlow resources to look into:
TensorFlow Tutorials for Beginners
TensorFlow Tutorial: Deep Learning for Beginners
One of the reasons that Google’s TensorFlow is the most popular deep learning framework available is because it’s built to be accessible for everyone, which is why it remains open-source. The TensorFlow library is capable of incorporating different APIs to build deep learning architecture, such as CNN or RNN. Additionally, it can be built to be deployed at scale. As a result, there are many algorithms available that are supported by TensorFlow. If you’re looking to implement deep learning into your data analysis capabilities, consider using TensorFlow.
Learn more about Tensorflow and deep learning by calling our team of experts today.