Neural networks are notoriously hard to program because of their high complexity and the multiple components in them. However, with the help of deep learning APIs, you can significantly reduce the requirements to produce, test and deploy neural networks.
One such API is Keras. However, after creating your neural network, how do you know if you are using the most optimum parameters? Luckily Keras provides a module called the Keras Tuner to tune your hyperparameters. In this tutorial titled ‘A guide to learning all about Keras Tuner’, you will learn how to implement Keras Tuner and find the best hyperparameters.
What are Neural Networks?
Neural Networks are deep learning algorithms designed after the human brain. They break down complex problems into smaller parts and solve them individually, by combing and refining the individual output to get the final output.
The neurons are mathematical operators, or algorithms used to solve a small part of the problem. The output of a neuron leads to another, where the output is further refined. The figure depicted below shows a neural network with its different layers. There are three primary layers in a neural network. The work of these layers is as shown:
- The input layer takes in the input and brings it into the system.
- The hidden layer processes the data through activation functions and adds weights according to their importance to the final output.
- The output layer combines all data processed by the hidden layer and gives the final output.
Figure 1: Neural network
What is Keras?
Keras is an open-source, high-level, deep learning python API for easy implementation, training, and deployment of neural networks and deep learning applications. It supports multiple deep learning APIs as a backend.
Figure 2: Keras Logo
Keras provides an easy-to-use backend for programmers while using a high-level API like Tensorflow as a backend. This helps provide fast computation power while being easy to use.
What is Keras Tuner?
The various variables which affect the rate of learning of the machine learning model are called hyperparameters. Hyperparameters control the performance of a model. Using Keras Tuner, you can find the best value of hyperparameters for the models. This process is also called Hyperparameter Tuning. The diagram shows the working of a Keras tuner :
Figure 3: Keras Tuner
Hyperparameter tuning is a hit and trial method where every combination of hyperparameters is tested and evaluated, and it selects the best model as the final model.
To work with the Tuner, you have first to install it. The process of installing Keras Tuner is simple. You can do directly it via the pip install command in the command prompt or can be installed from the GitHub repository.
You can also do the installation from git by cloning the Keras Tuner repository using the git clone command followed by the repository URL.
Figure 5: git installation
Optimizing the Number of Hidden Layers and Hidden Neurons Using Keras Tuner
Now, look at how to implement hyperparameter tuning using Keras tuner. You start by importing the necessary modules :
Figure 6: Importing necessary modules
You will work on the MNIST dataset, which consists of many images of handwritten digits and their labels.
Figure 7: MNIST Dataset
You will separate the images from the labels and convert the image into a float data type.
Figure 8: Dataset manipulation
You will then create our machine learning model.
- Create a 2-D convolution neural network with three layers where each layer consists of a MaxPooling and AveragePooling layer and relu activation function.
- If GlobalPooling is chosen as ‘hp,’ we will use GlobalMaxPooling; otherwise, if GlobalAveragePooling is selected as ‘hp,’ you will use GlobalAveragePooling.
- You will finally decide on a compiler and return the model output.
Figure 9: Neural networks model
After creating the model, create the tuner using the Hyperband module in Keras Tuner. You will give it your model as input and select the number of epochs as 8.
Figure 10: Keras Tuner
You will then see a summary of the model by using the search_space_summary() method.
Figure 11: Model summary
Finally, you will have to train the models and evaluate the outcome of each set of hyperparameters. Send in 600 steps and have 2 epochs on your mnist dataset.
Figure 12: Implementing Tuner
You can see the best set of epochs will give us an accuracy of 98%. The set of hyperparameters for which you obtain this accuracy is shown below.
Figure 13: Best hyperparameter set
The below method will give you a summary of the entire process.
Figure 14: Test results
The above image is just a tiny part of the total summary of the test done for each iteration.
Master Deep Learning, Machine Learning, and other programming languages with Artificial Intelligence Engineer Master’s Program
Conclusion
In this tutorial titled 'A guide to learning all about Keras Tuner', you looked at neural networks and the deep learning API, Keras. You then briefly look at Keras Tuner and how to install it. You finally learn to use python to implement a program to perform hypermeter tuning.
We hope this helped you understand Keras tuners in depth. To learn more about Keras, deep learning, and machine learning, check out Simplilearn’s Artificial Intelligence course. If you on the other hand need any clarifications on this Keras tuner tutorial, do share them with us by mentioning them in this page’s comments section. We will have our experts review them and reply to your comments at the earliest!
Happy learning!