One More Step Closer to a Self-Driving Car!
Self-Driving Cars is a field that continues to gain much popularity and interest, rightfully so. Numerous companies like Uber, are investing in self-driving cars and are pursuing to make self-driving cars a more common and reliable part of our future. Personally, I am really interested in different types of futuristic transportation systems, from self-driving cars to space elevators, to re-useable rockets. Other cool projects like Hyperloop, and the Boring Company are also really fascinating. In fact, I wrote an article on the different super cool futuristic technologies. Check that out here:
However recently, my interest in self-driving cars has spiked. I have realized how realistic self-driving cars have really become and the potential that self-driving cars have. Not only are we seeing many today, but in the future it is possible to only see self-driving cars on the road. So these autonomous vehicles are really interesting and I asked myself: How hard would it be to build one? The real answer to the question is, very hard. But I decided to break down the technological aspect of the self-driving car.
There are many different parts and systems involved in making a self-driving car. Some of the examples are, control (how the car is able to actually drive itself), path planning (how the car is able to decipher the best possible route to take), localization (how the car is capable of determining where it is located), sensor fusion (how the car is able to sense what is around it, its environment), and finally computer vision (how the car is able to see the different objects around it). In my last project, I decided to create a system that uses computer vision to detect different road signs. I created an article discussing the project and the code itself. You can check that out here:
I also created a Youtube video to discuss the code for that project. You can find that here:
After completing that project, I thought it would be fitting to attempt a similar project but conquering a different aspect of the self-driving car. I decided to choose one of the systems listed above, and I went with Path Planning!!
Path Planning plays an essential part in not only self-driving cars but also in GPS systems as path planning helps determine what is the optimal route for a vehicle or person to take. I therefore went ahead and gave a project a shot. Now, the project that I was able to complete involves path planning but is a much different system than those used in self-driving cars as those developers have to consider so many different variables when path planning. I wanted to use these projects as a method of getting better at coding and learning how different systems in a self-driving car work, so these projects will not be exactly how the systems in real self-driving cars work but rather a much, much smaller version.
The Project:
For this project, I was very fortunate to have learned how to code prior to the previous project. This project is essentially all coding and without knowing how to code the project would be simply impossible to code. If you are looking to learn Python, I highly suggest you take a look at this video:
In this article, I will go over my code which detects different objects and in part two of this article series, I will breakdown the path planning code for this project.
Now let me get into the project itself and break down the code for you! I will explain the major steps I took when coding the project and explain what it is used for and the importance of having it in my code.
Step 1:
The first thing that I did in this project, is once again get data to actually run the system. In this case I found 4 test samples of 10 x 10 grids with different shapes and colors on them.
Each color/shape represents a different object, or location in the case of self driving cars. The black squares represent obstacles. Each square on the grid is defined through an X, Y coordinate. Now our goal for this project is for our system to be able to detect these objects and obstacles and find pathways from one object to another.
Step 2:
The next step in the project was to start the actual code. Similar to the second step in the last project, the second step in this project required me to install and import all the useful libraries which would allow my code to run different functions.
For this project, I installed and imported openCV, numpy, time, and from skimage, I imported structural_similarity. OpenCV is very useful in projects like these as openCV does favors when working with image related problems. So when we were detecting different road signs, we also used openCV. Numpy is very useful for scientific computing, time, for measuring time and structural_similarity is very useful for comparing and contrasting two different images.
Now, I imported astarsearch and traversal. Astarsearch is super important in this project. It is a very powerful and one of the best pathfinding algorithms that exists. Astarsearch is used in many video games played today too. Traversal is a very important part of this project too as it will examine the image we give it, in this case the 10 x 10 grid, and scan it from left to right. All of these libraries/functions play a big part in the execution of this project.
Step 3:
The next thing I did, was create a bunch of lists and open the 10 x 10 image.
The image above demonstrates the different lists I created. Starting with a list of the coordinates of the squares that are occupied and then a dictionary to store the information that has to do with the actual planning of the optimal path. Next, I opened the image and adjusted it to a 60 x 60 scale. Now, I created another list, this time for the coordinates of the squares with obstacles in it. The next part of the code is to create a list of the 100 different images in the grid.
Step 4:
Our next step is to travers the image and actually start looking for occupied squares and be able to print those squares out. We can do this by taking the average colour of the row and the square. If the square is not a majority white, that means that it is an occupied square and it can be appended as a tuple into the list of occupied squares.
So at the top of this image of code, the traversing starts. The first thing we need to do to make sure we are getting the correct squares, is to code that if the squares do not match the correct size, 60 x 60, the system should ignore them.
Next, we make a copy of the image where we can crop different images and add it to the array of images. The next thing we want to do is check for the occupied squares by the colour of the grid and squares. We can check this by collecting the average colour of the row and then of the square. If the image is not a majority white, the square can then be determined as occupied.
We then do something similar but this time to check if the object is black. If the object is black, we can go ahead and append the coordinates of the object in the obstacles list.
Step 5:
Our next step is to compare images using the structural_similarity tool. Comparing the images will help us realize how similar the objects are and once it is determined that the objects are similar enough, we can proceed with the path planning algorithm.
Here, the very important step to consider is to convert both images to a grayscale. The reason this is done is to speed up the process and reduce the dimensionality of the images. We are able to do this as the objects in the 10 x 10 grid are different in colour, but also in shape. So, we only need to detect the different shape and not always colour. Therefore, changing both images to grayscale will quicken and simplify the process. In order to make sure the objects are similar enough, they must be 90% similar, and if they are, we can move on to the path planning phase.
If we were to print the code so far, we would get a list of the occupied grids. Our next step is to code the astarsearch algorithm to allow our system to find the optimal path from one object to another similar object. I will hopefully explain this part of my code in the second article!
So stay tuned for the second and final article in this short series. In the next article, I will aim to breakdown my astarsearch/path planning code to you and show you the final product of this project!
