Prototype Self Driving Car has been created to imitate human abilities in driving according to the road. The navigation systems uses the concept of Visual Servoing with camera sensor to process images and adjust motion control of the car’s position towards the road lines using Python Programming.

Prototype Self Driving Car using Visual Servoing and Python Programming

As a step toward the goal of developing autonomous self-driving car, i develop a prototype of self driving car using visual information and visual servoing technique for which the robot able to detect road lane and navigate autonomously.

In this project, Prototype Self Driving Car has been created to follow the road markings that have been detected by the camera directly mounted on robot and avoids obstacles in front of the car by using ultrasonic sensor. The Raspberry Pi camera is used to detect the road lane markings and gives visual information that processed by the Computer Vision method using OpenCV library such as Grayscale, Color Thresholding, Region Of Interest, Hough Lines Transform, Perspective Transform, and Road Lines Centering. The robot is tested and applied real-time using Python Programming on the Raspberry Pi 3 model B+.

Result of Image Processing show the center value of the x-axis pixel road marking and the middle value of the camera x-axis pixels when follow the road. The aim of the control system is to maintain the heading of the robot by adjust the motor servo angle for steering control. Experiments show the robot is able to navigate autonomously following both painted and unpainted road markings and avoids obstacles.

My name Tito Valiant Muhammad Fadillah and i'm a Mechatronics (Robotics) Engineering from Politeknik Elektronika Negeri Surabaya and i've finished my study as a cumlaude with GPA 3.53 of 4.00. I've been created several robots like transporter arm robot, line tracer robot, voice recognition mobile robot, prototype autonomous car, etc. I have become a public speaker at my university and I enjoy being a speaker with my competence in robotics.

In this paper, the original size of captured image is 448x208 pixels RGB color image. The aim of image processing method is to obtain the visual information data from continuous image in front view of vehicle as the input of decision module and get the data needed to control the car's motion. The procedures for Road Markings detection are described as follows:

Convert Image

The RGB value is a function of the overall brightness as well as the color of the object. In order to minimize the processing time, the RGB color image need to convert to grayscale image. The processing of grayscale image becomes minimal as compared to RGB color image, this function transforms a 24-bit, three-channel, color image to 8-bit, single-channel grayscale image.

grayscale = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

Color Thresholding

The Comparison between the image pixel point that has a difference in threshold intensity or high contrast will be detected as the road lines. In my experiments the low threshold value is 127 and the high threshold value is 255.

ret, thres= cv2.threshold(grayscale, 127, 255,cv2.THRESH_BINARY_INV)

Region of Interest

Selecting the region of interest (ROI) from the color thresholding image by concentrating on the Road Markings region can greatly reduce the time of algorithm and increase running process speed. In ROI, the system can set two detection lines to find a path near the middle of the ROI. If the lane mark position is located, it will be represented as vehicle trajectory data which is then used in the system decision module.

Hough Lines Transform

The image processing method of hough lines transform is used for the process of displaying road markings on initial images that have RGB format after processing using the Color Thresholding feature in binary format in the specific area that specified by the region of interest function. This method requires OpenCV function [ cv2.HoughLinesP() ].

Perspective Transform

The application of this feature is to obtain a two-dimensional view of the top “birds eye view” detected road lane by choosing a rectangular pixel point value as the new specific area using the OpenCV function [ cv2.getPerspectiveTransform() ] and [ cv2.warpPerspective() ]. The specific area according to the selected trapezoidal pixel points as the boundary to display the visual information. So that the process of obtaining parameter values ​​as feedback control of the car becomes easier because it only detects one part/side of the road markings.

Road Lines Centering

After getting a specific area from the Perspective Transform process, system needs [ cv2.moments () ] function to calculate centroid point of the middle value of the x-axis pixel image of a detected road markings. The purpose of this function is to calculate the difference value (error) between the center value from the x-axis pixel road marking and the middle value of the robot's x-axis pixels. The results of difference value calculation (error value) will be used to adjust the motor servo angle for steering control.