Lehigh University logo
Lehigh University logo
Lehigh University logo

Jack Circus, Logan Herr & Samuel Presti

Self-Driving Robotic Car Using Machine Learning And Image Recognition

Department: Intergrated Business & Engineering / Industrial and Systems Engineering
Advisor: Martin Takáč

Abstract:
Our goal for this project was to explore the use of machine learning and image recognition to manipulate a robotic car through an obstacle course. We were able to interface with and control the car through a combination of bluetooth, python, and arduino software. Next, we implemented a python module called YOLO, a popular image recognition tool which uses convolutional neural networks (CNN) to determine what is in an image. We also implemented Principal Components Analysis (PCA) techniques to take image recognition one step further and determine the orientation of an object from above. Given more resources and time, the next steps in our project would be to apply these image recognition technologies to an overhead course with obstacles, much like a parking lot, where our robotic car would be tracked in position and orientation from a live streaming bird’s eye view camera. We would then develop a deep and reinforcement learning system where states of the system environment would be fed into a decision-making algorithm which would determine which actions to take with the robotic car, such as move a small step in a specific direction towards a target location. Here we would implement machine learning techniques, where the reward and reinforcement includes getting closer to the target point on the course and maintaining distance from obstacles. Ultimately, real-world applications include optimizing systems where many self-driving robots would interact all at once over a unified network, such as warehouse picking robots, large parking lots with automated cars, and more.

About Jack Circus:
Jack Circus is a Senior majoring in Industrial & Systems Engineering in the Integrated Business and Engineering Program. He is originally from Mount Laurel, a suburb in south Jersey. During the summer after Sophomore year, he interned for Dolezych GmbH in Dortmund, Germany through the Iacocca Institute, and this past summer he worked in New York City as a Risk IT analyst for Jefferies Group. After graduation he will be moving to Chicago to work as an Analytics Consultant for IBM. At Lehigh, he is a teaching assistant in a freshman engineering and coding class, works at the school’s gym as a lifeguard, and was a member of the executive board of Delta Upsilon.

About Logan Herr:
Logan Herr is a senior in the Integrated Business and Engineering (IBE) program, pursuing dual degrees in Industrial & Systems Engineering and Finance. Logan has previously interned at Volvo as a manufacturing engineer, and most recently at PricewaterhouseCoopers as a technology consultant. At Lehigh, he is a Rossin Junior Fellow, one of a group of mentors working with prospective and first-year engineering students, and a Head Gryphon (Head RA) in university housing. Logan is also a member of the Martindale Student Associates Honors Program, where he is writing a research paper soon to be published in Volume 36 of the Perspectives on Business and Economics journal. In his free time he loves adventuring outdoors as well as competing on the men’s club rugby and volleyball teams. After graduation, Logan will begin working full time as a technology consultant for PwC, hoping to eventually go to graduate school for his MBA.

About Samuel Presti:
Samuel Presti, is a senior at Lehigh University, studying Industrial Engineering through the Integrated Business and Engineering Honors Program. On campus, he has been involved in the Gryphon Society, as a residential advisor and community organizer to floors of over 20 students. Sam also worked on TEDxLehighU, which is a student organized TED talk with speakers from the campus community, Bethlehem community, and Alumni. During the summer before his junior year Sam interned for Acsis Inc, a supply chain management software company based out of Mount Laurel, New Jersey. With a strong passion for technology, Sam was able to put his knowledge to use in the field last summer through an internship at PricewaterhouseCoopers while also gaining firsthand experience with SAP implementation on the job as a Technology Consultant. Sam is excited to start working for PwC full time after he graduates.