About Me

I have received my PhD from the Institute for Software Research at Carnegie Mellon University under supervision of Prof. Claire Le Goues. My main research interests are quality assurance, testing, and validation for robotics and cyberphysical systems. I also have research experience in automatic program repair, and program analysis. I worked at Nuro as a senior software engineer for 2 years, working on automated testing and validation of Nuro’s autonomous vehicle in simulation using a variety of heuristic and machine learning methods. I am now the Technical Lead of Autonomy Validation at Zipline, focusing on validation and risk assessment of autonomous drones. You can download my CV here.

Education

Doctor of Philosophy in Software Engineering

2021
Carnegie Mellon University

Master of Science in Software Engineering

2019
Carnegie Mellon University

Bachelor of Science in Software Engineering

2015
Sharif University of Technology

Thesis: Bisulfite Genome Sequencing

Experiences

Technical Lead

2023
Zipline Inc. (South San Francisco, CA)

Technical Lead of Autonomy Validation focusing on validation and risk assessment of autonomous drones.

Senior Software Engineer

2021 - 2023
Nuro Inc. (Mountain View, CA)

Member of the Simulation team focused on automating processes required for large scale testing and validation of AVs in simulation using a variety of heuristic and machine learning methods.

Product Integrity Intern

2018
Apple Inc. (Cupertino, CA)

Interned with Mac Systems Quality team at Apple Inc., where I developed a platform that automates many procedures of testing Mac systems.

Software Research Intern

2017
ABB Inc. (Raleigh, NC)

Interned at ABB corporate research center, where we developed and evaluated a robotics programming environment for novice programmers.

Senior Application Architect

2013 - 2015
Congenial Mobile Co. (Tehran, Iran)

Developed and managed the payment system of CafeBazaar Android application store.

Research

My research focuses on making robotics and cyberphysical systems safe, and improving the quality assurance of these systems. For this purpose I have done a number of empirical studies that identifies the challenges and bottlenecks of automated testing in robotics. In addition, I have developed tools and benchmarks towards building an automated testing pipeline for robotic systems. In addition, over the years I have also worked on automatic program repair, where given a faulty program and a set of test cases (some of which are failing), patches are automatically generated in a way that the final program passes all the test cases.

Robotics Quality Assurance - In this project, we discover and develop powerful methods to automatically detect, localize and fix defects in real-world autonomous systems using low-fidelity, software-based simulation.
SOSRepair - Automatic program repair tool which generates high quality patches by replacing the buggy fragment of code with a correct implementation that is already written by developers in maybe some other project.
CoBlox - In this project, with inspiration from block-based programming languages, initially designed as an introductory programming environment for education, we created CoBlox, a block-based interface for programming a one-armed industrial robot.

Presentations

Panel: Software Quality in Robotics
November 2020
ROS World

A robot is only as good as the software that runs it. This panel addresses questions related to software quality in robotics. While there have been vast improvements in many areas of robotics, the quality of the end product is always constrained by the quality of the software that underlies the system. Panelists have worked with software quality in robotics in general and ROS in particular and will include perspectives from both industry and academia. Topics include: the current state of the art in quality assurance practices for robotics; the role of simulation; promoting quality through good development practices; ways to improve testing; software quality in projects with participants of different backgrounds; and unique quality features of the ROS ecosystem.

Video
A developer perspective on the challenges of automated testing for robots
July 2020
ROS Quality Assurance Working Group

How do robotics developers test their systems? What are the main challenges of testing robots? Is it possible to automatically test robots in simulation? In this talk, I present the findings of several empirical studies with robotics developers, conducted by our research group, designed to identify both the state of the practice and challenges faced by developers when testing their systems. Considering the importance of test automation, and the potential of simulation-based testing, I will focus on presenting the challenges and bottlenecks of test automation for robotics, and how they are impacted by the capabilities of existing simulators.

Video

Publications

Simulation for Robotics Test Automation: Developer Perspectives, Afsoon Afzal, Deborah S. Katz, Claire Le Goues, and Christopher Steven Timperley, in International Conference on Software Testing, Validation and Verification, ICST ’21, 2021.
PDF · Data · BibTeX
@inproceedings{AfzalSimulation21,
  title = {Simulation for Robotics Test Automation: Developer Perspectives},
  author = {Afzal, Afsoon and Katz, Deborah S. and {Le~Goues}, Claire and Timperley, Christopher Steven},
  booktitle = {International Conference on Software Testing, Validation and
                    Verification},
  series = {ICST '21},
  year = {2021},
  month = apr
}

Mithra: Anomaly Detection as an Oracle for Cyberphysical Systems, Afsoon Afzal, Claire Le Goues, and Christopher Steven Timperley, IEEE Transactions on Software Engineering, pp. 1–1, 2021.
PDF · Code · Data · DOI · BibTeX
@article{AfzalMithra,
  author = {Afzal, Afsoon and Le Goues, Claire and Timperley, Christopher Steven},
  journal = {IEEE Transactions on Software Engineering},
  title = {Mithra: Anomaly Detection as an Oracle for Cyberphysical Systems},
  year = {2021},
  volume = {},
  number = {},
  pages = {1-1},
  doi = {10.1109/TSE.2021.3120680}
}

GzScenic: Automatic Scene Generation for Gazebo Simulator, Afsoon Afzal, Claire Le Goues, and Christopher S. Timperley, . 2021.
PDF · Code · BibTeX
@misc{AfzalGzScenic,
  title = {{GzScenic}: Automatic Scene Generation for Gazebo Simulator},
  author = {Afzal, Afsoon and {Le Goues}, Claire and Timperley, Christopher S.},
  year = {2021},
  eprint = {2104.08625},
  archiveprefix = {arXiv},
  primaryclass = {cs.RO}
}

A Study on Challenges of Testing Robotic Systems, Afsoon Afzal, Claire Le Goues, Michael Hilton, and Christopher Steven Timperley, in International Conference on Software Testing, Validation and Verification, ICST ’20, 2020.
PDF · Data · BibTeX
@inproceedings{AfzalQualitative20,
  title = {A Study on Challenges of Testing Robotic Systems},
  author = {Afzal, Afsoon and {Le~Goues}, Claire and Hilton, Michael and Timperley, Christopher Steven},
  booktitle = {International Conference on Software Testing, Validation and
                    Verification},
  series = {ICST '20},
  location = {Porto, Portugal},
  year = {2020},
  month = mar
}

It Takes a Village to Build a Robot: An Empirical Study of The ROS Ecosystem, Sophia Kolak, Afsoon Afzal, Claire Le Goues, Michael Hilton, and Christopher Steven Timperley, in International Conference on Software Maintenance and Evolution, ICSME ’20, 2020, vol. .
PDF · Data · BibTeX
@inproceedings{KolakROS20,
  author = {Kolak, Sophia and Afzal, Afsoon and {Le Goues}, Claire and Hilton, Michael and Timperley, Christopher Steven},
  booktitle = {International Conference on Software Maintenance and Evolution},
  title = {It Takes a Village to Build a Robot: An Empirical Study of The {ROS} Ecosystem},
  year = {2020},
  series = {ICSME '20},
  volume = { },
  number = {},
  pages = {}
}

SOSRepair: Expressive Semantic Search for Real-World Program Repair, Afsoon Afzal, Manish Motwani, Kathryn Stolee, Yuriy Brun, and Claire Le Goues, IEEE Transactions on Software Engineering, 2019.
PDF · Code · Data · Results · Presentation · DOI · BibTeX
@article{AfzalSOSRepair19,
  title = {{SOSRepair}: Expressive Semantic Search for Real-World Program Repair},
  author = {Afzal, Afsoon and Motwani, Manish and Stolee, Kathryn and Brun, Yuriy and {Le~Goues}, Claire},
  journal = {{IEEE} Transactions on Software Engineering},
  year = {2019},
  publisher = {IEEE},
  doi = {10.1109/TSE.2019.2944914},
  presentation = {https://youtu.be/PYBxq28o6UM}
}

A Study on the Use of IDE Features for Debugging, Afsoon Afzal and Claire Le Goues, in Proceedings of the Challenge Track at the International Conference on Mining Software Repositories, MSR ’18, 2018.
PDF · DOI · BibTeX
@inproceedings{AfzalMSRChallenge2018,
  title = {A Study on the Use of IDE Features for Debugging},
  author = {Afzal, Afsoon and {Le~Goues}, Claire},
  booktitle = {Proceedings of the Challenge Track at
                    the International Conference on Mining Software Repositories},
  series = {MSR '18},
  doi = {10.1145/3196398.3196468},
  month = may,
  year = {2018},
  location = {Gothenburg, Sweden}
}

Crashing Simulated Planes is Cheap: Can Simulation Detect Robotics Bugs Early?, Christopher Steven Timperley, Afsoon Afzal, Deborah Katz, Jam Marcos Hernandez, and Claire Le Goues, in International Conference on Software Testing, Validation and Verification, ICST ’18, 2018, pp. 331–342.
PDF · Data · DOI · BibTeX
@inproceedings{TimperleyCrashing2018,
  title = {Crashing Simulated Planes is Cheap: {C}an Simulation Detect
                    Robotics Bugs Early?},
  author = {Timperley, Christopher Steven and Afzal, Afsoon and Katz, Deborah and Hernandez, Jam Marcos and {Le~Goues}, Claire},
  booktitle = {International Conference on Software Testing, Validation and
                    Verification},
  series = {ICST '18},
  pages = {331--342},
  location = {V{\"{a}}ster{\aa}s, Sweden},
  doi = {10.1109/ICST.2018.00040},
  year = {2018},
  month = apr
}

Evaluating CoBlox: A Comparative Study of Robotics Programming Environments for Adult Novices, David Weintrop, Afsoon Afzal, Jean Salac, Patrick Francis, Boyang Li, David C. Shepherd, and Diana Franklin, in Conference on Human Factors in Computing Systems, CHI ’18, 2018, p. 366.
PDF · DOI · BibTeX
@inproceedings{WeintropCoBlox2018,
  title = {Evaluating CoBlox: A Comparative Study of Robotics Programming Environments
             for Adult Novices},
  author = {Weintrop, David and Afzal, Afsoon and Salac, Jean and Francis, Patrick and Li, Boyang and Shepherd, David C. and Franklin, Diana},
  booktitle = {Conference on Human Factors in Computing Systems},
  pages = {366},
  year = {2018},
  series = {CHI '18},
  doi = {10.1145/3173574.3173940},
  location = {Montreal, Canada}
}

A Turing Test for Genetic Improvement, Afsoon Afzal, Jeremy Lacomis, Claire Le Goues, and Christopher S. Timperley, in Genetic Improvement Workshop, GI ’18, 2018, pp. 17–18.
PDF · BibTeX
@inproceedings{AfzalTuringTest2018,
  author = {Afzal, Afsoon and Lacomis, Jeremy and {Le~Goues}, Claire and Timperley, Christopher S.},
  title = {A Turing Test for Genetic Improvement},
  booktitle = {Genetic Improvement Workshop},
  year = {2018},
  series = {GI '18},
  pages = {17--18}
}

Quality assurance automation in autonomous systems, Afsoon Afzal, in Proceedings of the Doctoral Symposium Track at the Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE ’18, 2018, pp. 948–951.
PDF · BibTeX
@inproceedings{AfzalDoctoralSymposium2018,
  title = {Quality assurance automation in autonomous systems},
  author = {Afzal, Afsoon},
  booktitle = {Proceedings of the Doctoral Symposium Track at the
                     Joint Meeting on European Software Engineering Conference
                     and Symposium on the Foundations of Software Engineering},
  series = {ESEC/FSE '18},
  pages = {948--951},
  year = {2018},
  location = {Lake Buena Vista, Florida}
}