Quality Assurance on One Million Lines of Code

The Starliner Failure

Boeing Corporation has had noted problems with safety and flight control issues in its 737-MAX commercial jetliners.  More recently, however, Boeing experienced a failure of another high-profile product in a crucial test. The Starliner space delivery system is Boeing’s entry into the return of manned spaceflight on American-built launch vehicles.  NASA and Boeing conducted an unmanned test flight of the Starliner capsule to demonstrate that it could enter orbit and dock with the International Space Station.

The Starliner never got there.  A software error caused its internal clock to synchronize with the launch vehicle’s clock improperly.  While engineers on the ground diagnosed and corrected that error, they found a second software error that could have caused the capsule to separate improperly from its command module and caused the two to collide.  As Boeing Senior Vice President of Space and Launch, Jim Chilton said in a press teleconference, “Nothing good can come from those two spacecraft bumping into one another.”

Fortunately, Boeing engineers on the ground detected and corrected both software errors before the Starliner’s reentry, and the capsule landed safely on Earth.  Without that intervention, the capsule would have been lost.

This failure cost Boeing millions of dollars.  Instead of starting to earn revenue from Starliner flights to the ISS, Boeing will now have to submit the Starliner to another test flight.  It also puts Boeing in a worse competitive position with respect to SpaceX and its Dragon capsule, which has successfully flown to and docked with the ISS.

When Software Quality Assurance Fails

How did this failure happen?  Doesn’t Boeing have a software quality assurance program?

Of course, it does.  Besides, NASA independently verifies the code in the software of the spaceflight systems it operates, on top of the verification that the equipment manufacturer (or systems integrator) does.  Any errors that slip past Boeing’s internal software quality program should, therefore, have been detected by NASA before the spacecraft was cleared to launch.

NASA notes that the software that controls Starliner’s flight contains about one million lines of code.  NASA’s engineers will now review all of that code to determine whether it has more errors than just the two that manifested during the test flight.  However, this review is after the fact: the quality assurance processes at Boeing and NASA should have prevented potentially fatal errors from making it into the final release of the spacecraft’s software.

In a statement, NASA said, “Software defects, particularly in complex spacecraft code, are not unexpected. However, there were numerous instances where the Boeing software quality processes either should have or could have uncovered the defects. Due to these breakdowns found in design, code, and test of the software, they will require systemic corrective actions.”

For these two errors (and any others that may come to light) to exist in the Starliner flight software, the quality assurance processes at both Boeing and NASA must have had significant gaps and shortcomings.  “To put it bluntly, the issue that we’re dealing with is that we have numerous process escapes in the software design, development, and test cycle for Starliner,” said Douglas Loverro, director of spaceflight operations at NASA Headquarters.

The challenge is not unique to Boeing.  Elon Musk, who owns SpaceX, also owns Tesla, and Tesla’s cars run on tens of millions of lines of code.  Bugs crop up in Tesla’s software from time to time and impact thousands of vehicle owners - sometimes with potentially fatal consequences, as when the autopilot feature disengages without warning the driver.

Software Quality: Designed In and Built-In, Not Just Tested

Software quality has to be engineered into software programs at every level:

• Proper analysis translates the system’s purpose and required functionality into requirements specifications
• Robust design and program management translate requirements into a design fit for purpose
• Well-managed implementation turns the design into a working system that has been verified at every step
• Thorough testing validates that the completed system fulfills the desired purpose and functionality reliably
• Careful deployment assures that the system functions in its real-world operating environment just as it did in the test environment

As we become ever more dependent on digital systems in our daily lives, software quality assurance, validation, and verification become more important.  Every company that develops software for use in its products or its operations has the same high degree of responsibility for quality assurance. That means every such organization needs a robust team of managers, developers, and quality assurance engineers to make quality awareness central to its software effort.

The Many Skills of Software Quality Assurance

Of course, software quality depends on good project management and proper design and development practices as much as it does on verification and validation testing.  Simplilearn offers training in each of these areas.

For project and program managers, Simplilearn offers a full Digital Project Manager master’s program.  This includes certifications in PMP, Agile & Scrum, Lean Six Sigma (Green Belt), and DevOps.  These skills focus on quality assurance in software projects, in addition to the timely and cost-effective delivery of software that is fit for purpose.

For architects and developers, Simplilearn offers programs like the DevOps Engineer master’s program to provide certification in DevOps, Agile & Scrum, and related practices for rigorous and high-quality software development.

Industry projections show the demand for engineers in software verification, validation, and testing will grow at a rate of 9 percent.  Simplilearn can help you gain the skills necessary to provide this vital function in your organization’s software development. The Automation Test Engineer master’s program is an excellent place to start in this aspect of software quality assurance.

Other Simplilearn courses can be applied to software quality in different ways.  Data analytics and AI can help detect anomalies in system operations in time to prevent failures. IT service and architecture design certification helps ensure that software systems are designed and built to meet requirements reliably.  Cyber security protects digital systems from errors that might be introduced from outside the organization.

The software you develop may not have life or death riding on it. But if it is mission-critical to your organization, it just might: if not the safety of your coworkers and customers, then their jobs and financial well-being may depend on your software’s reliability and proper functioning. Keep quality in mind in all of your software development.