The Satisfaction of Test Engineering

By:

Systems Engineering for a Complex Entrepreneurial Society

Since the advent of the first mechanical and electrical machines, improvements to their functionality, reliability, operability, safety, and cost have been an ongoing effort for corporations and their engineering staff. The American entrepreneurial spirit and free market forces during the twentieth century helped to bring about many new systems and enterprises. Competing and succeeding has required engineering companies to recruit very specialized engineers. For example, the companies (such as General Electric) that originally focused on the development of motors and the transfer of electricity evolved integrated engineering disciplines such as jet propulsion and nuclear energy. By bringing together electrical, chemical, mechanical, and aeronautical engineers, you can create a jet engine. These companies could not create these complex systems without bringing all of these disciplines together. So they had to recruit specialized engineers, and they had to get the engineers to work together in a coordinated fashion.

Coordinating diverse disciplines required a system engineering approach and the employment of a new type of engineer. The Systems Engineer assures that the final system is composed of properly integrated subsystems and components, and that it meets customer expectations. Systems Engineers support a mix of technical and managerial functions. The Systems Engineer is concerned with system requirements, system architecture (hardware and software), design, development, integration, testing, customer acceptance, and maintenance. The Systems Engineers operate at a level that encompasses the software and hardware architecture in combination with network elements, human operations, physical interfaces, computers, and non-computer hardware (for example, physical components such as solar panels, antennas, speed indicators, temperature gauges, switches, etc.) The Systems Engineer also operates as a liaison between corporate management and the specialized engineers and analysts, helping to coordinate resources, resolve issues, and plan project schedules. Additional information on Systems Engineering may be found at the website for the International Council on Systems Engineering (INCOSE), founded in 1990 to advance the state of the art and practice of systems engineering in industry, academia, and government.

Test Engineering as a Sub-Discipline of Systems Engineering

As systems and products became more complex, system engineering departments organized their test support engineers into groups dedicated to the test support tasks, and the Test Engineering profession was created. Gradually and through lessons learned it became apparent that these test support groups should operate independent from the development staff to assure objectivity in their testing and test results reporting. As a group separated from the designers and developers, the Test Engineers could perform an independent assessment of the system or products, and this provided an improvement to the quality. The discipline has had to evolve to meet the demands of complex systems with a low margin for error. Recent failures and system malfunctions reported in the news have been all too costly, and this confirms that the challenges ahead require a disciplined approach.

Complex systems come in many forms. Whether a company is building a system composed of hardware and software components for a one-time delivery, or a purely software system with planned revisions and upgrades, or a million widgets for mass distribution, proper test engineering applies. The Test Engineer is a specialized Systems Engineer, focused primarily on the verification of the system design and functionality. In many companies the Test Engineer also supports Quality Assurance functions related to process improvement. The larger companies may have a QA organization separate from the test organization, which allows the Test Engineer to remain specialized. However, the modern Test Engineer is fully aware of the system development life cycle components and issues and how they may affect the quality of the final product or system. The Test Engineer may also be considered a specialized Software Engineer, when the system being designed is composed of a purely software architecture. In this case the Test Engineer’s test environment and work bench is composed of software components only, but the duties and focus are the same – to discover and prevent defects from getting into the final system or products.

The Satisfaction of Test Engineering

Those fortunate enough to have supported a successful large-scale system project, from initial design through final acceptance, will likely have experienced a very satisfactory feeling of achievement. But they may also have experienced a range of emotions, from stress, frustration, disappointment, and dread, to amusement, pride, and triumph. Especially on development programs that run for several years, these emotions result from the long parade of urgent issues and milestones, and from the interaction with the personalities among the many engineers, analysts, and managers that make up a large program. More often than not, the negative days outnumber the positive days. Schedules are often tight or unrealistic. Test Engineers often have to report new system issues or product defects. So what is it then that keeps the Test Engineer plodding along, satisfied with completing today’s verification tasks and planning tomorrow’s?

Few professions have the opportunity to apply scientific and computational principles to solve everyday human problems. This is what engineers do. They engineer solutions that solve problems or make life more comfortable. The nuclear engineer uses principles of nuclear physics, thermal physics, and electricity to safely move energy to where it is needed by people. The civil engineer uses principles of structural mechanics and materials science to design a safe bridge. The systems engineer coordinates the work products of others to orchestrate the design, development, and integration of a system. To accomplish this, the system engineer applies principles composed largely of process and project control methods and tools.

System Test Engineers utilize a distinct set of methods and tools to verify that a system will function as intended and will meet customer expectations. Some of these methods and tools, to name a few, include simulators, test data generators, written test plans & scripts, peer reviews, automated regression tests, data analyzers, requirement traceability matrices (RTM), inspection checklists, and defect tracking tools. One of the most challenging items the Test Engineer will face is getting the test environment to look like the real world system. They may need to engineer the test environment in the laboratory to simulate the product in the real world. They may need to engineer input data to best simulate the natural conditions under which the system will operate. Basically, they engineer the necessary test conditions to simulate the real world operational environment. The reasons that the design and control of the test environment can be so challenging are many. An example for a challenging test environment was the NASA TDRSS ground station integration and test, for which I was test manager during the early 90’s. For this system we designed and end-to-end test (EET) system composed of equipment designed to simulate TDRSS space users such as the Space Shuttle or the Hubble Telescope. The test environment was composed of a 4.5 meter antenna, uplink & downlink radio frequency hardware, and simulation software, all situated at the ground station in New Mexico. Testing the new TDRSS system required setting up and operating forward and return radio channels with the simulated space user at the facility. This test setup satisfactorily verified the TDRSS forward and return signaling capabilities, but with a very important limitation. Since the EET was not actually in orbit as the Hubble and Shuttle, we could not adequately simulate Doppler effects on the transmitting and receiving signals. For testing true Doppler effects we had to test against a NASA test satellite already in orbit. It was very rewarding for all the hardware and software engineers involved with the TDRSS project to meet the challenges it posed and to deliver a working system to NASA.

In addition to the challenging work and the feeling of accomplishment that comes with the final acceptance of a system by the customers, the work can quite simply be fun. When things are not working properly, the Test Engineer becomes a sleuth, using deductive reasoning, test repetition, and fault isolation testing to discover the root cause. Furthermore, the Test Engineer climbs into the driver seat ahead of the customer, and gets a firsthand trial run of the new system, which is particularly rewarding if the system is using cutting edge technologies.

Full article...

Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/