Of course the intention here is not to go through every detail aspect of the fundamentals of software testing, but just try to shed some light in some concepts that some people, even taking part of testing a system, are not familiar all the time.

As a start, the discussion about what to test is usually very common as the project reaches that phase of implementation. No matter the methodology of development or project used, there are test activities involved in big complex products scenarios.

And how we can define testing in software? It's not only about running tests, it's a variety of activities and usually aiming to uncover bugs and from there generate quality information for decision making.

A traditional test strategy usually, in high level, establishes a planning phase, design phase, creation of test artifacts such as manual test cases or automation scripts, generation of test date, the execution of the tests artifacts, generation of reports and closure of activities for a next release.

The ISTQB® defines some principles for the activity of testing:

Principle 1 – Testing shows presence of defects
Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.

Principle 2 – Exhaustive testing is impossible
Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts.

Principle 3 – Early testing
To find defects early, testing activities shall be started as early as possible in the software or system development life cycle, and shall be focused on defined objectives.

Principle 4 – Defect clustering
Testing effort shall be focused proportionally to the expected and later observed defect density of modules. A small number of modules usually contains most of the defects discovered during pre-release testing, or is responsible for most of the operational failures.

Principle 5 – Pesticide paradox
If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this “pesticide paradox”, test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of
the software or system to find potentially more defects.

Principle 6 – Testing is context dependent
Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site.

Principle 7 – Absence-of-errors fallacy
Finding and fixing defects does not help if the system built is unusable and does not fulfil the users’ needs and expectations.

The test activities should focus on assessing if the product or service reaches out an acceptable quality expectation. Quality is something difficult to achieve and usually it is achieved specially by finding defects and fixing them in time to guarantee that the product or service will run and commit to its original expectations.

So, why test is necessary? Test is an important activity as part of the software development; it's used to assess a product or service quality and to improve it by finding issues or also generating enhancement ideas. So, it's not only about finding defects, despite this being the main goal most of the time. Software testing also aims to ensure that the customer needs are satisfied.

The cause of defects can have different sources, specially in a complex software environment like Maximo. Defects in software are hard to find sometimes, because it depends on a number of variations and how the product is implemented and other factors; time pressure, complexity of the code, complexity of the infrastructure (with continuous delivery things may be even worse), etc.

The testing activity must be rigorous, no matter if it's automated or manual. Testing can help to reduce the likelihood of problems to happen at any time. Also, testing may be required to meet contractual or legal requirements, or industry-specific standards such as 21 CFR Part 11.

With testing, usually the main intention is to collect data that will tell us if we are in good shape or not; usually number of defects found is the first metric to be established. Also, the risk in a system can be better understand with testing. We can also affirm that when defects are found, the quality of the software increases when those defects are fixed in a timely fashion.

It's difficult to know when to stop testing. If we ask this questions to our technical team, the answer will always be 'it's not good enough'. Deciding how much testing is enough should take into consideration the level of risk, this means then, a good risk mapping is very useful, regarding the software, environment, infrastructure, labor, etc. Usually risks are categorized into technical and business.

Testing should provide sufficient information to stakeholders to make informed decisions about the release of the software or system being tested, for the next development step or handover to customers.