Black Box Testing in Software Engineering

Black box test case is used to test the security, functional, and behavioral properties of an application by a tester with no knowledge of the internal structure of the application and inner workings of an application. The tester usually has no access to the code of an application. For a black box test, the test person, or team, will access an application as if they are users. During testing, the test person will try to assess the application as a hacker or malicious actor. For a black box testing, the focus is often on the input and output by the application.

Black box testing is also referred to as penetration testing or pen test where an application’s security is tested through authorized and simulated attacks on the system to find possible vulnerabilities that can be exploited.

For instance, a test person can attempt an SQL Injection attack on a system to try and see if a vulnerability exists. An SQL injection vulnerability is tested by modifying the input into an application through popular input fields such as login, account creation, contact us, and comments.

Common Black Box Testing Techniques

These are techniques used in black box testing to ensure comprehensive testing to identify application defects.

1. Equivalence Partitioning

In equivalence partitioning, the test data is divided and grouped by a certain criteria. Only one input is then tested from each group where data in each group is expected to yield similar results to all input data in the same partition. Basically, equivalence partitioning contains both valid and invalid data partitions where test cases are designed from each to assess the application’s behavior.

For instance, if an application accepts a person’s age as between 18 to 30, several age groups can be created, which include 1-17, 18-30, 31-50, 51-100. A sample input is then picked from each age group and tested. If in the first group, 1-17 the number 12 is picked as a test input, the results it yields should also be yielded by every other number in the same partition.

2. Boundary value analysis

Boundary value analysis focuses on the edges on input ranges to try and find errors. An assumption is made that one is likely to find errors at the extreme edge of values other than the middle.

Consider an application that allows users to input passwords between 6 and 10 characters. While it is normal, and recommended, to test for possible, and random, inputs of passwords with 1-5 and 11 and above characters, boundary value analysis requires testing even with numbers closest to the minimum and maximum valid values, 6 and 10 in our example.

Therefore, 4 test cases can be formed from the example above:

1. Test 1: Test with one character less than the valid length (5 characters)

2. Test 2: Test with the minimum valid length (6 characters) - lower boundary

3. Test 3: Test with the maximum valid length (10 characters) - upper boundary

4. Test 3: Test with one character more than the maximum valid length (11 characters)

3. Cause Effect Graphing

Basically, Cause effect Graphing establishes relationships between inputs and outputs of a system with the help of boolean operators. Further, just as the name suggests, test cases are developed from a graphical representation of cause-and-effect relationship.

In cause and effect graphing, logical conditions and relationships are represented through four basic symbols, which include Identity, negation, logical OR, and logical AND.

a) Identity:- A specified presence or absence of an input directly affects any corresponding outputs. It is often represented by an arrow in a cause-and-effect graph. For instance, a cause cold weather has an output variable wear sweater can be represented as:

Cold outside → Wear sweater

The representation above can be interpreted as "if it is cold, wear a sweater".

b) Negation:- Basically, an output occurs if the specified input condition is not met. It is represented by a circle in a cause-and-effect graph. For instance a representation

⚬ Raining → Go Play Outside

The representation above can be interpreted as "only play outside if it is not raining".

c) Logical OR:- If you are a programmer, you have heard and even used OR operators before. Logical OR simply states that where two inputs exist, either or both input conditions can be met for the output to occur. It is often represented by a diamond in the cause and effect graph.

For instance, a representation:

Comedy ♦ Romance → Go to the Movies

The example above can be represented as "If the movie theater is showing a comedy or a romantic movie, then I can go to the movie". Therefore, if the movie is strictly comedy, romance, or a romantic comedy then I can go see it.

d) Logical AND:- This operator is also very much used in programming. Unlike the Logical OR, both input conditions must be met for the output to occur. It is represented by a rectangle in the cause and effect graph.

For instance, the relationship:

New Membership Account ☐ Purchases over $100 → Give 10% Discount

The example above can be translated as: "If the customer is a new member and he or she has spent over $100, a 10% discount should be applied"

Types of Black Box Testing

There are different types of testing involved in black box testing. These types include function, non-functional, and regression testing.

Functional Testing:

The objective of functional testing is to ensure that a system is functioning as intended. A system’s test case based on functional testing can be {based} on the expected behavior and functionalities. Simply put, a test user inputs data into a system and the output (responses) are compared against the expected results.

Non-functional Testing:

Non-functional testing focuses on other aspects of an application that are not classified as functional components. It is also identified as quality attribute testing. This type of testing focuses on performance, scalability, reliability, usability, and security of an application - just to name a few. An example of a non-functional test case would be to simulate many “users” accessing the system at the same time to assess its performance and response times to user requests. Furthermore, penetration tests are used to find any vulnerabilities in an application and attempt to exploit them.

Regression Testing:

Regression testing is carried out to ensure that new code (fixes, updates, and upgrades) does not affect the already existing code, producing new issues that disrupt normal functionalities of an application. Basically, regression testing aims to verify that an application’s functionalities and features remain as-is after modifications.

Advantages of Black Box Testing

1. The test person, or team, does not need to have programming skills or prior knowledge of the internal mechanisms of a system.

2. Tests are done according to the system’s requirements from a customer’s perspective.

3. Black box testing is suitable for use in a large system.

Disadvantages of Black Box Testing

1. Test cases can be repeated without the tester's knowledge, causing redundancy of the test process.

2. Black box testing requires clear system requirements and functional specifications to implement properly.

3. Black box testing results in either pass or fail depending on an output. No reason is attributed to a system failure.

4. Time consuming when working with more input spaces.

Example of Simple Black Box Test of an ATM System

The table below demonstrates a simple ATM System Black box test with sample actions and their results, which are compared against the expected results. Depending on the comparison results, it is either determined as a pass or a fail.

RELATED ARTICLE

Software, Software Engineering

Developers's Guide To Designing Secure Software

Read on the principles that guide developers creating secure systems. These principles are a collection of properties, behaviors, designs, and implementation practices that can reduce the likelihood of threat realization.