Types of Software Testings

Explain Load, Performance, Stress Testing with an example

Load Testing and Performance Testing are commonly said as positive testing where as Stress Testing is said to be as negative testing.

Say for example there is an application which can handle 25 simultaneous user logins at a time. In load testing we will test the application for 25 users and check how application is working in this stage, in performance testing we will concentrate on the time taken to perform the operation. Where as in stress testing we will test with more users than 25 and the test will continue to any number and we will check where the application is cracking the Hardware resources.

What is Security testing?

It is a process used to look out whether the security features of a system are implemented as designed and also whether they are adequate for a proposed application environment. This process involves functional testing, penetration testing and verification.

What is Installation testing?

Installation testing is done to verify whether the hardware and software are installed and configured properly. This will ensure that all the system components were used during the testing process. This Installation testing will look out the testing for a high volume data, error messages as well as security testing.

What is AUT ?

AUT is nothing but “Application Under Test”. After the designing and coding phase in Software development life cycle, the application comes for testing then at that time the application is stated as Application Under Test.

What is Negative testing?

Negative testing – Testing the system using negative data is called negative testing, e.g. testing the password where it should be a minimum of 8 characters so testing it using 6 characters is negative testing.

What is Ad-hoc testing?

Ad hoc testing is concern with the Application Testing without following any rules or test cases.

For Ad hoc testing one should have strong knowledge about the Application.

Describe bottom-up and top-down approaches in Regression Testing.

Bottom-up approach : In this approach testing is conducted from sub module to main module, if the main module is not developed a temporary program called DRIVERS is used to simulate the main module.

Top-down approach : In this approach testing is conducted from main module to sub module. if the sub module is not developed a temporary program called STUB is used for simulate the submodule.

What is the difference between structural and functional testing?

Structural testing is a “white box” testing and it is based on the algorithm or code.

Functional testing is a “black box” (behavioral) testing where the tester verifies the functional specification.

What is Re- test ? What is Regression Testing ?

Re- test – Retesting means we testing only the certain part of an application again and not considering how it will effect in the other part or in the whole application.

Regression Testing – Testing the application after a change in a module or part of the application for testing that is the code change will affect rest of the application.

What is UAT testing? When it is to be done?

UAT testing – UAT stands for ‘User acceptance Testing. This testing is carried out with the user perspective and it is usually done before the release.

What are the basic solutions for the software development problems?

  • Basic requirements – clear, detailed, complete, achievable, testable requirements has to be developed. Use some prototypes to help pin down requirements. In nimble environments, continuous and close coordination with customers/end-users is needed.
  • Schedules should be realistic – enough time to plan, design, test, bug fix, re-test, change, and document in the given schedule.
  • Adequate testing – testing should be started early, it should be re-tested after the bug fixed or changed, enough time should be spent for testing and bug-fixing.
  • Proper study on initial requirements – be ready to look after more changes after the development has begun and be ready to explain the changes done to others. Work closely with the customers and end-users to manage expectations. This avoids excessive changes in the later stages.
  • Communication – conduct frequent inspections and walk-through in appropriate time period; ensure that the information and the documentation is available on up-to-date if possible electronic. More emphasize on promoting teamwork and cooperation inside the team; use prototypes and proper communication with the end-users to clarify their doubts and expectations.

What are the common problems in the software development process?

  • Inadequate requirements from the Client – if the requirements given by the client is not clear, unfinished and not testable, then problems may come.
  • Unrealistic schedules – Sometimes too much of work is being given to the developer and ask him to complete in a Short duration, then the problems are unavoidable.
  • Insufficient testing – The problems can arise when the developed software is not tested properly.
  • Given another work under the existing process – request from the higher management to work on another project or task will bring some problems when the project is being tested as a team.
  • Miscommunication – in some cases, the developer was not informed about the Clients requirement and expectations, so there can be deviations.

What software testing types can be considered?

Black box testing – This type of testing doesn’t require any knowledge of the internal design or coding. These Tests are based on the requirements and functionality.

White box testing – This kind of testing is based on the knowledge of internal logic of a particular application code. The Testing is done based on the coverage of code statements, paths, conditions.

Unit testing – the ‘micro’ scale of testing; this is mostly used to test the particular functions or code modules. This is typically done by the programmer and not by testers; it requires detailed knowledge of the internal program design and code. It cannot be done easily unless the application has a well-designed architecture with tight code; this type may require developing test driver modules or test harnesses.

Sanity testing or Smoke testing – This type of testing is done initially to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing the systems in every 5 minutes or corrupting databases, the software may not be in a ‘sound’ condition to proceed for further testing in its current state.

Functional testing – This a commonly used black-box testing geared to check the functional requirements of an application; this type of testing should be done by testers.

Integration testing – This testing is combining the ‘parts’ of an application to determine if they function together correctly. The ‘parts’ can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to the client/server and distributed systems.

Incremental Integration testing – This is continuous testing of an application when a new functionality is added the existing ones; it checks the application functionality by verifying whether it works separately before all parts of the program are completed, in this type it will be checked whether to introduce test drivers or not; this is done by programmers or by testers.

Regression testing – This is testing the whole application again after the fixes or the modifications are done on the software. This is mostly done at the end of the Software development life cycle. Mostly Automated testing tools are used for these type of testing.

System testing – This is a type of black-box type testing that is based on overall requirements specifications; covers all combined parts of a system.

End-to-end testing – This is similar to system testing; this involves testing of a complete application environment such as interacting with a database, using network communications, or interacting with other hardware, applications and so on.

UAT ( User Acceptance Testing ) – This type of testing comes on the final stage and mostly done on the specifications of the end-user or client.

Usability testing – This testing is done to check the ‘user-friendliness’ of the application. This depends on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.

Compatibility testing – Testing how well the software performs in a particular hardware, software, operating system, network etc.

Comparison testing – This is nothing comparing the software strengths and weakness with another competing product.

Mutation testing – This is another method for determining if a set of test data or test cases is useful, by purposely introducing various code changes or bugs and retesting with the original test data or cases to determine whether the ‘bugs’ are detected.

What is SAE J1939?

What is SAE J1939?

The Society of Automotive Engineers (SAE) developed the J1939 standard to be the preferred CAN for equipment used in industries ranging from agriculture, construction, and fire/rescue to forestry, materials handling as well as on and off-highway vehicles. It is a high-level protocol that defines how communication between nodes (modules) occurs on the bus. The J1939 network is a specific communication system, supporting specific sets of applications and a specific industry, rather than being generalized.

 What kinds of messages are sent on a J1939 network?

Any electronic control unit (ECU) using J1939 is permitted to transmit a message on the network when the bus is idle. Every message includes a 29-bit identifier, which defines the message priority, what data is contained within the 8-byte data array that follows the identifier, and which ECU sent the message.

What is a PGN?

A Parameter Group Number (PGN) is a part of the 29-bit identifier sent with every message. The PGN is a combination of the Reserved bit (always 0), the data page bit (currently only 0, 1 is reserved for future use), the PDU Format (PF) and PDU Specific (PS). PDU stands for Protocol Data Unit, and can also be read as the message format. The PF and PS are both a byte (8-bits) long.

 The PS is dependent on the value in the PF field. If the PF value is between 0 and 239, the PS field will contain the destination address of the node that will receive the message. If the Global Address (FF16) is used, then all nodes on the bus will receive the message. This type of message, ” one that can be directed to a specific ECU on the bus by sending the message to its address “, is called a PDU1 message.

 If the PF field is between 240 and 255, then the PS field will contain a Group Extension (GE). The GE provides a larger set of values to identify messages that are broadcasted to all nodes on the network. This type of message, ” one that is sent to all ECUs on the bus “, is called a PDU2 message.

 The PGN uniquely identifies the Parameter Group (PG) that is being transmitted in the message. Each PG (a grouping of specific parameters) has a definition that includes the assignment of each parameter within the 8-byte data field (size in bytes, location of LSB), and the transmission rate and priority of the message. The structure of a PGN permits a total of up to 8672 different parameter groups to be defined per page.

 When an ECU receives a message, it uses the PGN in the identifier to recognize the type of data that was sent in the message.

 What is an SPN?

Each parameter used in the J1939 network is described by the standard. A Suspect Parameter Number (SPN) is a number that has been assigned by the SAE committee to a specific parameter. Each SPN has the following detailed information associated with it: data length (in bytes); data type; resolution, offset; range; and a tag (label) for reference. SPNs that share common characteristics will be grouped into a Parameter Group (PG) and will be transmitted to the network using the same PGN.

What happens if my ECU has parameters that have not been assigned an SPN by the SAE Committee?

The J1939 standard has two types of messages (PGNs) that can be used by manufacturers to handle parameters or messages that are not already covered by the standard.

The Proprietary A PGN (00EF0016) is a PDU1 message, and is used where the manufacture wants to direct the message to a specific destination node. How the data field of this message is used is up to each manufacturer.

Alternatively, Proprietary B PGNs (00FF0016 to 00FFFF16) are PDU2 messages allowing the manufacture to define the GE fields, as they desire. The data length and how the data fields of these messages are used are up to each manufacturer. Two manufacturers may use the same GE value, and the receivers of the information would have to differentiate between the two manufactures.

What happens if I have to send more than 8-bytes of data?

The J1939 standard has defined a method of communicating more than 8 bytes of data by sending the data in packets as specified in the Transport Protocol (TP). There are two types of TP, one for broadcasting the data, and the other for sending it to a specific address.

A Broadcast Announce Message (BAM) is received by all ECUs on the network, and they do not have to send any messages back to the originator in order for the BAM session to proceed. In a BAM session the transmitting module simply sends the data packets one after another at a predefined rate.

A specific TP, however, it directed at a particular address. The ECU at that address must respond to the commands sent by the transmitting module in order for the TP session to proceed. To begin a session, the originator sends a Request to Send (RTS) and the responder sends a Clear to Send (CTS). For this reason, this type of TP is called an RTS/CTS session. Either the originator or the responder can abort an RTS/CTS session at any time.

It is not mandatory for all J1939 ECUs to support TP sessions. If, however, the module does support TP, it must be able to, at a minimum, support one BAM and one RTS/CTS session concurrently from the same source address. It can only originate one destination specific connection with a given destination at a time, but an ECU might be able to support simultaneous CTS/RTS sessions with different addresses. Check with the manufacturer whether or not a node can support multiple simultaneous TP sessions (RTS/CTS and/or BAM).

How are diagnostics supported?

A Diagnostic Message (DM) may be sent and/or received by an ECU. If the ECU supports diagnostic messaging, each type of potential fault in the module will have associated with it a Diagnostic Trouble Code (DTC).

A DTC is a combination of four independent fields: the Suspect Parameter Number (SPN) of the channel or feature that can have faults; a Failure Mode Identifier (FMI) of the specific fault; the occurrence count (OC) of the SPN/FMI combination; and the SPN conversion method (CM) which tells the receiving mode how to interpret the SPN. Together, the SPN, FMI, OC and CM form a number that a diagnostic tool can use to understand the failure that is being reported.

When an ECU detects a fault, it will send an Active Diagnostic Trouble Code, DM1, message. The DM1 message send by the ECU will also contain the status lamps of the module. While the fault is still present, the ECU will continue to broadcast the DM1 message to the network every second. When the fault clears, the ECU will send a final DM1 message showing that there are no further faults present. If multiple faults are present simultaneously, the ECU will send all the active faults in a single DM1 by using the Broadcast Announce Message (BAM) in a Transport Protocol session. Refer to a product’s datasheet to see if it will send a DM1, and what type of DTCs it supports.

An ECU on the bus may respond to the data in the DM1, or a diagnostic tool may be connected to the network to show an operator all the active faults on the network. Refer to a product’s datasheet to see if and how it will respond to a DM1.

There are lots of other features for diagnostics, such as retrieval of information from a log by requesting a DM2, Previously Active DTCs, which are supported by the J1939 standard.

What is a NAME, and why is it important?

A J1939 NAME is comprised of the following fields: Arbitrary Address Capable; Industry Group; Vehicle System Instance; Vehicle System; Reserved Bit; Function; Function Instance; ECU Instance; Manufacturer Code; and Identity Number.

Every ECU on a J1939 network has an exclusive combination of the above fields, which results in a unique NAME for each module. The NAME is not only used to identify the module to other nodes on the bus, but it is also used in Network Management. The combination of the NAME field will form a number, and the lower numeric value NAMEs have a higher priority. If more than one node on the network attempt to claim the same address, they will arbitrate for the address, and the module with the higher priority NAME will be able to claim the address.