Term
| What is the purpose of verification? |
|
Definition
| The prove requirements (i.e., shalls). |
|
|
Term
|
Definition
| Did we make the system right. I.e., was the system made to spec. |
|
|
Term
|
Definition
| Did we make the right system. I.e., does the system meet the mission need. |
|
|
Term
| Is V&V (V and V) internal or external to system development and operation? |
|
Definition
|
|
Term
| Is T&E (T and E) internal or external to system development and operation? |
|
Definition
|
|
Term
| What is IEEE Std. 1012-2004? |
|
Definition
| The IEEE Standard for Software Verification and Validation. It includes the processes to determine whether the development products of a given activity conform to the requirements of that activity and whether the software satisfies its intended use and user needs. |
|
|
Term
| What are the three types of "independence" is IV&V testing? |
|
Definition
1. Technical Independence
2. Managerial Independence
3. Financial Independence |
|
|
Term
| What is the meaning of Technical Independence, why is it useful, and how is it employed? |
|
Definition
1. Utilizes personnel who are not involved in the development of the software
2. “Fresh viewpoint” valuable in detecting subtle errors overlooked by those too close to the solution
3. The IV&V effort uses or develops its own set of analysis and test tools separate from the developer’s tools |
|
|
Term
| What is the meaning of Managerial Independence and how is it employed? |
|
Definition
1. Responsibility for the IV&V effort be vested in an organization separate from the development and program management organizations
2. IV&V selects the things to be analyzed and tested, defines the schedule for its activities, and selects the specific technical issues and problems upon which to act |
|
|
Term
| What is the meaning of Financial Independence and how is it employed? |
|
Definition
1. Control of the IV&V budget is vested in an organization independent of the development organization
2. Prevents situations where the IV&V effort cannot be completed because funds have been diverted or adverse financial pressures or influences have been exerted. |
|
|
Term
| Basic Verificaiton and Validation Activities |
|
Definition
1. Identify Verification and Validation Targets
2. Define Verification and Validation Approach
3. Perform Verification
4. Perform Validation
5. Provide Verification and Validation Results |
|
|
Term
| What is the typical main entrance criteria for Validation testing? |
|
Definition
| The successful completion of verification testing. |
|
|
Term
| What are leading indicators? |
|
Definition
| Security assessments that seem to indicate due diligence in security planning. |
|
|
Term
| What are concurrent indicators? |
|
Definition
| Technical target metrics that show plans were properly executed. |
|
|
Term
| What are lagging indicators? |
|
Definition
| Indicators that indicate after-the-fact that a security breach took place. E.g., viewing logs weekly to find intrusions that already happened. |
|
|
Term
| What are the types of validity? |
|
Definition
1. Face validity
2. Content validity
3. Criterion validity
4. Construct validity |
|
|
Term
|
Definition
| A Layman’s judgment given basic knowledge of a situation and the set of security controls in place. |
|
|
Term
| What does TTOA stand for? |
|
Definition
| Technical Target of Assessment |
|
|
Term
| What is content validity? |
|
Definition
Summary: Does the test cover everything it needs to cover?
From web: The items on the test represent the entire range of possible items the test should cover. Individual test questions may be drawn from a large pool of items that cover a broad range of topics.
In some instances where a test measures a trait that is difficult to define, an expert judge may rate each item’s relevance. Because each judge is basing their rating on opinion, two independent judges rate the test separately. Items that are rated as strongly relevant by both judges will be included in the final test.
From charts: Measures weather something represents all facets of a given construct.
E.g., if the construct is a "secure state" and the security of a given TTOA is dependent of the values in a set of variable configurations that make the system difficult to penetrate, then a content validity text would be to compare the variables in the configuration files to a set of values for those variables that have been previously determined to thwart perpetrators. The systems whose configuration files contain the expected content would be validated to warrant a label of “secure,” given this definition. |
|
|
Term
| What is Criterion Validity? |
|
Definition
Smmary: Does possession of a particular characteristic (e.g., passign a test or having some other property) correlate with the property being assessed?
Summary 2: Are the test results correlated to the desired state?
If the system meets a specific test and it can be shown that there is a correlation between that test and the desired state (e.g., security), then the system it said to have criterion validity for that state.
E.g., a comparison of operational test results and organizational practices to a set of standards that have been established to achieve security within the system of interest. An example of this type of test is a regulatory audit. This type of test would be considered to have this type of validity if there was a correlation between the ability to pass audit and the ability to thwart perpetrators who exploit vulnerabilities. |
|
|
Term
| What is construct validity? |
|
Definition
Refers to whether a scale measures or correlates with the theorized psychological scientific construct (e.g., "fluid intelligence") that it purports to measure. In other words, it is the extent to which what was to be measured was actually measured.
Or in more simple terms, are we actually measuring what we think we are measuring? |
|
|
Term
| Basic Validation strategy |
|
Definition
1. Create validation plan
2. Ensure that operator and facilities are ready for validation.
3. Conduct validation
4. If necessary, isolate part of system that is causing problems.
5. Re-validate by component and whole system
6. Analyze, record and report validation
7. Build required assurance cases. |
|
|
Term
|
Definition
| The average interval between when a target is first aware of the existence of a new threat and when it successfully deflects it. This measure depends mainly on the speed and effectiveness of a target's response capability. |
|
|
Term
|
Definition
| The median lifetime of malicious activity emanating from a specific source. This is useful to measure in situations where attackers must constantly create and abandon original points to evade detection. The shorter this median lifetime, the heavier is the burden on the attacker to continuously change its location to evade detection. |
|
|
Term
| What can you tell me about TTP/TTA? |
|
Definition
| The extent the ratio is minimized, the defenders are successfully thwarting attacks. To the extent it increases, the attackers are more successful. |
|
|
Term
| Example validation tests for a centralized identification and authentication function |
|
Definition
1. an active login in any electronic component of the system is randomly sampled and compared with a centralized Identification and Authentication function to ensure that it is represented and a random sample of identities is made from the centralized repository and compared with access that it has recorded.
2. insert an unauthorized login into a system electronic component and observe to ensure that a security incident was detected and reported, and the incident response process resulting in its removal.
3. Parse all logins out of audit logs and verify the correspond to active identities |
|
|
