Rules Introduction
The only rule is: there are no rules
A mature person is one who does not think only in absolutes, who is able to be objective even when deeply stirred emotionally, who has learned that there is both good and bad in all people and in all things, and who walks humbly and deals charitably with the circumstances of life, knowing that in this world no one is all knowing and therefore all of us need both love and charity.
--Eleanor Roosevelt
The purpose of Rules in VirtualMock
"Rules" are how VirtualMock allows you to specify general guidelines about how you would like your mock objects to behave. When you use use VirtualMock to write your unit tests, these rules are automatically enforced. Some are enforced during the Verify phase, and others are enforced in the Record or Playback phases. If a rule is broken, VirtualMock will either display a warning message, or cause the JUnit test to fail, depending on the "Enforcement Level" for the rule.
Rules Enforcement
It is understandable that you may want to ignore these rules in some cases (or at least prevent them from causing your unit test to fail). To account for this, VirtualMock is designed to allow you to selectively change the Enforcement Level for most rules (there are some rules that must always be enforced, to support the design and functionality of VirtualMock). The Enforcement Level for the rules can be set through the VirtualMock API (TODO: put links to appropriate methods here), or through a property file (TODO: link to instructions for using a property file).
There are three possible Enforcement Levels: IGNORE, WARN, and FAIL:
- IGNORE: This means the rule will be ignored - if broken, it will not print any warning message, or cause the test to fail.
- WARN: This means a warning will be issued if the rule is broken. The warning will be printed to system out, but the test will not fail (TODO: is there a more appropriate way to handle than system.out?).
- FAIL: This means that VirtualMock will cause the Junit to fail if the rule is broken.
Rule Descriptions
Below is a list of all the rules supported by VirtualMock. The following information is provided for each rule:
- Rule Name: The name of the rule.
- Key: The java constant (public static final variable) name that is used to refer to the rule within the unit test.
- Enforcement Level: The list of enforcement levels that are supported for this rule. The default enforcement level is bold and in brackets.
- Phase:
The phase(s) of the VirtualMock test in which this rule is enforced. The phases are Record, Playback, or Verify (TODO: put link to description of phases)
- Description: The description of this rule.
TODO: verify each entry as they are developed, to ensure consistency.
RULE: All Recorded Calls should be Invoked
- Key: ALL_RECORDED_CALLS_MUST_BE_INVOKED
- Enforcement Level: IGNORE, WARN, [FAIL]
- Phase: Verify
- Description:
This rule supports verification. It requires that for each call that has been mocked up in the Record phase, a corresponding call is expected to be invoked during the Playback phase. The matching or recorded calls is done based on signature and argument values (TODO: more details on this, ties in with ArgVerifier). This also applies when there are multiple invocations of the same method recorded (these are considered indivually recorded calls).
RULE: All Calls should be invoked in the order in which they were recorded
- Key: CALLS_MUST_BE_MADE_IN_CORRECT_ORDER
- Enforcement Level: [IGNORE], WARN, FAIL
- Phase: Verify
- Description:
(TODO: need to review approach to ordering calls. jMock allows ordering for calls to be specified "relative" to each other. This global rule ("strict" ordering) will probably never be implemented.) The rule requires that the Class Under Test invokes the mocked calls in the same order that they were recorded.
RULE: Only public methods on the Class Under Test should be called by the Unit Test
- Key: ONLY_CALL_PUBLIC_METHODS_ON_CLASS_BEING_TESTED
- Enforcement Level: IGNORE, [WARN], FAIL
- Phase: Playback
- Description:
This rule specifies that the Unit Test should not call any non-public (private, protected, or default) methods on the Class Under Test. This validity of this rule is debatable, but it is included anyway, with a WARN enforcement level. There are valid arguments on both sides of the issue. One very practical reason for testing protected or default methods is that you often end up having to mock up much fewer calls. However, if you are testing at that granularity, you can miss some bugs in your class. Also, the need to call private or protected methods in a unit test can be an indicator your class should be refactored to be more testable. Many people would recommend that if a method cannot easily be tested via the public API, it should be refactored to be a public method on a separate class. Consider this analogy: Functional testing can uncover problems with the interaction between classes that are missed (or cannot be caught) by Unit Tests. Likewise, if you test non-public methods in isolation, then you can miss some bugs caused by interaction of methods which might have otherwise been caught if you were testing only via the public api. (TODO: provide links to discussions on both sides of the issue). One notable exception to this may be singletons. However, in some cases singletons can be misused (TODO: link to c2 wiki pages and "Use Your Singletons Wisely" article).