[JBEHAVE-770] Ensure behaviour is reproducible in unit tests when retrieving methods from Class.getMethods() - jira.codehaus.org

Details

Type: Task
Status: Resolved
Priority: Minor
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 3.6.4, 3.7
Component/s: Core
Labels:
None
Environment:
Found the issue using Win7(x64) and Java7u2.

Testcase included:
yes
Patch Submitted:

Yes
Number of attachments :
1

Description

Testcases in CompositeStepCandidateBehaviour and StepCandidateBehaviour uses the resulting list from getMethods() and expected results to be in declaration order.
According to Javadoc of java.lang.Class.getMethods():
"Returns an array ... The elements in the array returned are not sorted and are not in any particular order..."

Correction attached, making use of an extraction method to find the desired CandidateStep in the test.

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending
- Download All

Attachments

0001-testcase-bugfix-Corrected-testcases-that-were-using-.patch

25/Apr/12 6:13 PM

9 kB

Joachim Nilsson

Activity

Hide

Permalink

Joachim Nilsson added a comment - 25/Apr/12 6:17 PM

Tricky error since unit tests fails in random order. See 5 run results below:
----------------------------------------------------------
Failed tests:
shouldBeFastUsingGroovy(org.jbehave.core.embedder.MetaFilterBehaviour): should be less than half a second for 1000 matches on a simple case
shouldMatchCompositeStepsWhenStepParameterIsProvided(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
shouldMatchAndIdentifyPendingAnnotatedSteps(org.jbehave.core.steps.StepCandidateBehaviour):
Tests run: 453, Failures: 5, Errors: 0, Skipped: 1
----------------------------------------------------------
Failed tests:
shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
Tests run: 453, Failures: 2, Errors: 0, Skipped: 1
----------------------------------------------------------
Failed tests:
shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
Tests in error:
shouldPerformStepsInDryRunMode(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst '

{GIVEN=Given, AND=And, IGNORABLE=!--, THEN=Then, WHEN=When}'
shouldCreateStepsOfDifferentTypesWithSameMatchingPattern(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst '{GIVEN=Given, AND=And, IGNORABLE=!--, THEN=Then, WHEN=When}

'
Tests run: 453, Failures: 2, Errors: 2, Skipped: 1
----------------------------------------------------------
Failed tests:
shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
shouldMatchAndIdentifyPendingAnnotatedSteps(org.jbehave.core.steps.StepCandidateBehaviour):
Tests run: 453, Failures: 3, Errors: 0, Skipped: 1
----------------------------------------------------------
Failed tests:
shouldMatchCompositeStepsWhenStepParameterIsProvided(org.jbehave.core.steps.CompositeStepCandidateBehaviour):
Tests in error:
shouldPerformStepsInDryRunMode(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst '

{AND=And, GIVEN=Given, IGNORABLE=!--, THEN=Then, WHEN=When}'
shouldCreateStepsOfDifferentTypesWithSameMatchingPattern(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst '{AND=And, GIVEN=Given, IGNORABLE=!--, THEN=Then, WHEN=When}

'
Tests run: 453, Failures: 1, Errors: 2, Skipped: 1
----------------------------------------------------------

Show

Joachim Nilsson added a comment - 25/Apr/12 6:17 PM Tricky error since unit tests fails in random order. See 5 run results below: ---------------------------------------------------------- Failed tests: shouldBeFastUsingGroovy(org.jbehave.core.embedder.MetaFilterBehaviour): should be less than half a second for 1000 matches on a simple case shouldMatchCompositeStepsWhenStepParameterIsProvided(org.jbehave.core.steps.CompositeStepCandidateBehaviour): shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): shouldMatchAndIdentifyPendingAnnotatedSteps(org.jbehave.core.steps.StepCandidateBehaviour): Tests run: 453, Failures: 5, Errors: 0, Skipped: 1 ---------------------------------------------------------- Failed tests: shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): Tests run: 453, Failures: 2, Errors: 0, Skipped: 1 ---------------------------------------------------------- Failed tests: shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): Tests in error: shouldPerformStepsInDryRunMode(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst ' {GIVEN=Given, AND=And, IGNORABLE=!--, THEN=Then, WHEN=When}' shouldCreateStepsOfDifferentTypesWithSameMatchingPattern(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst '{GIVEN=Given, AND=And, IGNORABLE=!--, THEN=Then, WHEN=When} ' Tests run: 453, Failures: 2, Errors: 2, Skipped: 1 ---------------------------------------------------------- Failed tests: shouldMatchCompositeStepsAndCreateComposedStepsUsingMatchedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): shouldMatchCompositeStepsAndCreateComposedStepsUsingNamedParameters(org.jbehave.core.steps.CompositeStepCandidateBehaviour): shouldMatchAndIdentifyPendingAnnotatedSteps(org.jbehave.core.steps.StepCandidateBehaviour): Tests run: 453, Failures: 3, Errors: 0, Skipped: 1 ---------------------------------------------------------- Failed tests: shouldMatchCompositeStepsWhenStepParameterIsProvided(org.jbehave.core.steps.CompositeStepCandidateBehaviour): Tests in error: shouldPerformStepsInDryRunMode(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst ' {AND=And, GIVEN=Given, IGNORABLE=!--, THEN=Then, WHEN=When}' shouldCreateStepsOfDifferentTypesWithSameMatchingPattern(org.jbehave.core.steps.StepCandidateBehaviour): No starting word found for step 'Given foo named xyz' of type 'WHEN' amongst '{AND=And, GIVEN=Given, IGNORABLE=!--, THEN=Then, WHEN=When} ' Tests run: 453, Failures: 1, Errors: 2, Skipped: 1 ----------------------------------------------------------

Mauro Talevi made changes - 26/Apr/12 5:24 AM

Field	Original Value	New Value
Status	Open [ 1 ]	Resolved [ 5 ]
Fix Version/s		3.6.4 [ 18434 ]
Resolution		Fixed [ 1 ]

Mauro Talevi made changes - 27/Apr/12 6:35 AM

Summary	Testcases expects order in result from getMethods()	Ensure behaviour is reproducible in unit tests when retrieving methods from Class.getMethods()
Issue Type	Bug [ 1 ]	Task [ 3 ]

Mauro Talevi made changes - 16/Sep/12 4:29 AM

Fix Version/s

3.7 [ 18390 ]

Ensure behaviour is reproducible in unit tests when retrieving methods from Class.getMethods()

Details

Description

Attachments

Activity

People

Dates