JBehave
  1. JBehave
  2. JBEHAVE-282

Using GivenScenarios causes failed tests to be forgotten

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.3
    • Fix Version/s: 2.5.4, 3.0
    • Component/s: Core
    • Labels:
      None
    • Testcase included:
      yes
    • Number of attachments :
      1

      Description

      We have multiple related scenarios within the same JBehave script. This seems to be supported functionality but a major issue came to light earlier: using GivenScenarios at the start of a later scenario will cause any failed scenarios thus far to be forgotten. For example:

      ""
      Story: Test showing that using a given scenario skews the overall result

      Scenario: Fail

      GivenScenarios: com/example/sandpit/jbehave/noop
      Given I do nothing
      Then I fail

      Scenario: Succeed

      GivenScenarios: com/example/sandpit/jbehave/noop
      Given I do nothing
      Then I pass
      ""

      The JUnit result for running a scenario with the above script is a pass, even though the "I fail" step threw an exception. If you remove the GivenScenarios line from the Succeed case then the JUnit result is a fail.

      Attached is an Eclipse project that demonstrates this. This is a major issue for us: we run the JBehave tests from our build farm in an automated process. This bug would give us the green light for the whole build despite tests failing.

        Activity

        Hide
        Mauro Talevi added a comment -

        Added failure_followed_by_given_scenarios.scenario in trader example reproducing issue, based on example project provided.

        The problem stems from the fact that the failed state is per story and running given scenarios resets the state.

        Show
        Mauro Talevi added a comment - Added failure_followed_by_given_scenarios.scenario in trader example reproducing issue, based on example project provided. The problem stems from the fact that the failed state is per story and running given scenarios resets the state.
        Mauro Talevi made changes -
        Field Original Value New Value
        Assignee Mauro Talevi [ maurotalevi ]
        Fix Version/s 2.5.4 [ 16442 ]
        Hide
        Mauro Talevi added a comment -

        Error state reset only for non-embedded stories.

        Show
        Mauro Talevi added a comment - Error state reset only for non-embedded stories.
        Mauro Talevi made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Mauro Talevi made changes -
        Fix Version/s 3.0 [ 16302 ]

          People

          • Assignee:
            Mauro Talevi
            Reporter:
            Martin Hutchinson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: