JBehave
  1. JBehave
  2. JBEHAVE-1053

Allow entire story to be restarted

    Details

    • Type: New Feature New Feature
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.9.5
    • Fix Version/s: 3.10
    • Component/s: Core
    • Labels:
      None
    • Number of attachments :
      0

      Description

      For selenium tests that keep the same browser / context throughout the story, there are countless things that can go wrong such as network latency, slow application, etc. Since a false positive can cause people to quickly lose trust in the tests, others advised that the story should restart "x" amount of times before failing the test.

      There is a way to restart a single scenario, but it is possible it could cause additional errors such as expecting to be on a specific page when the scenario starts, context variables have been changed, being in an unknown state with data.

        Activity

        Hide
        Brent Barker added a comment - - edited

        I've created a branch for this issue with some the basics implemented:
        https://github.com/bbarke/jbehave-core
        https://github.com/bbarke/jbehave-core/commit/c6c7020f04ee247619d530e76f742c9e9d229aae

        How it works is similar to the RestartingScenarioFailure, just throw RestartingStoryFailure somewhere within the jbehave framework and it will restart the story

        Some questions/problems I have are:
        reporter.get().restartedStory(story, e); does not get reported if running in multi-threaded mode (Story runner, line 254)
        Also, I am not sure how to get reporting to report if a story was restarted or not.

        Please advise how I can get the reporting to work correctly

        Thanks in advance!

        Show
        Brent Barker added a comment - - edited I've created a branch for this issue with some the basics implemented: https://github.com/bbarke/jbehave-core https://github.com/bbarke/jbehave-core/commit/c6c7020f04ee247619d530e76f742c9e9d229aae How it works is similar to the RestartingScenarioFailure, just throw RestartingStoryFailure somewhere within the jbehave framework and it will restart the story Some questions/problems I have are: reporter.get().restartedStory(story, e); does not get reported if running in multi-threaded mode (Story runner, line 254) Also, I am not sure how to get reporting to report if a story was restarted or not. Please advise how I can get the reporting to work correctly Thanks in advance!
        Hide
        Mauro Talevi added a comment -

        Thanks for the contribution! I'll have a look at it soon.

        As for the multi-threading reporting, it should get reported by the invokeDelayed() method in the ConcurrentReporter.

        Show
        Mauro Talevi added a comment - Thanks for the contribution! I'll have a look at it soon. As for the multi-threading reporting, it should get reported by the invokeDelayed() method in the ConcurrentReporter.
        Mauro Talevi made changes -
        Field Original Value New Value
        Fix Version/s 3.9.6 [ 20672 ]
        Hide
        Mauro Talevi added a comment -

        Pulled commit and added restarting.story in core examples.

        Can you please add a story that can verify the multi-threading behaviour?

        Show
        Mauro Talevi added a comment - Pulled commit and added restarting.story in core examples. Can you please add a story that can verify the multi-threading behaviour?
        Hide
        Mauro Talevi added a comment -

        Did some cleanup on the StoryRunner implementation. Can you please verify it works on your side?

        Please add any different behaviour to restarting.story.

        Show
        Mauro Talevi added a comment - Did some cleanup on the StoryRunner implementation. Can you please verify it works on your side? Please add any different behaviour to restarting.story.
        Hide
        Brent Barker added a comment -

        Great, thank you very much!

        I tried the code, and if an exception is thrown from a before/after annotated method, it does not restart the story. Exceptions that happen in that area only gets logged, and does not get saved off anywhere.

        The scenario would be like this: If a story fails, call the annotated method:

          @AfterScenario(uponType = ScenarioType.NORMAL, uponOutcome = Outcome.FAILURE)
          public void afterNormalScenario(UUIDExceptionWrapper uuidWrappedFailure)

        Inside of that method, would be logic to restart the story if it fails the first time. Other then that don't restart.

        I'll see if I can add that logic to the restarting.story, but I may have to edit a lifecycle method somewhere to do this.

        Show
        Brent Barker added a comment - Great, thank you very much! I tried the code, and if an exception is thrown from a before/after annotated method, it does not restart the story. Exceptions that happen in that area only gets logged, and does not get saved off anywhere. The scenario would be like this: If a story fails, call the annotated method: @AfterScenario(uponType = ScenarioType.NORMAL, uponOutcome = Outcome.FAILURE) public void afterNormalScenario(UUIDExceptionWrapper uuidWrappedFailure) Inside of that method, would be logic to restart the story if it fails the first time. Other then that don't restart. I'll see if I can add that logic to the restarting.story, but I may have to edit a lifecycle method somewhere to do this.
        Hide
        Brent Barker added a comment -

        I updated the restarting.story to reflect these changes, and are in pull request https://github.com/jbehave/jbehave-core/pull/71

        Show
        Brent Barker added a comment - I updated the restarting.story to reflect these changes, and are in pull request https://github.com/jbehave/jbehave-core/pull/71
        Hide
        Brent Barker added a comment -

        The above pull request doesn't work when running a story with an examples table. I'm working on resolving this

        Show
        Brent Barker added a comment - The above pull request doesn't work when running a story with an examples table. I'm working on resolving this
        Hide
        Brent Barker added a comment -

        The above problems have been resolved: https://github.com/jbehave/jbehave-core/pull/71

        Show
        Brent Barker added a comment - The above problems have been resolved: https://github.com/jbehave/jbehave-core/pull/71
        Hide
        Brent Barker added a comment -

        Anything I can do to help this pull request go in? I've been using this code for about a month now on our ci-server, and has been working out great.

        Show
        Brent Barker added a comment - Anything I can do to help this pull request go in? I've been using this code for about a month now on our ci-server, and has been working out great.
        Hide
        Mauro Talevi added a comment -

        Hi Brett, the changes are a bit tricky to track because they're spread over multiple commits.

        Could you please provide in a single commit the proposed changes since the pull of Nov 10?

        Thanks

        Show
        Mauro Talevi added a comment - Hi Brett, the changes are a bit tricky to track because they're spread over multiple commits. Could you please provide in a single commit the proposed changes since the pull of Nov 10? Thanks
        Hide
        Brent Barker added a comment -

        Hi Mauro,
        I have squished all of the change lists together, it should be much easier to read now. Let me know if there is anything else I can do to help.

        Show
        Brent Barker added a comment - Hi Mauro, I have squished all of the change lists together, it should be much easier to read now. Let me know if there is anything else I can do to help.
        Hide
        Brent Barker added a comment -

        Is there anything else I can do to help get this merged? Let me know if you want me to re-work any code

        Show
        Brent Barker added a comment - Is there anything else I can do to help get this merged? Let me know if you want me to re-work any code
        Hide
        Mauro Talevi added a comment -

        Hi Brett,

        apologies for the delay in pulling this request. I meant to do the corresponding changes to the 4.x branch but did not get time for it. Could you perhaps have a first stab at it? Essentially changing the PerformableTree in a similar way you've modified the StoryRunner?

        Cheers

        Show
        Mauro Talevi added a comment - Hi Brett, apologies for the delay in pulling this request. I meant to do the corresponding changes to the 4.x branch but did not get time for it. Could you perhaps have a first stab at it? Essentially changing the PerformableTree in a similar way you've modified the StoryRunner? Cheers
        Hide
        Brent Barker added a comment -

        Yep I can take a stab at it.

        The more I think about this, the more I'm wondering if it would be better to simply have a configuration that takes a number to define how many times to restart the story if any failure occurs. This would replace trying to keep track and see if the RestartingStoryFailure exception was thrown.

        Show
        Brent Barker added a comment - Yep I can take a stab at it. The more I think about this, the more I'm wondering if it would be better to simply have a configuration that takes a number to define how many times to restart the story if any failure occurs. This would replace trying to keep track and see if the RestartingStoryFailure exception was thrown.

          People

          • Assignee:
            Unassigned
            Reporter:
            Brent Barker
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated: