Details
-
Type:
Improvement
-
Status:
Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.9.5
-
Fix Version/s: 3.10
-
Component/s: None
-
Labels:None
-
Number of attachments :
Description
When running in multi-threading mode, the total is calculated as simply be the sum of each duration. If you have a single thread then it will coincide with the build duration (more or less), but if you use multiple threads it'll be off by a factor which should correspond to the number of threads.
The additional information of the average duration per thread, the total sum of the story durations divided by the number of threads, should also be available if you are in multi-threading mode.
Activity
Mauro Talevi
made changes -
| Field | Original Value | New Value |
|---|---|---|
| Summary | Story duration time in multi-threading execution mode is high than normal | Story duration time in multi-threading execution mode is higher than normal |
Mauro Talevi
made changes -
| Description |
From: Mauro Talevi [mailto:mauro.talevi@aquilonia.org]
Sent: Wednesday, April 01, 2015 4:03 PM To: user@jbehave.codehaus.org Subject: Re: [jbehave-user] Story duration are not calculated correctly in multi-thread execution Sure, easy enough to do! Can you please raise a JIRA issue for this usecase? Cheers On 1 Apr 2015, at 23:56, tran, phong <phong.tran@emc.com> wrote: Hi Mauro, Yes we’re running in multi-threading mode with 10 threads for concurrent story execution. EmbedderControls[batch=false,skip=false,generateViewAfterStories=true,ignoreFailureInStories=true,ignoreFailureInView=false,verboseFailures=true,verboseFiltering=false,storyTimeoutInSecs=108000,failOnStoryTimeout=false,threads=10] We’re trying to speed up the test execution (Selenium-based tests) with multiple threads (threads=10) targeting a Selenium Grid with 10 nodes. I noticed that the reported duration for each story is also high and that is why the sum of all story durations added up to a high total number. Does this mean the duration at the story level has been affected with multi-threading execution mode? Do we need to de-normalize (divided by 10 in this case) the duration for each story to get its accurate execution time? It would be great if the de-normalization for story duration can be at addressed at the JBehave framework level to handle the multi-threading execution mode. Thanks, Phong From: Mauro Talevi [mailto:mauro.talevi@aquilonia.org] Sent: Saturday, March 28, 2015 1:56 AM To: user@jbehave.codehaus.org Subject: Re: [jbehave-user] Story duration are not calculated correctly in multi-thread execution Hi Phong, I would imagine you're running in multi-threading mode. The total is calculated as simply be the sum of each duration. If you have a single thread then it will coincide with the build duration (more or less), but you use multiple threads it'll be off by a factor which should correspond to the number of threads. Arguably we could use the information of the threads to renormalise the total by the number of threads. This formula would still work in the case of a single thread. Can you confirm that you're using multiple threads? Cheers On 25/03/2015 08:22, tran, phong wrote: Hi Mauro, Thanks a lot for updating the core example template! We can access and integrate the durations from storyDurations.props into our custom HTML report now. However, we just notice another issue. The total duration reported in storyDurations.props for all stories is much higher (8 times) that what we usually see in our test run. Basically we have a configuration in TeamCity to run our Selenium-based tests and the test run took 5 hours and 25 minutes as reported by TeamCity. This is the normal time range we expected for our test run. Results Started Changes Started Duration #177 <image001.png> NGIS #: 6234 Beacon #: 3224 Pass Rate: 83.43949% passed: 262 failed: 52 pending: 51 <image002.png> View Changes (7) 24 Mar 15 09:55 5h:25m Beacon_Selenium_W2K8R2_13 None However, the total duration aggregated for all stories as reported by JBehave is unusually high (more than 48 hours). This duration is not correct! Reports Overview Stories Scenarios Steps Download as CSV Total Excluded Total Successful Pending Failed Excluded Total Successful Pending Failed Not Performed Ignorable Duration (hh:mm:ss.sss) 279 0 345 210 82 53 0 9,195 3,751 230 66 3,699 1,449 48:39:21.000 storyDurations.props: #org.jbehave.core.embedder.StoryManager #Tue Mar 24 15:20:14 PDT 2015 total=175161000 Do you have any idea what could be causing the much higher story duration reported in storyDurations.props? Is there any JBehave configuration I should check in our setup? Thanks, Phong |
When running in multi-threading mode, the total is calculated as simply be the sum of each duration. If you have a single thread then it will coincide with the build duration (more or less), but you use multiple threads it'll be off by a factor which should correspond to the number of threads.
|
| Fix Version/s | 3.9.6 [ 20672 ] | |
| Assignee | Mauro Talevi [ maurotalevi ] |
Mauro Talevi
made changes -
| Description |
When running in multi-threading mode, the total is calculated as simply be the sum of each duration. If you have a single thread then it will coincide with the build duration (more or less), but you use multiple threads it'll be off by a factor which should correspond to the number of threads.
|
When running in multi-threading mode, the total is calculated as simply be the sum of each duration. If you have a single thread then it will coincide with the build duration (more or less), but if you use multiple threads it'll be off by a factor which should correspond to the number of threads.
The additional information of the average duration per thread, the total sum of the story durations divided by the number of threads, should also be available if you are in multi-threading mode. |
| Issue Type | Bug [ 1 ] | Improvement [ 4 ] |
Mauro Talevi
made changes -
| Resolution | Fixed [ 1 ] | |
| Status | Open [ 1 ] | Resolved [ 5 ] |
