2015-10-16 94 views
1

我想獲得一些在詹金斯運行的片狀測試的可見性。詹金斯,Junit和片狀測試處理程序插件

爲了做到這一點,我安裝了https://wiki.jenkins-ci.org/display/JENKINS/Flaky+Test+Handler+Plugin這似乎有這個目的。

所有的插件頁面中指定的要求,被我安裝(詹金斯版本,Junit的插件,等...)

滿足安裝該插件後後,建議

check on 「Publish JUnit flaky test reports」 under 「Additional test report features.

和他們的屏幕截圖顯示

enter image description here

Maven項目的conf iguration不給我指定測試報告個XML的選項,看起來像這個:

enter image description here

雖然,當我運行詹金斯構建配置的項目,我得到的是一個空表:

enter image description here

項目配置的構建部分使用-Dsurefire.rerunFailingTestsCount=3選項。

我有點困惑,插件是不是與我的詹金斯安裝行爲或我失去了一些東西。

編輯:我一直試圖版本1.0.3包含在其中一個答案提及該修補程序,並在作業執行我得到以下錯誤的末尾:

ERROR: Publisher 'Publish JUnit test result report' aborted due to exception: 
java.io.IOException: Unable to serialize [email protected]e9 
    at hudson.remoting.UserRequest.serialize(UserRequest.java:169) 
    at hudson.remoting.UserRequest.perform(UserRequest.java:129) 
    at hudson.remoting.UserRequest.perform(UserRequest.java:49) 
    at hudson.remoting.Request$2.run(Request.java:326) 
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at java.lang.Thread.run(Thread.java:745) 
    at ......remote call to some.slave.com(Native Method) 
    at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413) 
    at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) 
    at hudson.remoting.Channel.call(Channel.java:778) 
    at com.google.jenkins.flakyTestHandler.plugin.JUnitFlakyTestDataPublisher.getTestData(JUnitFlakyTestDataPublisher.java:49) 
    at hudson.tasks.junit.TestDataPublisher.contributeTestData(TestDataPublisher.java:62) 
    at hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:166) 
    at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:75) 
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) 
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) 
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:726) 
    at hudson.model.Build$BuildExecution.post2(Build.java:185) 
    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:671) 
    at hudson.model.Run.execute(Run.java:1766) 
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 
    at hudson.model.ResourceController.execute(ResourceController.java:98) 
    at hudson.model.Executor.run(Executor.java:408) 
Caused by: java.io.NotSerializableException: com.google.jenkins.flakyTestHandler.junit.FlakyCaseResult$FlakyRunInformation 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) 
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) 
    at java.util.ArrayList.writeObject(ArrayList.java:762) 
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:483) 
    at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) 
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) 
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) 
    at java.util.ArrayList.writeObject(ArrayList.java:762) 
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:483) 
    at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) 
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) 
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) 
    at java.util.ArrayList.writeObject(ArrayList.java:762) 
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:483) 
    at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) 
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) 
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) 
    at hudson.remoting.UserRequest._serialize(UserRequest.java:158) 
    at hudson.remoting.UserRequest.serialize(UserRequest.java:167) 
    at hudson.remoting.UserRequest.perform(UserRequest.java:129) 
    at hudson.remoting.UserRequest.perform(UserRequest.java:49) 
    at hudson.remoting.Request$2.run(Request.java:326) 
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at java.lang.Thread.run(Thread.java:745) 

這似乎只發生在從機上執行作業時,似乎在主機上正常工作。

編輯2:1.0.4修正了上面的問題。

編輯3:儘管1.0.4固定的序列化問題,該插件報告似乎仍然被打破。在下列情況下,例如:

wrong reporting

  • 工作#21成功運行
  • 工作#22跑去,有一個片狀的測試,其中在第二次運行
  • 工作#23成功運行通過

該項目的報告是說有一個薄片測試,一個片狀,0失敗和0通過。我期望2次傳球,0次失敗和1次失誤。

我錯過了什麼嗎?

回答

2

我有同樣的問題。看着主日誌,事實證明,該插件不希望構建在從服務器上執行。這裏面有修復該問題,我拉的請求 - https://github.com/jenkinsci/flaky-test-handler-plugin/pull/2

+0

可以請你在這裏把解決方案「在修訂彙總統計測試」? – manetsus

+0

@bezda,我一直在嘗試1.0.3,其中應包括您的拉取請求,現在我得到一個新的錯誤,在我原來的帖子的底部描述。任何想法可能是由什麼造成的? – supercalifragilistichespirali

+0

@bezda,1.0.4修復了問題 – supercalifragilistichespirali

2

如果我沒有記錯,我們定義了一個片狀的測試是:

對於相同的代碼版本,它既失敗並傳遞。因此,如果你從不修改代碼,即使你運行了100次,結果是50次通過和50次失敗,我們仍然把它算作一個片斷;如果它全部通過100次,那麼它算作一次通過;否則如果全部失敗100次,那就是一次失敗。

如果您進行了一些更改並提交了另一個版本,然後再進行構建,那麼它將會是一個新數字。

我相信如果你點擊測試,那麼你可以看到詳細的通過/失敗,這是你想要的?

看到https://wiki.jenkins-ci.org/display/JENKINS/Flaky+Test+Handler+Plugin

+0

感謝您的解釋。 – supercalifragilistichespirali