1. 首頁
  2. 問答
  3. Play2中的滑動/ Postgres數據庫測試問題/規格2

Play2中的滑動/ Postgres數據庫測試問題/規格2

2017-07-07 48 views
0

我一直在構建應用程序測試時遇到了很多麻煩，並希望有更多Scala經驗的人可以指引我朝着正確的方向發展。Play2中的滑動/ Postgres數據庫測試問題/規格2

我有一些駐留在Postgres數據庫中的數據模型，映射到使用Slick的案例類。我的Play應用程序然後向所述數據模型提供基於JSON的REST端點。由於大多數實際代碼在每個端點之間是相似的，所以大部分代碼都是作爲一個特徵實現的，該特徵被混合到實際的控制器中，這些控制器覆蓋了必要的位。

這工作得很好，但是當我嘗試在他們每個人的運行單元測試，大多數控制器工作，那麼我最終錯誤：

[error] Can't find a constructor for class helpers.DatabaseHelper 
[warn] c.z.h.HikariConfig - The jdbcConnectionTest property is now deprecated, see the documentation for connectionTestQuery 
[error] 
[error] cannot create an instance for class FileControllerSpec 
[error] caused by java.sql.SQLTransientConnectionException: db - Connection is not available, request timed out after 1005ms. 
[error] caused by org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections 
[error] 
[error] STACKTRACE 
[error] sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
[error] sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 
[error] sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
[error] java.lang.reflect.Constructor.newInstance(Constructor.java:423) 
[error] org.specs2.reflect.Classes$$anonfun$org$specs2$reflect$Classes$$createInstanceForConstructor$1.apply(Classes.scala:104) 
[error] org.specs2.control.ActionT$$anonfun$safe$1.apply(ActionT.scala:88) 
[error] org.specs2.control.ActionT$$anonfun$reader$1$$anonfun$apply$6.apply(ActionT.scala:79) 
[error] org.specs2.control.Status$.safe(Status.scala:100) 
[error] org.specs2.control.StatusT$$anonfun$safe$1.apply(StatusT.scala:62) 
[error] org.specs2.control.StatusT$$anonfun$safe$1.apply(StatusT.scala:62) 
[error] scalaz.syntax.ToApplicativeOps$$anon$1.self$lzycompute(ApplicativeSyntax.scala:29) 
[error] scalaz.syntax.ToApplicativeOps$$anon$1.self(ApplicativeSyntax.scala:29) 
[error] scalaz.syntax.ToApplicativeOps$ApplicativeIdV$$anonfun$point$1.apply(ApplicativeSyntax.scala:33) 
[error] scalaz.WriterTApplicative$$anonfun$point$1.apply(WriterT.scala:282) 
[error] scalaz.WriterTApplicative$$anonfun$point$1.apply(WriterT.scala:282) 
[error] scalaz.effect.IO$$anonfun$apply$19$$anonfun$apply$20.apply(IO.scala:136) 
[error] scalaz.effect.IO$$anonfun$apply$19$$anonfun$apply$20.apply(IO.scala:136) 
[error] scalaz.FreeFunctions$$anonfun$return_$1.apply(Free.scala:326) 
[error] scalaz.FreeFunctions$$anonfun$return_$1.apply(Free.scala:326) 
[error] scalaz.std.FunctionInstances$$anon$1$$anonfun$map$1.apply(Function.scala:56) 
[error] scalaz.Free$$anonfun$run$1.apply(Free.scala:172) 
[error] scalaz.Free$$anonfun$run$1.apply(Free.scala:172) 
[error] scalaz.Free.go2$1(Free.scala:119) 
[error] scalaz.Free.go(Free.scala:122) 
[error] scalaz.Free.run(Free.scala:172) 
[error] scalaz.effect.IO$class.unsafePerformIO(IO.scala:22) 
[error] scalaz.effect.IOFunctions$$anon$6.unsafePerformIO(IO.scala:227) 
[error] org.specs2.reflect.Classes$$anonfun$createInstance$1$$anonfun$apply$1$$anonfun$3.apply(Classes.scala:37) 
[error] org.specs2.reflect.Classes$$anonfun$createInstance$1$$anonfun$apply$1$$anonfun$3.apply(Classes.scala:36) 
[error] scala.collection.immutable.List.map(List.scala:273) 
[error] org.specs2.reflect.Classes$$anonfun$createInstance$1$$anonfun$apply$1.apply(Classes.scala:36) 
[error] org.specs2.reflect.Classes$$anonfun$createInstance$1$$anonfun$apply$1.apply(Classes.scala:29) 
[error] scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52) 
[error] org.specs2.control.Status$class.fold(Status.scala:30) 
[error] org.specs2.control.Ok.fold(Status.scala:95) 
[error] org.specs2.control.Status$class.flatMap(Status.scala:48) 
[error] org.specs2.control.Ok.flatMap(Status.scala:95) 
[error] org.specs2.control.Status$class.map(Status.scala:45) 
[error] org.specs2.control.Ok.map(Status.scala:95) 
[error] org.specs2.control.StatusT$$anonfun$map$1.apply(StatusT.scala:16) 
[error] org.specs2.control.StatusT$$anonfun$map$1.apply(StatusT.scala:16) 
[error] scalaz.WriterT$$anonfun$map$1.apply(WriterT.scala:46) 
[error] scalaz.WriterT$$anonfun$map$1.apply(WriterT.scala:46) 
[error] scalaz.effect.IO$$anonfun$map$1$$anonfun$apply$8.apply(IO.scala:56) 
[error] scalaz.effect.IO$$anonfun$map$1$$anonfun$apply$8.apply(IO.scala:55) 
[error] scalaz.Free$$anonfun$map$1.apply(Free.scala:52) 
[error] scalaz.Free$$anonfun$map$1.apply(Free.scala:52) 
[error] scalaz.Free$$anonfun$flatMap$1$$anonfun$apply$1.apply(Free.scala:60) 
[error] scalaz.Free$$anonfun$flatMap$1$$anonfun$apply$1.apply(Free.scala:60) 
[error] scalaz.Free.resume(Free.scala:72) 
[error] scalaz.Free.go2$1(Free.scala:118) 
[error] scalaz.Free.go(Free.scala:122) 
[error] scalaz.Free.run(Free.scala:172) 
[error] scalaz.effect.IO$class.unsafePerformIO(IO.scala:22) 
[error] scalaz.effect.IOFunctions$$anon$6.unsafePerformIO(IO.scala:227) 
[error] org.specs2.runner.SbtRunner$$anonfun$newTask$1$$anon$4.execute(SbtRunner.scala:37) 
[error] sbt.ForkMain$Run$2.call(ForkMain.java:294) 
[error] sbt.ForkMain$Run$2.call(ForkMain.java:284) 
[error] java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[error] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[error] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[error] java.lang.Thread.run(Thread.java:745)

DatabaseHelper是一個對象，建立了我的測試數據。

減少規格的數量會導致錯誤消失，所以我知道這個問題不適用於特定的測試規範。

我使用Play文檔推薦的運行時DI，並覆蓋數據庫綁定進行測試，以避免消隱我的開發數據庫。

我使用Play evolutions來管理數據庫模式，但是爲了確保它在每個測試規範之前是原始的，故意在數據庫中清空數據庫並重新設置它。

我認爲發生的事情是我的所有測試規格都在同一時間初始化，這意味着它們都試圖同時連接到數據庫，因此它的連接不足。

我曾嘗試在sbt中使用parallelExecution和concurrentRestrictions設置來一次只設置一個進程，但這是無濟於事。我試着設置每個規格以順序運行，但這似乎不起作用。我也嘗試捕捉異常並重試設置，但這似乎不起作用。

我現在不知道該如何嘗試使我的測試工作！請幫忙。

非常感謝。

測試用例：

@RunWith(classOf[JUnitRunner]) 
class FileControllerSpec extends GenericControllerSpec { 
    sequential 

    override val componentName: String = "FileController" 
    override val uriRoot: String = "/file" 

    override def testParsedJsonObject(checkdata: JsLookupResult, parsed_test_json: JsValue) = { 
    val object_keys = Seq("filepath","user","ctime","mtime","atime") 
    val object_keys_int = Seq("storage","version") 

    object_keys.map(key=> 
     (checkdata \ key).as[String] must equalTo((parsed_test_json \ key).as[String]) 
    ) ++ object_keys_int.map(key=> 
     (checkdata \ key).as[Int] must equalTo((parsed_test_json \ key).as[Int]) 
    ) 
    } 

    override val testGetId: Int = 3 
    override val testGetDocument: String = """{"filepath":"/path/to/a/video.mxf","storage":1,"user":"me","version":1,"ctime":"1970-01-01T04:25:45.678+0100","mtime":"1970-01-01T04:25:45.678+0100","atime":"1970-01-01T04:25:45.678+0100"}""" 
    override val testCreateDocument: String = """{"filepath":"/path/to/some/other.project","storage":1,"user":"test","version":3,"ctime":"2017-03-17T13:51:00.123+0000","mtime":"2017-03-17T13:51:00.123+0000","atime":"2017-03-17T13:51:00.123+0000"}""" 
    override val minimumNewRecordId = 3 
    override val testDeleteId: Int = 2 
    override val testConflictId: Int = -1 
}

GenericControllerSpec：

@RunWith(classOf[JUnitRunner]) 
trait GenericControllerSpec extends Specification with BeforeAfterAll { 
    //can over-ride bindings here. see https://www.playframework.com/documentation/2.5.x/ScalaTestingWithGuice 
    val application:Application = new GuiceApplicationBuilder() 
    .overrides(bind[DatabaseConfigProvider].to[TestDatabase.testDbProvider]) 
    .build 

    val injector:Injector = new GuiceApplicationBuilder() 
     .overrides(bind[DatabaseConfigProvider].to[TestDatabase.testDbProvider]) 
     .injector() 

    def inject[T : ClassTag]: T = injector.instanceOf[T] 

    //needed for body.consumeData 
    implicit val system = ActorSystem("storage-controller-spec") 
    implicit val materializer = ActorMaterializer() 

    protected val databaseHelper:DatabaseHelper = inject[DatabaseHelper] 

    val logger: Logger = Logger(this.getClass) 

    override def beforeAll(): Unit ={ 
    logger.warn(">>>> before all <<<<") 
    val theFuture = databaseHelper.setUpDB().map({ 
     case Success(result)=>logger.info("DB setup successful") 
     case Failure(error)=>logger.error(s"DB setup failed: $error") 
    }) 

    Await.result(theFuture, 10.seconds) 
    } 

    override def afterAll(): Unit ={ 
    logger.warn("<<<< after all >>>>") 
    Await.result(databaseHelper.teardownDB(), 10.seconds) 
    } 

    val componentName:String 
    val uriRoot:String 

    def testParsedJsonObject(checkdata:JsLookupResult,test_parsed_json:JsValue):Seq[MatchResult[Any]] 

    val testGetId:Int 
    val testGetDocument:String 
    val testCreateDocument:String 
    val testDeleteId:Int 
    val testConflictId:Int 
    val minimumNewRecordId:Int 

    def bodyAsJsonFuture(response:Future[play.api.mvc.Result]) = response.flatMap(result=> 
    result.body.consumeData.map(contentBytes=> { 
     logger.debug(contentBytes.decodeString("UTF-8")) 
     Json.parse(contentBytes.decodeString("UTF-8")) 
    } 
    ) 
) 

    componentName should { 

    "return 400 on a bad request" in { 
     logger.debug(s"$uriRoot/boum") 
     val response = route(application,FakeRequest(GET, s"$uriRoot/boum")).get 
     status(response) must equalTo(BAD_REQUEST) 
    } 

    "return valid data for a valid record" in { 
     logger.warn(s"Test URL is $uriRoot/1") 
     val response:Future[play.api.mvc.Result] = route(application, FakeRequest(GET, s"$uriRoot/1")).get 

     status(response) must equalTo(OK) 
     val jsondata = Await.result(bodyAsJsonFuture(response), 5.seconds).as[JsValue] 
     (jsondata \ "status").as[String] must equalTo("ok") 
     (jsondata \ "result" \ "id").as[Int] must equalTo(1) 
     testParsedJsonObject(jsondata \ "result", Json.parse(testGetDocument)) 
    } 

    "accept new data to create a new record" in { 
     val response = route(application, FakeRequest(
     method="PUT", 
     uri=uriRoot, 
     headers=FakeHeaders(Seq(("Content-Type", "application/json"))), 
     body=testCreateDocument) 
    ).get 

     status(response) must equalTo(OK) 
     val jsondata = Await.result(bodyAsJsonFuture(response), 5.seconds).as[JsValue] 
     (jsondata \ "status").as[String] must equalTo("ok") 
     (jsondata \ "detail").as[String] must equalTo("added") 
     (jsondata \ "id").as[Int] must greaterThanOrEqualTo(minimumNewRecordId) //if we re-run the tests without blanking the database explicitly this goes up 

     val newRecordId = (jsondata \ "id").as[Int] 
     val checkResponse = route(application, FakeRequest(GET, s"$uriRoot/$newRecordId")).get 
     val checkdata = Await.result(bodyAsJsonFuture(checkResponse), 5.seconds) 


     (checkdata \ "status").as[String] must equalTo("ok") 
     (checkdata \ "result" \ "id").as[Int] must equalTo(newRecordId) 
     testParsedJsonObject(checkdata \ "result", Json.parse(testCreateDocument)) 
    } 

    "delete a record" in { 
     val response = route(application, FakeRequest(
     method="DELETE", 
     uri=s"$uriRoot/$testDeleteId", 
     headers=FakeHeaders(), 
     body="") 
    ).get 

     status(response) must equalTo(OK) 
     val jsondata = Await.result(bodyAsJsonFuture(response), 5.seconds).as[JsValue] 
     (jsondata \ "status").as[String] must equalTo("ok") 
     (jsondata \ "detail").as[String] must equalTo("deleted") 
     (jsondata \ "id").as[Int] must equalTo(testDeleteId) 
    } 

    "return conflict (409) if attempting to delete something with sub-objects" in { 
     val response = route(application, FakeRequest(
     method = "DELETE", 
     uri = s"$uriRoot/$testConflictId", 
     headers = FakeHeaders(), 
     body = "") 
    ).get 

     status(response) must equalTo(CONFLICT) 
     val jsondata = Await.result(bodyAsJsonFuture(response), 5.seconds).as[JsValue] 
     (jsondata \ "status").as[String] must equalTo("error") 
     (jsondata \ "detail").as[String] must equalTo("This is still referenced by sub-objects") 
    } 
    } 
}

DatabaseHelper：

class DatabaseHelper @Inject()(configuration: Configuration, dbConfigProvider: DatabaseConfigProvider) { 

    private val dbConfig = dbConfigProvider.get[JdbcProfile] 
    private val logger: Logger = Logger(this.getClass) 

    def setUpDB():Future[Try[Unit]] = { 
    logger.warn("In setUpDB") 
    dbConfig.db.run(
     DBIO.seq(
     (TableQuery[FileAssociationRow].schema ++ 
      TableQuery[FileEntryRow].schema ++ 
      TableQuery[ProjectEntryRow].schema ++ 
      TableQuery[ProjectTemplateRow].schema ++ 
      TableQuery[ProjectTypeRow].schema ++ 
      TableQuery[StorageEntryRow].schema 
     ).create, 
     TableQuery[StorageEntryRow] += StorageEntry(None,None,"filesystem",Some("me"),None,None,None), 
     TableQuery[StorageEntryRow] += StorageEntry(None,None,"omms",Some("you"),None,None,None), 
     TableQuery[FileEntryRow] += FileEntry(None,"/path/to/a/video.mxf",1,"me",1,new Timestamp(12345678),new Timestamp(12345678),new Timestamp(12345678)), 
     TableQuery[FileEntryRow] += FileEntry(None,"/path/to/secondtestfile",1,"tstuser",1,new Timestamp(123456789),new Timestamp(123456789),new Timestamp(123456789)), 
     //"""{"name": "Premiere test template 1","projectTypeId": 1,"filepath", "storageId": 1}""" 
     //"{"name":,"opensWith":"AdobePremierePro.app","targetVersion":"14.0"}" 
     TableQuery[ProjectTypeRow] += ProjectType(None,"Premiere 2014 test","AdobePremierePro.app","14.0"), 
     TableQuery[ProjectTypeRow] += ProjectType(None,"Cubase 7.0 test","Cubase.app","7.0"), 
     TableQuery[ProjectTemplateRow] += ProjectTemplate(Some(1),"Premiere test template 1",1,"/srv/projectfiles/ProjectTemplatesDev/Premiere/premiere_template_2014.prproj",1) 

    ).asTry 
    ) 
    } 

    def teardownDB():Future[Try[Unit]] = { 
    logger.warn("In teardownDB") 
    dbConfig.db.run(
     DBIO.seq(
     (
      TableQuery[FileAssociationRow].schema ++ 
      TableQuery[FileEntryRow].schema ++ 
      TableQuery[ProjectEntryRow].schema ++ 
      TableQuery[ProjectTemplateRow].schema ++ 
      TableQuery[ProjectTypeRow].schema ++ 
      TableQuery[StorageEntryRow].schema 
     ).drop 
    ).asTry 
    ) 
    } 
}

build.sbt設置：

concurrentRestrictions in Global := Seq(
    Tags.limit(Tags.Test, 1), 
    Tags.limitAll(1) 
) 

parallelExecution in Test := false

來源

2017-07-07 Andy Gallagher

回答

0

該異常告訴您您的連接池無法連接。

在這裏我看到兩個問題：

  1. 您不關閉在拆解的數據庫連接。
  2. 您可能會爲每個連接池打開多個連接，並且由於您的每個規格都在創建一個新的連接池，因此您的postgres服務器上配置的最大連接數不足。您可以通過減少slick config

的「numThreads」參數減少每個連接池的連接數。由於性能原因，我建議你反正要在Postgres中使用的設置H2DB驅動單元測試，只要你不使用postgres H2DB無法模擬的特定功能。

來源

2017-07-07 15:09:42

+0

嗨大衛，非常感謝您的幫助。我已經添加了接近拆解的東西，並且在本地工作但在CI中仍然存在問題。我已經開始使用H2DB驅動程序，但切換到Postgres，因爲我無法讓H2與數據庫Evolution一起工作，但是目前我已經決定放棄進化並讓應用程序工作，然後再擔心數據遷移。這真的有助於提高我的理解:) –

相關問題

 相關問題