MongoDB $或查詢本身的速度更快

我有一個集合中有16m文檔的mongo實例。我正在編寫一個查詢來搜索（索引）字段之一，我得到了一些奇怪的結果，這是我無法解釋的。

如果我執行一個查詢直接，如：

find({ "$and" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "active" : true}]}).limit(100).sort({ "_id" : 1})

甚至增加無謂$或查詢：

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1})

它返回71673ms

擷取的3結果

但是，如果我使用$或反對自己，如：

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1})

它返回：在4ms的

於是一個大的性能差異擷取的3結果。通過檢查查詢的explain（），我無法確定爲什麼存在如此大的性能差異。任何人都可以闡明我錯過了什麼或者mongo在這些方面做了什麼不同？

說明（）在單個$或採取> 60000毫秒

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1}).explain() 
{ 
    "queryPlanner" : { 
     "plannerVersion" : 1, 
     "namespace" : "CLS-TEST.Leases", 
     "indexFilterSet" : false, 
     "parsedQuery" : { 
      "$and" : [ 
       { 
        "active" : { 
         "$eq" : true 
        } 
       }, 
       { 
        "ipAddr" : /^01:172/ 
       } 
      ] 
     }, 
     "winningPlan" : { 
      "stage" : "SORT", 
      "sortPattern" : { 
       "_id" : 1 
      }, 
      "limitAmount" : 100, 
      "inputStage" : { 
       "stage" : "SORT_KEY_GENERATOR", 
       "inputStage" : { 
        "stage" : "FETCH", 
        "filter" : { 
         "active" : { 
          "$eq" : true 
         } 
        }, 
        "inputStage" : { 
         "stage" : "IXSCAN", 
         "keyPattern" : { 
          "ipAddr" : 1 
         }, 
         "indexName" : "ipAddr_1", 
         "isMultiKey" : false, 
         "isUnique" : false, 
         "isSparse" : false, 
         "isPartial" : false, 
         "indexVersion" : 1, 
         "direction" : "forward", 
         "indexBounds" : { 
          "ipAddr" : [ 
           "[\"01:172\", \"01:173\")", 
           "[/^01:172/, /^01:172/]" 
          ] 
         } 
        } 
       } 
      } 
     }, 
     "rejectedPlans" : [ 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "ipAddr" : /^01:172/ 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "sessionId" : 1, 
           "updateTime" : 1 
          }, 
          "indexName" : "active_1_sessionId_1_updateTime_1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "sessionId" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "updateTime" : [ 
            "[MinKey, MaxKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "ipAddr" : /^01:172/ 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "clientId" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_clientId_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "clientId" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "ipAddr" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_ipAddr_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "ipAddr" : [ 
            "[\"01:172\", \"01:173\")", 
            "[/^01:172/, /^01:172/]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "ipAddr" : /^01:172/ 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "macAddress" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_macAddress_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "macAddress" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "ipAddr" : /^01:172/ 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "remoteId" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_remoteId_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "remoteId" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "LIMIT", 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "FETCH", 
        "filter" : { 
         "$and" : [ 
          { 
           "active" : { 
            "$eq" : true 
           } 
          }, 
          { 
           "ipAddr" : /^01:172/ 
          } 
         ] 
        }, 
        "inputStage" : { 
         "stage" : "IXSCAN", 
         "keyPattern" : { 
          "_id" : 1 
         }, 
         "indexName" : "_id_", 
         "isMultiKey" : false, 
         "isUnique" : true, 
         "isSparse" : false, 
         "isPartial" : false, 
         "indexVersion" : 1, 
         "direction" : "forward", 
         "indexBounds" : { 
          "_id" : [ 
           "[MinKey, MaxKey]" 
          ] 
         } 
        } 
       } 
      } 
     ] 
    }, 
    "serverInfo" : { 
     "host" : "", 
     "port" : 27017, 
     "version" : "3.2.3", 
     "gitVersion" : "b326ba837cf6f49d65c2f85e1b70f6f31ece7937" 
    }, 
    "ok" : 1 
}

說明（）在$或自相這需要< 50ms的

find({ "$and" : [ { "$or" : [ { "ipAddr" : { "$regex" : "^01:172"}} , { "ipAddr" : { "$regex" : "^01:172"}}]} , { "active" : true}]}).limit(100).sort({ "_id" : 1}).explain() 
{ 
    "queryPlanner" : { 
     "plannerVersion" : 1, 
     "namespace" : "CLS-TEST.Leases", 
     "indexFilterSet" : false, 
     "parsedQuery" : { 
      "$and" : [ 
       { 
        "$or" : [ 
         { 
          "ipAddr" : /^01:172/ 
         }, 
         { 
          "ipAddr" : /^01:172/ 
         } 
        ] 
       }, 
       { 
        "active" : { 
         "$eq" : true 
        } 
       } 
      ] 
     }, 
     "winningPlan" : { 
      "stage" : "SORT", 
      "sortPattern" : { 
       "_id" : 1 
      }, 
      "limitAmount" : 100, 
      "inputStage" : { 
       "stage" : "SORT_KEY_GENERATOR", 
       "inputStage" : { 
        "stage" : "FETCH", 
        "filter" : { 
         "active" : { 
          "$eq" : true 
         } 
        }, 
        "inputStage" : { 
         "stage" : "IXSCAN", 
         "keyPattern" : { 
          "ipAddr" : 1 
         }, 
         "indexName" : "ipAddr_1", 
         "isMultiKey" : false, 
         "isUnique" : false, 
         "isSparse" : false, 
         "isPartial" : false, 
         "indexVersion" : 1, 
         "direction" : "forward", 
         "indexBounds" : { 
          "ipAddr" : [ 
           "[\"01:172\", \"01:173\")", 
           "[/^01:172/, /^01:172/]" 
          ] 
         } 
        } 
       } 
      } 
     }, 
     "rejectedPlans" : [ 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "$or" : [ 
           { 
            "ipAddr" : /^01:172/ 
           }, 
           { 
            "ipAddr" : /^01:172/ 
           } 
          ] 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "sessionId" : 1, 
           "updateTime" : 1 
          }, 
          "indexName" : "active_1_sessionId_1_updateTime_1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "sessionId" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "updateTime" : [ 
            "[MinKey, MaxKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "$or" : [ 
           { 
            "ipAddr" : /^01:172/ 
           }, 
           { 
            "ipAddr" : /^01:172/ 
           } 
          ] 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "clientId" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_clientId_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "clientId" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "$or" : [ 
           { 
            "ipAddr" : /^01:172/ 
           }, 
           { 
            "ipAddr" : /^01:172/ 
           } 
          ] 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "ipAddr" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_ipAddr_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "ipAddr" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "$or" : [ 
           { 
            "ipAddr" : /^01:172/ 
           }, 
           { 
            "ipAddr" : /^01:172/ 
           } 
          ] 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "macAddress" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_macAddress_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "macAddress" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "SORT", 
       "sortPattern" : { 
        "_id" : 1 
       }, 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "SORT_KEY_GENERATOR", 
        "inputStage" : { 
         "stage" : "FETCH", 
         "filter" : { 
          "$or" : [ 
           { 
            "ipAddr" : /^01:172/ 
           }, 
           { 
            "ipAddr" : /^01:172/ 
           } 
          ] 
         }, 
         "inputStage" : { 
          "stage" : "IXSCAN", 
          "keyPattern" : { 
           "active" : 1, 
           "remoteId" : 1, 
           "startTime" : -1, 
           "_id" : -1 
          }, 
          "indexName" : "active_1_remoteId_1_startTime_-1__id_-1", 
          "isMultiKey" : false, 
          "isUnique" : false, 
          "isSparse" : false, 
          "isPartial" : false, 
          "indexVersion" : 1, 
          "direction" : "forward", 
          "indexBounds" : { 
           "active" : [ 
            "[true, true]" 
           ], 
           "remoteId" : [ 
            "[MinKey, MaxKey]" 
           ], 
           "startTime" : [ 
            "[MaxKey, MinKey]" 
           ], 
           "_id" : [ 
            "[MaxKey, MinKey]" 
           ] 
          } 
         } 
        } 
       } 
      }, 
      { 
       "stage" : "LIMIT", 
       "limitAmount" : 100, 
       "inputStage" : { 
        "stage" : "FETCH", 
        "filter" : { 
         "$and" : [ 
          { 
           "$or" : [ 
            { 
             "ipAddr" : /^01:172/ 
            }, 
            { 
             "ipAddr" : /^01:172/ 
            } 
           ] 
          }, 
          { 
           "active" : { 
            "$eq" : true 
           } 
          } 
         ] 
        }, 
        "inputStage" : { 
         "stage" : "IXSCAN", 
         "keyPattern" : { 
          "_id" : 1 
         }, 
         "indexName" : "_id_", 
         "isMultiKey" : false, 
         "isUnique" : true, 
         "isSparse" : false, 
         "isPartial" : false, 
         "indexVersion" : 1, 
         "direction" : "forward", 
         "indexBounds" : { 
          "_id" : [ 
           "[MinKey, MaxKey]" 
          ] 
         } 
        } 
       } 
      } 
     ] 
    }, 
    "serverInfo" : { 
     "host" : "", 
     "port" : 27017, 
     "version" : "3.2.3", 
     "gitVersion" : "b326ba837cf6f49d65c2f85e1b70f6f31ece7937" 
    }, 
    "ok" : 1 
}

來源

2016-03-04 caverman

你可能會注意到，沒有任何的索引選擇包括"active"和"ipAddr"的組合，這將是在此定義的有用索引。

簡而言之，「較慢」查詢僅使用"ipAddr"的索引，因此需要更多的工作才能「過濾」出{ "active": true }條目。

顯然，當其他索引選擇使用帶有這些邊界的"active"鍵時，傳遞到正則表達式模式上的後續過濾器的結果較少。你在這裏似乎有很多索引，並且它們都不是真正的最佳查詢。

我給你的道具至少運行「解釋」對任何查詢輸出，但如果你仔細觀察的話你應該看到的是，「慢」查詢「錯誤地」選擇了"ipAddr"指數認爲它是最佳的。它可能不是，但對於優化器來說，考慮使用「錨定」正則表達式是一個合理的假設。

該$or力"index intersection"，它是聰明的，當$or只有「一個」參數不這樣做。「兩個」參數使得這種情況發生，優化器通過查找與其他查詢條件（"active"值）前面的索引進行另一個「猜測」。

這是有道理的，因爲現在正在運行「兩條」查詢，它將與結果「交叉」，因此任何條件之外的任何條件都是爲最佳選擇索引的合理選擇。

由於這些返回的結果可能較小，因此「過濾」出正則表達式匹配比查看所有正則表達式結果並篩選出「有效」值更快。

因此「最好」的指標來定義這個查詢將是：

.createIndex({ "active": 1, "ipAddr": 1 })

然後從任一查詢的結果是一致的，當然提供的優化器沒有得到通過存在的其他指標混淆並選擇一個。要強制索引選擇，請使用.hint()

來源

2016-03-05 00:25:37

感謝您提供詳細的響應和指向「索引交集」的指針。我希望避免創建一個額外的索引，因爲其他索引可以減少維護開銷，但也許這是必需的。 – caverman

@caverman關於索引的數量，您可能需要仔細查看所有查詢模式（並解釋輸出）並決定是否真的需要它們。將索引鍵「最優化」使用的複合索引可能是最好的，因此最好使用最常見的組合和他們可能過濾的結果。這裏的經驗主要是''active''是真正的價值減少可能的結果，並且對於這個查詢，至少該值將作爲索引內的前面條目做出最大差異。 –

MongoDB $或查詢本身的速度更快

回答

相關問題