2016-11-30 240 views
1

我目前有兩個大型數據集(大約200GB),我想將其存儲到ContextBroker中,以便對其執行數據分析。到目前爲止,我已經能夠設置ContextBrokerMongoDB。我還安裝了Cygnus訂閱ONCHANGE爲了堅持數據從ContextBroker。是否可以使用CKAN將我的外部數據集存儲到FIWARE平臺中,還是必須使用其他內容?將數據集導入ContextBroker

EDIT1:這些數據集是一組包含一系列事件的對象,是timestampedgeo-locatedjson文件。

EDIT2:由於請求的數據的一個小樣本我指的是:

{"track":[ 
{"time":"2015-11-16T00:45:29.016","midpoint":{"x":70.66,"y":188.90},"realworld":{"x":-7.49,"y":22.96},"RDCoordinate":{"x":161582.09,"y":383031.34},"UTM-WGS84":{"Latitude":51.4363489,"Longitude":5.4818640}}, 
{"time":"2015-11-16T00:45:29.022","midpoint":{"x":66.85,"y":189.61},"realworld":{"x":-7.53,"y":22.92},"RDCoordinate":{"x":161582.16,"y":383031.34},"UTM-WGS84":{"Latitude":51.4363489,"Longitude":5.4818649}}, 
{"time":"2015-11-16T00:45:29.029","midpoint":{"x":61.66,"y":189.92},"realworld":{"x":-7.60,"y":22.90},"RDCoordinate":{"x":161582.22,"y":383031.31},"UTM-WGS84":{"Latitude":51.4363486,"Longitude":5.4818658}}, 
{"time":"2015-11-16T00:45:29.082","midpoint":{"x":56.16,"y":190.47},"realworld":{"x":-7.66,"y":22.87},"RDCoordinate":{"x":161582.28,"y":383031.28},"UTM-WGS84":{"Latitude":51.4363483,"Longitude":5.4818667}}, 
{"time":"2015-11-16T00:45:29.090","midpoint":{"x":50.74,"y":191.64},"realworld":{"x":-7.71,"y":22.80},"RDCoordinate":{"x":161582.38,"y":383031.28},"UTM-WGS84":{"Latitude":51.4363483,"Longitude":5.4818681}}, 
{"time":"2015-11-16T00:45:29.112","midpoint":{"x":45.58,"y":192.07},"realworld":{"x":-7.78,"y":22.78},"RDCoordinate":{"x":161582.44,"y":383031.25},"UTM-WGS84":{"Latitude":51.4363480,"Longitude":5.4818690}}, 
{"time":"2015-11-16T00:45:29.151","midpoint":{"x":41.80,"y":193.42},"realworld":{"x":-7.80,"y":22.69},"RDCoordinate":{"x":161582.52,"y":383031.31},"UTM-WGS84":{"Latitude":51.4363486,"Longitude":5.4818701}}, 
{"time":"2015-11-16T00:45:29.197","midpoint":{"x":36.27,"y":194.43},"realworld":{"x":-7.86,"y":22.63},"RDCoordinate":{"x":161582.59,"y":383031.31},"UTM-WGS84":{"Latitude":51.4363486,"Longitude":5.4818712}} 
]} 
+0

你可以編輯你的問題,以闡述你的數據集是怎樣的?我的意思是,它們對應於上下文信息的「拍攝」,事件的歷史序列等。 – fgalan

+0

(查看Edit1之後)您可以編輯添加一個或兩個您引用的JSON對象嗎? – fgalan

回答

0

甲posibility可能是每個項目建模爲在獵戶Conext下列方式的Event類型的實體經紀人:

{ 
    "id": "Item1", 
    "type": "Event", 
    "time": { 
     "value": "2015-11-16T00:45:29.016", 
     "type": "DateTime" 
    }, 
    "midpoint": { 
     "value": { 
      "x": 70.66, 
      "y": 188.90 
     }, 
     "type": "Object" 
    }, 
    "realworld": { 
     "value": { 
      "x": -7.49, 
      "y": 22.96 
     }, 
     "type": "Object" 
    }, 
    "RDCoordinate": { 
     "value": { 
      "x": 161582.09, 
      "y": 383031.34 
     }, 
     "type": "Object" 
    }, 
    "UTM-WGS84": { 
     "value": "51.4363489, 5.4818640", 
     "type": "geo:point" 
    } 
} 

(以上可作爲有效載荷爲POST /v2/entities操作來創建實體)。

在這個模型中,timemidpointrealworldRDCoordinateUTM-WG84是實體屬性。請注意,特殊的屬性類型timeUTM-WG84

  • time使用DateTime,這是NGSIv2標準的方式來指定日期(見NSGIv2 specification document「特殊屬性類型」)。我建議加入時區,以獲得更高的精度(例如2015-11-16T00:45:29.016Z)。

  • UTM-WG84使用geo:point,這是NGSIv2標準方式來指定實體的點位置(請參閱NSGIv2 specification document中的「實體的地理空間屬性」)。

+0

嗨fgalan。謝謝您的回覆!這看起來確實是我正在尋找的東西,但是這需要很多工作來插入所有數據。是否有可能自動完成這項工作? –

+0

處理JSON文件中的每行數據並向Orion發送相應的創建實體命令的腳本應該不是很困難(即使您可以考慮使用POST/v2/op /更新'與APPEND actionType)。 – fgalan