{"id":8682,"date":"2015-12-21T09:00:54","date_gmt":"2015-12-21T00:00:54","guid":{"rendered":"http:\/\/www.techscore.com\/blog\/?p=8682"},"modified":"2018-11-14T16:33:46","modified_gmt":"2018-11-14T07:33:46","slug":"presto-connector-development-3","status":"publish","type":"post","link":"https:\/\/www.techscore.com\/blog\/2015\/12\/21\/presto-connector-development-3\/","title":{"rendered":"Presto \u30b3\u30cd\u30af\u30bf\u30fc\u3092\u5b9f\u88c5\u3059\u308b \u7b2c\u4e09\u56de"},"content":{"rendered":"
\u3053\u3093\u306b\u3061\u306f\u3002\u677e\u672c\u3067\u3059\u3002<\/p>\n
\u3053\u308c\u306f TECHSCORE Advent Calendar 2015<\/a> \u306e21\u65e5\u76ee\u306e\u8a18\u4e8b\u3067\u3059\u3002<\/p>\n \u524d\u56de<\/a>\u304b\u3089\u968f\u5206\u3068\u6642\u9593\u304c\u7a7a\u3044\u3066\u3057\u307e\u3044\u307e\u3057\u305f\u306d\u3002\u30b9\u30ba\u30ad\u7de8\u96c6\u9577\u306b\u300c\u307e\u3060\u304b\u300d\u3068\u8a00\u308f\u308c\u3064\u3064\u3082\u3001\u5b8c\u5168\u306b\u4ed5\u4e8b\u306b\u304b\u307e\u3051\u3066\u3055\u307c\u3063\u3066\u3057\u307e\u3044\u307e\u3057\u305f\u3002\u3054\u3081\u3093\u306a\u3055\u3044\u3002\u643a\u308f\u3063\u3066\u3044\u305f\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\uff08\u3053\u308c<\/a>\u3068\u304b\u3053\u308c<\/a>\uff09\u304c\u697d\u3057\u304f\u3066\u4ed5\u65b9\u304c\u306a\u304b\u3063\u305f\u306e\u3067\u3059\u30fb\u30fb\uff08\u672c\u5f53\u306b\uff09\u3002<\/p>\n \u3055\u3066\u3001Presto \u306e\u30d0\u30fc\u30b8\u30e7\u30f3\u306f\u65e2\u306b 0.131 \u307e\u3067\u4e0a\u304c\u3063\u305f\u3088\u3046\u3067\u3059\u304c\u3001\u672c\u8a18\u4e8b\u3067\u306f\u5f15\u304d\u7d9a\u304d 0.96 \u3067\u306e\u30b3\u30cd\u30af\u30bf\u30fc\u958b\u767a\u306b\u3064\u3044\u3066\u3054\u7d39\u4ecb\u3057\u3066\u3044\u304d\u307e\u3059\u3002<\/p>\n \u4eca\u56de\u306f\u30ec\u30b3\u30fc\u30c9\u64cd\u4f5c\u307e\u308f\u308a\u3067\u3059\u3002\u9593\u304c\u7a7a\u304d\u904e\u304e\u3066\u524d\u56de\u307e\u3067\u306e\u5185\u5bb9\u3092\u5fd8\u308c\u305f\u3068\u3044\u3046\u65b9\u306f\u3053\u3061\u3089\u3092\u53c2\u7167\u4e0b\u3055\u3044\u3002<\/p>\n Presto \u30b3\u30cd\u30af\u30bf\u30fc\u3092\u5b9f\u88c5\u3059\u308b \u7b2c\u4e00\u56de<\/a> Presto \u306f\u3001\u30e6\u30fc\u30b6\u30fc\u304b\u3089\u306e\u554f\u3044\u5408\u308f\u305b\u3092\u30c6\u30fc\u30d6\u30eb\u5358\u4f4d\u3067\u30b3\u30cd\u30af\u30bf\u30fc\u306b\u6e21\u3057\u307e\u3059\u3002\u30b3\u30cd\u30af\u30bf\u30fc\u306f\u3053\u306e\u554f\u3044\u5408\u308f\u305b\u306b\u5bfe\u3059\u308b\u51e6\u7406\u3092 ConnectorSplitManager \u30a4\u30f3\u30bf\u30d5\u30a7\u30fc\u30b9\u3092\u4f7f\u3044\u3001\u4efb\u610f\u306b\u5206\u5272\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\n 8.1. SPI Overview - Presto 0.96 Documentation<\/a><\/p>\n \nThe split manager partitions the data for a table into the individual chunks that Presto will distribute to workers for processing.\n<\/p><\/blockquote>\n \u4f8b\u3048\u3070\u3001\u554f\u3044\u5408\u308f\u305b\u5148\u3068\u306a\u308b\u30c6\u30fc\u30d6\u30eb\u306e\u30c7\u30fc\u30bf\u304c\u3001\u5b9f\u969b\u306b\u306f\u8907\u6570\u306e\u30d5\u30a1\u30a4\u30eb\u306b\u5206\u5272\u3055\u308c\u3066\u4fdd\u5b58\u3055\u308c\u3066\u3044\u308b\u5834\u5408\u3001\u30d5\u30a1\u30a4\u30eb\u6bce\u306b\u51e6\u7406\u3092\u5206\u5272\u3057\u3001\u305d\u308c\u3092\u4e26\u5217\u51e6\u7406\u3055\u305b\u308b\u3053\u3068\u3067\u52b9\u7387\u3088\u304f\u554f\u3044\u5408\u308f\u305b\u3092\u51e6\u7406\u3059\u308b\u3053\u3068\u304c\u53ef\u80fd\u306b\u306a\u308a\u307e\u3059\u3002Presto \u30b3\u30cd\u30af\u30bf\u30fc\u306e\u5b9f\u88c5\u4f8b\u3067\u3042\u308b Example HTTP Connector<\/a> \u304c\u3053\u306e\u4f8b\u3067\u3059\u3002<\/p>\n \u6b21\u306b\u7d39\u4ecb\u3059\u308b 3 \u3064\u306e\u30a4\u30f3\u30bf\u30d5\u30a7\u30fc\u30b9\u304c\u3053\u308c\u3089\u306e\u5f79\u5272\u3092\u62c5\u3044\u307e\u3059\u3002<\/p>\n ConnectorPartition \u306f\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u306e\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u3092\u8868\u3059\u30a4\u30f3\u30bf\u30d5\u30a7\u30fc\u30b9\u3067\u3059\u3002<\/p>\n ConnectorSplit \u306f\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u306b\u5bfe\u3059\u308b\u30a2\u30af\u30bb\u30b9\u51e6\u7406\u5358\u4f4d\u3092\u8868\u3057\u307e\u3059\u3002\u5f8c\u8ff0\u3059\u308b RecordCursor \u306f\u3053\u306e\u5358\u4f4d\u3067\u751f\u6210\u3055\u308c\u307e\u3059\u3002<\/p>\n ConnectorSplitManager \u306f\u3001\u554f\u3044\u5408\u308f\u305b\u306b\u5bfe\u3057 ConnectorPartition \u3078\u306e\u5206\u5272\u3068\u3001ConnectorSplit \u3078\u306e\u5206\u5272\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\n \u5c1a\u3001\u4eca\u56de\u306e\u30b5\u30f3\u30d7\u30eb\u30b3\u30fc\u30c9\u3067\u306f\u3001\u30b5\u30f3\u30d7\u30eb\u3068\u3057\u3066\u306e\u30b7\u30f3\u30d7\u30eb\u3055\u3092\u512a\u5148\u3057\u3001\u51e6\u7406\u306e\u5206\u5272\u306b\u306f\u5bfe\u5fdc\u3057\u3066\u3044\u307e\u305b\u3093\u3002\u554f\u3044\u5408\u308f\u305b\u306b\u5bfe\u3057\u5e38\u306b ConnectorSplit \u3072\u3068\u3064\u3067\u51e6\u7406\u3092\u884c\u3088\u3046\u5b9f\u88c5\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n \u3053\u3053\u3067\u5b9f\u88c5\u3059\u308b\u4e8b\u306b\u306a\u308b\u306e\u306f partitionId \u30d7\u30ed\u30d1\u30c6\u30a3\u3068 tupleDomain \u30d7\u30ed\u30d1\u30c6\u30a3\u306e Getter \u30e1\u30bd\u30c3\u30c9\u3067\u3059\u3002<\/p>\n partitionId \u30d7\u30ed\u30d1\u30c6\u30a3\u306f\u5bfe\u8c61\u30c6\u30fc\u30d6\u30eb\u306b\u5bfe\u3057\u3066\u4e00\u610f\u306b\u306a\u308b\u3088\u3046\u5272\u308a\u5f53\u3066\u305f\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u306e ID \u3067\u3059\u3002\u30b5\u30f3\u30d7\u30eb\u30b3\u30fc\u30c9\u306f\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u304c\u3072\u3068\u3064\u306a\u306e\u3067\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u6bce\u306b ID \u3092\u5272\u308a\u5f53\u3066\u308b\u5fc5\u8981\u304c\u306a\u304f\u3001\u30b9\u30ad\u30fc\u30de\u540d\u3068\u30c6\u30fc\u30d6\u30eb\u540d\u306e\u307f\u3092\u7d50\u5408\u3057\u305f\u6587\u5b57\u5217\u3092 ID \u3068\u3057\u3066\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059\u3002Getter \u30e1\u30bd\u30c3\u30c9\u306e\u5b9a\u7fa9\u306f lombok \u306e @Value \u30a2\u30ce\u30c6\u30fc\u30b7\u30e7\u30f3\u306b\u3088\u3063\u3066\u81ea\u52d5\u751f\u6210\u3055\u308c\u307e\u3059\u3002<\/p>\n tupleDomain \u306f\u3053\u306e\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u304c\u53d7\u3051\u6301\u3064\u30c7\u30fc\u30bf\u306e\u7bc4\u56f2\u3092\u8868\u3057\u307e\u3059\u3002\u30b5\u30f3\u30d7\u30eb\u30b3\u30fc\u30c9\u3067\u306f\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u304c\u3072\u3068\u3064\u306a\u306e\u3067\u3001\u5e38\u306b\u300c\u5168\u3066\u300d\u3092\u8868\u3059 TupleDomain<\/a> \u3092\u8fd4\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n \u672c\u30af\u30e9\u30b9\u3082\u524d\u56de\u306e ConnectorTableHandle \u3084 ConnectorColumn \u540c\u69d8\u3001@JsonCreator \u3084 @JsonProperty \u30a2\u30ce\u30c6\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u3063\u3066 JSON \u3067\u306e\u30b7\u30ea\u30a2\u30e9\u30a4\u30ba\uff0f\u30c7\u30b7\u30ea\u30a2\u30e9\u30a4\u30ba\u3092\u53ef\u80fd\u3068\u3057\u307e\u3059\u3002<\/p>\n address \u30d7\u30ed\u30d1\u30c6\u30a3\u3001remotelyAccessible \u30d7\u30ed\u30d1\u30c6\u30a3\u306f\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u3078\u306e\u30a2\u30af\u30bb\u30b9\u65b9\u6cd5\u306e\u30d2\u30f3\u30c8\u306b\u306a\u308b\u60c5\u5831\u3067\u3059\u304c\u3001\u30b5\u30f3\u30d7\u30eb\u30b3\u30fc\u30c9\u3067\u3042\u308b\u305f\u3081\u3001\u305d\u308c\u305e\u308c Getter \u30e1\u30bd\u30c3\u30c9\u304c\u56fa\u5b9a\u5024\u3092\u8fd4\u3059\u3088\u3046\u5b9f\u88c5\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n getPartitions() \u30e1\u30bd\u30c3\u30c9\u306f\u554f\u3044\u5408\u308f\u305b\u5185\u5bb9\u3092\u3082\u3068\u306b ConnectorPartition \u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\u30b5\u30f3\u30d7\u30eb\u3067\u306f\u554f\u3044\u5408\u308f\u305b\u306e\u5185\u5bb9\u306b\u95a2\u4fc2\u306a\u304f\u3001\u5358\u4e00\u306e TechscorePartition \u3092\u751f\u6210\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n \u3053\u3053\u3067\u3001\u5f15\u6570 tupleDomain \u304c\u554f\u3044\u5408\u308f\u305b\u6761\u4ef6\u3092\u4fdd\u6301\u3057\u3066\u3044\u307e\u3059\u3002tupleDomain \u304c\u3069\u306e\u3088\u3046\u306b\u6761\u4ef6\u3092\u4fdd\u6301\u3057\u3066\u3044\u308b\u304b\u306f\u3001\u6b21\u306e\u5b9f\u884c\u7d50\u679c\u3092\u898b\u308b\u3068\u30a4\u30e1\u30fc\u30b8\u3057\u6613\u3044\u3067\u3057\u3087\u3046\u3002\u3053\u306e\u5b9f\u884c\u7d50\u679c\u306f\u3001Presto \u306b\u5bfe\u3057\u3066\u30af\u30a8\u30ea\u3092\u5b9f\u884c\u3057\u305f\u969b\u306e\u3001\u5f15\u6570 tupleDomain \u306e\u5185\u5bb9\u3092\u51fa\u529b\u3057\u305f\u3082\u306e\u3067\u3059\u3002<\/p>\n getPartitionSplits() \u30e1\u30bd\u30c3\u30c9\u306f\u3001getPartitions() \u3067\u4f5c\u6210\u3057\u305f\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u60c5\u5831\u3092\u3082\u3068\u306b ConnectorSplit \u3092\u751f\u6210\u3057\u307e\u3059\u3002\u30b5\u30f3\u30d7\u30eb\u3067\u306f\u5e38\u306b\u5358\u4e00\u306e TechscoreSplit \u3092\u751f\u6210\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n \u524d\u8ff0\u306e\u901a\u308a Presto \u30b3\u30cd\u30af\u30bf\u30fc\u306f ConnectorSplitManager \u3067\u751f\u6210\u3055\u308c\u305f ConnectorSplit \u5358\u4f4d\u3067\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u306b\u30a2\u30af\u30bb\u30b9\u3057\u3001Presto \u306b\u30c7\u30fc\u30bf\u3092\u8fd4\u3057\u307e\u3059\u3002<\/p>\n \u3053\u306e\u5f79\u5272\u3092\u62c5\u3046\u306e\u304c\u3001\u6b21\u306e\u30a4\u30f3\u30bf\u30d5\u30a7\u30fc\u30b9\u3067\u3059\u3002<\/p>\n RecordCursor \u306f JDBC \u306e ResultSet \u306e\u3088\u3046\u306a\u3082\u306e\u3067\u3001\u554f\u3044\u5408\u308f\u305b\u7d50\u679c\u306b\u5bfe\u3057\u3001\u884c\u3092\u9032\u3081\u306a\u304c\u3089\u30d5\u30a3\u30fc\u30eb\u30c9\u30c7\u30fc\u30bf\u306b\u30a2\u30af\u30bb\u30b9\u3059\u308b\u30a4\u30f3\u30bf\u30d5\u30a7\u30fc\u30b9\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002<\/p>\n RecordSet \u306f RecordCursor \u3092\u751f\u6210\u3059\u308b\u5f79\u5272\u3092\u62c5\u3044\u307e\u3059\u3002<\/p>\n ConnectorRecordSetProvider \u306f ConnectorSplit \u5358\u4f4d\u3067 RecordSet \u3092\u751f\u6210\u3059\u308b\u5f79\u5272\u3092\u62c5\u3044\u307e\u3059\u3002<\/p>\n \u5c1a\u3001RecordSet\u3001ConnectorRecordSetProvider \u3092\u305d\u308c\u305e\u308c\u5b9f\u88c5\u3057\u305f TechscoreRecordSet \u3068 TechscoreRecordSetProvider \u30af\u30e9\u30b9\u306b\u3064\u3044\u3066\u306f\u3001\u305d\u308c\u305e\u308c TechscoreRecordCursor \u306e\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u3001TechscoreRecordSet \u306e\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u3092\u751f\u6210\u3057\u3066\u3044\u308b\u3060\u3051\u306a\u306e\u3067\u8aac\u660e\u3092\u7701\u304d\u3001\u30bd\u30fc\u30b9\u30b3\u30fc\u30c9\u3060\u3051\u3092\u63b2\u8f09\u3057\u307e\u3059\u3002<\/p>\n \u672c\u5b9f\u88c5\u3067\u306f\u3001\u30b3\u30f3\u30b9\u30c8\u30e9\u30af\u30bf\u306b\u554f\u3044\u5408\u308f\u305b\u7d50\u679c\u3068\u306a\u308b\u30c7\u30fc\u30bf\u304c\u6e21\u3055\u308c\u308b\u5b9f\u88c5\u3067\u3059\u3002<\/p>\n getTotalBytes(), getCompletedBytes(), getReadTimeNanos() \u30e1\u30bd\u30c3\u30c9\u306f\u5404\u7a2e\u7d71\u8a08\u60c5\u5831\u306b\u5229\u7528\u3059\u308b\u6570\u5024\u3092\u8fd4\u3057\u307e\u3059\u3002\u30b5\u30f3\u30d7\u30eb\u30b3\u30fc\u30c9\u3067\u306f\u5b9f\u88c5\u3092\u7701\u7565\u3057\u3001\u5e38\u306b 0 \u3092\u8fd4\u3059\u3088\u3046\u306b\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n getType() \u30e1\u30bd\u30c3\u30c9\u306f\u6307\u5b9a\u30d5\u30a3\u30fc\u30eb\u30c9\u306e\u30c7\u30fc\u30bf\u578b\u3092\u8fd4\u3057\u307e\u3059\u3002<\/p>\n advanceNextPosition() \u30e1\u30bd\u30c3\u30c9\u304c\u884c\u3092\u9032\u3081\u308b\u5f79\u5272\u3092\u6301\u3061\u307e\u3059\u3002<\/p>\n getBoolean(), getLong(), getDouble(), getSlice() \u30e1\u30bd\u30c3\u30c9\u304c\u30ab\u30ec\u30f3\u30c8\u884c\u306e\u30d5\u30a3\u30fc\u30eb\u30c9\u30c7\u30fc\u30bf\u306b\u30a2\u30af\u30bb\u30b9\u3059\u308b\u5f79\u5272\u3092\u62c5\u3044\u307e\u3059\u3002<\/p>\n Presto \u30b3\u30cd\u30af\u30bf\u30fc\u3067\u306f\u3001\u30c7\u30fc\u30bf\u3092\u6b21\u306e 4 \u3064\u306e Java \u30c7\u30fc\u30bf\u578b\u3067\u8868\u73fe\u3057\u307e\u3059\u3002\u7b2c\u4e8c\u56de\u3067\u3082\u3054\u7d39\u4ecb\u3057\u305f\u57fa\u672c\u7684\u306a\u30c7\u30fc\u30bf\u578b\u3068\u306e\u30de\u30c3\u30d4\u30f3\u30b0\u306f\u6b21\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n \u3053\u3053\u3067\u3001TimestampType.TIMESTAMP \u306f\u65e5\u6642\u3092 1970-01-01T00:00:00 UTC \u304b\u3089\u306e\u7d4c\u904e\u30df\u30ea\u79d2\u3068\u3057\u3066\u6271\u3044\u3001DateType.DATE \u306f\u65e5\u4ed8\u3092 1970-01-01 \u304b\u3089\u306e\u7d4c\u904e\u65e5\u6570\u3068\u3057\u3066\u6271\u3044\u307e\u3059\u3002<\/p>\n VarcharType.VARCHAR \u306f Slice \u3092\u4f7f\u3044\u307e\u3059\u3002\u4eca\u56de\u306f Slices.utf8Slice(String) \u30e1\u30bd\u30c3\u30c9\u3067 String \u304b\u3089 Slice \u3078\u306e\u5909\u63db\u3092\u884c\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n RecordCursor \u306b\u306f Presto \u306e\u30d0\u30fc\u30b8\u30e7\u30f3 0.113 \u304b\u3089 getObject(int) \u3068\u3044\u3046 Object \u3092\u8fd4\u3059\u30e1\u30bd\u30c3\u30c9\u304c\u8ffd\u52a0\u3055\u308c\u3066\u3044\u307e\u3059\u3002ARRAY \u578b\u3084 MAP \u578b\u7b49\u306f\u3053\u306e\u30e1\u30bd\u30c3\u30c9\u3067\u6271\u3046\u4ed5\u69d8\u306b\u5909\u66f4\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n isNull() \u306f\u30ab\u30ec\u30f3\u30c8\u884c\u306e\u6307\u5b9a\u30d5\u30a3\u30fc\u30eb\u30c9\u30c7\u30fc\u30bf\u304c null \u3067\u3042\u308b\u3053\u3068\u3092\u691c\u8a3c\u3059\u308b\u30e1\u30bd\u30c3\u30c9\u3067\u3059\u3002<\/p>\n \u30c7\u30d7\u30ed\u30a4\u65b9\u6cd5\u306f\u3001\u30d3\u30eb\u30c9\u3057\u305f JAR \u30d5\u30a1\u30a4\u30eb\u3001\u304a\u3088\u3073\u4f9d\u5b58\u3059\u308b JAR \u30d5\u30a1\u30a4\u30eb\u3092\u6240\u5b9a\u306e\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u306b\u653e\u308a\u8fbc\u3080\u3060\u3051\u3067\u3059\u3002\u4eca\u56de\u306e\u30b5\u30f3\u30d7\u30eb\u3067\u306f Presto \u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u76f4\u4e0b\u306e plugin \u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u306b techscore \u3068\u3044\u3046\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u3092\u4f5c\u6210\u3057\u3001\u305d\u3053\u306b\u30d3\u30eb\u30c9\u7d50\u679c\u3068\u306a\u308b techscore-presto-plugin-1.0.jar \u3092\u5165\u308c\u308b\u306e\u307f\u3067\u3059\u3002\u30c7\u30d7\u30ed\u30a4\u5f8c\u306f Presto \u3092\u518d\u8d77\u52d5\u3059\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n \u8d77\u52d5\u5b8c\u4e86\u5f8c\u3001Presto \u306e\u30b3\u30de\u30f3\u30c9\u30e9\u30a4\u30f3\u30c4\u30fc\u30eb\u3092\u4f7f\u3044\u3001\u52d5\u4f5c\u78ba\u8a8d\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\n \u6765\u5e74\u306f\u5fc3\u3092\u5165\u308c\u66ff\u3048\u3001\u4ed5\u4e8b\u306f\u307b\u3069\u307b\u3069\u306b\u3057\u3066\u30d6\u30ed\u30b0\u306b\u529b\u3092\u5165\u308c\u307e\u3059\uff01\u7de8\u96c6\u9577\uff01<\/p>\n","protected":false},"excerpt":{"rendered":" \u3053\u3093\u306b\u3061\u306f\u3002\u677e\u672c\u3067\u3059\u3002<\/p>\n \u3053\u308c\u306f TECHSCORE Advent Calendar 2015 \u306e21\u65e5\u76ee\u306e\u8a18\u4e8b\u3067\u3059\u3002<\/p>\n
\nPresto \u30b3\u30cd\u30af\u30bf\u30fc\u3092\u5b9f\u88c5\u3059\u308b \u7b2c\u4e8c\u56de<\/a><\/p>\n\u51e6\u7406\u306e\u5206\u5272<\/h2>\n
\n
com.techscore.example.presto.plugin.TechscorePartition<\/h3>\n
\r\npackage com.techscore.example.presto.plugin;\r\n\r\nimport lombok.NonNull;\r\nimport lombok.Value;\r\n\r\nimport com.facebook.presto.spi.ConnectorColumnHandle;\r\nimport com.facebook.presto.spi.ConnectorPartition;\r\nimport com.facebook.presto.spi.SchemaTableName;\r\nimport com.facebook.presto.spi.TupleDomain;\r\n\r\n@Value\r\npublic class TechscorePartition implements ConnectorPartition {\r\n\r\n private final SchemaTableName schemaTableName;\r\n private final String partitionId;\r\n\r\n public TechscorePartition(@NonNull SchemaTableName schemaTableName) {\r\n this.schemaTableName = schemaTableName;\r\n this.partitionId = new StringBuilder()\r\n .append(schemaTableName.getSchemaName()).append(':')\r\n .append(schemaTableName.getTableName()).toString();\r\n }\r\n\r\n @Override\r\n public TupleDomain
com.techscore.example.presto.plugin.TechscoreSplit<\/h3>\n
\r\npackage com.techscore.example.presto.plugin;\r\n\r\nimport java.util.List;\r\n\r\nimport lombok.EqualsAndHashCode;\r\nimport lombok.NonNull;\r\nimport lombok.ToString;\r\n\r\nimport com.facebook.presto.spi.ConnectorSplit;\r\nimport com.facebook.presto.spi.HostAddress;\r\nimport com.facebook.presto.spi.SchemaTableName;\r\nimport com.fasterxml.jackson.annotation.JsonCreator;\r\nimport com.fasterxml.jackson.annotation.JsonIgnore;\r\nimport com.fasterxml.jackson.annotation.JsonProperty;\r\nimport com.google.common.collect.ImmutableList;\r\n\r\n@EqualsAndHashCode\r\n@ToString\r\npublic class TechscoreSplit implements ConnectorSplit {\r\n\r\n private final String connectorId;\r\n\r\n private final SchemaTableName schemaTableName;\r\n\r\n @JsonCreator\r\n public TechscoreSplit(\r\n @JsonProperty(\"connectorId\") @NonNull String connectorId,\r\n @JsonProperty(\"schemaTableName\") @NonNull SchemaTableName schemaTableName) {\r\n this.connectorId = connectorId;\r\n this.schemaTableName = schemaTableName;\r\n }\r\n\r\n @JsonProperty\r\n public String getConnectorId() {\r\n return connectorId;\r\n }\r\n\r\n @JsonProperty\r\n public SchemaTableName getSchemaTableName() {\r\n return schemaTableName;\r\n }\r\n\r\n @JsonIgnore\r\n @Override\r\n public List
com.techscore.example.presto.plugin.TechscoreSplitManager<\/h3>\n
\r\npackage com.techscore.example.presto.plugin;\r\n\r\nimport static com.google.common.base.Preconditions.checkArgument;\r\nimport static com.google.common.base.Preconditions.checkNotNull;\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.Collections;\r\nimport java.util.List;\r\n\r\nimport lombok.Getter;\r\nimport lombok.NonNull;\r\nimport lombok.val;\r\n\r\nimport com.facebook.presto.spi.ConnectorColumnHandle;\r\nimport com.facebook.presto.spi.ConnectorPartition;\r\nimport com.facebook.presto.spi.ConnectorPartitionResult;\r\nimport com.facebook.presto.spi.ConnectorSplit;\r\nimport com.facebook.presto.spi.ConnectorSplitManager;\r\nimport com.facebook.presto.spi.ConnectorSplitSource;\r\nimport com.facebook.presto.spi.ConnectorTableHandle;\r\nimport com.facebook.presto.spi.FixedSplitSource;\r\nimport com.facebook.presto.spi.TupleDomain;\r\nimport com.google.common.collect.ImmutableList;\r\nimport com.google.inject.Inject;\r\nimport com.google.inject.name.Named;\r\n\r\npublic class TechscoreSplitManager implements ConnectorSplitManager {\r\n\r\n @Getter\r\n private final String connectorId;\r\n\r\n @Getter\r\n private final TechscoreHandleResolver tHandleResolver;\r\n\r\n @Getter\r\n private final TechscoreConnectorConfig tConnectorConfig;\r\n\r\n @Inject\r\n public TechscoreSplitManager(\r\n @Named(\"connectorId\") @NonNull String connectorId,\r\n @NonNull TechscoreHandleResolver tHandleResolver,\r\n @NonNull TechscoreConnectorConfig tConnectorConfig) {\r\n this.connectorId = connectorId;\r\n this.tHandleResolver = tHandleResolver;\r\n this.tConnectorConfig = tConnectorConfig;\r\n }\r\n\r\n @Override\r\n public ConnectorPartitionResult getPartitions(\r\n ConnectorTableHandle tableHandle,\r\n TupleDomain
select * from techscore.schema1.authors;<\/code><\/p>\n
TupleDomain:ALL<\/pre>\n
select * from techscore.schema1.authors where id = 1;<\/code><\/p>\n
TupleDomain:{TechscoreColumnHandle(connectorId=techscore, caseSensitiveName=id, name=id, type=bigint, ordinalPosition=0)=[[1]]}<\/pre>\n
select * from techscore.schema1.authors where id in (1, 2);<\/code><\/p>\n
TupleDomain:{TechscoreColumnHandle(connectorId=techscore, caseSensitiveName=id, name=id, type=bigint, ordinalPosition=0)=[[1], [2]]}<\/pre>\n
select * from techscore.schema1.authors where id >= 1;<\/code><\/p>\n
TupleDomain:{TechscoreColumnHandle(connectorId=techscore, caseSensitiveName=id, name=id, type=bigint, ordinalPosition=0)=[[1,
\u30c7\u30fc\u30bf\u30a2\u30af\u30bb\u30b9<\/h2>\n
\n
com.techscore.example.presto.plugin.TechscoreRecordCursor<\/h3>\n
\r\npackage com.techscore.example.presto.plugin;\r\n\r\nimport static com.facebook.presto.spi.type.BigintType.BIGINT;\r\nimport static com.facebook.presto.spi.type.BooleanType.BOOLEAN;\r\nimport static com.facebook.presto.spi.type.TimestampType.TIMESTAMP;\r\nimport static com.facebook.presto.spi.type.VarcharType.VARCHAR;\r\nimport io.airlift.slice.Slice;\r\nimport io.airlift.slice.Slices;\r\n\r\nimport java.time.LocalDateTime;\r\nimport java.time.ZoneId;\r\nimport java.util.List;\r\n\r\nimport lombok.Getter;\r\nimport lombok.NonNull;\r\nimport lombok.val;\r\n\r\nimport com.facebook.presto.spi.RecordCursor;\r\nimport com.facebook.presto.spi.type.Type;\r\n\r\npublic class TechscoreRecordCursor implements RecordCursor {\r\n\r\n @Getter\r\n private final List
> data;\r\n\r\n protected int index;\r\n\r\n public TechscoreRecordCursor(\r\n @NonNull List
> data) {\r\n this.tColumnHandles = tColumnHandles;\r\n this.data = data;\r\n this.index = -1;\r\n }\r\n\r\n @Override\r\n public long getTotalBytes() {\r\n return 0L;\r\n }\r\n\r\n @Override\r\n public long getCompletedBytes() {\r\n return 0L;\r\n }\r\n\r\n @Override\r\n public long getReadTimeNanos() {\r\n return 0L;\r\n }\r\n\r\n @Override\r\n public Type getType(int field) {\r\n return getTColumnHandles().get(field).getType();\r\n }\r\n\r\n protected Object getValue(int field) {\r\n val ordinalPosition = getTColumnHandles().get(field)\r\n .getOrdinalPosition();\r\n return data.get(index).get(ordinalPosition);\r\n }\r\n\r\n @Override\r\n public boolean advanceNextPosition() {\r\n int index = this.index + 1;\r\n if (index >= data.size()) {\r\n return false;\r\n }\r\n this.index = index;\r\n return true;\r\n }\r\n\r\n @Override\r\n public boolean getBoolean(int field) {\r\n val type = getType(field);\r\n if (BOOLEAN.equals(type)) {\r\n return ((Boolean) getValue(field)).booleanValue();\r\n }\r\n throw new IllegalArgumentException();\r\n }\r\n\r\n @Override\r\n public long getLong(int field) {\r\n val type = getType(field);\r\n if (BIGINT.equals(type)) {\r\n return ((Long) getValue(field)).longValue();\r\n }\r\n if (TIMESTAMP.equals(type)) {\r\n val dateTime = (LocalDateTime) getValue(field);\r\n return dateTime.atZone(ZoneId.of(\"Asia\/Tokyo\")).toEpochSecond();\r\n }\r\n throw new IllegalArgumentException();\r\n }\r\n\r\n @Override\r\n public double getDouble(int field) {\r\n throw new UnsupportedOperationException();\r\n }\r\n\r\n @Override\r\n public Slice getSlice(int field) {\r\n val type = getType(field);\r\n if (VARCHAR.equals(type)) {\r\n return Slices.utf8Slice((String) getValue(field));\r\n }\r\n throw new IllegalArgumentException();\r\n }\r\n\r\n @Override\r\n public boolean isNull(int field) {\r\n val value = getValue(field);\r\n if (value == null) {\r\n return true;\r\n }\r\n if (value instanceof String && value.equals(\"\")) {\r\n return true;\r\n }\r\n return false;\r\n }\r\n\r\n @Override\r\n public void close() {\r\n ;\r\n }\r\n\r\n}\r\n<\/pre>\n
\n
com.techscore.example.presto.plugin.TechscoreRecordSet<\/h3>\n
\r\npackage com.techscore.example.presto.plugin;\r\n\r\nimport java.util.List;\r\nimport java.util.stream.Collectors;\r\n\r\nimport lombok.Getter;\r\nimport lombok.NonNull;\r\n\r\nimport com.facebook.presto.spi.RecordCursor;\r\nimport com.facebook.presto.spi.RecordSet;\r\nimport com.facebook.presto.spi.type.Type;\r\nimport com.google.common.collect.ImmutableList;\r\n\r\npublic class TechscoreRecordSet implements RecordSet {\r\n\r\n @Getter\r\n private final TechscoreClient tClient;\r\n\r\n @Getter\r\n private final TechscoreSplit tSplit;\r\n\r\n @Getter\r\n private final List
com.techscore.example.presto.plugin.TechscoreRecordSetProvider<\/h3>\n
\r\npackage com.techscore.example.presto.plugin;\r\n\r\nimport static com.google.common.base.Preconditions.checkArgument;\r\n\r\nimport java.util.List;\r\nimport java.util.stream.Collectors;\r\n\r\nimport lombok.Getter;\r\nimport lombok.NonNull;\r\nimport lombok.val;\r\n\r\nimport com.facebook.presto.spi.ConnectorColumnHandle;\r\nimport com.facebook.presto.spi.ConnectorRecordSetProvider;\r\nimport com.facebook.presto.spi.ConnectorSplit;\r\nimport com.facebook.presto.spi.RecordSet;\r\nimport com.google.inject.Inject;\r\nimport com.google.inject.name.Named;\r\n\r\npublic class TechscoreRecordSetProvider implements ConnectorRecordSetProvider {\r\n\r\n @Getter\r\n private final String connectorId;\r\n\r\n @Getter\r\n private final TechscoreClient tClient;\r\n\r\n @Getter\r\n private final TechscoreHandleResolver tHandleResolver;\r\n\r\n @Getter\r\n private final TechscoreConnectorConfig tConnectorConfig;\r\n\r\n @Inject\r\n public TechscoreRecordSetProvider(\r\n @Named(\"connectorId\") @NonNull String connectorId,\r\n @NonNull TechscoreClient tClient,\r\n @NonNull TechscoreHandleResolver tHandleResolver,\r\n @NonNull TechscoreConnectorConfig tConnectorConfig) {\r\n this.connectorId = connectorId;\r\n this.tClient = tClient;\r\n this.tHandleResolver = tHandleResolver;\r\n this.tConnectorConfig = tConnectorConfig;\r\n }\r\n\r\n @Override\r\n public RecordSet getRecordSet(ConnectorSplit split,\r\n List extends ConnectorColumnHandle> columns) {\r\n val tSplit = getTHandleResolver().convertSplit(split);\r\n checkArgument(tSplit.getConnectorId().equals(getConnectorId()));\r\n val tColumnHandles = columns.stream()\r\n .map(getTHandleResolver()::convertColumnHandle)\r\n .collect(Collectors.toList());\r\n return new TechscoreRecordSet(getTClient(), tSplit, tColumnHandles);\r\n }\r\n}\r\n<\/pre>\n
\u52d5\u4f5c\u78ba\u8a8d<\/h2>\n
\r\n$ ls plugin\/techscore\r\ntechscore-presto-plugin-1.0.jar\r\n$ .\/bin\/launcher restart\r\n<\/pre>\n
\r\npresto:default> show catalogs;\r\n Catalog\r\n-----------\r\n jmx\r\n techscore\r\n(2 rows)\r\n\r\nQuery 20151214_103249_00014_4wdq9, FINISHED, 1 node\r\nSplits: 2 total, 2 done (100.00%)\r\n0:00 [2 rows, 24B] [9 rows\/s, 109B\/s]\r\n\r\npresto:default> show schemas from techscore;\r\n Schema\r\n--------------------\r\n information_schema\r\n schema1\r\n sys\r\n(3 rows)\r\n\r\nQuery 20151214_103303_00015_4wdq9, FINISHED, 1 node\r\nSplits: 2 total, 2 done (100.00%)\r\n0:00 [3 rows, 79B] [29 rows\/s, 782B\/s]\r\n\r\npresto:default> show tables from techscore.schema1;\r\n Table\r\n---------\r\n authors\r\n entries\r\n(2 rows)\r\n\r\nQuery 20151214_103312_00016_4wdq9, FINISHED, 1 node\r\nSplits: 2 total, 2 done (100.00%)\r\n0:00 [2 rows, 120B] [14 rows\/s, 868B\/s]\r\n\r\npresto:default> desc techscore.schema1.authors;\r\n Column | Type | Null | Partition Key | Comment\r\n--------+---------+------+---------------+---------\r\n id | bigint | true | false |\r\n name | varchar | true | false |\r\n(2 rows)\r\n\r\nQuery 20151214_103325_00017_4wdq9, FINISHED, 1 node\r\nSplits: 2 total, 2 done (100.00%)\r\n0:00 [2 rows, 1.4KB] [10 rows\/s, 7.54KB\/s]\r\n\r\n\r\npresto:default> select * from techscore.schema1.authors;\r\n id | name\r\n----+------------\r\n 1 | suzuki-kei\r\n 2 | ter@\r\n(2 rows)\r\n\r\nQuery 20151214_103401_00018_4wdq9, FINISHED, 1 node\r\nSplits: 2 total, 2 done (100.00%)\r\n0:00 [2 rows, 0B] [11 rows\/s, 0B\/s]\r\n<\/pre>\n
\u6700\u5f8c\u306b<\/h2>\n