{"id":4710,"date":"2013-04-11T09:01:00","date_gmt":"2013-04-11T00:01:00","guid":{"rendered":"http:\/\/www.moonmile.net\/blog\/?p=4710"},"modified":"2013-04-11T00:06:00","modified_gmt":"2013-04-10T15:06:00","slug":"mysql%e3%81%a7mysqlbulkloader%e3%81%ae%e3%82%b9%e3%83%94%e3%83%bc%e3%83%89%e3%82%92hddssd%e3%81%a7%e6%af%94%e8%bc%83%e3%81%99%e3%82%8b","status":"publish","type":"post","link":"http:\/\/www.moonmile.net\/blog\/archives\/4710","title":{"rendered":"MySQL\u3067MySqlBulkLoader\u306e\u30b9\u30d4\u30fc\u30c9\u3092HDD\/SSD\u3067\u6bd4\u8f03\u3059\u308b"},"content":{"rendered":"

SSD\u306b\u3059\u308b\u3068Blukcopy\u306f5\u500d\u3050\u3089\u3044\u65e9\u304f\u306a\u308b – Moonmile Solutions Blog<\/a> \u3068 MySQL\u3067HDD\/SSD\u306e\u30a2\u30af\u30bb\u30b9\u30b9\u30d4\u30fc\u30c9\u3092\u6bd4\u8f03\u3059\u308b – Moonmile Solutions Blog<\/a> \u306e\u7d9a\u304d\u3067\u3059\u3002@h141gm \u3055\u3093\u304b\u3089\u3001MySqlBulkLoader \u3082\u3042\u308b\u3088\u3001\u3068\u3044\u3046\u3053\u3068\u306a\u306e\u3067\u518d\u5b9f\u9a13\u3002\u3055\u3059\u304c\u306b\u3001insert \u306e\u9023\u7d9a\u3067\u306f SQL Server \u3068\u306e\u6bd4\u8f03\u304c\u7121\u7406\u304c\u3042\u308b\u3057\u3001insert \u306e\u9023\u7d9a\u304c SSD \u306e\u51e6\u7406\u306b\u8ffd\u3044\u4ed8\u3044\u3066\u3044\u308b\u304b\u3069\u3046\u304b\u3082\u3042\u3084\u3057\u3044\u306e\u3067\u3001\u3061\u3087\u3063\u3068\u4ed5\u5207\u308a\u76f4\u3057\u307e\u3059\u3002<\/p>\n

# my.ini \u306e\u5024\u306f\u3001xampp \u306e\u540a\u308b\u3057\u306e\u307e\u307e\u4f7f\u3063\u3066\u3044\u307e\u3059\u3002\u3072\u3068\u307e\u305a\u65e2\u88fd\u54c1\u72b6\u614b\u3067\u3069\u306e\u304f\u3089\u3044\u306e\u5dee\u304c\u3067\u308b\u306e\u304b\u306a\uff1f\u3063\u3066\u3053\u3068\u3067\u3002\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u3068\u3069\u306e\u4f4d\u306a\u306e\u304b\uff1f\u3063\u3066\u306e\u3082\u8208\u5473\u3042\u308b\u3068\u3053\u308d\u306a\u306e\u3067\u3059\u304c\u300c\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3057\u306a\u304f\u3066\u3082\u3001SSD \u3092\u63db\u88c5\u3059\u308c\u3070\u300c\u624b\u8efd\u306b\u300d\u65e9\u304f\u306a\u308b\u3001\u3063\u3066\u306e\u304c\u7406\u60f3\u7684\u306a\u306e\u3067\u3059\u3002<\/p>\n

\u4f8b\u306b\u3088\u3063\u3066\u3001100\u4e07\u4ef6\u306e\u30a2\u30af\u30bb\u30b9\u751f\u30ed\u30b0\u3092\u8aad\u307f\u8fbc\u305b\u307e\u3059\u3002<\/p>\n

\u25a0MySqlBulkLoader \u3067 insert \u3059\u308b<\/h3>\n

SSD \u306e BulkLoader \u3067\u300111.8 sec<\/p>\n

\"image\"<\/a><\/p>\n

\"image\"<\/a><\/p>\n

HDD \u3067 26.8 sec<\/p>\n

\"image\"<\/a><\/p>\n

\"image\"<\/a><\/p>\n

SSD \u306e\u307b\u3046\u304c HDD \u3088\u308a\u3082 2.5 \u500d\u3050\u3089\u3044\u65e9\u304f\u306a\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n\n\n
BulkCopy\/BulkLoader<\/td>\nSSD<\/td>\nHDD<\/td>\n<\/tr>\n
SQL Server<\/td>\n7.8 sec<\/td>\n32.1 sec<\/td>\n<\/tr>\n
MySQL<\/td>\n11.8 sec<\/td>\n26.8 sec<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

SQL Server \u306e BulkCopy \u306f\u3001\u30e1\u30e2\u30ea\u4e0a\u306e DataTable \u304b\u3089\u4e00\u6c17\u306b\u7a81\u3063\u8fbc\u3080\u306e\u3067\u3001\u30d5\u30a1\u30a4\u30eb\u3092\u89e3\u6790\u3057\u306a\u304c\u30ed\u30fc\u30c9\u3059\u308b MySQL \u306e BulkLoader \u3088\u308a\u3082\u65e9\u3044\u306e\u306f\u5f53\u7136\u2026\u306a\u306e\u3067\u3059\u304c\u3001\u3048\u3048\u3001HDD \u306e\u307b\u3046\u304c\u9006\u8ee2\u3057\u3066\u3044\u308b\u3051\u3069\u7121\u8996\u3067\uff08\u82e6\u7b11\uff09\u3002\u30ed\u30fc\u30c9\u3059\u308b\u30d5\u30a1\u30a4\u30eb\u81ea\u4f53\u306f SSD \u306b\u7f6e\u3044\u3066\u3042\u308b\u306e\u3067\u3001\u3053\u306e\u30d5\u30a1\u30a4\u30eb\u304c HDD \u4e0a\u306b\u3042\u308b\u3068 MySQL \u306e BulkLoader \u306e\u307b\u3046\u304c\u9045\u304f\u306a\u308b\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059\u3002\u3053\u306e\u8a08\u6e2c\u304b\u3089\u3044\u3051\u3070\u3001\u5358\u7d14\u306b HDD \u304b\u3089 SSD \u306b\u5207\u308a\u66ff\u3048\u305f\u3060\u3051\u3067\u3082\u30012.5 \u500d\u304b\u3089 4 \u500d\u3050\u3089\u3044\u306e\u52b9\u679c\u304c\u3042\u308a\u307e\u3059\u306d\u3001\u3063\u3066\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3059\u3002<\/p>\n

\u25a0500\u4e07\u4ef6\u306e\u30c7\u30fc\u30bf\u304b\u3089 SELECT \u3059\u308b<\/p>\n

SQL Server \u306e\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u30d5\u30a1\u30a4\u30eb\u3092 SSD \u306b\u79fb\u3059 – Moonmile Solutions Blog<\/a> \u3068\u4f3c\u305f\u3088\u3046\u306a\u30af\u30a8\u30ea\u3092\u4f7f\u3063\u3066\u8a08\u6e2c\u3057\u307e\u3059\u3002<\/p>\n

\u521d\u56de\u306e SELECT \u304c\u3001SSD \u3067 5.1 sec\u3001HDD \u304c 7.9 sec<\/p>\n

\"image\"<\/a>\"image\"<\/a><\/p>\n

\uff12\u56de\u76ee\u4ee5\u964d\u306f\u30ad\u30e3\u30c3\u30b7\u30e5\u304c\u52b9\u304f\u306e\u304b\u3068\u601d\u3063\u3066\u3044\u305f\u3089\u3001SSD \u3067 5.0 sec\u3001HDD \u3067 7.9 sec \u3068\u5909\u308f\u308a\u307e\u305b\u3093\u3002\u3053\u306e\u3042\u305f\u308a\u306f\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3067\u5909\u308f\u308b\u306e\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u3002<\/p>\n

\"image\"<\/a>\"image\"<\/a><\/p>\n\n\n\n\n\n
<\/td>\nSSD\u521d\u56de<\/td>\nSSD\u4e8c\u56de\u76ee<\/td>\nHDD\u521d\u56de<\/td>\nHDD\uff12\u56de\u76ee<\/td>\n<\/tr>\n
SQL Server<\/td>\n4.6 sec<\/td>\n0.4 sec<\/td>\n15.1 sec<\/td>\n0.4 sec<\/td>\n<\/tr>\n
MySQL<\/td>\n5.1 sec<\/td>\n5.1 sec<\/td>\n7.9 sec<\/td>\n7.9 sec<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

\u4f7f\u7528\u30e1\u30e2\u30ea\u3092\u898b\u308b\u3068\u3001SQL Server \u306f\uff12\u56de\u76ee\u306e\u3068\u304d\u306b 4GB \u7a0b\u5ea6\u53d6\u3063\u3066\u3044\u307e\u3059\u304c\u3001MySQL \u306e\u5834\u5408\u306f 36 MB \u7a0b\u5ea6\u3068\u8b19\u865a\u3067\u3059\u3002\u304c\u3063\u3064\u308a\u30e1\u30e2\u30ea\u3092\u5272\u308a\u5f53\u3066\u3066\u3084\u308c\u3070\u3001MySQL \u306e\u5834\u5408\u3082\u8d85\u65e9\u306b\u306a\u308b\u306e\u304b\u306a\u3068\u3002\u305f\u3060\u3057\u3001SQL Server + HDD \u306e\u7d44\u307f\u5408\u308f\u305b\u304c\u9045\u3059\u304e\u3067\u3059\u3088\u306d\u2026MySQL \u304c\u65e9\u3044\u306e\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u304c\u3002<\/p>\n

\u3053\u306e\u7d50\u679c\u3092\u898b\u308b\u3068\u3001SELECT \u306e\u5834\u5408\u306f\u3001HDD \u304b\u3089 SSD \u306b\u63db\u88c5\u3057\u3066\u3082\u3042\u307e\u308a\u30e1\u30ea\u30c3\u30c8\u304c\u306a\u3044\u304b\u306a\u3001\u3063\u3066\u611f\u3058\u3082\u3057\u307e\u3059\u3002\u3053\u306e\u5834\u5408\u306f\u3001\u5358\u7d14\u306a\u30a2\u30af\u30bb\u30b9\u30ed\u30b0\u3067\u30ef\u30f3\u30c6\u30fc\u30d6\u30eb\u306e\u691c\u7d22\u3067\u3057\u304b\u306a\u3044\u306e\u3067\u3001\u8907\u96d1\u306a join \u3092\u3057\u305f\u3068\u304d\u306b\u306f\u3069\u3046\u304b\u5206\u304b\u308a\u307e\u305b\u3093\u304c\u3001\u5927\u91cf\u30c7\u30fc\u30bf\uff06\u5358\u7d14\u306a\u5168\u691c\u7d22\u306e\u30d1\u30bf\u30fc\u30f3\u3060\u3068\u3001SELECT \u6642\u306e\u6069\u6075\u306f\u3042\u307e\u308a\u306a\u3044\u304b\u3082\u3057\u308c\u307e\u305b\u3093\u3002\u3048\u3048\u3068\u3001\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3059\u308c\u3070\u5225\u306e\u3088\u3046\u306a\u6c17\u3082\u3057\u307e\u3059\u304c\u3002<\/p>\n

\u25a0\u5b9f\u9a13\u30b3\u30fc\u30c9\u306e\u629c\u7c8b<\/p>\n

MySqlBulkLoader \u3092\u5229\u7528<\/p>\n

\r\nvoid GoBulkLoad(string CNSTR)\r\n{\r\n    MySqlConnection cn = new MySqlConnection(CNSTR);\r\n    MySqlBulkLoader bl = new MySqlBulkLoader(cn);\r\n    bl.TableName = "logs";\r\n    bl.FieldTerminator = "t";  \/\/ \u30bf\u30d6\u533a\u5207\u308a\r\n    bl.LineTerminator = "n";   \/\/ \u6539\u884c\r\n    bl.FileName = @"D:sitelogsout.csv";\r\n\r\n    DateTime start = DateTime.Now;\r\n\r\n    cn.Open();\r\n    bl.Load();\r\n    cn.Clone();\r\n\r\n    DateTime end = DateTime.Now;\r\n    label2.Text = string.Format("BulkCopy\u5b8c\u4e86 {0:#.0} sec",\r\n        ((TimeSpan)(end - start)).TotalSeconds);\r\n}\r\n<\/pre>\n
MySqlCommand \u3067\u691c\u7d22<\/p>\n
\r\nvoid GoSelect(string CNSTR)\r\n{\r\n    var ip = textBox1.Text;\r\n    var cn = new MySqlConnection(CNSTR);\r\n    var cmd = new MySqlCommand(\r\n        "select count(*) from logs where ip = @ip", cn);\r\n    cmd.Parameters.Add(new MySqlParameter("@ip", ip));\r\n    DateTime start = DateTime.Now;\r\n    cn.Open();\r\n    object cnt = cmd.ExecuteScalar();\r\n    cn.Clone();\r\n\r\n    DateTime end = DateTime.Now;\r\n    label4.Text = string.Format("\u691c\u7d22\u7d50\u679c {0}\u4ef6 {1:#.0} sec", cnt,\r\n        ((TimeSpan)(end - start)).TotalSeconds);\r\n}\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
SSD\u306b\u3059\u308b\u3068Blukcopy\u306f5\u500d\u3050\u3089\u3044\u65e9\u304f\u306a\u308b – Moonmile Solutions Blog \u3068 MySQL\u3067HDD\/SSD\u306e\u30a2\u30af\u30bb\u30b9\u30b9\u30d4\u30fc\u30c9\u3092\u6bd4\u8f03\u3059\u308b – Moonmile Solut … \u7d9a\u304d\u3092\u8aad\u3080 →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[38,54],"tags":[],"class_list":["post-4710","post","type-post","status-publish","format-standard","hentry","category-mysql","category-performance"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/posts\/4710","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/comments?post=4710"}],"version-history":[{"count":2,"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/posts\/4710\/revisions"}],"predecessor-version":[{"id":4711,"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/posts\/4710\/revisions\/4711"}],"wp:attachment":[{"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/media?parent=4710"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/categories?post=4710"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.moonmile.net\/blog\/wp-json\/wp\/v2\/tags?post=4710"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}