-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
1499 lines (1002 loc) · 154 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE HTML>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<title>Promise John</title>
<meta name="viewport" content="width=device-width, initial-scale=1,user-scalable=no">
<meta name="author" content="promise john">
<meta name="description" content="A place for my writing about work, thoughts, and techs.">
<meta property="og:type" content="website">
<meta property="og:title" content="Promise John">
<meta property="og:url" content="http://promisejohn.github.io/index.html">
<meta property="og:site_name" content="Promise John">
<meta property="og:description" content="A place for my writing about work, thoughts, and techs.">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Promise John">
<meta name="twitter:description" content="A place for my writing about work, thoughts, and techs.">
<meta name="twitter:creator" content="@promisejohn19">
<link rel="alternative" href="/atom.xml" title="Promise John" type="application/atom+xml">
<link rel="icon" href="/img/favicon.ico">
<link rel="apple-touch-icon" href="/img/jacman.jpg">
<link rel="apple-touch-icon-precomposed" href="/img/jacman.jpg">
<link rel="stylesheet" href="/css/style.css" type="text/css">
</head>
<body>
<header>
<div>
<div id="textlogo">
<h1 class="site-name"><a href="/" title="Promise John">Promise John</a></h1>
<h2 class="blog-motto">My Blog</h2>
</div>
<div class="navbar"><a class="navbutton navmobile" href="#" title="菜单">
</a></div>
<nav class="animated">
<ul>
<ul>
<li><a href="/">Home</a></li>
<li><a href="/archives">Archives</a></li>
<li><a href="/tags">Tags</a></li>
<li><a href="/categories">categories</a></li>
<li><a href="/about">About</a></li>
<li>
<form class="search" action="//google.com/search" method="get" accept-charset="utf-8">
<label>Search</label>
<input type="search" id="search" name="q" autocomplete="off" maxlength="20" placeholder="搜索" />
<input type="hidden" name="q" value="site:promisejohn.github.io">
</form>
</li>
</ul>
</nav>
</div>
</header>
<div id="container">
<div id="main">
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/06/13/NodeJSdevEnvSetting/" title="NodeJS开发环境配置" itemprop="url">NodeJS开发环境配置</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-06-13T07:09:27.000Z" itemprop="datePublished"> 发表于 2015-06-13</time>
</p>
</header>
<div class="article-content">
<h1 id="NodeJS开发环境配置">NodeJS开发环境配置</h1><h2 id="安装nvm、node和npm">安装nvm、node和npm</h2><p>nvm是nodejs的多版本管理利器，node是nodejs的解释器，npm是nodejs的包管理工具。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># ubuntu安装之后会自动添加配置到~/.profile，可以直接cut到自己喜欢的比如~/.bashrc</span></span><br><span class="line">$ wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.<span class="number">25.4</span>/install.sh | bash</span><br><span class="line">$ nvm install <span class="number">0.12</span>.<span class="number">4</span></span><br><span class="line">$ nvm use <span class="number">0.12</span>.<span class="number">4</span></span><br><span class="line">$ nvm <span class="built_in">alias</span> default <span class="number">0.12</span>.<span class="number">4</span></span><br><span class="line">$ node --version</span><br></pre></td></tr></table></figure>
<p>由于网速问题，国内可以使用<a href="https://npm.taobao.org/" target="_blank" rel="external">taobao的npm镜像</a>：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ npm install -g cnpm --registry=https://registry.npm.taobao.org</span><br><span class="line">$ cnpm install [package name]</span><br></pre></td></tr></table></figure>
<h2 id="Build工具：grunt/gulp,_bower,_yeoman">Build工具：grunt/gulp, bower, yeoman</h2><p>自动编译、依赖管理、自动化测试、打包发布、项目模板工具、文档自动生成…… ，这些基本上属于每个项目（不论语言差异）构建的标配，javascript目前也配齐了，于是乎开发效率直线提升:)</p>
<ul>
<li>grunt是javascript的自动化构建工具，类似于Java的gradle/maven/ant，Python的tox，C/C++的make和scons等。</li>
<li>gulp也是构建工具，它采用类似jQuery的流式配置简化了任务的编写。</li>
<li>bower是一个依赖管理的工具，可以自动化安装bootstrap、angulajs、jquery等包，解决他们之间的依赖关系。</li>
<li>yeoman是一个生成项目框架scaffolding工具，遵循Convention over Configuration，用过RoR或者Django的应该都知道快速开发形式.d可以配合generator-webapp、generator-angular等模板快速生成项目结构。</li>
</ul>
<h3 id="安装上述工具到本地：">安装上述工具到本地：</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ cnpm install -g yo bower grunt-cli gulp</span><br><span class="line"><span class="comment"># 安装yo项目模板</span></span><br><span class="line">$ cnpm install -g generator-angular</span><br></pre></td></tr></table></figure>
<h3 id="生成一个HelloWorld">生成一个HelloWorld</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ mkdir -p ~/dev/nodejs/helloworld && <span class="built_in">cd</span> <span class="variable">$_</span></span><br><span class="line">$ yo angular</span><br><span class="line">$ grunt serve</span><br><span class="line">$ grunt <span class="built_in">test</span></span><br><span class="line">$ grunt</span><br></pre></td></tr></table></figure>
<p>这样就把一个前段项目的框架生成完毕了，可以进入开发阶段。</p>
<h2 id="编辑器：Atom">编辑器：Atom</h2><p>使用chrome和nodejs开发的Atom，几个月前看还是离sublimetext挺远，现在看几乎快要完全超越！——<strong>ubuntu下的中文算是个麻烦事，自定义以后还凑合</strong>。</p>
<ul>
<li>安装可以直接从<a href="https://atom.io/" target="_blank" rel="external">官网</a>开始。</li>
<li>开源字体从<a href="http://wenq.org/" target="_blank" rel="external">文泉驿</a>开始。</li>
</ul>
<p>参考配置：</p>
<figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line">@font-family: "DejaVu Sans Mono", "WenQuanYi Zen Hei";</span><br><span class="line"></span><br><span class="line">.tree-view, .title, .current-path, .tooltip {</span><br><span class="line"> font-family: @font-family;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">.terminal {</span><br><span class="line"> font-family: @font-family !important;</span><br><span class="line"> div {</span><br><span class="line"> white-space: nowrap;</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">// style the background and foreground colors on the atom-text-editor-element</span><br><span class="line">// itself</span><br><span class="line">atom-text-editor {</span><br><span class="line"> font-family: @font-family;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">// To style other content in the text editor's shadow DOM, use the ::shadow</span><br><span class="line">// expression</span><br><span class="line">atom-text-editor::shadow .cursor {</span><br><span class="line"> font-family: @font-family;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">.markdown-preview {</span><br><span class="line"> font-family: @font-family;</span><br><span class="line"> atom-text-editor {</span><br><span class="line"> font-family: @font-family;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>Atom也有大量的插件可以使用，比如把hexo集成进来，可以少开个term了：）</p>
<ul>
<li>VIM开启了编辑器的多模式状态，让敲键盘更尽兴；</li>
<li>SublimeText方便了编辑器的DIY，性能也很好；</li>
<li>Atom让Web开发更彻底，它本身就是个基于浏览器内核的工具；此外，它来自于Github:)</li>
</ul>
<h2 id="参考">参考</h2><ol>
<li><a href="https://atom.io/" title="Atom Offical site" target="_blank" rel="external">Atom Offical</a></li>
<li><a href="http://wenq.org/" title="文泉驿官网" target="_blank" rel="external">文泉驿官网</a></li>
<li><a href="https://npm.taobao.org/" title="Taobao NPM" target="_blank" rel="external">Taobao NPM</a></li>
<li><a href="http://gruntjs.com/" title="Grunt Offical" target="_blank" rel="external">Grunt Offical</a></li>
<li><a href="http://bower.io/" title="Bower Offical" target="_blank" rel="external">Bower Offical</a></li>
<li><a href="http://yeoman.io/" title="Yeoman Offical" target="_blank" rel="external">Yeoman Offical</a></li>
</ol>
<p class="article-more-link">
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/dev/">dev</a><a href="/tags/nodejs/">nodejs</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/06/13/NodeJSdevEnvSetting/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/06/13/NodeJSdevEnvSetting/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/06/13/HelloHexo/" title="Hello Hexo" itemprop="url">Hello Hexo</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-06-13T07:06:01.239Z" itemprop="datePublished"> 发表于 2015-06-13</time>
</p>
</header>
<div class="article-content">
<p>Welcome to <a href="http://hexo.io/" target="_blank" rel="external">Hexo</a>! This is your very first post. Check <a href="http://hexo.io/docs/" target="_blank" rel="external">documentation</a> for more info. If you get any problems when using Hexo, you can find the answer in <a href="http://hexo.io/docs/troubleshooting.html" target="_blank" rel="external">troubleshooting</a> or you can ask me on <a href="https://github.com/hexojs/hexo/issues" target="_blank" rel="external">GitHub</a>.</p>
<h2 id="Quick_Start">Quick Start</h2><h3 id="Create_a_new_post">Create a new post</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new <span class="string">"My New Post"</span></span><br></pre></td></tr></table></figure>
<p>More info: <a href="http://hexo.io/docs/writing.html" target="_blank" rel="external">Writing</a></p>
<h3 id="Run_server">Run server</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo server</span><br></pre></td></tr></table></figure>
<p>More info: <a href="http://hexo.io/docs/server.html" target="_blank" rel="external">Server</a></p>
<h3 id="Generate_static_files">Generate static files</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo generate <span class="comment"># 'hexo g' for short</span></span><br></pre></td></tr></table></figure>
<p>More info: <a href="http://hexo.io/docs/generating.html" target="_blank" rel="external">Generating</a></p>
<h3 id="Deploy_to_remote_sites">Deploy to remote sites</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo deploy <span class="comment"># 'hexo d' for short</span></span><br></pre></td></tr></table></figure>
<p>More info: <a href="http://hexo.io/docs/deployment.html" target="_blank" rel="external">Deployment</a></p>
<h3 id="Useful_hexo_plugins">Useful hexo plugins</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ npm install hexo-generator-feed --save</span><br></pre></td></tr></table></figure>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ npm install hexo-generator-sitemap --save</span><br></pre></td></tr></table></figure>
<h3 id="Put_the_whole_source_folder_into_git">Put the whole source folder into git</h3><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">... # Dependency directory node_modules # hexo generated files, deployed to github.io pages public .deploy_git ...</span><br></pre></td></tr></table></figure>
<h3 id="Some_notes_about_doc_syntax">Some notes about doc syntax</h3><ol>
<li>use <code><!-- more --></code> to hide blog details below.</li>
<li>use <code>toc: false</code> to hide the topic menu about the blog.</li>
<li>use <code>mathjax: true</code> to enable LaTex math prents.</li>
</ol>
<h3 id="Awsome_markdown_editors:">Awsome markdown editors:</h3><ul>
<li>vim, sublimetext, …</li>
<li><a href="http://25.io/mou/" target="_blank" rel="external"><strong>Mou</strong></a></li>
<li><a href="http://macdown.uranusjr.com/" target="_blank" rel="external"><strong>MacDown</strong></a></li>
</ul>
<p class="article-more-link">
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/hexo/">hexo</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/06/13/HelloHexo/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/06/13/HelloHexo/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/06/04/HelloHadoop/" title="HelloHadoop" itemprop="url">HelloHadoop</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-06-04T02:54:16.000Z" itemprop="datePublished"> 发表于 2015-06-04</time>
</p>
</header>
<div class="article-content">
<h1 id="Hadoop系列">Hadoop系列</h1><p>大数据依旧在热炒，hadoop虽然不是唯一的代表，却也是各家必谈之资本，玩玩大数据。</p>
<h2 id="通过ambari部署hadoop集群">通过ambari部署hadoop集群</h2><p>在cloudlab119-123上部署集群，其中119作为ambari server控制端。</p>
<p>安装部署：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /etc/yum.repo.d</span><br><span class="line">$ wget http://public-repo-<span class="number">1</span>.hortonworks.com/ambari/centos6/<span class="number">2</span>.x/updates/<span class="number">2.0</span>.<span class="number">0</span>/ambari.repo</span><br><span class="line">$ yum install -y ambari-server</span><br><span class="line">$ ambari-server setup <span class="comment"># 按指示操作即可</span></span><br><span class="line">$ ambari-server start <span class="comment"># *:8080端口</span></span><br></pre></td></tr></table></figure>
<p>配置，打开<a href="http://cloudlab119:8080，默认密码admin:admin。在cloudlab119上对所有节点做免密码认证：" target="_blank" rel="external">http://cloudlab119:8080，默认密码admin:admin。在cloudlab119上对所有节点做免密码认证：</a></p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ vi /etc/hosts</span><br><span class="line">$ ssh-keygen -t rsa -P <span class="string">''</span> <span class="operator">-f</span> ~/.ssh/hadoop119</span><br><span class="line">$ <span class="keyword">for</span> i <span class="keyword">in</span> {<span class="number">119</span>,<span class="number">120</span>,<span class="number">121</span>,<span class="number">122</span>,<span class="number">123</span>}; <span class="keyword">do</span> ssh-copy-id -i ~/.ssh/hadoop119.pub root@cloudlab<span class="variable">$i</span>; <span class="keyword">done</span>;</span><br></pre></td></tr></table></figure>
<p>按提示建议操作，如ntpd、iptables、关闭THP：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ chkconfig ntpd on</span><br><span class="line">$ service ntpd start</span><br><span class="line">$ service iptables stop <span class="comment"># 生产环境中参照文档把对应端口打开</span></span><br><span class="line">$ chkconfig iptables off</span><br><span class="line">$ <span class="built_in">echo</span> never > /sys/kernel/mm/redhat_transparent_hugepage/enabled <span class="comment"># 关闭THP，如果提示文件系统readonly，重启机器再执行</span></span><br></pre></td></tr></table></figure>
<p>按提示一步步操作即可，但中间如果出现下载包失败，则会导致整个安装失败，所以最好提前把官方源同步到本地，做镜像后安装。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">$ yum install -y yum-utils createrepo</span><br><span class="line">$ mkdir -p /var/www/html/ambari/centos6 && <span class="built_in">cd</span> <span class="variable">$_</span></span><br><span class="line">$ reposync -r Updates-ambari-<span class="number">2.0</span>.<span class="number">0</span></span><br><span class="line">$ createrepo Updates-ambari-<span class="number">2.0</span>.<span class="number">0</span></span><br><span class="line">$ mkdir -p /var/www/html/hdp/centos6 && <span class="built_in">cd</span> <span class="variable">$_</span></span><br><span class="line">$ reposync -r HDP-<span class="number">2.2</span></span><br><span class="line">$ reposync -r HDP-UTILS-<span class="number">1.1</span>.<span class="number">0.20</span></span><br><span class="line">$ createrepo HDP-<span class="number">2.2</span></span><br><span class="line">$ createrepo HDP-UTILS-<span class="number">1.1</span>.<span class="number">0.20</span></span><br><span class="line">$ vim /etc/yum.repo.d/ambari.repo <span class="comment"># 修改baseurl指向本地镜像</span></span><br></pre></td></tr></table></figure>
<p>安装之后，发现某几台服务器内存占用率非常高，可以通过Ambari增加Host节点，然后迁移部分的组件到新的机器。</p>
<h2 id="通过Docker部署集群">通过Docker部署集群</h2><p>可以自己做docker image，简单起见可以先用sequenceiq的image。</p>
<p>单节点测试：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ docker pull sequenceiq/hadoop-docker:<span class="number">2.7</span>.<span class="number">0</span></span><br><span class="line">$ docker run -it sequenceiq/hadoop-docker:<span class="number">2.7</span>.<span class="number">0</span> /etc/bootstrap.sh -bash</span><br><span class="line">$ <span class="built_in">cd</span> <span class="variable">$HADOOP_PREFIX</span> <span class="comment"># /usr/local/hadoop</span></span><br><span class="line">$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-<span class="number">2.7</span>.<span class="number">0</span>.jar grep input output <span class="string">'dfs[a-z.]+'</span> <span class="comment"># 统计key出现次数</span></span><br><span class="line">$ bin/hdfs dfs -cat output/* <span class="comment"># 在用户Home下的output下</span></span><br></pre></td></tr></table></figure>
<p>多节点部署，可以用ambari镜像，准备好blueprint.json：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">{ "host_groups" : [ { "name" : "host_group_1", "components" : [ { "name" : "ZOOKEEPER_SERVER" }, { "name" : "ZOOKEEPER_CLIENT" }, { "name" : "AMBARI_SERVER" }, { "name" : "HDFS_CLIENT" }, { "name" : "NODEMANAGER" }, { "name" : "MAPREDUCE2_CLIENT" }, { "name" : "APP_TIMELINE_SERVER" }, { "name" : "DATANODE" }, { "name" : "YARN_CLIENT" }, { "name" : "RESOURCEMANAGER" } ], "cardinality" : "1" }, { "name" : "host_group_2", "components" : [ { "name" : "ZOOKEEPER_SERVER" }, { "name" : "ZOOKEEPER_CLIENT" }, { "name" : "SECONDARY_NAMENODE" }, { "name" : "NODEMANAGER" }, { "name" : "YARN_CLIENT" }, { "name" : "DATANODE" }], "cardinality" : "1" }, { "name" : "host_group_3", "components" : [ { "name" : "ZOOKEEPER_SERVER" }, { "name" : "ZOOKEEPER_CLIENT" }, { "name" : "NAMENODE" }, { "name" : "NODEMANAGER" }, { "name" : "YARN_CLIENT" }, { "name" : "DATANODE" }], "cardinality" : "1" } ], "Blueprints" : { "blueprint_name" : "blueprint-c1", "stack_name" : "HDP", "stack_version" : "2.2" } }</span><br></pre></td></tr></table></figure>
<p>可以在ambari server上查找：<code>http://172.17.0.13:8080/api/v1/blueprints</code>。</p>
<p>创建集群：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">$ docker pull sequenceiq/ambari:<span class="number">1.7</span>.<span class="number">0</span></span><br><span class="line">$ curl -Lo .amb https://github.com/sequenceiq/docker-ambari/raw/master/ambari-functions && . .amb</span><br><span class="line">$ amb-start-cluster <span class="number">3</span> <span class="comment"># 注意默认使用的是sequenceiq/ambari:1.7.0-warmup，可以修改 .amb</span></span><br><span class="line">$ amb-shell <span class="comment"># 又启了个container</span></span><br><span class="line">ambshell > host list</span><br><span class="line">ambshell > blueprint add --url http://<span class="number">172.17</span>.<span class="number">42.1</span>/bp.json <span class="comment"># 准备好json保存到某个能访问的http服务器</span></span><br><span class="line">ambshell > cluster build --blueprint blueprint-c1</span><br><span class="line">ambshell > cluster assign --hostGroup host_group_1 --host amb0.mycorp.kom</span><br><span class="line">ambshell > cluster assign --hostGroup host_group_2 --host amb1.mycorp.kom</span><br><span class="line">ambshell > cluster assign --hostGroup host_group_3 --host amb2.mycorp.kom</span><br><span class="line">ambshell > blueprint show --id blueprint-c1</span><br><span class="line">ambshell > cluster preview</span><br><span class="line">ambshell > cluster create</span><br></pre></td></tr></table></figure>
<p>中间安装过程如果出现失败，可以到ambari-server上看详细情况，当然也可以直接向之前方式一样，通过GUI安装部署；或者开着浏览器看amb-shell执行过程。<br>如果实在VM上远程使用，可以在docker所在机器上做NAT映射直接访问：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ iptables -t nat -A PREROUTING <span class="operator">-d</span> <span class="number">10.101</span>.<span class="number">29.26</span> -p tcp --dport <span class="number">8000</span> -j DNAT --to-destination <span class="number">172.17</span>.<span class="number">0.20</span>:<span class="number">8080</span> <span class="comment"># 可以用端口转发直接远程浏览器访问</span></span><br></pre></td></tr></table></figure>
<p>做个简单的测试：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ docker <span class="built_in">exec</span> -it <span class="number">9572938</span>bb253 /bin/bash <span class="comment"># 进入到容器内</span></span><br><span class="line">bash <span class="comment"># su hdfs # 切换到HDFS的超级用户</span></span><br><span class="line">bash <span class="comment"># hdfs dfsadmin -report</span></span><br><span class="line">bash <span class="comment"># hdfs dfs -ls /</span></span><br></pre></td></tr></table></figure>
<h2 id="HDFS">HDFS</h2><p>HDFS由NameNode、SNameNode和DataNode组成，HA的时候还有另一个NameNode（StandBy）。SNameNode专门从NameNode获取FSImage和Edits，为NameNode合并生成新的FSImage。</p>
<p>HDFS有个超级用户，就是启动NameNode的那个linux账号。</p>
<h3 id="基本Shell操作">基本Shell操作</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">$ ps -elf | grep NameNode <span class="comment">#看看哪个账号启动了NameNode，那个账号就是超级用户</span></span><br><span class="line">$ su <span class="operator">-s</span> /bin/sh -c <span class="string">'hadoop fs -ls /'</span> hdfs <span class="comment">#使用超级账号执行命令</span></span><br><span class="line">$ su hdfs <span class="comment">#切换到超级账号</span></span><br><span class="line">$ useradd -G hdfs promise</span><br><span class="line">$ hdfs dfs -mkdir /user/promise</span><br><span class="line">$ hdfs dfs -chown promise:hdfs /user/promise</span><br><span class="line">$ hdfs dfs -ls /user/</span><br><span class="line">$ hdfs dfs -mkdir -p /user/promise/dev/hello</span><br><span class="line">$ hdfs dfs -chmod -R <span class="number">777</span> /user/promise/dev</span><br><span class="line">$ hdfs dfs -ls /user/promise/dev/</span><br><span class="line">$ <span class="built_in">echo</span> “helloworld” > hello.txt</span><br><span class="line">$ hdfs dfs -put hello.txt /user/promise/helloworld.txt</span><br><span class="line">$ hdfs dfs -put - hdfs://<span class="number">192.168</span>.<span class="number">182.119</span>/user/promise/test.txt <span class="comment"># 从stdin输入，Ctrl-D结束</span></span><br><span class="line">$ hdfs dfs -cat hdfs://<span class="number">192.168</span>.<span class="number">182.119</span>/user/promise/test.txt</span><br><span class="line">$ hdfs dfs -cat /user/promise/helloworld.txt</span><br><span class="line">$ hdfs dfs -get /user/promise/helloworld.txt hello2.txt</span><br><span class="line">$ hdfs dfs -tail <span class="operator">-f</span> /user/promise/helloworld.txt <span class="comment"># 查看追加的文件写入</span></span><br><span class="line">$ hdfs dfs -appendToFile - /user/promise/helloworld.txt</span><br></pre></td></tr></table></figure>
<h3 id="webhdfs操作">webhdfs操作</h3><p>启用了webhdfs之后（<code>hdfs-site</code>中<code>dfs.webhdfs.enabled=true</code>），可以通过HTTP方式访问HDFS：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ curl -i <span class="string">"http://192.168.182.119:50070/webhdfs/v1/user/promise/?op=LISTSTATUS"</span></span><br><span class="line">$ curl -i -L <span class="string">"http://192.168.182.119:50070/webhdfs/v1/user/promise/helloworld.txt?op=OPEN"</span> <span class="comment"># 通过重定向到DataNode获取文件</span></span><br><span class="line">$ curl -i -X PUT <span class="string">"http://192.168.182.119:50070/webhdfs/v1/user/promise/hello?user.name=promise&op=MKDIRS"</span></span><br></pre></td></tr></table></figure>
<p>通过HTTP方式创建和追加文件都需要通过2阶段实现：先在NameNode上创建，获得DataNode URI后再向URI上传文件。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ curl -i -X PUT <span class="string">"http://192.168.182.119:50070/webhdfs/v1/user/promise/hello/hi.txt?op=CREATE"</span></span><br><span class="line">$ curl -i -X PUT -T hi.txt <span class="string">"http://192.168.182.119:50075/webhdfs/v1/user/promise/hello/hi.txt?user.name=promise&op=CREATE&namenoderpcaddress=192.168.182.119:8020&overwrite=false"</span> <span class="comment"># 需要声明账号</span></span><br></pre></td></tr></table></figure>
<h3 id="查看离线的FSImage和Edits：">查看离线的FSImage和Edits：</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ hdfs oiv -p XML -i /hadoop/hdfs/namenode/current/fsimage_0000000000000018887 -o fsimage.xml <span class="comment"># 生成XML格式</span></span><br><span class="line">$ hdfs oiv -i /hadoop/hdfs/namenode/current/fsimage_0000000000000018887 <span class="comment"># 在线查看形式</span></span><br><span class="line">$ hdfs dfs -ls webhdfs://<span class="number">127.0</span>.<span class="number">0.1</span>:<span class="number">5978</span>/</span><br><span class="line">$ hdfs oev -i /hadoop/hdfs/namenode/current/edits_0000000000000000001-<span class="number">0000000000000005150</span> -o edits.xml <span class="comment"># 查看edits</span></span><br></pre></td></tr></table></figure>
<h3 id="使用Java_API">使用Java API</h3><p>Java在大型软件系统开发中有利于更清晰的架构设计和团队分工，而且有大量的第三方优质框架可用；可惜在命令里写HelloWorld很啰嗦，方便起见，直接用maven生成基本文件结构。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">export</span> PATH=<span class="variable">$PATH</span>:/opt/apache-maven-<span class="number">3.3</span>.<span class="number">3</span>/bin/</span><br><span class="line">$ <span class="built_in">export</span> JAVA_HOME=/usr/jdk64/jdk1.<span class="number">7.0</span>_67/</span><br><span class="line">$ <span class="built_in">export</span> MAVEN_OPTS=<span class="string">"-Xms256m -Xmx512m"</span></span><br><span class="line">$ mkdir ~/dev && <span class="built_in">cd</span> <span class="variable">$_</span></span><br><span class="line">$ mvn archetype:generate -DgroupId=org.tecstack -DartifactId=hellohdfs -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=<span class="literal">false</span></span><br><span class="line">$ <span class="built_in">cd</span> hellohdfs && mvn package</span><br><span class="line">$ vim ./src/main/java/org/tecstack/App.java</span><br></pre></td></tr></table></figure>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> org.tecstack;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.conf.Configuration;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.FileStatus;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.FileSystem;</span><br><span class="line"><span class="keyword">import</span> org.apache.hadoop.fs.Path;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span><br><span class="line"> * Hello HDFS</span><br><span class="line"> *</span><br><span class="line"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">App</span></span><br><span class="line"></span>{</span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">( String[] args )</span></span><br><span class="line"> </span>{</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Configuration conf=<span class="keyword">new</span> Configuration();</span><br><span class="line"> FileSystem hdfs=FileSystem.get(conf);</span><br><span class="line"> Path dst =<span class="keyword">new</span> Path(<span class="string">"/user/promise/helloworld.txt"</span>);</span><br><span class="line"> FileStatus files[]=hdfs.listStatus(dst);</span><br><span class="line"> <span class="keyword">for</span>(FileStatus file:files) {</span><br><span class="line"> System.out.println(file.getPath());</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>在<code>pom.xml</code>增加hadoop依赖包:</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">project</span> <span class="attribute">xmlns</span>=<span class="value">"http://maven.apache.org/POM/4.0.0"</span> <span class="attribute">xmlns:xsi</span>=<span class="value">"http://www.w3.org/2001/XMLSchema-instance"</span></span><br><span class="line"> <span class="attribute">xsi:schemaLocation</span>=<span class="value">"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">modelVersion</span>></span>4.0.0<span class="tag"></<span class="title">modelVersion</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">groupId</span>></span>org.tecstack<span class="tag"></<span class="title">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">artifactId</span>></span>hellohdfs<span class="tag"></<span class="title">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">packaging</span>></span>jar<span class="tag"></<span class="title">packaging</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">version</span>></span>1.0-SNAPSHOT<span class="tag"></<span class="title">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hellohdfs<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">url</span>></span>http://maven.apache.org<span class="tag"></<span class="title">url</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">dependencies</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">groupId</span>></span>org.apache.hadoop<span class="tag"></<span class="title">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">artifactId</span>></span>hadoop-common<span class="tag"></<span class="title">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">version</span>></span>2.6.0<span class="tag"></<span class="title">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">groupId</span>></span>org.apache.hadoop<span class="tag"></<span class="title">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">artifactId</span>></span>hadoop-hdfs<span class="tag"></<span class="title">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">version</span>></span>2.6.0<span class="tag"></<span class="title">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">groupId</span>></span>org.apache.hadoop<span class="tag"></<span class="title">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">artifactId</span>></span>hadoop-client<span class="tag"></<span class="title">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">version</span>></span>2.6.0<span class="tag"></<span class="title">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">groupId</span>></span>junit<span class="tag"></<span class="title">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">artifactId</span>></span>junit<span class="tag"></<span class="title">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">version</span>></span>3.8.1<span class="tag"></<span class="title">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">scope</span>></span>test<span class="tag"></<span class="title">scope</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">dependency</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">dependencies</span>></span></span><br><span class="line"><span class="tag"></<span class="title">project</span>></span></span><br></pre></td></tr></table></figure>
<p>打包运行：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ mvn package</span><br><span class="line">$ mkdir -p src/main/resources</span><br><span class="line">$ scp cloudlab119:/etc/hadoop/conf/core-site.xml src/main/resources</span><br><span class="line">$ mvn <span class="built_in">exec</span>:java -Dexec.mainClass=<span class="string">"org.tecstack.App"</span> -Dexec.cleanupDaemonThreads=<span class="literal">false</span></span><br><span class="line">$ mvn dependency:copy-dependencies <span class="comment"># 导出依赖的包</span></span><br><span class="line">$ mvn resources:resources <span class="comment"># 导出资源</span></span><br><span class="line">$ mvn eclipse:eclipse <span class="comment"># 生成eclipse工程文件，.project, .classpath</span></span><br></pre></td></tr></table></figure>
<p>关于maven的exec插件，也可以通过配置<code>pom.xml</code>的plugin的参数简化执行：</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">build</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">plugins</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">groupId</span>></span>org.codehaus.mojo<span class="tag"></<span class="title">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">artifactId</span>></span>exec-maven-plugin<span class="tag"></<span class="title">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">version</span>></span>1.4.0<span class="tag"></<span class="title">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">goal</span>></span>java<span class="tag"></<span class="title">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">goals</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">mainClass</span>></span>org.tecstack.App<span class="tag"></<span class="title">mainClass</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">cleanupDaemonThreads</span>></span>false<span class="tag"></<span class="title">cleanupDaemonThreads</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">configuration</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">plugin</span>></span></span><br><span class="line"> <span class="tag"></<span class="title">plugins</span>></span></span><br><span class="line"><span class="tag"></<span class="title">build</span>></span></span><br></pre></td></tr></table></figure>
<p>之后运行只需要<code>mvn exec:java</code>，如果需要定制更多参数，比如JVM内存，单独启动进程执行等，可以使用<code>exec:exec</code>插件，具体看[codehaus官网][<a href="http://mojo.codehaus.org/exec-maven-plugin/usage.html]。" target="_blank" rel="external">http://mojo.codehaus.org/exec-maven-plugin/usage.html]。</a></p>
<h2 id="HUE交互界面">HUE交互界面</h2><p>Hue是一个基于Django开发的webapp，通过WebHDFS或HttpFS的一种访问HDFS的数据，在HDFS HA部署方式中只能使用HttpFS。<br>如通过WebHDFS方式访问，需要修改<code>hdfs-site.xml</code>：</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>dfs.webhdfs.enabled<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>true<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br></pre></td></tr></table></figure>
<p>修改<code>core-site.xml</code>使Hue账号可以代理其他用户：</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hadoop.proxyuser.hue.hosts<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>*<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hadoop.proxyuser.hue.groups<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>*<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br></pre></td></tr></table></figure>
<p>修改完参数需要重新启动HDFS，通过Ambari操作很方便。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ yum install -y hue hue-server</span><br><span class="line">$ vim /etc/hue/conf/hue.ini <span class="comment"># 修改各个服务的URI，可以通过Ambari看到服务所在的服务器位置；修改默认监听端口不与其它冲突</span></span><br><span class="line">$ service hue start</span><br></pre></td></tr></table></figure>
<p>默认登陆账号为<code>admin:admin</code>，可以通过hdfs为该账号创建一个专用的文件夹<code>/user/hue</code>进行测试，正式环境中可以考虑与其他身份系统对接。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ su hdfs</span><br><span class="line">$ hdfs dfs -mkdir /user/hue</span><br><span class="line">$ hdfs dfs -chown -R admin:hadoop /user/hue</span><br><span class="line">$ hdfs dfs -ls /user/</span><br></pre></td></tr></table></figure>
<h2 id="导入数据到HBase">导入数据到HBase</h2><p>通过hcat建立数据库表结构，通过pig导入数据。</p>
<p>建立数据文件<code>data.tsv</code>:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">row1 c1 c2 row2 c1 c2 row3 c1 c2 row4 c1 c2 row5 c1 c2 row6 c1 c2 row7 c1 c2 row8 c1 c2 row9 c1 c2 row10 c1 c2</span><br></pre></td></tr></table></figure>
<p>准备建表DDL，<code>simple.ddl</code>：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">CREATE TABLE simple_hcat_load_table (id STRING, c1 STRING, c2 STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( 'hbase.columns.mapping' = 'd:c1,d:c2' ) TBLPROPERTIES ( 'hbase.table.name' = 'simple_hcat_load_table' );</span><br></pre></td></tr></table></figure>
<p>准备导入脚本，<code>simple.bulkload.pig</code>：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">A = LOAD 'hdfs:///tmp/data.tsv' USING PigStorage('\t') AS (id:chararray, c1:chararray, c2:chararray); -- DUMP A; STORE A INTO 'hbase://simple_hcat_load_table' USING org.apache.hive.hcatalog.pig.HCatStorer();</span><br></pre></td></tr></table></figure>
<p>把文件放到HDFS，建表，导入：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ su - hdfs</span><br><span class="line">$ hdfs dfs -put data.tsv /tmp/</span><br><span class="line">$ hcat <span class="operator">-f</span> simple.ddl</span><br><span class="line">$ pig -useHCatalog simple.bulkload.pig</span><br></pre></td></tr></table></figure>
<p>通过Hue的HCat可以查看数据库和表数据，也可以通过HBase Shell：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ su - hdfs</span><br><span class="line">$ hbase shell</span><br><span class="line">hbase > list <span class="comment"># 查询所有表</span></span><br><span class="line">hbase > scan <span class="string">'simple_hcat_load_table'</span> <span class="comment"># 查询所有数据</span></span><br><span class="line">hbase > describe <span class="string">'simple_hcat_load_table'</span></span><br></pre></td></tr></table></figure>
<p>也可以通过Hive Shell：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ su - hdfs</span><br><span class="line">$ hive</span><br><span class="line">hive > show tables;</span><br><span class="line">hive > select count(*) from simple_hcat_load_table;</span><br><span class="line">hive > desc simple_hcat_load_table;</span><br></pre></td></tr></table></figure>
<p>如果建表过程中出现类似<code>java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration</code>如下的找不到类情况，检查hcat的配置文件是否有HBase目录：<code>vim /usr/bin/hcat</code>。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">...</span><br><span class="line"><span class="built_in">export</span> HBASE_HOME=/usr/hdp/<span class="number">2.2</span>.<span class="number">4.2</span>-<span class="number">2</span>/hbase</span><br><span class="line">...</span><br></pre></td></tr></table></figure>
<p>此外，由于Hue集成的HCat调用的是Hive下的hcat，需要在<code>/etc/hive-hcatalog/conf/hcat-env.sh</code>中指定<code>HBASE_HOME</code>:</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">...</span><br><span class="line">HBASE_HOME=<span class="variable">${HBASE_HOME:-/usr/hdp/current/hbase-client}</span></span><br><span class="line">...</span><br></pre></td></tr></table></figure>
<p>pig在使用MapReduce模式执行时，可以根据log打开MR Job跟踪，如果也出现<code>java.lang.NoClassDefFoundError</code>类错误，说明mapreduce服务的classpath有缺漏，可以通过Ambari修改MapReduce服务<code>Advanced mapred-site</code>配置中的<code>mapreduce.application.classpath</code>，比如出现HBase相关类找不到，则添加HBase相关的库<code>/usr/hdp/2.2.4.2-2/hbase/lib/*</code>，然后重启MapReduce服务即可。</p>
<p>如果出现执行权限错误，需要检查如下配置是否存在：<br><code>hive-site.xml</code>:</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hive.security.metastore.authorization.manager<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</span><br><span class="line"><span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hive.security.metastore.authenticator.manager<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator</span><br><span class="line"><span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hive.metastore.pre.event.listeners<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</span><br><span class="line"><span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hive.metastore.execute.setugi<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>true<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br></pre></td></tr></table></figure>
<p><code>webhcat-site.xml</code>:<br>正式环境中可以明确具体使用的账号名称，最小化权限。</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>webhcat.proxyuser.hue.hosts<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>*<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>webhcat.proxyuser.hue.groups<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>*<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br></pre></td></tr></table></figure>
<p><code>core-site.xml</code>:</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hadoop.proxyuser.hcat.group<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>*<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br><span class="line"><span class="tag"><<span class="title">property</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">name</span>></span>hadoop.proxyuser.hcat.hosts<span class="tag"></<span class="title">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="title">value</span>></span>*<span class="tag"></<span class="title">value</span>></span></span><br><span class="line"><span class="tag"></<span class="title">property</span>></span></span><br></pre></td></tr></table></figure>
<h2 id="使用小结">使用小结</h2><p>Hadoop系架构经过引入YARN，真正实现了数据和应用的分离，在YARN架构上可以实现出了原来的MapReduce之外的更多App，比如流式处理、实时处理、图计算等，如此可以真正实现“把应用挪到数据旁边高效执行”，构建大数据平台。此外，做数据分析时可以发现周边工具很丰富，比如hue统一的综合管理界面、从数据文件识别结构并管理数据的hcat、数据处理高级语言pig、支持SQL的Hive，使得在Hadoop平台上做数据分析非常方便（可以参考<a href="http://zh.hortonworks.com/hadoop-tutorial/hello-world-an-introduction-to-hadoop-hcatalog-hive-and-pig/" title="hcat & hive & pig" target="_blank" rel="external">这里</a>）。<br>另外，对比YARN利用OS进程隔离分配资源之外，<a href="http://mesos.apache.org/" title="Apache Mesos" target="_blank" rel="external">Mesos</a>结合了Container技术实现容器隔离分配资源，以此实现更通用的框架（用各种语言写的各种计算框架）。当然YARN也进入了<a href="http://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/DockerContainerExecutor.html" title="YARN DCE" target="_blank" rel="external">DCE</a>，通过docker实现容器隔离。不过正式应用还是有待Linux kernel本身功能的成熟，以及docker之类管理工具的完善。<br>最后，数据处理优选python scikit-learn系工具，当大到一定程度，或需要多人同时工作时，hadoop系平台是个不错的选择。</p>
<h2 id="参考">参考</h2><ol>
<li><a href="http://docs.hortonworks.com/HDPDocuments/Ambari-2.0.0.0/Ambari_Doc_Suite/ADS_v200.html" title="Ambari Official Docs" target="_blank" rel="external">Ambari Official Docs</a></li>
<li><a href="http://hadoop.apache.org/docs/stable2/hadoop-project-dist/hadoop-common/FileSystemShell.html" title="HDFS Official Doc" target="_blank" rel="external">HDFS Official Doc</a></li>
<li><a href="https://github.com/sequenceiq/hadoop-docker" title="hadoop-docker ffrom sequenceiq" target="_blank" rel="external">Hadoop-docker from sequenceiq</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/AMBARI/Blueprints#Blueprints-Step1:CreateBlueprint" title="Ambari docs on apache" target="_blank" rel="external">Ambari Docs on Apache</a></li>
<li><a href="https://blog.codecentric.de/en/2014/05/lambda-cluster-provisioning/" title="Sample Blueprint" target="_blank" rel="external">Sample BluePrint</a></li>
<li><a href="http://ju.outofmemory.cn/entry/128881" title="安装配置Hue" target="_blank" rel="external">安装配置Hue</a></li>
<li><a href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.4/HDP_Man_Install_v224/index.html" title="手动安装HDP" target="_blank" rel="external">手动安装HDP</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/AMBARI/Blueprints" title="Ambari Blueprint" target="_blank" rel="external">Ambari Blueprint</a></li>
<li><a href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.4/Importing_Data_HBase_v224/index.html" title="导入数据到HBase" target="_blank" rel="external">导入数据到HBase</a></li>
<li><a href="http://mesos.apache.org/" title="Apache Mesos" target="_blank" rel="external">Apache Mesos</a></li>
<li><a href="http://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/DockerContainerExecutor.html" title="YARN DCE" target="_blank" rel="external">YARN DCE</a></li>
</ol>
<p class="article-more-link">
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/ambari/">ambari</a><a href="/tags/hadoop/">hadoop</a><a href="/tags/hbase/">hbase</a><a href="/tags/hdfs/">hdfs</a><a href="/tags/hive/">hive</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/06/04/HelloHadoop/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/06/04/HelloHadoop/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/06/04/iSCSIwithTgtd/" title="使用Linux的tgtd提供iscsi服务" itemprop="url">使用Linux的tgtd提供iscsi服务</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-06-04T02:54:10.000Z" itemprop="datePublished"> 发表于 2015-06-04</time>
</p>
</header>
<div class="article-content">
<h2 id="安装部署">安装部署</h2><table>
<thead>
<tr>
<th>IP</th>
<th>角色</th>
</tr>
</thead>
<tbody>
<tr>
<td>192.168.182.156</td>
<td>tgtd</td>
</tr>
<tr>
<td>192.168.182.157</td>
<td>client1</td>
</tr>
<tr>
<td>192.168.182.158</td>
<td>client2</td>
</tr>
</tbody>
</table>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ yum install -y scsi-target-utils</span><br><span class="line">$ chkconfig tgtd on</span><br><span class="line">$ service tgtd start</span><br></pre></td></tr></table></figure>
<p>配置方法2种：</p>
<ol>
<li>tgtadm，在线修改</li>
<li>conf配置文件</li>
</ol>
<h2 id="在服务端增加一个Target">在服务端增加一个Target</h2><p>主要流程：</p>
<ol>
<li>建立target</li>
<li>为target增加backstorage</li>
<li>配置客户端访问target的控制策略</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">$ tgtadm --lld iscsi --op new --mode target --tid <span class="number">1</span> --targetname iqn.<span class="number">2015</span>-<span class="number">05</span>-<span class="number">04</span>.org.tecstack.storage.tg1 <span class="comment"># add target</span></span><br><span class="line"><span class="comment"># 用losetup映射块设备作为backstorage</span></span><br><span class="line">$ mkdir /opt/tgtstorage</span><br><span class="line">$ dd <span class="keyword">if</span>=/dev/zero of=/opt/tgtstorage/disk0.img bs=<span class="number">1</span>M count=<span class="number">5120</span></span><br><span class="line">$ losetup <span class="operator">-f</span> /opt/tgtstorage/disk0.img <span class="comment"># 映射为设备</span></span><br><span class="line">$ losetup <span class="operator">-a</span> <span class="comment"># /dev/loop0</span></span><br><span class="line">$ tgtadm --lld iscsi --op new --mode logicalunit --tid <span class="number">1</span> --lun <span class="number">1</span> --backing-store /dev/loop0</span><br><span class="line"><span class="comment"># 直接用块文件作为backstorage添加第二个块</span></span><br><span class="line">$ dd <span class="keyword">if</span>=/dev/zero of=/opt/tgtstorage/disk1.img bs=<span class="number">1</span>M count=<span class="number">5120</span></span><br><span class="line">$ tgtadm --lld iscsi --op new --mode logicalunit --tid <span class="number">1</span> --lun <span class="number">2</span> --backing-store /opt/tgtstorage/disk1.img</span><br><span class="line"><span class="comment"># 绑定客户端ip</span></span><br><span class="line">$ tgtadm --lld iscsi --mode target --op <span class="built_in">bind</span> --tid <span class="number">1</span> --initiator-address=<span class="number">192.168</span>.<span class="number">182.157</span></span><br><span class="line">$ tgtadm --lld iscsi --mode target --op <span class="built_in">bind</span> --tid <span class="number">1</span> --initiator-address=<span class="number">192.168</span>.<span class="number">182.158</span></span><br><span class="line">$ tgt-admin --dump |grep -v default-driver > /etc/tgt/conf.d/my-targets.conf <span class="comment"># 通过tgt-admin保存为配置文件，注意与tgtadm的区别！</span></span><br><span class="line">$ tgtadm --lld iscsi --mode target --op show</span><br><span class="line">$ tgt-admin --show <span class="comment"># 和上一个命令一样，tgt-admin是tgtadm的perl封装</span></span><br></pre></td></tr></table></figure>
<p>在服务端禁用某个客户端：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ tgtadm --lld iscsi --mode target --op unbind --tid <span class="number">1</span> --initiator-address=<span class="number">192.168</span>.<span class="number">182.158</span> <span class="comment"># unbind，使不可发现</span></span><br></pre></td></tr></table></figure>
<h2 id="在客户端使用target">在客户端使用target</h2><p>iscsi存储使用的主要流程：</p>
<ol>
<li>发现target</li>
<li>login到target</li>
<li>使用存储管理工具使用块设备</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">$ iscsiadm --mode discovery --type sendtargets --portal <span class="number">192.168</span>.<span class="number">182.156</span></span><br><span class="line">$ ls -lh /var/lib/iscsi/nodes/ <span class="comment"># 可以查看到对应的target和卷资料</span></span><br><span class="line">$ iscsiadm -m node <span class="comment"># 查看当前机器上所有target</span></span><br><span class="line">$ iscsiadm -m node -T iqn.<span class="number">2015</span>-<span class="number">05</span>-<span class="number">04</span>.org.tecstack.storage.tg1 --login <span class="comment"># 登陆target</span></span><br><span class="line">$ fdisk <span class="operator">-l</span> <span class="comment"># 登陆后就可以看到块设备</span></span><br><span class="line">$ iscsiadm -m node -T iqn.<span class="number">2015</span>-<span class="number">05</span>-<span class="number">04</span>.org.tecstack.storage.tg1 --logout <span class="comment">#登出target</span></span><br><span class="line">$ fdisk <span class="operator">-l</span> <span class="comment"># 登出后设备移除</span></span><br><span class="line">$ iscsiadm -m node -o delete -T iqn.<span class="number">2015</span>-<span class="number">05</span>-<span class="number">04</span>.org.tecstack.storage.tg1 <span class="comment"># 删除target</span></span><br><span class="line">$ ls -lh /var/lib/iscsi/nodes/</span><br><span class="line">$ iscsiadm -m node -T iqn.<span class="number">2015</span>-<span class="number">05</span>-<span class="number">04</span>.org.tecstack.storage.tg1 -p <span class="number">192.168</span>.<span class="number">182.156</span> -- op update -n node.startup -v automatic <span class="comment"># 自动login</span></span><br></pre></td></tr></table></figure>
<p>当login到target之后，就可以使用，比如通过LVM：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">$ fdisk <span class="operator">-l</span> <span class="comment"># 增加了一个/dev/sdb块设备</span></span><br><span class="line">$ pvcreate /dev/sdb</span><br><span class="line">$ pvdisplay</span><br><span class="line">$ vgcreate myiscsi /dev/sdb</span><br><span class="line">$ vgdisplay</span><br><span class="line">$ lvcreate <span class="operator">-l</span> <span class="number">1024</span> -n vdisk0 myiscsi</span><br><span class="line">$ lvdisplay</span><br><span class="line">$ ls -lh /dev/myiscsi/vdisk0 <span class="comment"># 生成的块设备位置</span></span><br><span class="line">$ mkfs.ext4 /dev/myiscsi/vdisk0</span><br><span class="line">$ mkdir -p /opt/myiscsidata</span><br><span class="line">$ mount /dev/myiscsi/vdisk0 /opt/myiscsidata/</span><br><span class="line">$ df -h</span><br></pre></td></tr></table></figure>
<p>运行过程中为target添加后端存储，客户端需要logout后重新login才能看到。但是如果重新login后块设备名会变化，比如变成<code>/dev/sdc</code>。可以通过文件系统的UUID来识别设备并挂载：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ tune2fs <span class="operator">-l</span> /dev/sdc</span><br></pre></td></tr></table></figure>
<h3 id="GFS2测试">GFS2测试</h3><h4 id="创建和挂载iscsi块存储">创建和挂载iscsi块存储</h4><p>在target端创建LUN</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ dd <span class="keyword">if</span>=/dev/zero of=/opt/tgtstorage/disk_gfs.img bs=<span class="number">1</span>M count=<span class="number">5120</span></span><br><span class="line">$ tgtadm --lld iscsi --op new --mode logicalunit --tid <span class="number">1</span> --lun <span class="number">3</span> --backing-store /opt/tgtstorage/disk_gfs.img</span><br><span class="line">$ tgtadm --lld iscsi --mode target --op <span class="built_in">bind</span> --tid <span class="number">1</span> --initiator-address=<span class="number">192.168</span>.<span class="number">182.157</span></span><br><span class="line">$ tgtadm --lld iscsi --mode target --op <span class="built_in">bind</span> --tid <span class="number">1</span> --initiator-address=<span class="number">192.168</span>.<span class="number">182.158</span></span><br><span class="line">$ tgt-admin --show</span><br></pre></td></tr></table></figure>
<p>在client1和client2上挂载块设备：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ iscsiadm --mode discovery --type sendtargets --portal <span class="number">192.168</span>.<span class="number">182.156</span></span><br><span class="line">$ iscsiadm -m node</span><br><span class="line">$ iscsiadm -m node -T iqn.<span class="number">2015</span>-<span class="number">05</span>-<span class="number">04</span>.org.tecstack.storage.tg1 --login</span><br><span class="line">$ fdisk <span class="operator">-l</span></span><br></pre></td></tr></table></figure>
<h4 id="使用luci配置集群：">使用luci配置集群：</h4><p>在target端配置：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ yum -y install luci</span><br><span class="line">$ service luci start</span><br></pre></td></tr></table></figure>
<p>在client1和client2上配置：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ yum install -y ricci</span><br><span class="line">$ service ricci start</span><br><span class="line">$ passwd ricci <span class="comment"># 123456，节点密码</span></span><br></pre></td></tr></table></figure>
<p>通过访问target的https（默认端口8084，账号同Linux本地root账号），创建集群，新增节点，选择下载包，勾选启用共享文件系统。管理工具会自动帮助安装：<code>cman rgmanager lvm2-cluster sg3_utils gfs2-utils</code>，并启动相关服务。</p>
<h4 id="在一台集群节点上创建LVM逻辑卷，格式化为GFS文件系统">在一台集群节点上创建LVM逻辑卷，格式化为GFS文件系统</h4><p>识别scsi和块设备的对应关系：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ ls <span class="operator">-l</span> /dev/disk/by-path/*tecstack*</span><br><span class="line">$ scsi_id -gu /dev/sdb</span><br></pre></td></tr></table></figure>
<p>在client1上执行：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ pvcreate /dev/sdb</span><br><span class="line">$ vgcreate gfstest /dev/sdb</span><br><span class="line">$ lvcreate <span class="operator">-l</span> <span class="number">1024</span> -n gfsdisk0 gfstest</span><br><span class="line">$ mkfs.gfs2 -j2 -p lock_dlm -t gfstest:gfs2 /dev/gfstest/gfsdisk0</span><br></pre></td></tr></table></figure>
<h4 id="在两台集群节点上同时挂载GFS文件系统">在两台集群节点上同时挂载GFS文件系统</h4><p>在client1和client2上执行：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ mkdir /mnt/gfstest</span><br><span class="line">$ mount /dev/gfstest/gfsdisk0 /mnt/gfstest</span><br></pre></td></tr></table></figure>
<p>参考：</p>
<ol>
<li><a href="http://blog.delouw.ch/2013/07/07/creating-and-managing-iscsi-targets/" title="creating and managing iscsi targets" target="_blank" rel="external">creating and managing iscsi targets</a></li>
<li><a href="http://stgt.sourceforge.net/manpages/tgtadm.8.html" title="tgtadm official man page" target="_blank" rel="external">tgtadm man page</a></li>
<li><a href="http://linux.vbird.org/linux_server/0460iscsi.php" title="iscsi使用案例" target="_blank" rel="external">iscsi使用案例</a></li>
<li><a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Global_File_System/s1-config-tasks.html" title="GFS配置" target="_blank" rel="external">GFS配置</a></li>
</ol>
<p class="article-more-link">
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/iscsi/">iscsi</a><a href="/tags/linux/">linux</a><a href="/tags/tgtd/">tgtd</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/06/04/iSCSIwithTgtd/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/06/04/iSCSIwithTgtd/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/05/13/HelloDocker/" title="HelloDocker" itemprop="url">HelloDocker</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-05-13T07:56:19.000Z" itemprop="datePublished"> 发表于 2015-05-13</time>
</p>
</header>
<div class="article-content">
<h1 id="Docker概述">Docker概述</h1><p>Docker是基于Go语言开发的容器管理工具，而且抽象级别比lxc这些管理工具高，目前官方默认通过libcontainer管理容器。lxc是一些kernel patch（namespaces）和userspace tool（cgroup）的集合，通过cgroup这个用户态管理工具实现对隔离资源的管理。Docker相比LXC增加了镜像服务（通过UnionFS和DeviceMapper），同时简化了操作复杂度，可以很方便实现应用分发部署，甚至是扩容。目前也有一些周边管理工具（Compose、Flocker、weaver）在创新，在docker基础上实现更多实用功能，如迁移，流程编排，网络管理等。跟openstack相比，docker更年轻，也更轻量级，但两者在某些场景下又可以很好结合起来，比如通过docker将openstack的管理节点实现高可用、可扩展。</p>
<p>docker的部署在发行版上比较简单，但对内核版本有不同要求，比如在centos6.5上要求内核版本至少2.6.32-431。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ yum install -y docker-io</span><br><span class="line">$ chkconfig docker on</span><br><span class="line">$ service docker start</span><br><span class="line">$ docker run -it ubuntu /bin/bash</span><br><span class="line">$ docker info <span class="comment">#查看docker信息</span></span><br></pre></td></tr></table></figure>
<h2 id="Docker基本用法">Docker基本用法</h2><h3 id="基本命令使用：">基本命令使用：</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">$ docker run ubuntu:<span class="number">14.04</span> /bin/<span class="built_in">echo</span> <span class="string">'Hello world'</span></span><br><span class="line">$ docker run <span class="operator">-d</span> ubuntu:<span class="number">14.04</span> /bin/sh -c <span class="string">"while true; do echo hello world; sleep 1; done"</span> <span class="comment"># 后台运行</span></span><br><span class="line">$ docker ps <span class="operator">-a</span></span><br><span class="line">$ docker logs instance_name</span><br><span class="line">$ docker logs <span class="operator">-f</span> instance_name <span class="comment"># 持续输出</span></span><br><span class="line">$ docker stop/start/restart instance_name <span class="comment"># stop容器后还能启动，但网络信息会变</span></span><br><span class="line">$ docker ps -aq | xargs docker rm <span class="comment"># 删除所有容器，运行中的会保留</span></span><br><span class="line">$ docker images ＃ -tree参数可以看层级关系，最大<span class="number">128</span>层</span><br><span class="line">$ docker save -o ubuntu.tar ubuntu:<span class="number">14.04</span> <span class="comment"># 把镜像保存到本地</span></span><br><span class="line">$ tar -tf ubuntu.tar <span class="comment"># 可以看到image的结构，包含了多层FS</span></span><br><span class="line">$ docker rmi ubuntu</span><br><span class="line">$ docker run <span class="operator">-d</span> -P training/webapp python app.py <span class="comment"># 映射所有端口</span></span><br><span class="line">$ docker port instance_name <span class="number">5000</span> <span class="comment">#查看被映射的端口</span></span><br><span class="line">$ docker run <span class="operator">-d</span> -p <span class="number">5000</span>:<span class="number">5000</span> training/webapp python app.py <span class="comment"># 映射5000端口</span></span><br><span class="line">$ docker run <span class="operator">-d</span> -p <span class="number">127.0</span>.<span class="number">0.1</span>:<span class="number">5000</span>:<span class="number">5000</span>/udp training/webapp python app.py <span class="comment"># 映射本机UDP端口</span></span><br><span class="line">$ docker top instance_name</span><br><span class="line">$ docker inspect instance_name <span class="comment"># 查看容器配置</span></span><br><span class="line">$ docker inspect <span class="operator">-f</span> <span class="string">'{{ .NetworkSettings.IPAddress }}'</span> instance_name <span class="comment"># 查看具体的配置信息，如IP</span></span><br><span class="line">$ docker pull centos</span><br><span class="line">$ docker commit -m <span class="string">"some modify hints"</span> <span class="operator">-a</span> <span class="string">"author info"</span> </span><br><span class="line">contain_id repo_name/image_name:tags</span><br><span class="line">$ docker tag container_id repo_name/image_name:tags</span><br><span class="line">$ docker <span class="built_in">exec</span> -it CONTAINER_NAME /bin/bash <span class="comment"># 1.3开始支持在运行的容器内执行命令</span></span><br></pre></td></tr></table></figure>
<h3 id="使用Dockerfile创建image">使用Dockerfile创建image</h3><p><code>mkdir myimage && cd $_ && vim Dockerfile</code>:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"># This is a comment FROM ubuntu:14.04 MAINTAINER Promise John <promise.jon@gmail.com> RUN apt-get update && apt-get install -y nginx</span><br></pre></td></tr></table></figure>
<h3 id="使用Linking">使用Linking</h3><p>Docker主要通过环境变量和<code>/etc/hosts</code>文件来提供访问途径：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ docker run <span class="operator">-d</span> --name db training/postgres</span><br><span class="line">$ docker run <span class="operator">-d</span> -P --name web --link db:db training/webapp python app.py</span><br><span class="line">$ docker inspect <span class="operator">-f</span> <span class="string">"{{ .Name }}"</span> instance_id <span class="comment"># 查看容器名</span></span><br><span class="line">$ docker inspect <span class="operator">-f</span> <span class="string">"{{ .HostConfig.Links }}"</span> web</span><br><span class="line">$ docker run --rm --name web2 --link db:db training/webapp env</span><br><span class="line">$ docker run --rm --name web2 --link db:db training/webapp cat /etc/hosts</span><br></pre></td></tr></table></figure>
<h3 id="Storage">Storage</h3><p>主要包括独立的卷和共享本地文件系统：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ docker run <span class="operator">-d</span> -P --name web -v /webapp training/webapp python app.py <span class="comment"># volumn不会随容器消失，作为数据独立存在</span></span><br><span class="line">$ docker inspect web <span class="comment"># 查看/var/lib/docker/volumes和vfs</span></span><br><span class="line">$ docker run <span class="operator">-d</span> -P --name web -v /src/webapp:/opt/webapp training/webapp python app.py <span class="comment"># 挂载</span></span><br><span class="line">$ docker run <span class="operator">-d</span> -P --name web -v /src/webapp:/opt/webapp:ro training/webapp python app.py <span class="comment"># 只读挂载</span></span><br><span class="line">$ docker run --rm -it -v ~/.bash_<span class="built_in">history</span>:/.bash_<span class="built_in">history</span> ubuntu /bin/bash <span class="comment"># 只挂载某个文件</span></span><br></pre></td></tr></table></figure>
<p>Container间共享数据：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ docker create -v /dbdata --name dbdata training/postgres /bin/<span class="literal">true</span></span><br><span class="line">$ docker run <span class="operator">-d</span> --volumes-from dbdata --name db1 training/postgres</span><br><span class="line">$ docker run <span class="operator">-d</span> --volumes-from dbdata --name db2 training/postgres</span><br><span class="line">$ docker run --volumes-from dbdata -v $(<span class="built_in">pwd</span>):/backup ubuntu tar cvf /backup/backup.tar /dbdata <span class="comment"># 备份容器数据</span></span><br><span class="line">$ docker run --volumes-from dbdata2 -v $(<span class="built_in">pwd</span>):/backup busybox tar xvf /backup/backup.tar <span class="comment"># 恢复数据</span></span><br></pre></td></tr></table></figure>
<h2 id="Docker本地仓库">Docker本地仓库</h2><p>Docker Hub基本用法：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ docker search ubuntu</span><br><span class="line">$ docker pull ubuntu:<span class="number">14.04</span></span><br><span class="line">$ docker login <span class="comment"># 配置文件在~/.dockercfg</span></span><br><span class="line">$ docker push yourname/newimage</span><br></pre></td></tr></table></figure>
<p>配置本地的registry：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">$ docker run <span class="operator">-d</span> -p <span class="number">5000</span>:<span class="number">5000</span> registry</span><br><span class="line">$ docker tag ubuntu:<span class="number">14.04</span> localhost:<span class="number">5000</span>/ubuntu:<span class="number">14.04</span></span><br><span class="line">$ docker push localhost:<span class="number">5000</span>/ubuntu:<span class="number">14.04</span></span><br><span class="line">$ curl -v http://localhost:<span class="number">5000</span>/v1/repositories/ubuntu/tags/<span class="number">14.04</span> <span class="comment"># 查看image id</span></span><br><span class="line">$ docker run <span class="operator">-d</span> -p <span class="number">5000</span>:<span class="number">5000</span> \</span><br><span class="line"> <span class="operator">-e</span> STANDALONE=<span class="literal">false</span> \</span><br><span class="line"> <span class="operator">-e</span> MIRROR_SOURCE=https://registry-<span class="number">1</span>.docker.io \</span><br><span class="line"> <span class="operator">-e</span> MIRROR_SOURCE_INDEX=https://index.docker.io \</span><br><span class="line"> registry <span class="comment"># 作为官网镜像启动，代理模式</span></span><br></pre></td></tr></table></figure>
<p>使用本地的registry：</p>
<p>可以使用<code>--registry-mirror</code>参数启动：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker --registry-mirror=http://<span class="number">10.101</span>.<span class="number">29.26</span> <span class="operator">-d</span></span><br></pre></td></tr></table></figure>
<h2 id="Docker_Compose">Docker Compose</h2><p>Docker Compose可以方便地实现应用的组合。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ pip install docker-compose</span><br><span class="line">$ mkdir compose && <span class="built_in">cd</span> <span class="variable">$_</span></span><br></pre></td></tr></table></figure>
<p>增加<code>app.py</code>：</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> flask <span class="keyword">import</span> Flask</span><br><span class="line"><span class="keyword">from</span> redis <span class="keyword">import</span> Redis</span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line">app = Flask(__name__)</span><br><span class="line">redis = Redis(host=<span class="string">'redis'</span>, port=<span class="number">6379</span>)</span><br><span class="line"></span><br><span class="line"><span class="decorator">@app.route('/')</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">hello</span><span class="params">()</span>:</span></span><br><span class="line"> redis.incr(<span class="string">'hits'</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="string">'Hello World! I have been seen %s times.'</span> % redis.get(<span class="string">'hits'</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">"__main__"</span>:</span><br><span class="line"> app.run(host=<span class="string">"0.0.0.0"</span>, debug=<span class="keyword">True</span>)</span><br></pre></td></tr></table></figure>
<p>增加<code>requirements.txt</code>：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">flask redis</span><br></pre></td></tr></table></figure>
<p>增加<code>Dockerfile</code>：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">FROM python:2.7 ADD . /code WORKDIR /code RUN pip install -r requirements.txt</span><br></pre></td></tr></table></figure>
<p>增加<code>docker-compose.yml</code>：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">web: build: . command: python app.py ports: - "5000:5000" volumes: - .:/code links: - redis redis: image: redis</span><br></pre></td></tr></table></figure>
<p>启动并管理服务：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ docker-compose up <span class="operator">-d</span></span><br><span class="line">$ docker-compose ps</span><br><span class="line">$ docker-compose run web env</span><br><span class="line">$ docker-compose stop</span><br></pre></td></tr></table></figure>
<h2 id="小结">小结</h2><p>Docker在快速应用分发、扩展等方面有很高的自动化能力，但是跟主机虚拟化一样，网络也是需要重点考虑的难点。默认情况下都是通过本地的一个bridge提供容器之间的通信，通过NAT为应用提供地址，一般的应用部署足以应付，但如果要实现更复杂的网络场景，如多主机上container组成大规模的L3负载均衡集群、应用无缝迁移等，还是需要大量的定制，目前这方面的周边产品开发比较活跃。</p>
<h2 id="参考">参考</h2><ol>
<li><a href="https://docs.docker.com" title="Official Docs" target="_blank" rel="external">Official Docs</a></li>
<li><a href="http://www.infoq.com/cn/articles/analysis-of-docker-file-system-aufs-and-devicemapper" title="Docker fs with aufs and devicemapper" target="_blank" rel="external">Docker with aufs and devicemapper</a></li>
<li><a href="http://www.ibm.com/developerworks/cn/linux/l-lxc-containers/" title="LXC TOOL" target="_blank" rel="external">LXC Tools</a></li>
<li><a href="https://docs.docker.com/compose" title="Docker Compose" target="_blank" rel="external">Docker Compose</a></li>
</ol>
<p class="article-more-link">
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/docker/">docker</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/05/13/HelloDocker/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/05/13/HelloDocker/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/05/11/BuildKVMImages/" title="BuildKVMImages" itemprop="url">BuildKVMImages</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-05-11T07:33:37.000Z" itemprop="datePublished"> 发表于 2015-05-11</time>
</p>
</header>
<div class="article-content">
<h2 id="手动创建KVM虚拟机镜像">手动创建KVM虚拟机镜像</h2><p>通过libvirt系的本地命令行管理工具，也可以方便地创建虚拟机。</p>
<ul>
<li>qemu-img：生成虚拟机磁盘文件</li>
<li>virsh：命令行虚拟机管理工具</li>
</ul>
<h3 id="生成Domain_XML文件">生成Domain XML文件</h3><p>如果要用X11远程到物理机使用GUI工具，需要配置sshd，安装<code>xauth</code>：</p>
<p><code>/etc/ssh/sshd_config</code>:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">... X11Forwarding yes X11UseLocalhost no ...</span><br></pre></td></tr></table></figure>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ /etc/init.d/sshd reload</span><br><span class="line">$ yum install -y xauth</span><br><span class="line">$ ssh -X user@host</span><br><span class="line">$ virt-manager&</span><br></pre></td></tr></table></figure>
<p>也可以直接用官方的example修改配置：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><domain type = 'kvm'> <name>centos66</name> <memory>1048576</memory> <vcpu>1</vcpu> <os> <type arch = 'x86_64'machine = 'pc'>hvm</type> <boot dev = 'cdrom'/> <boot dev = 'hd'/> </os> <features> <acpi/> <apic/> <pae/> </features> <clock offset = 'utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type = 'file' device = 'disk'> <driver name = 'qemu' type = 'raw' cache='none'/> <source file = '/opt/vmdisks/centos66.raw'/> <target dev='vda' bus='virtio'/> </disk> <disk type = 'file' device = 'cdrom'> <source file = '/opt/CentOS-6.6-x86_64-bin-DVD1.iso'/> <target dev = 'hdb' bus = 'ide'/> </disk> "centos66.xml" 48L, 1243C written 8,20-34 Top <target dev = 'hdb' bus = 'ide'/> </disk> <interface type='network'> <source network='default'/> <model type='virtio'/> </interface> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> </devices> </domain></span><br></pre></td></tr></table></figure>
<h3 id="创建虚拟机">创建虚拟机</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ mkdir -p /opt/vmdisks</span><br><span class="line">$ qemu-img create <span class="operator">-f</span> raw /opt/vmdisks/centos66.raw <span class="number">8</span>G</span><br><span class="line">$ virsh create centos66.xml</span><br><span class="line">$ ip addr</span><br><span class="line">$ virsh vncdisplay centos66 <span class="comment">#获取ip和vnc端口登陆</span></span><br></pre></td></tr></table></figure>
<p>接下来就进入安装操作系统的界面，一路按指印即可。</p>
<h3 id="远程管理libvirt主机">远程管理libvirt主机</h3><p>可以用qemu+ssh，节点间互信后迁移；也可以用qemu+tcp，修改<code>/etc/libvirt/libvirtd.conf</code>:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">listen_tls = 0 listen_tcp = 1 tcp_port = "16509" listen_addr = "0.0.0.0" auth_tcp = "none" # 生产环境建议加上授权验证</span><br></pre></td></tr></table></figure>
<p>修改<code>/etc/init.d/libvirtd</code>:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">LIBVIRTD_CONFIG=/etc/libvirt/libvirtd.conf LIBVIRTD_ARGS="--listen"</span><br></pre></td></tr></table></figure>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ service libvirtd restart</span><br><span class="line">$ ss -ln | grep <span class="number">16509</span></span><br></pre></td></tr></table></figure>
<p>在其他节点就可以连接（注意iptables）：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ virsh -c qemu+tcp://<span class="number">192.168</span>.<span class="number">182.156</span>/system</span><br></pre></td></tr></table></figure>
<p>克隆虚拟机：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ virsh define centos66.xml</span><br><span class="line">$ virt-clone --original=centos66 --name=centos66_01 <span class="operator">-f</span> centos66_01.raw</span><br><span class="line">$ virsh list --all</span><br></pre></td></tr></table></figure>
<h3 id="参考">参考</h3><ol>
<li><a href="http://libvirt.org/formatdomain.html" title="libvirt官方虚拟机规格XML格式说明" target="_blank" rel="external">libvirt官方虚拟机规格XML格式说明</a></li>
</ol>
<p class="article-more-link">
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/kvm/">kvm</a><a href="/tags/libvirt/">libvirt</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/05/11/BuildKVMImages/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/05/11/BuildKVMImages/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/05/07/HelloOpenstack/" title="Openstack安装部署" itemprop="url">Openstack安装部署</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-05-07T07:12:56.000Z" itemprop="datePublished"> 发表于 2015-05-07</time>
</p>
</header>
<div class="article-content">
<h2 id="参考架构及部署规划">参考架构及部署规划</h2><p><img src="http://docs.openstack.org/icehouse/install-guide/install/yum/content/figures/1/a/common/figures/openstack_havana_conceptual_arch.png" alt="参考架构"></p>
<ul>
<li>操作系统：CentOS6.5</li>
<li>第三方yum源：epel, rdo</li>
</ul>
<p>节点部署角色：</p>
<table>
<thead>
<tr>
<th>节点名</th>
<th>internal ip</th>
<th>public ip</th>
<th>Role</th>
</tr>
</thead>
<tbody>
<tr>
<td>oscontroller</td>
<td>10.0.100.145</td>
<td>192.168.182.150</td>
<td>nova, glance, cinder, image, neutron, dashboard, heat</td>
</tr>
<tr>
<td>osnetwork</td>
<td>10.0.100.146</td>
<td>192.168.182.151</td>
<td>ML2, OVS, L2 Agent, L3 Agent, DHCP Agent</td>
</tr>
<tr>
<td>oscompute1</td>
<td>10.0.100.147</td>
<td>192.168.182.152</td>
<td>nova-compute</td>
</tr>
<tr>
<td>oscompute2</td>
<td>10.0.100.148</td>
<td>192.168.182.153</td>
<td>nova-compute</td>
</tr>
<tr>
<td>oskeystone</td>
<td>10.0.100.149</td>
<td>192.168.182.154</td>
<td>qpid/rabbitmq, keystone, mysql， memcached</td>
</tr>
<tr>
<td>osmeter</td>
<td>10.0.100.150</td>
<td>192.168.182.155</td>
<td>ceilometer, mongodb</td>
</tr>
<tr>
<td>osswift0</td>
<td>10.0.100.139</td>
<td>192.168.182.144</td>
<td>swift0, swift-proxy-server</td>
</tr>
<tr>
<td>osswift1</td>
<td>10.0.100.140</td>
<td>192.168.182.145</td>
<td>swift1</td>
</tr>
<tr>
<td>osswift2</td>
<td>10.0.100.141</td>
<td>192.168.182.146</td>
<td>swift2</td>
</tr>
<tr>
<td>osceph0</td>
<td>10.0.100.142</td>
<td>192.168.182.147</td>
<td>ceph0</td>
</tr>
<tr>
<td>osceph1</td>
<td>10.0.100.143</td>
<td>192.168.182.148</td>
<td>ceph1 # 暂时不用</td>
</tr>
<tr>
<td>osceph2</td>
<td>10.0.100.144</td>
<td>192.168.182.149</td>
<td>ceph2 # 暂时不用</td>
</tr>
</tbody>
</table>
<p><img src="http://docs.openstack.org/icehouse/install-guide/install/yum/content/figures/1/figures/installguide_arch-neutron.png" alt="参考部署架构"></p>
<p class="article-more-link">
<a href="/2015/05/07/HelloOpenstack/#more">Read More</a>
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/ceilometer/">ceilometer</a><a href="/tags/cinder/">cinder</a><a href="/tags/glance/">glance</a><a href="/tags/neutron/">neutron</a><a href="/tags/nova/">nova</a><a href="/tags/openstack/">openstack</a><a href="/tags/swift/">swift</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/05/07/HelloOpenstack/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/05/07/HelloOpenstack/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/04/17/RubyDevEnvSetting/" title="Ruby开发环境搭建" itemprop="url">Ruby开发环境搭建</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-04-17T02:48:49.000Z" itemprop="datePublished"> 发表于 2015-04-17</time>
</p>
</header>
<div class="article-content">
<h3 id="Ruby开发环境搭建">Ruby开发环境搭建</h3><p>多版本管理<code>RVM</code>：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">$ gpg --keyserver hkp://keys.gnupg.net --recv-keys <span class="number">409</span>B6B1796C275462A1703113804BB82D39DC0E3</span><br><span class="line">$ curl <span class="operator">-s</span>SL https://get.rvm.io | bash <span class="operator">-s</span> stable</span><br><span class="line">$ <span class="built_in">source</span> /etc/profile.d/rvm.sh</span><br><span class="line">$ sed -i <span class="string">'s!cache.ruby-lang.org/pub/ruby!ruby.taobao.org/mirrors/ruby!'</span> <span class="variable">$rvm_path</span>/config/db <span class="comment"># 使用taobao源下载ruby</span></span><br><span class="line">$ rvm list known</span><br><span class="line">$ rvm install <span class="number">2.1</span>.<span class="number">4</span></span><br><span class="line">$ rvm docs generate-ri <span class="comment"># 生成ruby文档</span></span><br><span class="line">$ rvm use <span class="number">2.1</span>.<span class="number">4</span> --default <span class="comment"># 设定默认ruby版本</span></span><br><span class="line">$ rvm list <span class="comment"># 查询已安装版本</span></span><br></pre></td></tr></table></figure>
<p>用<code>gemset</code>建立独立环境：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ rvm use <span class="number">2.1</span>.<span class="number">4</span></span><br><span class="line">$ rvm gemset create rails42</span><br><span class="line">$ rvm use <span class="number">2.1</span>.<span class="number">4</span>@rails42</span><br><span class="line">$ rvm gemset list</span><br></pre></td></tr></table></figure>
<p><code>Gem</code>管理ruby包：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ gem sources --remove https://rubygems.org/</span><br><span class="line">$ gem sources <span class="operator">-a</span> https://ruby.taobao.org/</span><br><span class="line">$ gem sources <span class="operator">-l</span></span><br><span class="line">$ gem search rails <span class="comment"># search，install, etc</span></span><br></pre></td></tr></table></figure>
<p>参考：</p>
<ol>
<li><a href="https://rvm.io/" title="RVM official site" target="_blank" rel="external">RVM official site</a></li>
<li><a href="https://ruby-china.org/wiki/rvm-guide" title="RVM Guide from ruby-china.org" target="_blank" rel="external">RVM Guide from ruby-china.org</a></li>
<li><a href="http://guides.rubygems.org/rubygems-basics/" title="Gem Official Guide" target="_blank" rel="external">Gem Official Guide</a></li>
<li><a href="http://ruby.taobao.org/" title="Ruby@Taobao.org" target="_blank" rel="external">Ruby@Taobao.org</a></li>
</ol>
<p class="article-more-link">
</p>
</div>
<footer class="article-footer clearfix">
<div class="article-catetags">
<div class="article-categories">
<span></span>
<a class="article-category-link" href="/categories/Tech/">Tech</a>
</div>
<div class="article-tags">
<span></span> <a href="/tags/dev/">dev</a><a href="/tags/ruby/">ruby</a>
</div>
</div>
<div class="comments-count">
<span></span>
<a href="/2015/04/17/RubyDevEnvSetting/#comments" class="ds-thread-count comments-count-link" data-thread-key="2015/04/17/RubyDevEnvSetting/" data-count-type="comments"> </a>
</div>
</footer>
</article>
<article class="post-expand post" itemprop="articleBody">
<header class="article-info clearfix">
<h1 itemprop="name">
<a href="/2015/04/17/HelloVagrant/" title="Hello Vagrant" itemprop="url">Hello Vagrant</a>
</h1>
<p class="article-author">By
<a href="http://promisejohn.github.io/about" title="promise john" target="_blank" itemprop="author">promise john</a>
<p class="article-time">
<time datetime="2015-04-17T02:22:06.000Z" itemprop="datePublished"> 发表于 2015-04-17</time>
</p>
</header>
<div class="article-content">
<h3 id="Vagrant部署">Vagrant部署</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/dev</span><br><span class="line">$ wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.<span class="number">7.2</span>_x86_64.rpm</span><br><span class="line">$ rpm -ivh vagrant_1.<span class="number">7.2</span>_x86_64.rpm</span><br></pre></td></tr></table></figure>
<h3 id="添加vagrant_boxes">添加vagrant boxes</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ vagrant box add hashicorp/precise32 <span class="comment"># 支持virtualbox Hypervisor</span></span><br><span class="line">$ vagrant box add centos64 http://citozin.com/centos64.box ＃ 支持 libvirt KVM Hypervisor</span><br></pre></td></tr></table></figure>
<h3 id="编辑Vagrantfile">编辑Vagrantfile</h3><p>默认使用的是virtualbox作为Hypervisor：</p>
<figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="constant">Vagrant</span>.configure(<span class="string">"2"</span>) <span class="keyword">do</span> |config|</span><br><span class="line"> config.vm.box = <span class="string">"hashicorp/precise32"</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure>
<h3 id="运行虚拟机">运行虚拟机</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ vagrant up</span><br></pre></td></tr></table></figure>
<h3 id="Vagrant使用KVM_Hypervisor">Vagrant使用KVM Hypervisor</h3><p>安装vagrant-libvirt plugin：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ yum install libxslt-devel libxml2-devel libvirt-devel</span><br><span class="line">$ vagrant plugin install vagrant-libvirt --plugin-source https://ruby.taobao.org/</span><br></pre></td></tr></table></figure>
<p>配置Vagrantfile：</p>
<figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="constant">Vagrant</span>.configure(<span class="string">"2"</span>) <span class="keyword">do</span> |config|</span><br><span class="line"> config.vm.define <span class="symbol">:node1</span> <span class="keyword">do</span> |node1|</span><br><span class="line"> node1.vm.box = <span class="string">"centos64"</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure>
<p>启动虚拟机：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ vagrant up --provider=libvirt</span><br><span class="line">$ <span class="built_in">export</span> VAGRANT_DEFAULT_PROVIDER=libvirt <span class="comment"># Another way</span></span><br></pre></td></tr></table></figure>
<h3 id="小结">小结</h3><p>Vagrant的box不是对所有hypervisor都通用，比如官方的<code>hashicorp/precise32</code>就不支持KVM，如果环境只有KVM，那么可能需要自己制作对应的box，可以参考<a href="https://github.com/pradels/vagrant-libvirt/" title="Vagrant-Libvirt" target="_blank" rel="external">Vagrant-Libvirt Plugin</a>，借助工具<code>create_box.sh</code>从qcow2 image制作。</p>
<p>此外，需要注意的是，KVM和VirtualBox不能同时启动虚拟机，否则会报类似如下错误：</p>
<p>先启动virtualbox虚拟机再启动KVM虚拟机：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">There was an error talking to Libvirt. The error message is shown below: Call to virDomainCreateWithFlags failed: internal error Process exited while reading console log output: char device redirected to /dev/pts/2 kvm_create_vm: Device or resource busy failed to initialize KVM: Operation not permitted No accelerator found!</span><br></pre></td></tr></table></figure>
<p>先启动KVM虚拟机再启动virtualbox虚拟机：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">The guest machine entered an invalid state while waiting for it to boot. Valid states are 'starting, running'. The machine is in the 'poweroff' state. Please verify everything is configured properly and try again. If the provider you're using has a GUI that comes with it, it is often helpful to open that and watch the machine, since the GUI often has more helpful error messages than Vagrant can retrieve. For example, if you're using VirtualBox, run `vagrant up` while the VirtualBox GUI is open.</span><br></pre></td></tr></table></figure>
<p>参考：</p>
<ol>
<li><a href="https://docs.vagrantup.com/v2/" title="Vagrant Official Site" target="_blank" rel="external">Vagrant Official Site</a></li>
<li><a href="https://github.com/pradels/vagrant-libvirt/" title="Vagrant-Libvirt" target="_blank" rel="external">Vagrant-Libvirt Plugin</a></li>
</ol>