Puppet整合Foreman(四)：整合Mcollective
一、功能说明
- Marionette Collective（MCollective）是一个与Puppet关系密切的服务运行框架。
- MC依赖Ruby1.9.3或1.8.7，通过Stomp协议通信，因此依赖rubygem 1.2.2+
- 在puppet客户端上安装Mcollective服务端
- 在puppet服务端上安装Mcollective客户端
- 在puppet服务端或单独安装MQ中间件
参考官方文档：https://docs.puppetlabs.com/mcollective/deploy/standard.html
二、安装ActiveMQ
1. 说明
RabbitMQ是一个实现了高级消息排队协议（AMQP）的消息队列服务。 RabbitMQ基于OTP(Open Telecom Platform，开发电信平台)进行构建， 并使用Erlang语言和运行时环境来实现。
ActiveMQ 是Apache出品，最流行的，能力强劲的开源消息总线。 ActiveMQ 是一个完全支持JMS1.1和J2EE 1.4规范的 JMS Provider实现
如果是单独安装需要设置主机名、HOSTS、puppet认证，这里是安装在puppet服务端。
2. 安装
| 1 2 | [root@puppet ~]# yum install -y tanukiwrapper activemq-info-provider [root@puppet ~]# yum install -y activemq |
下载慢，失败了几次，多试试
3. 配置
修改配置前最好备份下，并启动服务查看下进程是否运行
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | [root@puppet ~]# cp /etc/activemq/activemq.xml /etc/activemq/activemq.xml.bak [root@puppet ~]# vi /etc/activemq/activemq.xml #<>标签要按字母顺序排列，在标签<persistenceAdapter>后添加 <plugins> <simpleAuthenticationPlugin> <users> <authenticationUser username=”admin” password=”admin@123″ groups=”mcollective,admins,everyone”/> #账号”mcollective”将用于mc客户端配置 <authenticationUser username=”mcollective” password=”mcopwd123″ groups=”mcollective,everyone”/> </users> </simpleAuthenticationPlugin> <authorizationPlugin> <map> <authorizationMap> <authorizationEntries> <authorizationEntry queue=”>” write=”admins” read=”admins” admin=”admins” /> <authorizationEntry topic=”>” write=”admins” read=”admins” admin=”admins” /> <authorizationEntry topic=”mcollective.>” write=”mcollective” read=”mcollective” admin=”mcollective” /> <authorizationEntry queue=”mcollective.>” write=”mcollective” read=”mcollective” admin=”mcollective” /> <authorizationEntry topic=”ActiveMQ.Advisory.>” read=”everyone” write=”everyone” admin=”everyone”/> </authorizationEntries> </authorizationMap> </map> </authorizationPlugin> </plugins> #修改连接端口标签，只保留需要的即可（另外要注意的是注释附号内部不能包含注释符号） <transportConnectors> <transportConnector name=”stomp” uri=”stomp://0.0.0.0:61613″/> </transportConnectors> |
Reloading httpd: not reloading due to configuration syntax error
自动生成的httpd配置会导致httpd启动报错，proxy配置不对，可以不用这个配置也能访问
| 1 | [root@puppet ~]# mv /etc/httpd/conf.d/activemq-httpd.conf /etc/httpd/conf.d/activemq-httpd.conf.bak |
4. 启动
| 1 2 3 4 5 6 7 8 9 10 | [root@puppet ~]# service activemq restart [root@puppet ~]# chkconfig activemq on [root@puppet ~]# netstat -tnlp |grep java tcp 0 0 :::61613 :::* LISTEN 5644/java tcp 0 0 :::47248 :::* LISTEN 5644/java tcp 0 0 :::8161 :::* LISTEN 5644/java [root@puppet ~]# tailf /var/log/activemq/activemq.log [root@puppet ~]# iptables -I INPUT -p tcp –dport 8161 -j ACCEPT [root@puppet ~]# iptables -I INPUT -p tcp –dport 61613 -j ACCEPT [root@puppet ~]# service iptables save |
5. 访问
http://IP:8161/admin/
三、安装MC客户端
1. 安装
安装在puppetmaster服务器上，使用root账号运行MC相关命令，采用puppetmaster的证书
MC客户端不需要启动进程
| 1 | [root@puppet ~]# yum -y install mcollective-* |
2. 配置
修改配置前备份下配置文件
| 1 | [root@puppet ~]# mv /etc/mcollective/client.cfg /etc/mcollective/client.cfg.bak |
这里不使用ssl连接activemq，因为生成keystory.jks证书时失败，待解决
| 1 | [root@puppet ~]# vi /etc/mcollective/client.cfg |
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | main_collective = mcollective collectives = mcollective libdir = /usr/libexec/mcollective logfile = /var/log/mcollective.log loglevel = info daemonize = 1 #Active MQ connector settings: connector = activemq direct_addressing = 1 plugin.activemq.pool.size = 1 #activemq服务器地址、端口、用户账号 plugin.activemq.pool.1.host = puppetmaster.ewin.com plugin.activemq.pool.1.port = 61613 plugin.activemq.pool.1.user = mcollective plugin.activemq.pool.1.password = mcopwd123 #启用ssl连接activemq，使用MC服务器的puppet证书 #plugin.activemq.pool.1.ssl = 1 #plugin.activemq.pool.1.ssl.ca = /var/lib/puppet/ssl/certs/ca.pem #plugin.activemq.pool.1.ssl.cert = /var/lib/puppet/ssl/certs/puppetmaster.ewin.com.pem #plugin.activemq.pool.1.ssl.key = /var/lib/puppet/ssl/private_keys/puppetmaster.ewin.com.pem #plugin.activemq.pool.1.ssl.fallback = 0 #SSL seurity plugin settings 安全插件默认为PSK: securityprovider = ssl plugin.ssl_server_public = /var/lib/puppet/ssl/certs/mcollective-servers.pem plugin.ssl_client_public = /var/lib/puppet/ssl/certs/puppetmaster.ewin.com.pem plugin.ssl_client_private = /var/lib/puppet/ssl/private_keys/puppetmaster.ewin.com.pem # Facts factsource = yaml plugin.yaml = /etc/mcollective/facts.yaml |
mcollective-servers.pem证书在下一步中生成。
将文件中的puppet服务器主机名修改成你自己的:
| 1 | :%s/puppetmaster.ewin.com/puppet.yourdomain.cn/g |
四、安装MC服务端
说明：既然有puppet了，就不必一台一台来安装MC服务端了，这里写个mcollective模块推送到puppet agent。
I、Linux系统
1、 创建目录
在puppet master服务器中创建mcollective模块需要的目录
| 1 2 | [root@puppet ~]# mkdir -p /etc/puppet/modules/mcollective/{manifests,templates,files} [root@puppet ~]# mkdir -p /etc/puppet/modules/mcollective/files/pem/clients |
2、 创建类
| 1 | [root@puppet ~]# vi /etc/puppet/modules/mcollective/manifests/init.pp |
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | class mcollective{ #/etc/activemq/activemq.xml中配置的mcollective用户的密码，变量会传递到erb模块中 $activemq_server = “puppetmaster.ewin.com” $mcollective_password = “mcopwd123” package { [‘mcollective’,’mcollective-puppet-agent’,’mcollective-service-agent’]: ensure => installed, } service { ‘mcollective’: ensure => running, enable => true, require => Package[‘mcollective’], } #通过SVN提交的，需要过滤.svn目录 file { ‘/etc/mcollective’: ensure => directory, source => ‘puppet:///modules/mcollective/pem’, ignore => ‘.svn’, owner => root, group => root, mode => ‘0640’, recurse => remote, notify => Service[‘mcollective’], } file { ‘/etc/mcollective/server.cfg’: ensure => file, owner => root, group => root, mode => 400, content => template(“mcollective/server.cfg.erb”), notify => Service[‘mcollective’], } } |
3、 创建配置文件模板
| 1 | [root@puppet ~]# vi /etc/puppet/modules/mcollective/templates/server.cfg.erb |
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | main_collective = mcollective collectives = mcollective libdir = /usr/libexec/mcollective logfile = /var/log/mcollective.log loglevel = info daemonize = 1 connector = activemq direct_addressing = 1 plugin.activemq.pool.size = 1 plugin.activemq.pool.1.host = <%= @activemq_server %> plugin.activemq.pool.1.port = 61613 plugin.activemq.pool.1.user = mcollective plugin.activemq.pool.1.password = <%= @mcollective_password %> securityprovider = ssl plugin.ssl_client_cert_dir = /etc/mcollective/clients plugin.ssl_server_private = /etc/mcollective/server_private.pem plugin.ssl_server_public = /etc/mcollective/server_public.pem #plugin.puppet.resource_allow_managed_resources = true #plugin.puppet.resource_type_whitelist = exec,file factsource = yaml plugin.yaml = /etc/mcollective/facts.yaml registerinterval = 600 |
4、 生成证书
生成共享证书，将会同步给节点
| 1 2 3 4 5 6 7 8 | [root@puppet ~]# puppet cert generate mcollective-servers Notice: mcollective-servers has a waiting certificate request Notice: Signed certificate request for mcollective-servers Notice: Removing file Puppet::SSL::CertificateRequest mcollective-servers at ‘/var/lib/puppet/ssl/ca/requests/mcollective-servers.pem’ Notice: Removing file Puppet::SSL::CertificateRequest mcollective-servers at ‘/var/lib/puppet/ssl/certificate_requests/mcollective-servers.pem’ [root@puppet ~]# cp /var/lib/puppet/ssl/private_keys/mcollective-servers.pem /etc/puppet/modules/mcollective/files/pem/server_private.pem [root@puppet ~]# cp /var/lib/puppet/ssl/public_keys/mcollective-servers.pem /etc/puppet/modules/mcollective/files/pem/server_public.pem [root@puppet ~]# chmod 644 /etc/puppet/modules/mcollective/files/pem/server_* |
将MC客户端（puppetmaster）证书同步给节点
| 1 2 | [root@puppet ~]# cp /var/lib/puppet/ssl/certs/puppetmaster.ewin.com.pem /etc/puppet/modules/mcollective/files/pem/clients/ [root@puppet ~]# chmod 644 /etc/puppet/modules/mcollective/files/pem/clients/* |
5、 节点更新
（1）、未安装foreman
| 1 2 3 4 5 6 7 8 9 10 | [root@puppet ~]# vi /etc/puppet/manifests/puppetagent.pp node “puppetagent.ewin.com” { include mcollective } [root@puppet ~]# vi /etc/puppet/manifests/site.pp import “puppetagent.pp” [root@puppet ~]# vi /etc/puppet/puppet.conf [agent] report = true server = puppetmaster.ewin.com |
节点上Puppet agent进程需要关闭， 会导致agent不主动同步master， 需要在master设置crontab定期执行mco puppet runonce
| 1 2 3 | [root@puppet ~]# service puppet stop [root@puppet ~]# chkconfig puppet off [root@puppet ~]# puppet agent -t |
（2）、已安装foreman
如果事先部署了foreman，需要在foreman页面->配置->Puppet类-从Puppet-Proxy导入->选择联接的环境->主机中编辑puppetagent->类标签中添加mcollective模块->在节点上执行puppet agent -t
| 1 2 3 | [root@puppet ~]# service puppet stop [root@puppet ~]# chkconfig puppet off [root@puppet ~]# puppet agent -t |
II、Windows系统
1、 软件说明
（1）、 ruby
windows上需要安装ruby来运行mcollective
下载rubyinstaller-1.9.3-p551.exe
（2）、 mcollective
mcollective本身并没有相应的windows服务端安装包
http://repo.mornati.net/mcollective/
下载2.3.2下的mcollective_2_3_2_Setup.exe
先安装ruby才能运行bat脚本安装gem包和安装服务
（3）、 puppet插件
https://github.com/puppetlabs/mcollective-puppet-agent/releases
1.7.1版本无法正常加载puppet agent，插件 1.6.1就没问题
解压后将里面的文件复制粘贴到c:\mcollective\plugins\mcollective，提示替换时同意替换
（4）、 ssl密钥
密钥文件将被上传到客户端的以下路径
c:\mcollective\etc\ssl\clients\puppetmaster.ewin.com.pem
c:\mcollective\etc\ssl\server_private.pem
c:\mcollective\etc\ssl\server_public.pem
2、 win-mcollective模块
（1）、模块目录结构
在puppet的module下创建win-mcollective模块目录：
在files目录下还有两个程序没有在上图显示：
mcollective_2_3_2_Setup.exe
rubyinstaller-1.9.3-p551.exe
（2）、 files
下载ruby、mcollective程序，以及解压后的mcollective-puppet-agent插件。
将puppet服务端证书文件存放在files下的相应位置。
（3）、 manifests
init.pp
| 1 2 3 4 5 6 | class win-mcollective{ $activemq_server = “puppetmaster.ewin.com” $mcollective_password = “mcopwd123” $mcollective_path = “C:\Program Files\mcollective” include win-mcollective::install,win-mcollective::config,win-mcollective::service } |
install.pp
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | class win-mcollective::install { if $operatingsystem == “windows” { $filepath = “C:/Users/Administrator/Downloads” $installdir = “C:/Program Files” Exec{ path => [“${installdir}/Ruby193/bin”,”C:/Windows/system32″,”${installdir}/Puppet Labs/Puppet/bin” ] } #安装ruby file { ‘ruby’: path => “${filepath}/rubyinstaller-1.9.3-p551.exe”, ensure => file, owner => ‘Administrator’, mode => ‘0755’, group => ‘Administrators’, source => ‘puppet:///modules/win-mcollective/rubyinstaller-1.9.3-p551.exe’, } exec { ‘ruby’: command => ‘cmd.exe /c Start “puppet-install” /w “C:/Users/Administrator/Downloads/rubyinstaller-1.9.3-p551.exe” /SILENT /DIR=”C:/Program Files/Ruby193″‘, provider => ‘windows’, creates => “${installdir}/Ruby193/bin/ruby.exe”, require=> File[‘ruby’], } #安装mcollective file { ‘mcollective’: path => “${filepath}/mcollective_2_3_2_Setup.exe”, ensure => file, owner => ‘Administrator’, mode => ‘0755’, group => ‘Administrators’, source => ‘puppet:///modules/win-mcollective/mcollective_2_3_2_Setup.exe’, } exec { ‘mcollective’: command => ‘cmd.exe /c Start “puppet-install” /w “C:/Users/Administrator/Downloads/mcollective_2_3_2_Setup.exe” /SILENT /DIR=”C:/Program Files/mcollective”‘, provider => ‘windows’, creates => “${installdir}/mcollective/bin/mco.bat”, require => File[‘mcollective’], } # 安装gem包 exec { ‘install_gems’: command => ‘cmd.exe /c gem install -l -f “C:/Program Files/mcollective/gems/*.gem”‘, provider => ‘windows’, unless => ‘cmd.exe /c gem list –local | findstr win32-dir’, require => [Exec[‘ruby’],Exec[‘mcollective’]], } # 安装服务 exec { ‘install_service’: command => ‘cmd.exe /c C:/Progra~1/mcollective/bin/register_service.bat’, provider => ‘windows’, unless => ‘cmd.exe /c net start | find “The Marionette Collective”‘, require => Exec[‘install_gems’], } #puppet插件和aq连接认证文件 file { “${installdir}/mcollective/plugins/mcollective”: ensure => directory, ignore => ‘.svn’, source_permissions => ignore, source => ‘puppet:///modules/win-mcollective/mcollective-puppet-agent-1.6.1’, recurse => true, require => Exec[‘mcollective’], } file { “${installdir}/mcollective/etc/ssl”: ensure => directory, ignore => ‘.svn’, source_permissions => ignore, source => ‘puppet:///modules/win-mcollective/pem’, recurse => true, require => Exec[‘mcollective’], } } } |
config.pp
| 1 2 3 4 5 6 7 8 | class win-mcollective::config { file { ‘C:\\Progra~1\\mcollective\\etc\\server.cfg’: ensure => present, content => template(“win-mcollective/server.cfg.erb”), notify => Class[‘win-mcollective::service’], require => Class[‘win-mcollective::install’], } } |
service.pp
| 1 2 3 4 5 6 7 | class win-mcollective::service { service { ‘mcollectived’: ensure => running, enable => true, require => Class[‘win-mcollective::config’], } } |
（3）、 templates
server.cfg.erb
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | main_collective = mcollective collectives = mcollective libdir = <%= @mcollective_path %>\plugins;C:\Program Files\Puppet Labs\Puppet\puppet\lib;C:\Program Files\Puppet Labs\Puppet\facter\lib logfile = <%= @mcollective_path %>\mcollective.log loglevel = info daemonize = 1 securityprovider = ssl plugin.ssl_client_cert_dir = <%= @mcollective_path %>\etc\ssl\clients plugin.ssl_server_private = <%= @mcollective_path %>\etc\ssl\server_private.pem plugin.ssl_server_public = <%= @mcollective_path %>\etc\ssl\server_public.pem connector = activemq plugin.activemq.pool.size = 1 plugin.activemq.pool.1.host = <%= @activemq_server %> plugin.activemq.pool.1.port = 61613 plugin.activemq.pool.1.user = mcollective plugin.activemq.pool.1.password = <%= @mcollective_password %> plugin.puppet.command = “C:\Program Files\Puppet Labs\Puppet\bin\puppet.bat” agent plugin.puppet.config = C:\ProgramData\PuppetLabs\puppet\etc\puppet.conf factsource = yaml plugin.yaml = <%= @mcollective_path %>\etc\facts.yaml |
小结：以上完成后就可以使用MCO命令了，详见下一编文章。
五、整合Foreman
1、 说明
要求：Mcollective+ActiveMQ部署完成，且mco puppet -v runonce命令运行成功
实现：在foreman上通过mcollective触发puppet agent命令
2、 设置
在foreman页面上设置puppetrun为true
3、 配置
| 1 | [root@puppet ~]# vi /etc/foreman-proxy/settings.d/puppet.yml |
| 1 2 3 | :enabled: true :puppet_conf: /etc/puppet/puppet.conf :puppet_provider: mcollective |
| 1 2 | [root@puppet ~]# chmod u+w /etc/sudoers.d/foreman-proxy [root@puppet ~]# vi /etc/sudoers.d/foreman-proxy |
| 1 2 | foreman-proxy ALL = NOPASSWD : /usr/bin/puppet cert *, /usr/bin/mco puppet runonce * Defaults:foreman-proxy !requiretty |
重启服务
| 1 | [root@puppet ~]# /etc/init.d/foreman-proxy restart |
4、 计划
前文提到要关闭agent的puppet服务，需要使用计划来执行mco命令，执行同步
| 1 | [root@puppet ~]# crontab -e |
| 1 | SHELL=/bin/bash */1 * * * * /usr/bin/mco puppet runonce * |
测试下每分钟执行命令； 正常后改为*/30每半小时
| 1 | [root@puppet ~]# service crond reload |
查看计划日志
| 1 | [root@puppet ~]# tailf -n 3 /var/log/cron |
| 1 2 3 | Apr 25 16:22:01 puppetmaster CROND[11613]: (root) CMD (/usr/bin/mco puppet runonce *) Apr 25 16:23:01 puppetmaster CROND[11713]: (root) CMD (/usr/bin/mco puppet runonce *) Apr 25 16:24:01 puppetmaster CROND[11823]: (root) CMD (/usr/bin/mco puppet runonce *) |
查看foreman报告
5、 测试
foreman页面上－主机－选择执行－运行puppet：（不选择主机时，“选择执行”下拉菜单不会显示）
查看日志
| 1 2 | [root@puppet ~]# tailf /var/log/foreman-proxy/proxy.log 10.188.1.53 – – [23/Apr/2015 16:19:19] “POST /run HTTP/1.1” 200 – 0.5008 |
开启debug模式，显示更详细的信息，排错的时候使用
| 1 2 | [root@puppet ~]# vi /etc/foreman-proxy/settings.yml :log_level: DEBUG |
六、遇到过的错误
1、 activemq启动不了
activemq not runnin
| 1 | [root@puppet ~]# |
没有装java工具tanukiwrapper
2、节点连接activemq失败
| 1 2 | [root@puppetagent ~]# tailf /var/log/mcollective.log I, [2015-04-23T14:21:44.337723 #11451] INFO — : activemq.rb:129:in `on_connectfail’ TCP Connection to stomp://mcollective@puppetmaster.ewin.com:61613 failed on attempt 9 |
puppet模块mcollective中类定义里的activemq_server值写错了
3、mco命令执行失败
/etc/activemq/activemq.xml和 /etc/mcollective/client.cfg中的用户密码必须一致
4、运行PUPPET失败排错
（1)、查看日志
| 1 | [root@puppet ~]# t |
这是由于在puppetmaster上应用了puppet模块，把puppet.conf配置冲掉了
| 1 | WARN — : Non-null exit code when executing ‘/usr/bin/sudo/usr/bin/mcopuppetrunonce-It-search01-55.ewin.com’ |
http://www.pythian.com/blog/some-observations-of-puppetrun-with-foreman/
（2）、检查puppetmaster
mcollective mcollective-common 是否安装
foreman-proxy 是否正常启动
activemq 是否连接正常
mco ping 是否检测到客户端
mco puppet runonce -I agent.ewin.com 是否成功执行
mco inventory puppetagent.ewin.com 是否成功安装puppet插件
（3）、检查puppetagent
mcollective-puppet mcollective-client 是否安装
/var/log/mcollective.log 查看日志是否有错误信息
系统时间是否正常
1. 本站所有资源来源于用户上传和网络，因此不包含技术服务请大家谅解！如有侵权请邮件联系客服！10210454@qq.com
2. 本站不保证所提供所有下载的资源的准确性、安全性和完整性，资源仅供下载学习之用！如有链接无法下载、失效或广告，请联系客服处理，有奖励！
3. 您必须在下载后的24个小时之内，从您的电脑中彻底删除上述内容资源！如用于商业或者非法用途，与本站无关，一切后果请用户自负！
4. 如果您也有好的资源或教程，您可以投稿发布，成功分享后有RB奖励和额外RMB收入！
磊宇堂正在使用的服务器 维护管理由磊宇云服务器提供支持
磊宇堂 » Puppet整合Foreman(四)：整合Mcollective
2. 本站不保证所提供所有下载的资源的准确性、安全性和完整性，资源仅供下载学习之用！如有链接无法下载、失效或广告，请联系客服处理，有奖励！
3. 您必须在下载后的24个小时之内，从您的电脑中彻底删除上述内容资源！如用于商业或者非法用途，与本站无关，一切后果请用户自负！
4. 如果您也有好的资源或教程，您可以投稿发布，成功分享后有RB奖励和额外RMB收入！
磊宇堂正在使用的服务器 维护管理由磊宇云服务器提供支持
磊宇堂 » Puppet整合Foreman(四)：整合Mcollective
