• 配置crmsh的yum仓库，此仓库的RPM包有openSUSE提供，将这个network:ha-clustering:Stable.repo文件直接下载到本地并且命名为crmsh.repo

wget -O /etc/yum.repos.d/crmsh.repo http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/network:ha-clustering:Stable.repo

yum repolist

yum install -y crmsh

crmsh是pacemaker的命令行接口工具，执行help命令，可以查看shell接口所有的一级命令和二级命令，使用cd 可以切换到二级子命令的目录中去，可以执行二级子命令

在集群中的资源有四类：primitive主资源、group组资源、clone克隆资源、ms/master主从资源

在集群中的约束有三种：

①：location位置约束：资源间的依赖/互斥性，定义资源是否运行在同一节点上。score，正值表示要运行在同一节点上，负值则不可。 
②：colocation排列约束：每个节点都有一个score值，正值则倾向于本节点，负值倾向于其他节点，所有节点score比较，倾向于最大值的节点。 
③：order顺序约束：定义资源执行动作的次序，例如vip应先配置，httpd服务后配置。特殊的score值，-inf 负无穷，inf 正无穷。

Overview Help overview for crmsh

Topics Available topics

Description Program description

CommandLine Command line options

Introduction Introduction

Interface User interface

Completion Tab completion

Shorthand Shorthand syntax

Features Features

Shadows Shadow CIB usage

Checks Configuration semantic checks

Templates Configuration templates

Testing Resource testing

Security Access Control Lists (ACL)

Resourcesets Syntax: Resource sets

AttributeListReferences Syntax: Attribute list references

AttributeReferences Syntax: Attribute references

RuleExpressions Syntax: Rule expressions

Reference Command reference

      cd            切换命令层级，例如 cd cluster：切换到cluster二级命令

      help          查看帮助

      ls            列出命令层级和命令

      quit          退出

      report        报告

      status        查看集群状态，可以有参数 help status 可以查看

      up            回到上一个命令层级

      verify        验证集群状态

      assist/       Configuration assistant 配置集群代理

           template Create template for primitives

           weak-bond Create a weak bond between resources

      cib/          CIB shadow management

           cibstatus      CIB status management and editing (CIB状态管理和编辑)

           commit         Copy a shadow CIB to the cluster  (将影子CIB复制到集群)

           delete         Delete a shadow CIB   (删除阴影CIB)

           diff           Diff between the shadow CIB and the live CIB (阴影CIB和活动CIB的区别)

           import         Import a CIB or PE input file to a shadow  (将CIB或PE输入文件导入到阴影中)

           list           List all shadow CIBs    (列出所有阴影CIB)

           new            Create a new shadow CIB  (创建一个新的CIB)

           reset          Copy live cib to a shadow CIB  (将活动CIB复制到影子CIB)

           use            Change working CIB   (改变工作CIB)

      cibstatus/   CIB status management and editing

           load           Load the CIB status section

           node           Change node status

           op             Edit outcome of a resource operation

           origin         Display origin of the CIB status section

           quorum         Set the quorum

           run            Run policy engine

           save           Save the CIB status section

           show           Show CIB status section

           simulate       Simulate cluster transition

           ticket         Manage tickets

      cluster/    Cluster setup and management （重要）

           add            给集群添加节点

           copy           拷贝文件到另外一个集群的节点

           diff           Diff file across cluster

           geo_init       Configure cluster as geo cluster

           geo_init_arbitrator Initialize node as geo cluster arbitrator

           geo_join       Join cluster to existing geo cluster

           health         集群的健康状态检查

           init           初始化集群

           join           加入到已经存在的集群

           remove         从集群中移除一个节点

           run            在所有集群节点执行一条命令

           start          启动集群服务

           status         集群状态检查

           stop           停止集群服务

           wait_for_startup Wait for cluster to start

      configure/   CIB configuration cib 集群配置（很重要，最重要的命令用中文注释了）

           acl_target     Define target access rights

           alert          Event-driven alerts

           cib            CIB shadow management

           cibstatus      CIB status management and editing

           clone          定义克隆资源

           colocation     设置排列约束

           commit         Commit the changes to the CIB

           default-timeouts Set timeouts for operations to minimums from the meta-data

           delete         Delete CIB objects

           edit           编辑集群配置文件，cib文件

           erase          Erase the CIB

           fencing_topology Node fencing order

           filter         Filter CIB objects

           get_property   Get property value

           graph          Generate a directed graph

           group          定义组资源

           load           Import the CIB from a file

           location       设置位置约束

           modgroup       Modify group

           monitor        Add monitor operation to a primitive

           ms             定义主从资源

           node           Define a cluster node

           op_defaults    Set resource operations defaults

           order          设置顺序约束

           primitive      定义主资源

           property       设置集群的属性

           ptest          Show cluster actions if changes were committed

           refresh        Refresh from CIB

           rename         Rename a CIB object

           role           Define role access rights

           rsc_defaults   Set resource defaults

           rsc_template   Define a resource template

           rsc_ticket     Resources ticket dependency

           rsctest        Test resources as currently configured

           save           Save the CIB to a file

           schema         Set or display current CIB RNG schema

           set            Set an attribute value

           show           Display CIB objects

           tag            Define resource tags

           template       Edit and import a configuration from a template

           upgrade        Upgrade the CIB

           user           Define user access rights

           validate-all   Help for command validate-all

           validate_all   Call agent validate-all for resource

           verify         Verify the CIB with crm_verify

           xml            Raw xml

      corosync/ Corosync management Corosync的管理

           add-node       Add a corosync node

           del-node       Remove a corosync node

           diff           Diffs the corosync configuration

           edit           Edit the corosync configuration

           get            Get a corosync configuration value

           log            Show the corosync log file

           pull           Pulls the corosync configuration

           push           Push the corosync configuration

           reload         Reload the corosync configuration

           set            Set a corosync configuration value

           show           Display the corosync configuration

           status         Display the corosync status

      history/       Cluster history 设置集群的历史信息

           detail         Set the level of detail shown

           diff           Cluster states/transitions difference

           events         Show events in log

           exclude        Exclude log messages

           graph          Generate a directed graph from the PE file

           info           Cluster information summary

           latest         Show latest news from the cluster

           limit          Limit timeframe to be examined

           log            Log content

           node           Node events

           peinputs       List or get PE input files

           refresh        Refresh live report

           resource       Resource events

           session        Manage history sessions

           setnodes       Set the list of cluster nodes

           show           Show status or configuration of the PE input file

           source         Set source to be examined

           transition     Show transition

           transitions    List transitions

           wdiff          Cluster states/transitions difference

     maintenance/  Maintenance mode commands 维护模式的命令

          action         Invoke a resource action

          off            Disable maintenance mode

          on             Enable maintenance mode

     node/        Node management 节点管理（很重要）

         attribute      管理、查看节点属性

         clearstate     Clear node state

         delete         删除节点

         fence          Fence 一个节点

         maintenance    Put node into maintenance mode

         online         让节点上线, 例如：node online node2

         ready          Put node into ready mode

         server         Show node hostname or server address

         show           查看集群中的节点

         standby        让一个节点standy，例如：node standby node1

         status         查看集群中节点的状态

         status-attr    管理集群中节点的属性

         utilization    Manage utilization attributes

     options/     User preferences

         add-quotes     Add quotes around parameters containing spaces

         check-frequency When to perform semantic check

         check-mode     How to treat semantic errors

         colorscheme    Set colors for output

         editor         Set preferred editor program

         manage-children How to handle children resource attributes

         output         Set output type

         pager          Set preferred pager program

         reset          Reset user preferences to factory defaults

         save           Save the user preferences to the rc file

         set            Set the value of a given option

         show           Show current user preference

         skill-level    Set skill level

         sort-elements  Sort CIB elements

         user           Set the cluster user

         wait           Synchronous operation

     ra/       资源代理管理操作（很重要）

         classes        列出资源类别和提供者，例如：ra classes

         info           查看脚本的使用方法，例如：ra info IPaddr2

         list           列出一个类别中有多少可用的资源代理，例如：ra list lsb 或 ra list ocf

         providers      查看特定资源代理有哪个提供者提供，例如：ra providers IPaddr2

         validate       Validate parameters for RA

     resource/    Resource management 资源管理（重要）

         ban            Ban a resource from a node

         cleanup        Cleanup resource status

         clear          Clear any relocation constraint

         constraints    Show constraints affecting a resource

         demote         Demote a master-slave resource

         failcount      Manage failcounts

         locate        Show the location of resources

         maintenance   Enable/disable per-resource maintenance mode

         manage        Put a resource into managed mode

         meta          Manage a meta attribute

         move          将某个资源迁移到哪个节点上去

         operations    Show active resource operations

         param         Manage a parameter of a resource

         promote       Promote a master-slave resource

         refresh       Refresh CIB from the LRM status

         reprobe       Probe for resources not started by the CRM

         restart       Restart resources

         scores        Display resource scores

         secret        Manage sensitive parameters

         start         Start resources

         status        Show status of resources

         stop          Stop resources

         trace         Start RA tracing

        unmanage       Put a resource into unmanaged mode

        untrace        Stop RA tracing

        utilization    Manage a utilization attribute

script/       Cluster script management

        json          JSON API for cluster scripts

        list          List available scripts

        run           Run the script

        show          Describe the script

        verify        Verify the script

site/         GEO clustering site support

        ticket       Manage site tickets

template/     Edit and import a configuration from a template

       apply          Process and apply the current configuration to the current CIB

       delete         Delete a configuration

       edit           Edit a configuration

       list           List configurations/templates

       load           Load a configuration

       new            Create a new configuration from templates

       show           Show the processed configuration