以下所有操作均使用root用户 1 解压cloudera-manager-el6-cm5.12.1_x86_64.tar.gz [root@hadoop102 module]# mkdir /opt/module/cloudera-manager 3 配置CM Agent 修改文件/opt/module/cloudera-manager/cm-5.12.1/etc/cloudera-scm-agent/ config.ini [root@hadoop102 cloudera-scm-agent]# vim /opt/module/cloudera-manager/cm-5.12.1/etc/cloudera-scm-agent/config.ini server_host [root@hadoop102 module]# scp -r /opt/module/cloudera-manager/ hadoop103:/opt/module/ [root@hadoop102 module]# scp -r /opt/module/cloudera-manager/ hadoop104:/opt/module/ 6 创建Parcel-repo 目录 1.
Cloudera Manager安装 3.1 部署CM Server & Agent 在所有节点创建CM目录/opt/cloudera-manager mkdir /opt/cloudera-manager /目录 tar -zxvf /usr/local/src/cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/cloudera-manager / 在所有slave节点修改/opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini的server_host参数为master 目录赋权 chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager 3.3 设置开机启动 在master节点设置系统服务 # 设置使用" cloudera-scm-server"为系统启动服务 cp /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-server /etc/rc.d
、MySql node2 (Agents) 192.168.252.122 jdk、cloudera-manager node3 (Agents) 192.168.252.123 jdk、cloudera-manager node4 (Agents) 192.168.252.124 jdk、cloudera-manager node5 (Agents) 192.168.252.125 jdk、cloudera-manager node6 (Agents) 192.168.252.126 jdk、cloudera-manager node7 (Agents) 192.168.252.127 jdk、cloudera-manager mkdir /opt/cloudera-manager 把下载好的cloudera-manager-centos7-cm5.12.1_x86_64.tar.gz安装包上传至 node1 节点/opt *.tar.gz -C /opt/cloudera-manager 2、创建用户 cloudera-scm(所有节点) cloudera-scm 用户说明,摘自官网: Cloudera Manager
、MySql node2 (Agents) 192.168.252.122 jdk、cloudera-manager node3 (Agents) 192.168.252.123 jdk、cloudera-manager node4 (Agents) 192.168.252.124 jdk、cloudera-manager node5 (Agents) 192.168.252.125 jdk、cloudera-manager node6 (Agents) 192.168.252.126 jdk、cloudera-manager node7 (Agents) 192.168.252.127 jdk、cloudera-manager mkdir /opt/cloudera-manager 把下载好的cloudera-manager-centos7-cm5.12.1_x86_64.tar.gz安装包上传至 node1 节点/opt *.tar.gz -C /opt/cloudera-manager 2、创建用户 cloudera-scm(所有节点) cloudera-scm 用户说明,摘自官网: Cloudera Manager
五、安装Clouder Manager及CDH 所有节点操作: 1.新建目录 $ sudo mkdir /opt/cloudera-manager 2.将下载的Clouder Manager解压到此目录下 $ sudo tar xzf cloudera-manager*.tar.gz -C /opt/cloudera-manager 3.创建用户cloudera-scm 由于Cloudera Manager 和Managed Services默认使用cloudera-scm,所以需要创建此用户 $ sudo useradd --system --home=/opt/cloudera-manager/cm-5.14.1 /cm-5.14.1/etc/cloudera-scm-server Creating SCM configuration file in /opt/cloudera-manager/cm-5.14.1 可以使用命令: /opt/cloudera-manager/cm-5.14.1/share/cmf/schema/scm_prepare_database.sh mysql -utemp -ptemp
tar xvzf cloudera-manager*.tar.gz -C /opt/cloudera-manager 2、创建用户cloudera-scm(node1,node2,node3) useradd grant all on *.* to 'temp'@'%' identified by 'temp' with grant option; cd /opt/cloudera-manager/cm-5.4.3 7、启动CM Server、Agent 启动server cd /opt/cloudera-manager/cm-5.4.3/etc/init.d/(node1) . 可以监控目录log cd /opt/cloudera-manager/cm-5.4.3/log/cloudera-scm-server ? 直到如上信息方位启动成功,时间可能有点长需要耐心等待!!! 启动agent(node1,node2,node3) cd /opt/cloudera-manager/cm-5.4.3/etc/init.d/ .
tar -zxvf /data01/cloudera-manager-centos7-cm5.7.2_x86_64.tar.gz -C /opt/cloudera-manager # 授权 useradd --system --home=/opt/cloudera-manager/cm-5.7.2/run/cloudera-scm-server --no-create-home --shell=/bin */server_host=hadoop1/g" /opt/cloudera-manager/cm-5.7.2/etc/cloudera-scm-agent/config.ini # 确认修改成 显示 # 设置数据库驱动(创建软连接) ln -s /usr/share/java/mysql-connector-java.jar /opt/cloudera-manager/cm-5.7.2/share status # 在所有节点上 /opt/cloudera-manager/cm-5.7.2/etc/init.d/cloudera-scm-agent start /opt/cloudera-manager
psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb 创建目录: mkdir /opt/cloudera-manager 把前面准备好的文件cloudera-manager-centos7-cm5.7.2_x86_64.tar.gz上传到master,执行以下命令将该文件解压到/opt/cloudera-manager目录 : tar -zxvf cloudera-manager-centos7-cm5.7.2_x86_64.tar.gz -C /opt/cloudera-manager/ 创建账号cloudera-scm no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm 编辑文件config.ini: vi /opt/cloudera-manager
cloudera-manager.repo 安装相应的rpm包构建工具: yum install createrepo yum-utils httpd 缓存repo中的所有软件包: reposync -r cloudera-manager 命令执行完后会在当前目录生成cloudera-manager/RPMS/x86_64/目录。 此目录下为缓存的rpm包 将生成的rpm包用来构建本地repo源: cd cloudera-manager/ ;createrepo .
四、安装 cloudera-manager-daemons (1)创建 /opt/cloudera-manager 目录,用于存放 cdh的安装文件 mkdir /opt/cloudera-manager 如果安装不成功,可能是Java版本不合适或者Java环境变量没配成功 推荐官方版本:oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm cd /opt/cloudera-manager 六、安装 cloudera-manager-agent(离线) (1)如果直接使用 rpm -ivh 命令安装agent,会出现报错:Linux缺少依赖 cd /opt/cloudera-manager (2)如果直接安装不成功,则使用 --nodeps --force 命令安装 --nodeps:安装时不检查依赖关系 --force:强制安装 cd /opt/cloudera-manager HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" 七、主节点上安装 cloudera-manager-server 1、安装 cloudera-manager-server cd /opt/cloudera-manager
【所有节点】创建目录及解压 mkdir /opt/cloudera-manager tar -xzvf cm6.3.1-redhat7.tar.gz -C /opt/cloudera-manager/ 1.2.选择sc01为cm server,不下载依赖包直接部署(服务节点) cd /opt/cloudera-manager/cm6.3.1/RPMS/x86_64 rpm -ivh cloudera-manager-daemons 6.3.1-1466458.el7.x86_64.rpm --nodeps --force 1.3.所有节点(包含sc01)为cm agent,不下载依赖包直接部署(cm_agent节点) cd /opt/cloudera-manager
.* to 'temp'@'%' identified by 'temp' with grant option; flush privileges; 退出mariadb交互; 进入目录/opt/cloudera-manager /cm-5.7.2/share/cmf/schema/: cd /opt/cloudera-manager/cm-5.7.2/share/cmf/schema/ 执行以下命令即可初始化CM的数据: . 启动server(master机器) SSH登录master; 进入启动命令所在目录: cd /opt/cloudera-manager/cm-5.7.2/etc/init.d/ 启动server: . 启动agent(worker1和worker2机器) SSH登录worker1; 进入启动命令所在目录: cd /opt/cloudera-manager/cm-5.7.2/etc/init.d/ 启动
1、安装CM 解压cm tar包到指定目录,先创建目录,命令操作如下: [root@cdh01 ~]mkdir /opt/cloudera-manager [root@cdh01 ~]tar - zxvf cloudera-manager-centos7-cm5.7.2_x86_64.tar.gz -C /opt/cloudera-manager 创建cloudera-scm用户 #创建 4.初始化数据库 此操作在主节点上进行,初始脚本配置数据库scm_prepare_database.sh,操作命令如下: [root@cdh01 ~]/opt/cloudera-manager/ 进入/opt/cloudera-manager/cm-5.7.2/etc/init.d/目录中,执行如下命令: . 进入/opt/cloudera-manager/cm-5.7.2/etc/init.d/目录中,执行如下命令: .
[cloudera-manager] name = Cloudera Manager, Version baseurl = http://master/cloudera-repos/cm6/ gpgcheck root@master cm5.16]# yum clean all && yum makecache Loaded plugins: fastestmirror Cleaning repos: base cloudera-manager 00:00:00 (1/15): base/primary_db | 6.0 MB 00:00:00 (2/15): base/other_db | 2.6 MB 00:00:00 (3/15): cloudera-manager /filelists_db | 88 kB 00:00:00 (4/15): cloudera-manager/primary_db | 11 kB 00:00:00 (5/15): base/filelists_db | 7.1 MB 00:00:00 (6/15): cloudera-manager/other_db | 1.1 kB 00:00:00 (7/15): epel/filelists_db | 14
.* TO 'cmf'@'localhost' IDENTIFIED BY 'cmfpassword'";" 阶段3:安装Cloudera Manager服务器 # 配置cloudera-cdh源和cloudera-manager 源 # cloudera-manager wget http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/cloudera-manager.repo # cloudera-cdh wget https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/cloudera-cdh5.repo # 安装jdk和cloudera-manager
.* to 'oozie'@'%' identified by 'password'; 2.3 CDH安装 mkdir /opt/cloudera-manager 将cloudera-manager-agent cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm 、cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm 复制到/opt/cloudera-manager
/root/cdh1/mysql-connector-java-8.0.22-1.el7.noarch.rpm 安装完成后,在/usr/share/java下有如下文件 image.png 3.安装cloudera-manager 同时该目录下会多一个文件夹 12.png 创建文件cloudera-manager.repo(需要所有节点都有这个文件) vim /etc/yum.repos.d/cloudera-manager.repo 加入以下内容 [cloudera-manager cloudera-manager.repo cdh03:/etc/yum.repos.d/ 清楚yum缓存 yum clean all 将安装包进行缓存 yum makecache image.png 这三个出现说明成功 3.2安装cloudera-manager parcel | awk '{ print $1 }' > CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha 校验完成后会出现图中的文件 13.png 4.启动cloudera-manager 5. cloudera-manager界面配置 登陆http://cdh:7180 默认密码:admin/admin image.png 点击右下角继续 14.png 勾选,点击继续 15.png 选择免费版
请务必注意CDP Data Center的安装前置条件,请到https://docs.cloudera.com/cloudera-manager/7.1.1/installation/topics/cdpdc-requirements-supported-versions.html 关于特定组件的硬件需求的更多信息,请参考https://docs.cloudera.com/cloudera-manager/7.1.1/installation/topics/cdpdc-hardware-requirements.html 操作系统 目前支持的操作系统为:RHEL/CentOS/ OL with RHCK Kernel 7.6/7.7 https://docs.cloudera.com/cloudera-manager 更多详细信息参考:https://docs.cloudera.com/cloudera-manager/7.1.1/installation/topics/cdpdc-networking-security-requirements.html 参考:https://docs.cloudera.com/cloudera-manager/7.1.1/installation/topics/cdpdc-requirements-supported-versions.html
Agent * * * # node1上传所需要的所有资料( 图1 ) # 1、安装Cloudera Manager Server、Agent(所有节点: 这个是需要自己推敲的) mkdir /opt/cloudera-manager # (node01,node02,node03) tar zxf cloudera-manager*.tar.gz -C /opt/cloudera-manager(图2) # 2、配置CM Agent # (node1) 修改文件/opt/cloudera-manager/cm-5.4.3/etc/cloudera-scm-agent/config.ini中server_host server.host ## 授权 grant all on *.* to 'temp'@'%' identified by 'temp' with grant option; ## 进入到cm的指定目录 cd /opt/cloudera-manager # 7、分发 cd /opt/cloudera-manager/ scp -r ./* root@node02:`pwd` scp -r ./* root@node03:`pwd` ## 在复制时
* --nodeps >>>更改yum源 # cd /etc/yum.repos.d/ # rm -rf * # vi /etc/yum.repos.d/cloudera-manager.repo [cloudera-manager