第零套(流水线springcloud)-docer
该公司决定采用GitLab +Jenkins来构建CICD环境，以缩短新功能开发上线周期，及时满足客户的需求，实现DevOps的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成GitLab +Jenkins的CICD环境部署。
【题目1】安装Jenkins环境
在master节点上使用镜像jenkins/jenkins:2.262-centos部署Jenkins服务，具体要求如下：
（1）容器名称：jenkins；
（2）端口映射：8080:80；
（3）使用root身份生成容器；
（4）离线安装Jenkins插件；
（5）设置Jenkins用户：chinaskill；密码：000000；
（6）配置权限“任何用户可以做任何事(没有任何限制)”。
使用chinaskill用户登录Jenkins，完成后提交master节点的用户名、密码和IP到答题框。
#Harbor仓库有jenkins镜像，可以直接运行
docker run -itd -p 8080:8080 --name jenkins jenkins/jenkins:2.262-centos
`#Harbor仓库没有jenkins镜像，可以直接解压jenkins_offline.tar
tar -zxvf jenkins_offline.tar -C /opt/
cd /opt/
docker load -i jenkins.tar
docker run -d --name jenkins -p 8080:8080 -u root \
-v /home/jenkins_home:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $(which docker):/usr/bin/docker \
-v /usr/bin/kubectl:/usr/local/bin/kubectl \
-v /root/.kube:/root/.kube \
jenkins/jenkins:2.262-centos
复制插件到Jenkins
cp -frv plugins/ /home/jenkins_home/
复制完重启容器
挂载后 不行就直接docker cp /usr/bin/docker jenkins:/usr/bin/
docker restart jenkins
查看密码 进入容器或者查看挂在在宿主的目录
docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword
docker exec -it jenkins cat /var/jenkins_home/secrets/initialAdminPassword
cat /home/jenkins_home/secrets/initialAdminPassword
-v 宿主机：容器目录
docker image prune 删除临时镜像
【题目2】安装Gitlab环境
在master节点上使用镜像gitlab/gitlab-ce:12.9.2-ce.0部署Gitlab服务，具体要求如下：
（1）容器名称：mygitlab；
（2）端口映射：1022:22、81:80、443:443；
（3）重启策略：always；
（4）设置Gitlab用户：root；密码：00000000；
（5）新建项目：ChinaskillProject；
（6）将/opt/ChinaskillProject中的代码上传到ChinaskillProject项目中。
docker run -itd --name gitlab --hostname 192.168.100.101 -p 1022:22 -p 81:80 -p 443:443 --restart always -u root gitlab/gitlab-ce:12.9.2-ce.0
一响应超时 1.修改站点
docker exec -it gitlab bash
vi /etc/gitlab/gitlab.rb
gitlab.rb的[字段] ：external_url 'http://192.168.100.101
修改后，出现502的解决方案：
执行：docker exec gitlab gitlab-ctl reconfigure
2. 端口占用修改:`782` unicorn['port'] = 8080为unicorn['port'] = 9090
3.今早用管理帐号登录gitlab,登录后页面提示 422
Error 422 The change you requested was rejected on login
搜索前人经验，无果，尝试用其他浏览器登录，发现登录正常！
后来回想是昨天清空了chrome浏览器缓存,而且登录页面一直没有关闭直接再登录引起的。关闭登录页面后过一段时间再次登录就OK了
mkdir ChinaskillProject
# cd /opt/ChinaskillProject/
# yum install -y git
# rm -rf .git
# git config --global user.name "Administrator"
# git config --global user.email"admin@example.com"
# git init
Reinitialized existing Git repository in /opt/ChinaskillProject/.git/
`# git remote add origin http://192.168.100.101:81/root/chinaskillproject.git `
# git add .
# git commit -m "Initial commit"
On branch master
nothing to commit, working directory clean
[root@master ChinaskillProject]# git push -u origin master
Username for 'http://192.168.200.3:81': root
Password for 'http://root@192.168.200.3:81':
Counting objects: 3194, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (1430/1430), done.
Writing objects: 100% (3194/3194), 1.40 MiB | 2.61 MiB/s, done.
Total 3194 (delta 1235), reused 3009 (delta 1207)
remote: Resolving deltas: 100% (1235/1235), done.
To http://192.168.200.3:81/root/chinaskillproject.git
* [new branch] master -> master
Branch master set up to track remote branch master from origin.
推送报错git remote add origin git@192.168.100.101:root/d.git git remote -v 查看 origin是否正确 git remote remove origin删除
1.端口修改git remote add origin git@192.168.100.101:81/root/d.git
2.站点修改git remote add origin http://192.168.100.101:81/root/d.git
【题目3】配置Jenkins连接Gitlab
配置Jenkins连接Gitlab，具体要求如下：
（1）设置Outbound requests；
（2）生成“Access Tokens”；
（3）设置Jenkins取消对'/project' end-point进行身份验证；
（4）测试Jenkins与Gitlab的连通性。
AdminArea-->Settings-->Network-->Outbound requests
User Settings-->Access Tokens
系统管理-->系统配置-->Gitlab-->添加
#### 【题目4】配置Jenkins连接maven
配置Jenkins连接maven，具体要求如下：
（1）采用docker in docker的方式在Jenkins内安装maven；
（2）在Jenkins中配置maven信息。
[root@master jenkins]# tar -xvf apache-maven-3.6.3-bin.tar.gz
`[root@master jenkins]# mv apache-maven-3.6.3 maven
`[root@master jenkins]# docker cp maven jenkins:/usr/local/
`[root@master jenkins]# docker cp repository/ jenkins:/root/.m2/repository
和下面一样
`#cp -rf /opt/apache-maven-3.6.3-bin.tar.gz /home/jenkins_home/
[root@master jenkins]# docker exec -it jenkins /bin/bash
`#tar -zxvf /var/jenkins_home/apache-maven-3.6.3-bin.tar.gz -C .
`mv apache-maven-3.6.3/ /usr/local/maven
[root@e9a5662aca40 ~]# vi /etc/profile # 行末添加两行
export M2_HOME=/usr/local/maven
export PATH=$PATH:$M2_HOME/bin
[root@e9a5662aca40 ~]# vi /root/.bashrc
# .bashrc
# User specific aliases and functions
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
source /etc/profile #<---此处添加
fi
[root@e9a5662aca40 ~]# source /etc/profile
[root@e9a5662aca40 ~]# mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/maven
Java version: 1.8.0_265, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.265.b01-0.el8_2.x86_64/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.10.0-862.el7.x86_64", arch: "amd64", family: "unix"
连接Maven
切换至Jenkins首页，单击“系统管理→全局工具配置”
【题目5】配置CI/CD
配置CI/CD，具体要求如下：
（1）新建一个流水线任务ChinaskillProject；
（2）编写流水线脚本；
（3）配置Webhook；
（4）在Harbor中新建公开项目chinaskillproject。
新建任务
点击流水线语法
单击“添加→Jenkins”添加凭据，如图2-35所示。类型选择“Username with password”，用户名和密码为GitLab仓库的用户名和密码。
生成流水线脚本
项目地址按的 路径查找
选择Projects：1
导航栏中选择项目
记录生成的值，并将其写入流水线脚本中，完整的流水线脚本如下：
node{
stage('git clone'){
git credentialsId: '96caabb9-6b40-49e8-b897-283bb851a1fa', url: 'http://192.168.100.101:81/root/chinaskillproject.git'
}
stage('maven build'){
sh '''
/usr/local/maven/bin/mvn package -DskipTests -f config
/usr/local/maven/bin/mvn package -DskipTests -f gateway
'''
}
stage('image build'){
sh '''
echo $BUILD_ID
docker build -t 192.168.100.101/chinaskillproject/config:$BUILD_ID -f config/Dockerfile config
docker build -t 192.168.100.101/chinaskillproject/gateway:$BUILD_ID -f gateway/Dockerfile gateway
'''
}
stage('upload image'){
sh '''
docker login 192.168.100.101 -uadmin -pHarbor12345
docker push 192.168.100.101/chinaskillproject/config:$BUILD_ID
docker push 192.168.100.101/chinaskillproject/gateway:$BUILD_ID
'''
}
stage('deploy Rancher'){
sh '''
sed -i "s/sqshq\\/piggymetrics-config/192.168.100.101\\/chinaskillproject\\/config:$BUILD_ID/g" yaml/deployment/config-deployment.yaml
sed -i "s/sqshq\\/piggymetrics-gateway/192.168.100.101\\/chinaskillproject\\/gateway:$BUILD_ID/g" yaml/deployment/gateway-deployment.yaml
kubectl create ns chinaskillproject
kubectl apply -f yaml/deployment/config-deployment.yaml
kubectl apply -f yaml/deployment/gateway-deployment.yaml
kubectl apply -f yaml/svc/config-svc.yaml --kubeconfig=/root/.kube/config
kubectl apply -f yaml/svc/gateway-svc.yaml --kubeconfig=/root/.kube/config
'''
}
}
流水线2
\[
node{
stage('git clone'){
//check CODE
git credentialsId: '5c5bc5ac-1bc6-4302-9c45-0edc1762c2a4', url: 'http://192.168.100.121:81/root/chinaskillproject'
}
stage('maven build'){
sh '''/usr/local/maven/bin/mvn package -DskipTests -f /var/jenkins_home/workspace/ChinaskillProject'''
}
stage('image build'){
sh '''
echo $BUILD_ID
docker build -t 192.168.100.121/chinaskillproject/gateway:$BUILD_ID -f /var/jenkins_home/workspace/chinaskillproject/gateway/Dockerfile /var/jenkins_home/workspace/chinaskillproject/gateway
docker build -t 192.168.100.121/chinaskillproject/config:$BUILD_ID -f /var/jenkins_home/workspace/chinaskillproject/config/Dockerfile
/var/jenkins_home/workspace/chinaskillproject/config'''
}
stage('upload registry'){
sh '''docker login 192.168.100.121 -u=admin -p=Harbor12345
docker push 192.168.100.121/chinaskillproject/gateway:$BUILD_ID
docker push 192.168.100.121/chinaskillproject/config:$BUILD_ID'''
}
stage('deploy Rancher'){
//执行部署脚本
sh 'sed -i "s/sqshq\\/piggymetrics-gateway/192.168.100.121\\/chinaskillproject\\/gateway:$BUILD_ID/g" /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/gateway-deployment.yaml'
sh 'sed -i "s/sqshq\\/piggymetrics-config/192.168.100.121\\/chinaskillproject\\/config:$BUILD_ID/g" /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/config-deployment.yaml'
sh 'kubectl create ns springcloud'
sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/gateway-deployment.yaml --kubeconfig=/root/.kube/config'
sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/deployment/config-deployment.yaml --kubeconfig=/root/.kube/config'
sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/svc/gateway-svc.yaml --kubeconfig=/root/.kube/config'
sh 'kubectl apply -f /var/jenkins_home/workspace/ChinaskillProject/yaml/svc/config-svc.yaml --kubeconfig=/root/.kube/config'
}
}
\]
脚本中所有网段IP均为Harbor仓库的地址，需要根据实际修改。
复制脚本内容至流水线脚本中，删除上述代码中斜体字的内容，并将上述代码中加粗的内容修改为生成的流水线脚本中的内容。
Projects-->chinaskillproject-->Settings-->Webhooks
将 chinaskillproject项目中的代码上传到Gitlab 中触发构建， 。
1）触发构建
上传代码触发自动构建：
[root@master ~]# docker cp /opt/repository/ jenkins:/root/.m2/
[root@master ~]# cd /opt/ChinaskillProject/
[root@master ChinaskillProject]# git add .
[root@master ChinaskillProject]# git commit -m "Initial commit"
[master ec88bf4] Initial commit
1 file changed, 33 insertions(+)
create mode 100644 test
[root@master ChinaskillProject]# git push -u origin master
Username for 'http://10.24.2.37:81': root
Password for 'http://root@10.24.2.37:81':
Branch master set up to track remote branch master from origin.
Everything up-to-date
查看端口 kubectl get svc -n springcloud
查看pods
kubectl -n springcloud get pods
查看services
kubectl -n springcloud get service
第一套(2021-2022广东省/1)(spingcloud)
GitLab +Jenkins
该公司决定采用 GitLab +Jenkins 来构建CICD 环境，以缩短新功能开发上线周期，及时满足客户的需求，实现 DevOps 的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成 GitLab + Jenkins + Kubernetes 的CICD 环境部署。CICD 应用系统架构如图 2 所示。
在 master 节点上编写/root/jenkins/docker-compose.yaml 文件编排部署 Jenkins 服务， 具体要求为
- 容器名称：jenkins；
- 端口映射：8080:8080；
- 使用 root 身份生成容器；
- 离线安装Jenkins 插件；
- 设置Jenkins 用户：springcloud；密码：000000；
- 在授权策略中配置“任何用户可以做任何事(没有任何限制)”。
[root@master jenkins]# vi docker-compose.yaml version: "3" services: jenkins: container_name: jenkins image: jenkins/jenkins:2.262-centos ports: - 8080:8080 user: root volumes: - /home/jenkins_home:/var/jenkins_home - /usr/bin/docker:/usr/bin/docker - /usr/bin/kubectl:/usr/local/bin/kubectl - /var/run/docker.sock:/var/run/docker.sock - /root/.kube:/root/.kube[root@master jenkins]# cp -rf /opt/plugins/* /home/jenkins_home/plugins/ [root@master jenkins]# docker-compose restart[root@master gitlab]# cat /home/jenkins_home/secrets/initialAdminPassword bb46b54d0713425bbe4c955cbbcb4896
系统管理-->管理用户-->创建用户
系统管理-->全局安全配置-->授权策略
在 master 节点上编写/root/gitlab/docker-compose.yaml 文件编排部署 Gitlab 服务，具体要求为
- 容器名称：gitlab；
- 端口映射：1022:22、81:80、443:443；
- 容器重启策略：always；
- 设置 root 用户及密码；
- 使用 root 用户登录 Gitlab，密码：00000000；
- 新建项目Springcloud，将/opt/Springcloud 中的代码上传到Springcloud 项目中。
[root@master gitlab]# vim docker-compose.yaml version: "3" services: gitlab: container_name: gitlab image: gitlab/gitlab-ce:12.9.2-ce.0 ports: - 1022:22 - 81:80 - 443:443 restart: always
Create a project
[root@master opt]# cd /opt/ChinaskillProject/ [root@master ChinaskillProject]# yum install -y git [root@master ChinaskillProject]# rm -rf .git
[root@master ChinaskillProject]# git config --global user.name "Administrator" [root@master ChinaskillProject]# git config --global user.email "zhang1287609286@qq.com" [root@master ChinaskillProject]# git init [root@master ChinaskillProject]# git remote add origin http://192.168.100.101:81:81/root/springcloud.git [root@master ChinaskillProject]# git add . [root@master ChinaskillProject]# git commit -m "Initial commit" [root@master ChinaskillProject]# git push -u origin master
配置Jenkins 连接 Gitlab，具体要求为
- 设置Outbound requests；
- 生成“Access Tokens”并命名为 jenkins；
- 设置Jenkins 取消对'/project' end-point 进行身份验证；
- 测试Jenkins 与 Gitlab 的连通性。
AdminArea-->Settings-->Network-->Outbound requests
User Settings-->Access Tokens
系统管理-->系统配置-->Gitlab-->添加
配置Jenkins 连接 maven，具体要求为
- 采用docker in docker 的方式在Jenkins 内安装maven；
- 在 Jenkins 中配置 maven 信息。
[root@master jenkins]# ll total 4630368 drwxr-xr-x. 6 root root 99 Jan 22 21:59 apache-maven-3.6.3 -rw-r--r--. 1 root root 9506321 Nov 19 2019 apache-maven-3.6.3-bin.tar.gz dr-xr-xr-x. 13 root root 4096 Nov 4 2020 ChinaskillProject -rw-r--r--. 1 root root 390 Jan 22 04:03 docker-compose.yaml -rw-r--r--. 1 root root 1776615318 Jan 22 08:31 jenkins_offline.tar -rw-------. 1 root root 2955348480 Nov 4 2020 jenkins.tar drwxr-xr-x. 108 root root 8192 Nov 4 2020 plugins drwxr-xr-x. 32 root root 4096 Nov 4 2020 repository [root@master jenkins]# tar -xvf apache-maven-3.6.3-bin.tar.gz [root@master jenkins]# mv apache-maven-3.6.3 maven [root@master jenkins]# docker cp maven jenkins:/usr/local/ [root@master jenkins]# docker cp repository/ jenkins:/root/.m2/repository [root@master jenkins]# docker exec -it jenkins /bin/bash [root@e9a5662aca40 ~]# vi /etc/profile ... export MAVEN_HOME=/usr/local/maven export PATH=$PATH:$MAVEN_HOME/bin [root@e9a5662aca40 ~]# vi /root/.bashrc # .bashrc if [ -f /etc/bashrc ]; then . /etc/bashrc source /etc/profile #<---添加 fi [root@e9a5662aca40 ~]# source /etc/profile [root@e9a5662aca40 ~]# mvn -v Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f) Maven home: /usr/local/maven Java version: 1.8.0_265, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.265.b01-0.el8_2.x86_64/jre Default locale: en_US, platform encoding: ANSI_X3.4-1968 OS name: "linux", version: "3.10.0-862.el7.x86_64", arch: "amd64", family: "unix"系统管理-->全局工具配置-->Maven
配置CI/CD，具体要求为
- 新建一个流水线任务 Springcloud；
- 编写流水线脚本，构建 Springcloud 项目中的 gateway 和 config 服务，将构建后的镜像自动上传到 Harbor 仓库的 springcloud 项目，并自动发布 gateway 和 config 服务到Kubernetes 集群的 springcloud 命名空间下；
- 配置Webhook；
- 在 Harbor 中新建公开项目springcloud。
新建任务
流水线语法
node{ stage('git clone'){ git credentialsId: '67868c28-ce9b-411a-89a7-bdc6d1c257cd', url: 'http://192.168.100.101:81/root/springcloud.git' } stage('maven build'){ sh ''' /usr/local/maven/bin/mvn package -DskipTests -f config /usr/local/maven/bin/mvn package -DskipTests -f gateway ''' } stage('image build'){ sh ''' echo $BUILD_ID docker build -t 192.168.100.101/springcloud/config:$BUILD_ID -f config/Dockerfile config docker build -t 192.168.100.101/springcloud/gateway:$BUILD_ID -f gateway/Dockerfile gateway ''' } stage('upload image'){ sh ''' docker login 192.168.100.101 -uadmin -pHarbor12345 docker push 192.168.100.101/springcloud/config:$BUILD_ID docker push 192.168.100.101/springcloud/gateway:$BUILD_ID ''' } stage('deploy Rancher'){ sh ''' sed -i "s/sqshq\\/piggymetrics-config/192.168.100.101\\/springcloud\\/config:$BUILD_ID/g" yaml/deployment/config-deployment.yaml sed -i "s/sqshq\\/piggymetrics-gateway/192.168.100.101\\/springcloud\\/gateway:$BUILD_ID/g" yaml/deployment/gateway-deployment.yaml kubectl create ns springcloud kubectl apply -f yaml/deployment/config-deployment.yaml kubectl apply -f yaml/deployment/gateway-deployment.yaml kubectl apply -f yaml/svc/config-svc.yaml --kubeconfig=/root/.kube/config kubectl apply -f yaml/svc/gateway-svc.yaml --kubeconfig=/root/.kube/config ''' } }\[ node{ stage('git clone'){ git credentialsId: '67868c28-ce9b-411a-89a7-bdc6d1c257cd', url: 'http://192.168.100.101:81/root/springcloud.git' } stage('maven build'){ sh ''' /usr/local/maven/bin/mvn package -DskipTests -f config /usr/local/maven/bin/mvn package -DskipTests -f gateway ''' } stage('image build'){ sh ''' echo $BUILD_ID docker build -t 192.168.100.101:81/springcloud/config:$BUILD_ID -f config/Dockerfile config docker build -t 192.168.100.101:81/springcloud/gateway:$BUILD_ID -f gateway/Dockerfile gateway ''' } stage('upload image'){ sh ''' docker login 192.168.100.101:81 -uadmin -pHarbor12345 docker push 192.168.100.101:81/springcloud/config:$BUILD_ID docker push 192.168.100.101:81/springcloud/gateway:$BUILD_ID ''' } stage('deploy Rancher'){ sh ''' sed -i "s/sqshq\\/piggymetrics-config/192.168.100.101:81\\/springcloud\\/config:$BUILD_ID/g" yaml/deployment/config-deployment.yaml sed -i "s/sqshq\\/piggymetrics-gateway/192.168.100.101:81\\/springcloud\\/gateway:$BUILD_ID/g" yaml/deployment/gateway-deployment.yaml kubectl create ns springcloud kubectl apply -f yaml/deployment/config-deployment.yaml kubectl apply -f yaml/deployment/gateway-deployment.yaml kubectl apply -f yaml/svc/config-svc.yaml --kubeconfig=/root/.kube/config kubectl apply -f yaml/svc/gateway-svc.yaml --kubeconfig=/root/.kube/config ''' } } \]Projects-->Springcloud-->Settings-->Webhooks
将 Springcloud 项目中的代码上传到Gitlab 中触发构建，完成后提交 master 节点的用户名、密码和 IP 到答题框。
kubectl get svc -n springcloud
第二套（gpmall）
GitLab +Jenkins
该公司决定采用 GitLab +Jenkins 来构建CICD 环境，以缩短新功能开发上线周期，及时满足客户的需求，实现 DevOps 的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成 GitLab + Jenkins + Kubernetes 的CICD 环境部署。CICD 应用系统架构如图 2 所示。
在 master 节点上编写/root/jenkins/docker-compose.yaml 文件编排部署 Jenkins 服务， 具体要求为
- 容器名称：jenkins；
- 端口映射：8080:8080；
- 使用 root 身份生成容器；
- 离线安装Jenkins 插件；
- 设置Jenkins 用户：gpmall；密码：000000；
- 在授权策略中配置“任何用户可以做任何事(没有任何限制)”。
[root@master jenkins]# vi docker-compose.yaml version: '3' services: jenkins: container_name: jenkins image: jenkins/jenkins:2.262-centos ports: - 8080:8080 user: root volumes: - /home/jenkins_home:/var/jenkins_home - /usr/bin/kubectl:/usr/bin/kubectl - /usr/bin/docker:/usr/bin/docker - /var/run/docker.sock:/var/run/docker.sock - /root/.kube:/root/.kube[root@master jenkins]# cp -rf plugins/* /home/jenkins_home/plugins/ [root@master jenkins]# docker-compose restart系统管理-->管理用户-->新建用户
系统管理-->全局安全配置-->授权策略
在 master 节点上编写/root/gitlab/docker-compose.yaml 文件编排部署 Gitlab 服务，具体要求为
- 容器名称：gitlab；
- 端口映射：1022:22、81:80、443:443；
- 容器重启策略：always；
- 设置 root 用户及密码；
- 使用 root 用户登录 Gitlab，密码：00000000；
- 新建项目gpmall，将 gpmall 中的代码上传到gpmall 项目中；
- 实现通过SSH 链接克隆项目。
[root@master gitlab]# vi docker-compose.yaml version: '3' services: gitlab: container_name: gitlab image: gitlab/gitlab-ce:12.9.2-ce.0 ports: - 1022:22 - 81:80 - 443:443 restart: always
[root@master gitlab]# yum install -y git [root@master gitlab]# cd gpmall/ [root@master gpmall]# git config --global user.name "Administrator" [root@master gpmall]# git config --global user.email "admin@example.com" [root@master gpmall]# git init [root@master gpmall]# git remote add origin http://192.168.100.101:81/root/gpmall.git [root@master gpmall]# git add . [root@master gpmall]# git commit -m "Initial commit" [root@master gpmall]# git push -u origin master [root@master gitlab]# ll ~/.ssh/ total 16 -rw------- 1 root root 784 Feb 15 04:55 authorized_keys -rw------- 1 root root 1675 Feb 15 03:40 id_rsa -rw-r--r-- 1 root root 393 Feb 15 03:40 id_rsa.pub -rw-r--r-- 1 root root 1018 Feb 15 03:40 known_hosts [root@master gitlab]# cat ~/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKVQ0U+HTPczSK0QCjFMfAD7UBRVISW3FcIlV9xESZFaMxoy0HNCWQnOtzRcEZkSvJCo9Sxndr5h4I4xmcRfPS65oDWKGI7MrDBE2aqk5fjkOX4ygSYA/0GDVtLNeJx/PDF4FdYfOF/jnsF29MAmFKhYb7T+6cKb+ynfBOO/pZdWQv9mnJA/tOWUCbbuKjJ3QSIkXCHrjn0LNgXplC8DNMPO4W5g6oz0WatZbXwWcDw/hYzK8X03Jbo/ueeX5nk3FUK9ZjTFPvVyTbEpUywEGxXcWuyTaIP8AEdEPM1SkHQ0MDtAUSre/LlaM22O4bAvHEC8bsCStzgpBxiYeQ4U0D root@masterUserSettings-->SSH Keys
[root@master ~]# ssh -T git@192.168.100.101 -p 1022 Welcome to GitLab, @root!配置Jenkins 连接 Gitlab，具体要求为
- 设置Outbound requests；
- 生成“Access Tokens”并命名为 jenkins；
- 设置Jenkins 取消对'/project' end-point 进行身份验证；
- 测试Jenkins 与 Gitlab 的连通性。
Adminarea-->Settings-->Network-->Outbound requests
Usersetting-->AccessTokens
复制
系统管理-->系统配置-->Gitlab
配置CI/CD，具体要求为
- 新建一个流水线任务 gpmall；
- 编写流水线脚本，构建 gpmall 项目中的商城服务，将构建后的镜像自动上传到
- Harbor 仓库的 gpmall 项目，并自动发布服务到 Kubernetes 集群的 default 命名空间下；
- 配置Webhook；
- 在 Harbor 中新建公开项目gpmall。
新建任务
node{ stage('git clone'){ git credentialsId: 'ba7744af-6870-4720-96c9-35f4c73711d1', url: 'http://192.168.100.101:81/root/gpmall' } stage('image build'){ sh ''' docker build -t 192.168.100.101/gpmall/chinaskill-mariadb:$BUILD_ID -f mariadb/Dockerfile mariadb docker build -t 192.168.100.101/gpmall/chinaskill-redis:$BUILD_ID -f redis/Dockerfile redis docker build -t 192.168.100.101/gpmall/chinaskill-kafka:$BUILD_ID -f kafka/Dockerfile kafka docker build -t 192.168.100.101/gpmall/chinaskill-zookeeper:$BUILD_ID -f zookeeper/Dockerfile zookeeper docker build -t 192.168.100.101/gpmall/chinaskill-nginx:$BUILD_ID -f nginx/Dockerfile nginx ''' } stage('upload image'){ sh ''' docker login -u admin -p Harbor12345 192.168.100.101 docker push 192.168.100.101/gpmall/chinaskill-mariadb:$BUILD_ID docker push 192.168.100.101/gpmall/chinaskill-redis:$BUILD_ID docker push 192.168.100.101/gpmall/chinaskill-kafka:$BUILD_ID docker push 192.168.100.101/gpmall/chinaskill-zookeeper:$BUILD_ID docker push 192.168.100.101/gpmall/chinaskill-nginx:$BUILD_ID ''' } stage('deploy project'){ sh ''' sed -i "s/chinaskill-mariadb:v1.1/192.168.100.101\\/gpmall\\/chinaskill-mariadb:$BUILD_ID/g" gpmall.yaml sed -i "s/chinaskill-redis:v1.1/192.168.100.101\\/gpmall\\/chinaskill-redis:$BUILD_ID/g" gpmall.yaml sed -i "s/chinaskill-kafka:v1.1/192.168.100.101\\/gpmall\\/chinaskill-kafka:$BUILD_ID/g" gpmall.yaml sed -i "s/chinaskill-zookeeper:v1.1/192.168.100.101\\/gpmall\\/chinaskill-zookeeper:$BUILD_ID/g" gpmall.yaml sed -i "s/chinaskill-nginx:v1.1/192.168.100.101\\/gpmall\\/chinaskill-nginx:$BUILD_ID/g" gpmall.yaml kubectl apply -f gpmall.yaml ''' } }Settings-->Webhooks
将 gpmall 项目中的 gpmall.yaml 文件补全，然后将gpmall 项目代码重新上传到 Gitlab中触发构建，完成后提交master 节点的用户名、密码和 IP 到答题框。
[root@master gpmall]# git add . [root@master gpmall]# git commit -m "Second Commit" [root@master gpmall]# git push -u origin master
第三套（GZ-2021040/1）
GitLab + GitLab-CI + Harbor + Kubernetes
云梦公司决定采用GitLab + GitLab-CI + Harbor + Kubernetes架构来构建CICD环境，以缩短新功能开发上线周期，及时满足客户的需求，实现DevOps的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成CICD环境部署。CICD应用系统架构如下：
从私有仓库中拉取gitlab:latest镜像，创建gitlab.yaml文件，基于Kubernetes启动GitLab服务，实现web浏览器正常访问GitLab服务。（2分）
apiVersion: v1 kind: Service metadata: name: gitlab spec: selector: app: gitlab type: NodePort ports: - name: http port: 80 nodePort: 30080 - name: https port: 443 nodePort: 30443 - name: ssh port: 22 nodePort: 30022 --- apiVersion: v1 kind: Pod metadata: name: gitlab labels: app: gitlab spec: nodeName: master restartPolicy: Always volumes: - name: config nfs: server: 10.4.7.20 path: /root/data/gitlab/config - name: log nfs: server: 10.4.7.20 path: /root/data/gitlab/log - name: data nfs: server: 10.4.7.20 path: /root/data/gitlab/data - name: docker hostPath: path: /usr/bin/docker - name: dockersock hostPath: path: /var/run/docker.sock - name: kubectl hostPath: path: /usr/bin/kubectl - name: kube hostPath: path: /root/.kube containers: - name: gitlab image: 10.4.7.10/library/gitlab-ce:12.9.2-ce.0 ports: - name: http containerPort: 80 - name: https containerPort: 443 - name: ssh containerPort: 22 volumeMounts: - name: config mountPath: /etc/gitlab - name: log mountPath: /var/logs/gitlab - name: data mountPath: /var/opt/gitlab - name: docker mountPath: /usr/bin/docker - name: dockersock mountPath: /var/run/docker.sock - name: kubectl mountPath: /usr/bin/kubectl - name: kube mountPath: /root/.kube创建gitlab用户（用户名：Chinaskill），创建gitlab项目（项目名：SpringCloud），配置gitlab和master节点SSH免密，实现通过SSH链接克隆项目。（2分）
Add people
CreateProject
[root@master gitlab]# ll ~/.ssh/ total 16 -rw------- 1 root root 784 Feb 15 04:55 authorized_keys -rw------- 1 root root 1675 Feb 15 03:40 id_rsa -rw-r--r-- 1 root root 393 Feb 15 03:40 id_rsa.pub -rw-r--r-- 1 root root 1018 Feb 15 03:40 known_hosts [root@master gitlab]# cat ~/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKVQ0U+HTPczSK0QCjFMfAD7UBRVISW3FcIlV9xESZFaMxoy0HNCWQnOtzRcEZkSvJCo9Sxndr5h4I4xmcRfPS65oDWKGI7MrDBE2aqk5fjkOX4ygSYA/0GDVtLNeJx/PDF4FdYfOF/jnsF29MAmFKhYb7T+6cKb+ynfBOO/pZdWQv9mnJA/tOWUCbbuKjJ3QSIkXCHrjn0LNgXplC8DNMPO4W5g6oz0WatZbXwWcDw/hYzK8X03Jbo/ueeX5nk3FUK9ZjTFPvVyTbEpUywEGxXcWuyTaIP8AEdEPM1SkHQ0MDtAUSre/LlaM22O4bAvHEC8bsCStzgpBxiYeQ4U0D root@masterSettings-->SSH Keys
[root@master gitlab]# yum install -y git [root@master gitlab]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE gitlab NodePort 10.107.179.207 <none> 80:30080/TCP,443:30443/TCP,22:30022/TCP 16m kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20d [root@master gitlab]# ssh -T git@10.107.179.207 The authenticity of host '10.107.179.207 (10.107.179.207)' can't be established. ECDSA key fingerprint is SHA256:7ZLi+9JA5yKtkvm58Cj0PzKTan0ulZLUaiL5BWaf7Ig. ECDSA key fingerprint is MD5:60:99:26:2f:fa:f4:9f:0d:fc:33:c7:0a:7f:9b:0a:27. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.107.179.207' (ECDSA) to the list of known hosts. Welcome to GitLab, @root![root@master SpringCloud]# git config --global user.name "Administrator" [root@master SpringCloud]# git config --global user.email "admin@example.com" [root@master SpringCloud]# git init Reinitialized existing Git repository in /root/gitlab/SpringCloud/.git/ [root@master SpringCloud]# git remote add origin git@10.107.179.207:root/springcloud.git [root@master SpringCloud]# git commit -m "Initial commit" [root@master SpringCloud]# git push -u origin master采用docker in docker的方式启动Runner，并注册Runner。（2分）
安装maven
[root@master gitlab]# ll total 190644 -rwxr-xr-x 1 root root 49683094 Mar 7 07:54 gitlab-runner -rw-r--r-- 1 root root 1487 Mar 7 22:55 gitlab.yaml drwxr-xr-x 8 10143 10143 273 Jun 9 2021 jdk1.8.0_301 drwxr-xr-x 6 root root 99 Mar 7 23:00 maven drwxr-xr-x 32 root root 4096 Mar 7 22:45 repository drwxr-xr-x 13 root root 4096 Mar 7 21:01 SpringCloud [root@master gitlab]# kubectl cp maven gitlab:/usr/local/ [root@master gitlab]# kubectl cp jdk1.8.0_301 gitlab:/usr/local [root@master gitlab]# kubectl cp repository gitlab:/usr/local [root@master gitlab]# kubectl exec -it gitlab bash root@gitlab:/usr/local# vim /etc/profile export JAVA_HOME=/usr/local/jdk1.8.0_301 export MAVEN_HOME=/usr/local/maven export PATH=$PATH:$MAVEN_HOME/bin:/usr/gitlab/embedded/bin root@gitlab:/# mv /usr/local/repository/ /root/.m2/ root@gitlab:/usr/local# . /etc/profile root@gitlab:/usr/local# mvn -v root@gitlab:/opt# mvn -v Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f) Maven home: /usr/local/maven Java version: 1.8.0_301, vendor: Oracle Corporation, runtime: /opt/jdk1.8.0_301/jre Default locale: en, platform encoding: UTF-8 OS name: "linux", version: "5.4.182-1.el7.elrepo.x86_64", arch: "amd64", family: "unix"向GitLab-CI注册一个Runner需要两样东西：GitLab-CI的url和注册token、复制token
安装runner
\[ $ mkdir -p /data/gitlab-runner/config $ docker run -itd --restart=always --name gitlab-runner \ -v /data/gitlab-runner/config:/etc/gitlab-runner \ -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest \][root@master gitlab]# ll total 48528 -rwxr-xr-x 1 root root 49683094 Mar 7 07:54 gitlab-runner -rw-r--r-- 1 root root 586 Mar 7 07:07 gitlab.yaml drwxr-xr-x 13 root root 4096 Mar 7 07:18 SpringCloud [root@master gitlab]# kubectl get pod NAME READY STATUS RESTARTS AGE gitlab 1/1 Running 0 65m [root@master gitlab]# kubectl cp gitlab-runner gitlab:/opt/ [root@master gitlab]# kubectl exec -it gitlab bash root@gitlab:/# cd /opt/ root@gitlab:/opt# ls gitlab gitlab-runner-linux-amd64 root@gitlab:/opt# chmod +x gitlab-runner root@gitlab:/opt# useradd -c gitlab-runner -m gitlab-runner -s /bin/bash #创建GitLab CI用户 安装并作为服务运行 root@gitlab:/# su gitlab-runner gitlab-runner@gitlab:/$ vim ~/.bash . /etc/profile root@gitlab:/opt# ./gitlab-runner install --user=ru --working-directory=/home/gitlab-runner root@gitlab:/opt# ./gitlab-runner start root@gitlab:/opt# ./gitlab-runner register Enter the GitLab instance URL (for example, https://gitlab.com/): # http://192.168.100.101 Enter the registration token: # X8HgEf7n65zqvEekGwqs Enter a description for the runner: [192.168.100.101]:# runner Enter tags for the runner (comma-separated): Enter optional maintenance note for the runner: Registering runner... succeeded runner=X8HgEf7n Enter an executor: docker, shell, docker+machine, kubernetes, virtualbox, docker-ssh+machine, custom, docker-ssh, parallels, ssh: # shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
编写构建项目所需的Gitlab CI Runner资源清单文件并创建Runner资源对象。（2分）
[root@master ~]# kubectl create ns springcloudharbor
Set up CI/CD
runner的使用
\[ stages: - test - build - deploy test: stage: test script: echo "Running tests" build: stage: build script: echo "Building the app" deploy_staging: stage: deploy script: - echo "Deploy to staging server" environment: name: staging url: https://staging.example.com only: - master \]# 全局变量 variables: IP: 10.4.7.10 HARBOR_PROJECT: $IP/springcloud DOCKER_USER: admin DOCKER_PASSWORD: Harbor12345 TAG: v1 # 任务执行顺序 stages: - "build" - "upload" - "deploy" # 任务 build: stage: "build" script: - pwd - mvn package -DskipTests -f config - mvn package -DskipTests -f gateway - docker build -t $HARBOR_PROJECT/config:$TAG -f config/Dockerfile config - docker build -t $HARBOR_PROJECT/gateway:$TAG -f gateway/Dockerfile gateway upload: stage: "upload" script: - docker login $IP -u$DOCKER_USER -p$DOCKER_PASSWORD - docker push $HARBOR_PROJECT/config:$TAG - docker push $HARBOR_PROJECT/gateway:$TAG deploy: stage: "deploy" script: - sed -i "s|sqshq/piggymetrics-config|$HARBOR_PROJECT/config:$TAG|g" yaml/deployment/config-deployment.yaml - sed -i "s|sqshq/piggymetrics-gateway|$HARBOR_PROJECT/gateway:$TAG|g" yaml/deployment/gateway-deployment.yaml - kubectl apply -f yaml/deployment/config-deployment.yaml - kubectl apply -f yaml/deployment/gateway-deployment.yaml - kubectl apply -f yaml/svc/config-svc.yaml - kubectl apply -f yaml/svc/gateway-svc.yaml将SpringCloud项目中的代码推送到Gitlab中触发构建任务，完成镜像的构建、推送和服务发布。（2分）
第四套（GZ-2021040/2）
GitLab + Jenkins + Harbor + Kubernetes
云梦公司决定采用GitLab + Jenkins + Harbor + Kubernetes架构来构建CICD环境，以缩短新功能开发上线周期，及时满足客户的需求，实现DevOps的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成CICD环境部署（构建持续集成所需要的所有软件包在附件Jenkins_offline.tar中）。CICD应用系统架构如下：
从私有仓库中拉取jenkins:2.262-centos镜像，创建jenkins.yaml文件，基于Kubernetes启动Jenkins服务，实现web浏览器正常访问Jnekins服务。（2分）
apiVersion: v1 kind: ServiceAccount metadata: name: jenkins --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: jenkins rules: - apiGroups: ["extensions","apps"] resources: ["deployments"] verbs: ["create","delete","get","list","watch","patch","update"] - apiGroups: [""] resources: ["services","pods","pods/exec","pod/log","secrets"] verbs: ["create","delete","get","list","watch","patch","update"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: jenkins roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: jenkins subjects: - kind: ServiceAccount name: jenkins namespace: default --- apiVersion: apps/v1 kind: Deployment metadata: name: jenkins-deploy spec: selector: matchLabels: app: jenkins template: metadata: name: jenkins labels: app: jenkins spec: nodeName: master serviceAccount: jenkins volumes: - name: jenkins-home nfs: server: 10.4.7.20 path: /root/data/jenkins - name: docker hostPath: path: /usr/bin/docker - name: docker-sock hostPath: path: /var/run/docker.sock - name: kubectl hostPath: path: /usr/bin/kubectl - name: kube hostPath: path: /root/.kube containers: - name: jenkins image: jenkins:maven securityContext: runAsUser: 0 ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home - name: docker mountPath: /usr/bin/docker - name: docker-sock mountPath: /var/run/docker.run - name: kubectl mountPath: /usr/bin/kubectl - name: kube mountPath: /root/.kube --- apiVersion: v1 kind: Service metadata: name: jenkins spec: selector: app: jenkins type: NodePort ports: - name: http port: 8080 nodePort: 30080 - name: agent port: 50000[root@master cicd]# kubectl cp plugins/ jenkins:/var/jenkins_home/ [root@master cicd]# kubectl delete -f jenkins.yaml [root@master cicd]# kubectl apply -f jenkins.yaml [root@master cicd]# kubectl exec -it jenkins cat /var/jenkins_home/secrets/initialAdminPassword 9194b0124da24f5cb211fb7bcb0c38ca
配置动态jenkins-slave用于分担jenkins-master的压力。（1分）
系统管理-->节点管理-->Configure Clouds
[root@master cicd]# kubectl lable node master job=jenkins创建一个自由风格
echo "测试 Kubernetes 动态生成 jenkins slave" echo "===========mvn===========" mvn --version echo "==============docker in docker===========" docker version echo "=============kubectl=============" kubectl get pods
从私有仓库中拉取gitlab:latest镜像，创建gitlab.yaml文件，基于Kubernetes启动GitLab服务，实现web浏览器正常访问GitLab服务。（2分）
apiVersion: v1 kind: Service metadata: name: gitlab spec: selector: app: gitlab type: NodePort ports: - name: http port: 80 nodePort: 30080 - name: https port: 443 nodePort: 30443 - name: ssh port: 22 nodePort: 30022 --- apiVersion: apps/v1 kind: Deployment metadata: name: gitlab-deploy spec: selector: matchLabels: app: gitlab template: metadata: labels: app: gitlab spec: containers: - name: gitlab image: 10.4.7.10/library/gitlab-ce:12.9.2-ce.0 ports: - containerPort: 80 - containerPort: 443 - containerPort: 22
创建gitlab用户（用户名：Chinaskill），创建gitlab项目（项目名：SpringCloud），实现通过HTTP链接克隆项目。（2分）
[root@master SpringCloud]# git config --global user.name "Chinaskill"
[root@master SpringCloud]# git config --global user.email "zhang1287609286@qq.com"
[root@master SpringCloud]# git init
Initialized empty Git repository in /root/cicd/SpringCloud/.git/
[root@master SpringCloud]# git remote add origin http://10.111.230.41/Chinaskill/springcloud.git
[root@master SpringCloud]# git add .
[root@master SpringCloud]# git commit -m "Initial commit"
[root@master SpringCloud]# git push -u origin master
添加SSH凭证和Kubernetes凭证配置Jenkins和GitLab挂钩。（1分）
[root@master SpringCloud]# cat ~/.ssh/id_rsa -----BEGIN RSA PRIVATE KEY----- MIIEowIBAAKCAQEAylUNFPh0z3M0itEAoxTHwA+1AUVSEltxXCJVfcREmRWjMaMt BzQlkJzrc0XBGZEryQqPUsZ3a+YeCOMZnEXz0uuaA1ihiOzKwwRNmqpOX45Dl+Mo EmAP9Bg1bSzXicfzwxeBXWHzhf457BdvTAJhSoWG+0/unCm/sp3wTjv6WXVkL/Z q7/VEMFgo9BGMMKNcrOdx57+mZGBUH5QffQpqG5ouHUEjSS/0ho6AUPc8Fv4QpLn B50ly13faA+ojg7PMdXMoB5Sn7lEMJS/OQLDJq+X0gB1KM38ic2c -----END RSA PRIVATE KEY-----
[root@master SpringCloud]# cat ~/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKVQ0U+HTPczSK0QCjFMfAD7UBRVISW3FcIlV9xESZFaMxoy0HNCWQnOtzRcEZkSvJCo9Sxndr5h4I4xmcRfPS65oDWKGI7MrDBE2aqk5fjkOX4ygSYA/0GDVtLNeJx/PDF4FdYfOF/jnsF29MAmFKhYb7T+6cKb+ynfBOO/pZdWQv9mnJA/tOWUCbbuKjJ3QSIkXCHrjn0LNgXplC8DNMPO4W5g6oz0WatZbXwWcDw/hYzK8X03Jbo/ueeX5nk3FUK9ZjTFPvVyTbEpUywEGxXcWuyTaIP8AEdEPM1SkHQ0MDtAUSre/LlaM22O4bAvHEC8bsCStzgpBxiYeQ4U0D root@master
[root@master SpringCloud]# ssh -T git@10.111.230.41 The authenticity of host '10.111.230.41 (10.111.230.41)' can't be established. ECDSA key fingerprint is SHA256:UmH9ScY0gZdMpg87pxH2ThswlJPQ70gnyKZKTkAQMgI. ECDSA key fingerprint is MD5:ad:a5:82:17:20:e6:b6:c7:4a:e9:57:f7:1c:5d:3a:f7. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.111.230.41' (ECDSA) to the list of known hosts. Welcome to GitLab, @Chinaskill!
将SpringCloud项目中的代码推送到Gitlab中触发构建，完成镜像的构建、推送和服务发布。（2分）
node('slave'){ stage('git clone'){ git credentialsId: '29ebba78-095c-4d06-b9a1-1c42604601ab', url: 'http://10.111.230.41/root/springcloud.git' } stage('maven build'){ sh ''' mvn package -DskipTests -f config mvn package -DskipTests -f gateway ''' } stage('image build'){ sh ''' docker build -t 10.4.7.10/springcloud/config:$BUILD_ID -f config/Dockerfile config docker build -t 10.4.7.10/springcloud/gateway:$BUILD_ID -f gateway/Dockerfile gateway ''' } stage('image upload'){ sh ''' docker login 10.4.7.10 -uadmin -pHarbor12345 docker push 10.4.7.10/springcloud/config:$BUILD_ID docker push 10.4.7.10/springcloud/gateway:$BUILD_ID ''' } stage('deploy project'){ sh ''' sed -i "s|sqshq/piggymetrics-config|10.4.7.10/springcloud/config:$BUILD_ID|g" yaml/deployment/config-deployment.yaml sed -i "s|sqshq/piggymetrics-gateway|10.4.7.10/springcloud/gateway:$BUILD_ID|g" yaml/deployment/gateway-deployment.yaml kubectl apply -f yaml/deployment/config-deployment.yaml kubectl apply -f yaml/deployment/gateway-deployment.yaml kubectl apply -f yaml/svc/config-svc.yaml kubectl apply -f yaml/svc/gateway-svc.yaml ''' } }
第五套（GZ-2021040/3）
GitLab + GitLab-CI + Harbor + Kubernetes
云梦公司决定采用GitLab + GitLab-CI + Harbor + Kubernetes架构来构建CICD环境，以缩短新功能开发上线周期，及时满足客户的需求，实现DevOps的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成CICD环境部署。CICD应用系统架构如下：
从私有仓库中拉取gitlab:latest镜像，创建gitlab.yaml文件，基于Kubernetes启动GitLab服务，实现web浏览器正常访问GitLab服务。（2分）
apiVersion: v1 kind: Service metadata: name: gitlab spec: selector: app: gitlab type: NodePort ports: - name: http port: 80 nodePort: 30080 - name: https port: 443 nodePort: 30443 - name: ssh port: 22 nodePort: 30022 --- apiVersion: v1 kind: Pod metadata: name: gitlab labels: app: gitlab spec: containers: - name: gitlab image: 192.168.100.10/library/gitlab-ce:12.9.2-ce.0 ports: - containerPort: 80 - containerPort: 443 - containerPort: 22
创建gitlab用户（用户名：Chinaskill），创建gitlab项目（项目名：SpringCloud），配置gitlab和master节点SSH免密，实现通过SSH链接克隆项目。（2分）
[root@master cicd]# cat /root/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKVQ0U+HTPczSK0QCjFMfAD7UBRVISW3FcIlV9xESZFaMxoy0HNCWQnOtzRcEZkSvJCo9Sxndr5h4I4xmcRfPS65oDWKGI7MrDBE2aqk5fjkOX4ygSYA/0GDVtLNeJx/PDF4FdYfOF/jnsF29MAmFKhYb7T+6cKb+ynfBOO/pZdWQv9mnJA/tOWUCbbuKjJ3QSIkXCHrjn0LNgXplC8DNMPO4W5g6oz0WatZbXwWcDw/hYzK8X03Jbo/ueeX5nk3FUK9ZjTFPvVyTbEpUywEGxXcWuyTaIP8AEdEPM1SkHQ0MDtAUSre/LlaM22O4bAvHEC8bsCStzgpBxiYeQ4U0D root@masterUserSettings-->SSH Keys
[root@master cicd]# ssh -T git@10.4.7.10:30022 ssh: Could not resolve hostname 10.4.7.10:30022: Name or service not known [root@master cicd]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE gitlab NodePort 10.102.228.97 <none> 80:30080/TCP,443:30443/TCP,22:30022/TCP 8m38s kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 21d [root@master cicd]# ssh -T git@10.102.228.97 The authenticity of host '10.102.228.97 (10.102.228.97)' can't be established. ECDSA key fingerprint is SHA256:xzUdVuXy3YGiggNRLTN4V1B3stf+1y7ThVoHBDROXDM. ECDSA key fingerprint is MD5:e7:38:87:b2:eb:c5:68:14:bb:09:a5:be:6a:9c:13:17. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.102.228.97' (ECDSA) to the list of known hosts. Welcome to GitLab, @Chinaskill![root@master SpringCloud]# yum install -y git [root@master SpringCloud]# git config --global user.name "Chinaskill" [root@master SpringCloud]# git config --global user.email "Chinaskill@qq.com" [root@master SpringCloud]# git init Initialized empty Git repository in /root/cicd/SpringCloud/.git/ [root@master SpringCloud]# git remote add origin git@10.102.228.97:Chinaskill/springcloud.git [root@master SpringCloud]# git add . [root@master SpringCloud]# git commit -m "Initial commit" [root@master SpringCloud]# git push -u origin master在Kubernetes集群中安装GitLab Runner，由于GitLab Runner对缓存方案的支持有限，请使用挂载Volume的方式做缓存。（2分）
apiVersion: v1 kind: Pod metadata: name: gitlab-runner spec: nodeName: master volumes: - name: cache nfs: server: 10.4.7.20 path: /root/data/gitlab-runner - name: docker hostPath: path: /usr/bin/docker - name: docker-sock hostPath: path: /var/run/docker.sock - name: kubectl hostPath: path: /usr/bin/kubectl - name: kube hostPath: path: /root/.kube containers: - name: gitlab-runner image: 10.4.7.10/library/gitlab-runner securityContext: runAsUser: 0 command: ["sh","-c","gitlab-runner uninstall && gitlab-runner install --user=root --working-directory=/root && tail -f /etc/shadow"] volumeMounts: - name: cache mountPath: /root - name: docker mountPath: /usr/bin/docker - name: docker-sock mountPath: /var/run/docker.sock - name: kubectl mountPath: /usr/bin/kubectl - name: kube mountPath: /root/.kubeProject-->Settings-->CICD
[root@master cicd]# kubectl cp repository/ gitlab-runner:/root/.m2/ [root@master cicd]# kubectl exec -it gitlab-runner bash root@gitlab-runner:/# gitlab-runner register Runtime platform arch=amd64 os=linux pid=365 revision=e95f89a0 version=13.4.1 Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): http://gitlab Please enter the gitlab-ci token for this runner: JcWoSYi5y45b9JjWBKAf Please enter the gitlab-ci description for this runner: [gitlab-runner]: runner Please enter the gitlab-ci tags for this runner (comma separated): Registering runner... succeeded runner=JcWoSYi5 Please enter the executor: kubernetes, docker-ssh, shell, ssh, docker-ssh+machine, docker+machine, custom, docker, parallels, virtualbox: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! root@gitlab-runner:/# gitlab-runner start
编写构建项目所需的Gitlab CI Runner资源清单文件gitlab-ci.yml，完成源码项目的编译构建、镜像推送和应用部署。（2分）
SetupCI
variables: IP: 10.4.7.10 HARBOR_PROJECT: $IP/springcloud DOCKER_USER: admin DOCKER_PASSWORD: Harbor12345 stages: - "build" - "upload" - "deploy" package: stage: "build" script: - mvn package -DskipTests -f config - mvn package -DskipTests -f gateway - docker build -t $HARBOR_PROJECT/config:$CI_PIPELINE_ID -f config/Dockerfile config - docker build -t $HARBOR_PROJECT/gateway:$CI_PIPELINE_ID -f gateway/Dockerfile gateway harbor: stage: "upload" script: - docker login $IP -u$DOCKER_USER -p$DOCKER_PASSWORD - docker push $HARBOR_PROJECT/config:$CI_PIPELINE_ID - docker push $HARBOR_PROJECT/gateway:$CI_PIPELINE_ID kubernetes: stage: "deploy" script: - sed -i "s|sqshq/piggymetrics-config|$HARBOR_PROJECT/config:$CI_PIPELINE_ID|g" yaml/deployment/config-deployment.yaml - sed -i "s|sqshq/piggymetrics-gateway|$HARBOR_PROJECT/gateway:$CI_PIPELINE_ID|g" yaml/deployment/gateway-deployment.yaml - kubectl apply -f yaml/deployment/config-deployment.yaml - kubectl apply -f yaml/deployment/gateway-deployment.yaml - kubectl apply -f yaml/svc/config-svc.yaml - kubectl apply -f yaml/svc/gateway-svc.yaml将SpringCloud项目中的代码推送到Gitlab中触发构建任务，完成镜像的构建、推送和服务发布。（2分）
第六套 （GZ-2021040/5）
GitLab + Jenkins + Harbor + Kubernetes
云梦公司决定采用GitLab + Jenkins + Harbor + Kubernetes架构来构建CICD环境，以缩短新功能开发上线周期，及时满足客户的需求，实现DevOps的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成CICD环境部署（构建持续集成所需要的所有软件包在附件Jenkins_offline.tar中）。CICD应用系统架构如下：
从私有仓库中拉取jenkins:2.262-centos镜像，创建jenkins.yaml文件，基于Kubernetes集群编排部署Jenkins服务，实现web浏览器正常访问Jnekins服务。（2分）
apiVersion: v1 kind: Service metadata: name: jenkins spec: selector: app: jenkins type: NodePort ports: - name: http port: 8080 nodePort: 30080 --- apiVersion: apps/v1 kind: Deployment metadata: name: jenkins-deploy spec: selector: matchLabels: app: jenkins template: metadata: labels: app: jenkins spec: volumes: - name: jenkins-home nfs: server: 10.4.7.20 path: /root/data/jenkins - name: docker hostPath: path: /usr/bin/docker - name: docker-sock hostPath: path: /var/run/docker.sock - name: kubectl hostPath: path: /usr/bin/kubectl - name: kube hostPath: path: /root/.kube containers: - name: jenkins image: jenkins:maven securityContext: runAsUser: 0 ports: - containerPort: 8080 volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home - name: docker mountPath: /usr/bin/docker - name: docker-sock mountPath: /var/run/docker.sock - name: kubectl mountPath: /usr/bin/kubectl - name: kube mountPath: /root/.kube[root@master cicd]# kubectl cp plugins/ jenkins-deploy-567d97d6b5-92295:/var/jenkins_home/ [root@master cicd]# kubectl delete pod jenkins-deploy-567d97d6b5-92295 [root@master cicd]# kubectl exec -it jenkins-deploy-567d97d6b5-xhnm4 cat /var/jenkins_home/secrets/initialAdminPassword 7524f1e135f44cd1affad9f2f32624bc
从私有仓库中拉取gitlab:latest镜像，创建gitlab.yaml文件，基于Kubernetes集群编排部署GitLab服务，实现web浏览器正常访问GitLab服务。（2分）
apiVersion: v1 kind: Service metadata: name: gitlab spec: selector: app: gitlab type: NodePort ports: - name: http port: 80 nodePort: 30081 - name: https port: 443 nodePort: 30443 - name: ssh port: 22 nodePort: 30022 --- apiVersion: apps/v1 kind: Deployment metadata: name: gitlab-deploy spec: selector: matchLabels: app: gitlab template: metadata: labels: app: gitlab spec: containers: - name: gitlab image: 10.4.7.10/library/gitlab-ce:12.9.2-ce.0 ports: - containerPort: 80 - containerPort: 443 - containerPort: 22
创建gitlab用户（用户名：Chinaskill），创建gitlab项目（项目名：SpringCloud），实现通过HTTP链接克隆项目。（2分）
[root@master cicd]# yum install -y git [root@master SpringCloud]# git config --global user.name "Chinaskill" [root@master SpringCloud]# git config --global user.email "zhang1287609286@qq.com" [root@master SpringCloud]# git remote add origin http://10.98.127.54/Chinaskill/springcloud.git [root@master SpringCloud]# git add . [root@master SpringCloud]# git commit -m "Initial commit" [root@master SpringCloud]# git push -u origin master新建一个自由风格的软件项目，并编写构建项目所需的shell脚本。（2分）
Projects-->Settings-->Webhooks
shell
# variables IP=10.4.7.10 HARBOR_PROJECT=$IP/springcloud DOCKER_USER=admin DOCKER_PASSWORD=Harbor12345 # maven build starting... mvn package -DskipTests -f config mvn package -DskipTests -f gateway # docker build staring... docker build -t $HARBOR_PROJECT/config:$BUILD_ID -f config/Dockerfile config docker build -t $HARBOR_PROJECT/gateway:$BUILD_ID -f gateway/Dockerfile gateway # docker upload starting... docker login $IP -u$DOCKER_USER -p$DOCKER_PASSWORD docker push $HARBOR_PROJECT/config:$BUILD_ID docker push $HARBOR_PROJECT/gateway:$BUILD_ID # kubernetes deploy starting... sed -i "s|sqshq/piggymetrics-config|$HARBOR_PROJECT/config:$BUILD_ID|g" yaml/deployment/config-deployment.yaml sed -i "s|sqshq/piggymetrics-gateway|$HARBOR_PROJECT/gateway:$BUILD_ID|g" yaml/deployment/gateway-deployment.yaml kubectl apply -f yaml/deployment/config-deployment.yaml kubectl apply -f yaml/deployment/gateway-deployment.yaml kubectl apply -f yaml/svc/config-svc.yaml kubectl apply -f yaml/svc/gateway-svc.yaml将SpringCloud项目中的代码推送到Gitlab中触发构建，完成服务的自动发布。（2分）
[root@master SpringCloud]# kubectl create ns springcloud
第0套（GZ-2021040/10）
GitLab + Jenkins + Harbor + Kubernetes
该公司决定采用GitLab + Jenkins + Harbor + Kubernetes架构来构建CICD环境，以缩短新功能开发上线周期，及时满足客户的需求，实现DevOps的部分流程，来减轻部署运维的负担，实现可视化容器生命周期管理、应用发布和版本迭代更新，请完成CICD环境部署（构建持续集成所需要的所有软件包在附件Jenkins_offline.tar中）。CICD应用系统架构如下：
1．从私有仓库中拉取jenkins:2.262-centos镜像，基于docker run方式启动Jenkins服务，实现web浏览器正常访问Jnekins服务。
2．从私有仓库中拉取gitlab:latest镜像，基于docker run方式启动GitLab服务，实现web浏览器正常访问GitLab服务。
3．创建gitlab用户（用户名：Chinaskill），创建gitlab项目（项目名：ChinaskillProject），实现通过HTTP链接克隆项目。
4.采用docker in docker的方式在Jenkins内安装maven，并配置Jenkins连接maven。
5.新建一个maven项目ChinaskillProject，并编写构建所需的shell脚本。
6.将ChinaskillProject项目中的代码推送到Gitlab中触发构建，完成镜像的构建、推送和服务发布。
