交付Apollo至Kubernetes集群
Apollo简介
Apollo（阿波罗）是携程框架部门研发的分布式配置中心，能够集中化管理应用不同环境、不同集群的配置，配置修改后能够实时推送到应用端，并且具备规范的权限、流程治理等特性，适用于微服务配置管理场景。
官方GitHub地址
Apollo官方地址https://github.com/ctripcorp/apollo
官方release包https://github.com/apolloconfig/apollo/releases
简化模型
交付apollo-configservice
准备软件包
在运维主机k8s-7-200.host.top上：
下载官方release包https://github.com/ctripcorp/apollo/releases/download/v1.3.0/apollo-configservice-1.3.0-github.zip
/opt/src
[root@k8s-7-200 src]# ls -l|grep apollo
-rw-r--r-- 1 root root 53M 8月 5 13:11 apollo-configservice-1.4.0-github.zip
[root@k8s-7-200 src]# mkdir /data/dockerfile/apollo-configservice && unzip -o apollo-configservice-1.4.0-github.zip -d /data/dockerfile/apollo-configservice
Archive: apollo-configservice-1.4.0-github.zip
creating: /data/dockerfile/apollo-configservice/scripts/
inflating: /data/dockerfile/apollo-configservice/config/application-github.properties
inflating: /data/dockerfile/apollo-configservice/scripts/shutdown.sh
inflating: /data/dockerfile/apollo-configservice/apollo-configservice-1.4.0-sources.jar
inflating: /data/dockerfile/apollo-configservice/config/app.properties
inflating: /data/dockerfile/apollo-configservice/apollo-configservice-1.4.0.jar
inflating: /data/dockerfile/apollo-configservice/scripts/startup.sh
inflating: /data/dockerfile/apollo-configservice/apollo-configservice.conf
执行数据库脚本
在数据库主机k8s-7-11.host.top上：
注意：MySQL版本应为5.6或以上！
更新yum源
#vim /etc/yum.repos.d/MariaDB.repo
[mariadb]
name = MariaDB
baseurl = https://mirrors.ustc.edu.cn/mariadb/yum/10.1/centos7-amd64/
gpgkey=https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB
gpgcheck=1
导入GPG-KEY
[root@k8s-7-11 ~]# rpm --import https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB
更新数据库版本
[root@k8s-7-11 ~]# yum list mariadb-server --show-duplicates #查看版本
[root@k8s-7-11 ~]# yum update MariaDB-server -y
配置my.cnf
vim /etc/my.cnf
[mysql]
default-character-set = utf8mb4
[mysqld]
character_set_server = utf8mb4
collation_server = utf8mb4_general_ci
init_connect = "SET NAMES 'utf8mb4'"
启动数据库
#systemctl start mariadb
#systemctl enable mariadb
设置数据库root密码
#mysqladmin -uroot password
pwd:123456
检查数据库编码,确保为utf8mb4,否则会出现乱码
> \s
数据库脚本地址
https://raw.githubusercontent.com/ctripcorp/apollo/master/scripts/db/migration/configdb/V1.0.0__initialization.sql
导入初始化数据
[root@k8s-7-11 ~]# mysql -uroot -p
mysql> create database ApolloConfigDB;
mysql> use ApolloConfigDB;
mysql> source ./apolloconfig.sql
数据库用户授权
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";
mysql> select user,host from mysql.user;
修改初始数据
mysql> update ApolloConfigDB.ServerConfig set ServerConfig.Value="http://config.itdo.top/eureka" where ServerConfig.Key="eureka.service.url";
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select * from ServerConfig\G
*************************** 1. row ***************************
Id: 1
Key: eureka.service.url
Cluster: default
Value: http://config.itdo.top/eureka
Comment: Eureka服务Url，多个service以英文逗号分隔
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2019-04-10 15:07:34
DataChange_LastModifiedBy:
DataChange_LastTime: 2019-04-11 16:28:57
制作Docker镜像
在运维主机k8s-7-200.host.top上：
配置数据库连接串
/data/dockerfile/apollo-configservice
[root@k8s-7-200 apollo-configservice]# cat config/application-github.properties
# DataSource
spring.datasource.url = jdbc:mysql://10.4.7.11:3306/ApolloConfigDB?characterEncoding=
utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
更新startup.sh
#rm -rf /data/dockerfile/apollo-configservice/scripts/shutdown.sh #k8s中用不到，直接删除pod就行了
#vim /data/dockerfile/apollo-configservice/scripts/startup.sh #替换成applo提供的k8s环境启动脚本，官网可下载
#!/bin/bash
SERVICE_NAME=apollo-configservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-config-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_CONFIG_SERVICE_NAME=$(hostname -i) #添加这句
SERVER_URL="http://${APOLLO_CONFIG_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#调整内存大小
export JAVA_OPTS="-Xms128m -Xmx128m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null
写Dockerfile
vim /data/dockerfile/apollo-configservice/Dockerfile
FROM harbor.itdo.top/base/jre8:8u112
ENV VERSION 1.4.0
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar
ADD config/ /apollo-configservice/config
ADD scripts/ /apollo-configservice/scripts
CMD ["/apollo-configservice/scripts/startup.sh"]
制作镜像并推送
[root@k8s-7-200 apollo-configservice]# docker build . -t harbor.itdo.top/infra/apollo-configservice:v1.4.0
Sending build context to Docker daemon 61.96MB
Step 1/7 : FROM stanleyws/jre8:8u112
---> fa3a085d6ef1
Step 2/7 : ENV VERSION 1.4.0
---> Running in 565232e461a3
Removing intermediate container 565232e461a3
---> e0da6ad6db15
Step 3/7 : RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
---> Running in 894a95fa9534
Removing intermediate container 894a95fa9534
---> a16c66a2e00f
Step 4/7 : ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar
---> 280fc1726f6b
Step 5/7 : ADD config/ /apollo-configservice/config
---> a88492d4d3dd
Step 6/7 : ADD scripts/ /apollo-configservice/scripts
---> 694f1826b0fc
Step 7/7 : CMD ["/apollo-configservice/scripts/startup.sh"]
---> Running in aa1aaae47d64
Removing intermediate container aa1aaae47d64
---> 4d4e43fcd2dc
Successfully built 4d4e43fcd2dc
Successfully tagged harbor.itdo.top/infra/apollo-configservice:v1.4.0
[root@k8s-7-200 apollo-configservice]# docker push harbor.itdo.top/infra/apollo-configservice:v1.4.0
The push refers to repository [harbor.itdo.top/infra/apollo-configservice]
d67c6e21a063: Pushed
439c7dff6eca: Pushed
7958ed7986d4: Pushed
7edb47e620b9: Pushed
0690f10a63a5: Mounted from base/jre8
c843b2cf4e12: Mounted from base/jre8
fddd8887b725: Mounted from base/jre7
42052a19230c: Mounted from base/jre7
8d4d1ab5ff74: Mounted from base/jre7
v1.4.0: digest: sha256:4093430f2723b260e1a087ed2f73d3b78e452c48f352570ac750f3bdfb8df2e4 size: 2201
解析域名
DNS主机k8s-7-11.host.top上：
/var/named/itdo.top.zone
mysql 60 IN A 10.4.7.11
config 60 IN A 10.4.7.10
解析验证
#dig -t A config.itdo.top @192.168.0.2 +short
准备资源配置清单
在运维主机k8s-7-200.host.top上
/data/k8s-yaml
[root@k8s-7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-configservice && cd /data/k8s-yaml/apollo-configservice
- Deployment
- Service
- Ingress
- ConfigMap
vi deployment.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: infra
labels:
name: apollo-configservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-configservice
template:
metadata:
labels:
app: apollo-configservice
name: apollo-configservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-configservice-cm
containers:
- name: apollo-configservice
image: harbor.itdo.top/infra/apollo-configservice:v1.3.0 #修改镜像地址
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-configservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
vi svc.yaml
kind: Service
apiVersion: v1
metadata:
name: apollo-configservice
namespace: infra
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-configservice
clusterIP: None
type: ClusterIP
sessionAffinity: None
vi ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: infra
spec:
rules:
- host: config.itdo.top #修改域名
http:
paths:
- path: /
backend:
serviceName: apollo-configservice
servicePort: 8080
vi configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: apollo-configservice-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.itdo.top:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config.itdo.top/eureka
app.properties: |
appId=100003171
应用资源配置清单
在任意一台k8s运算节点执行：
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-configservice/configmap.yaml
configmap/apollo-configservice-cm created
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-configservice/deployment.yaml
deployment.extensions/apollo-configservice created
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-configservice/svc.yaml
service/apollo-configservice created
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-configservice/ingress.yaml
ingress.extensions/apollo-configservice created
浏览器访问
http://config.itdo.top
验证下mysql连接的ip，证明是使用host node ip去跟DB通讯
思考：如果要使用真实pod ip 跟mysql通讯？
方法1：可在mysql主机做NAT的优化规则，参考之前flannet配置部分即可；注意mysql授权的ip也要修改放开
方法2：mysql部署在k8s群集内
交付apollo-adminservice
准备软件包
在运维主机k8s-200.host.top上：
下载官方release包https://github.com/ctripcorp/apollo/releases/download/v1.3.0/apollo-adminservice-1.3.0-github.zip
[root@k8s-7-200 src]# ls -l|grep apollo
-rw-r--r-- 1 root root 51868604 8月 5 13:13 apollo-adminservice-1.4.0-github.zip
-rw-r--r-- 1 root root 55169267 8月 5 13:11 apollo-configservice-1.4.0-github.zip
[root@k8s-7-200 src]# mkdir /data/dockerfile/apollo-adminservice && unzip -o apollo-adminservice-1.4.0-github.zip -d /data/dockerfile/apollo-adminservice
Archive: apollo-adminservice-1.4.0-github.zip
creating: /data/dockerfile/apollo-ad/scripts/
inflating: /data/dockerfile/apollo-ad/config/app.properties
inflating: /data/dockerfile/apollo-ad/apollo-adminservice-1.4.0-sources.jar
inflating: /data/dockerfile/apollo-ad/scripts/shutdown.sh
inflating: /data/dockerfile/apollo-ad/apollo-adminservice.conf
inflating: /data/dockerfile/apollo-ad/scripts/startup.sh
inflating: /data/dockerfile/apollo-ad/config/application-github.properties
inflating: /data/dockerfile/apollo-ad/apollo-adminservice-1.4.0.jar
制作Docker镜像
在运维主机k8s-7-200.host.top上：
配置数据库连接串–这里可以不配置，因为后面用configmap来部署配置
cd /data/dockerfile/apollo-adminservice
[root@k8s-7-200 apollo-adminservice]# cat config/application-github.properties
# DataSource
spring.datasource.url = jdbc:mysql://mysql.itdo.top:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
更新starup.sh
#rm -rf /data/dockerfile/apollo-adminservice/scripts/shutdown.sh
#vim /data/dockerfile/apollo-adminservice/scripts/startup.sh
#!/bin/bash
SERVICE_NAME=apollo-adminservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-adminservice
## Adjust server port if necessary
SERVER_PORT=8080 #注意，官方给的是8090端口，因为我们放在k8s环境，网络空间是隔离的，所以我们这里配置8080不会冲突
APOLLO_ADMIN_SERVICE_NAME=$(hostname -i) #添加这句
# SERVER_URL="http://localhost:${SERVER_PORT}"
SERVER_URL="http://${APOLLO_ADMIN_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null
写Dockerfile
#vim /data/dockerfile/apollo-adminservice/Dockerfile
FROM harbor.itdo.top/base/jre8:8u112
ENV VERSION 1.4.0
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
ADD config/ /apollo-adminservice/config
ADD scripts/ /apollo-adminservice/scripts
CMD ["/apollo-adminservice/scripts/startup.sh"]
制作镜像并推送
[root@k8s-7-200 apollo-adminservice]# docker build . -t harbor.itdo.top/infra/apollo-adminservice:v1.4.0
Sending build context to Docker daemon 58.31MB
Step 1/7 : FROM harbor.itdo.top/base/jre8:8u112
---> 8c2c60b7922a
Step 2/7 : ENV VERSION 1.4.0
---> Running in 9e43a188b349
Removing intermediate container 9e43a188b349
---> 32417873619e
Step 3/7 : RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
---> Running in 2f483f1918f9
Removing intermediate container 2f483f1918f9
---> fac9f94f70cc
Step 4/7 : ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
---> 8b35e5a85db0
Step 5/7 : ADD config/ /apollo-adminservice/config
---> 63d62d6f7a62
Step 6/7 : ADD scripts/ /apollo-adminservice/scripts
---> 6bef5d6a4c47
Step 7/7 : CMD ["/apollo-adminservice/scripts/startup.sh"]
---> Running in c53f8ceb97fe
Removing intermediate container c53f8ceb97fe
---> 6d10ee95aa64
Successfully built 6d10ee95aa64
Successfully tagged harbor.itdo.top/infra/apollo-adminservice:v1.4.0
[root@k8s-7-200 apollo-adminservice]# docker push harbor.itdo.top/infra/apollo-adminservice:v1.4.0
The push refers to repository [harbor.itdo.top/infra/apollo-adminservice]
512fcae56667: Pushed
7ec0787281d5: Pushed
cbe21aecb371: Pushed
c5e006c68ecd: Pushed
f36a38bc906f: Mounted from base/jre8
18e69df50d3c: Mounted from base/jre8
a94a5d0907fe: Mounted from base/jre8
5b2615bb2614: Mounted from base/jre8
78bd98074182: Mounted from base/jre8
0690f10a63a5: Mounted from infra/apollo-configservice
c843b2cf4e12: Mounted from infra/apollo-configservice
fddd8887b725: Mounted from infra/apollo-configservice
42052a19230c: Mounted from infra/apollo-configservice
8d4d1ab5ff74: Mounted from infra/apollo-configservice
v1.4.0: digest: sha256:175c08f89390d87c7a436ef4855ed7aedecd05928c27c808d63c1faae4c3a8b1 size: 3240
准备资源配置清单
在运维主机k8s-7-200.host.top上
/data/k8s-yaml
[root@k8s-7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-adminservice && cd /data/k8s-yaml/apollo-adminservice
- Deployment
- ConfigMap
注：因为adminservice不需要提供外部访问，所以不用svc
vi deployment.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-adminservice
namespace: infra
labels:
name: apollo-adminservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-adminservice
template:
metadata:
labels:
app: apollo-adminservice
name: apollo-adminservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-adminservice-cm
containers:
- name: apollo-adminservice
image: harbor.itdo.top/infra/apollo-adminservice:v1.3.0 #修改镜像
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-adminservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
vim configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-adminservice-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.itdo.top:3306/ApolloConfigDB?characterEncoding=utf8 #修改mysql地址
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config.itdo.top/eureka #修改地址
app.properties: |
appId=100003172
应用资源配置清单
在任意一台k8s运算节点执行：
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-adminservice/configmap.yaml
configmap/apollo-adminservice-cm created
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-adminservice/deployment.yaml
deployment.extensions/apollo-adminservice created
浏览器访问
http://config.itdo.top
检查apollo-adminservice接口
[root@k8s-7-21 ~]# curl http://172.7.22.9:8080/info
交付apollo-portal
准备软件包
在运维主机k8s-200.host.top上：
下载官方release包https://github.com/ctripcorp/apollo/releases/download/v1.3.0/apollo-portal-1.3.0-github.zip
[root@k8s-7-200 src]#
ls -l|grep apollo
-rw-r--r-- 1 root root 51868604 8月 5 13:13 apollo-adminservice-1.4.0-github.zip
-rw-r--r-- 1 root root 55169267 8月 5 13:11 apollo-configservice-1.4.0-github.zip
-rw-r--r-- 1 root root 38925211 8月 5 13:11 apollo-portal-1.4.0-github.zip
[root@k8s-7-200 src]# mkdir /data/dockerfile/apollo-portal && unzip -o apollo-portal-1.4.0-github.zip -d /data/dockerfile/apollo-portal
Archive: apollo-portal-1.4.0-github.zip
creating: /data/dockerfile/apollo-portal/scripts/
inflating: /data/dockerfile/apollo-portal/config/application-github.properties
inflating: /data/dockerfile/apollo-portal/scripts/startup.sh
inflating: /data/dockerfile/apollo-portal/config/apollo-env.properties
inflating: /data/dockerfile/apollo-portal/scripts/shutdown.sh
inflating: /data/dockerfile/apollo-portal/config/app.properties
inflating: /data/dockerfile/apollo-portal/apollo-portal-1.4.0-sources.jar
inflating: /data/dockerfile/apollo-portal/apollo-portal.conf
inflating: /data/dockerfile/apollo-portal/apollo-portal-1.4.0.jar Archive: apollo-portal-1.3.0-github.zip
删除没用的文件
#rm -rf apollo-portal-1.4.0-sources.jar
#rm -rf apollo-portal.conf
#rm -rf scripts/shutdown.sh
执行数据库脚本
在数据库主机k8s-7-11.host.top上：
数据库脚本地址https://raw.githubusercontent.com/ctripcorp/apollo/master/scripts/db/migration/portaldb/V1.0.0__initialization.sql
[root@k8s-7-11 ~]# mysql -uroot -p
mysql> create database ApolloPortalDB;
mysql> use ApolloPortalDB;
mysql> source ./apolloportaldb.sql;
数据库用户授权
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloPortalDB.* to "apolloportal"@"10.4.7.%" identified by "123456";
mysql> select user,host from mysql.user;
+--------------+-------------------+
| user | host |
+--------------+-------------------+
| apolloconfig | 10.4.7.% |
| apolloportal | 10.4.7.% |
| root | 127.0.0.1 |
| root | ::1 |
| | k8s-7-11.host.top |
| root | k8s-7-11.host.top |
| | localhost |
| root | localhost |
+--------------+-------------------+
8 rows in set (0.00 sec)
制作Docker镜像
在运维主机k8s-7-200.host.top上：
配置数据库连接串
/data/dockerfile/apollo-portal
[root@k8s-7-200 apollo-portal]# cat config/application-github.properties
# DataSource
spring.datasource.url = jdbc:mysql://mysql.itdo.top:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloportal
spring.datasource.password = 123456
配置Portal的meta service
#vim /data/dockerfile/apollo-portal/config/apollo-env.properties #这里我们在configmap里配置
dev.meta=http://config.itdo.top
更新starup.sh
/data/dockerfile/apollo-portal/scripts/startup.sh
#!/bin/bash
SERVICE_NAME=apollo-portal
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-portal-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_PORTAL_SERVICE_NAME=$(hostname -i) #添加这句
# SERVER_URL="http://localhost:$SERVER_PORT"
SERVER_URL="http://${APOLLO_PORTAL_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null
写Dockerfile
/data/dockerfile/apollo-portal/Dockerfile
FROM harbor.itdo.top/base/jre8:8u112
ENV VERSION 1.4.0
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-portal-${VERSION}.jar /apollo-portal/apollo-portal.jar
ADD config/ /apollo-portal/config
ADD scripts/ /apollo-portal/scripts
CMD ["/apollo-portal/scripts/startup.sh"]
制作镜像并推送
[root@k8s-7-200 apollo-portal]# docker build . -t harbor.itdo.top/infra/apollo-portal:v1.4.0
Sending build context to Docker daemon 42.21MB
Step 1/7 : FROM harbor.itdo.top/base/jre8:8u112
---> 8c2c60b7922a
Step 2/7 : ENV VERSION 1.4.0
---> Using cache
---> 32417873619e
Step 3/7 : RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone
---> Using cache
---> fac9f94f70cc
Step 4/7 : ADD apollo-portal-${VERSION}.jar /apollo-portal/apollo-portal.jar
---> 971b5645a588
Step 5/7 : ADD config/ /apollo-portal/config
---> 54c2985f98e5
Step 6/7 : ADD scripts/ /apollo-portal/scripts
---> 669fdeafdc61
Step 7/7 : CMD ["/apollo-portal/scripts/startup.sh"]
---> Running in 755fc331a764
Removing intermediate container 755fc331a764
---> e19c63620079
Successfully built e19c63620079
Successfully tagged harbor.itdo.top/infra/apollo-portal:v1.4.0
[root@k8s-7-200 apollo-portal]# docker push harbor.itdo.top/infra/apollo-portal:v1.4.0
The push refers to repository [harbor.itdo.top/infra/apollo-portal]
b6b7105139e1: Pushed
82d2e7c91506: Pushed
5c5adaf604ee: Pushed
c5e006c68ecd: Mounted from infra/apollo-adminservice
f36a38bc906f: Mounted from infra/apollo-adminservice
18e69df50d3c: Mounted from infra/apollo-adminservice
a94a5d0907fe: Mounted from infra/apollo-adminservice
5b2615bb2614: Mounted from infra/apollo-adminservice
78bd98074182: Mounted from infra/apollo-adminservice
0690f10a63a5: Mounted from infra/apollo-adminservice
c843b2cf4e12: Mounted from infra/apollo-adminservice
fddd8887b725: Mounted from infra/apollo-adminservice
42052a19230c: Mounted from infra/apollo-adminservice
8d4d1ab5ff74: Mounted from infra/apollo-adminservice
v1.4.0: digest: sha256:54d85807ee92b5c64ba4ed49b8de79a19560bc634f47997c212cd5ceccac6f10 size: 3240
解析域名
DNS主机k8s-11.host.top上：
/var/named/idto.top.zone
portal 60 IN A 10.4.7.10
准备资源配置清单
在运维主机k8s-7-200.host.top上
/data/k8s-yaml
[root@k8s-7-200 k8s-yaml]# mkdir /data/k8s-yaml/apollo-portal && cd /data/k8s-yaml/apollo-portal
- Deployment
- Service
- Ingress
- ConfigMap
vi deployment.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-portal
namespace: infra
labels:
name: apollo-portal
spec:
replicas: 1
selector:
matchLabels:
name: apollo-portal
template:
metadata:
labels:
app: apollo-portal
name: apollo-portal
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-portal-cm
containers:
- name: apollo-portal
image: harbor.itdo.top/infra/apollo-portal:v1.3.0 #修改镜像地址
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-portal/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
vi svc.yaml
kind: Service
apiVersion: v1
metadata:
name: apollo-portal
namespace: infra
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-portal
clusterIP: None
type: ClusterIP
sessionAffinity: None
vi ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-portal
namespace: infra
spec:
rules:
- host: portal.itdo.top #修改域名
http:
paths:
- path: /
backend:
serviceName: apollo-portal
servicePort: 8080
vi configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-portal-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.itdo.top:3306/ApolloPortalDB?characterEncoding=utf8 #修改mysql地址
spring.datasource.username = apolloportal
spring.datasource.password = 123456
app.properties: |
appId=100003173
apollo-env.properties: |
dev.meta=http://config.itdo.top #修改地址
应用资源配置清单
在任意一台k8s运算节点执行：
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-portal/configmap.yaml
configmap/apollo-portal-cm created
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-portal/deployment.yaml
deployment.extensions/apollo-portal created
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-portal/svc.yaml
service/apollo-portal created
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/apollo-portal/ingress.yaml
ingress.extensions/apollo-portal created
浏览器访问
http://portal.itdo.top
用户名：apollo
密码： admin
修改密码为admin123
修改系统参数，增加部门
key:organizations
value:[{"orgId":"itdo01","orgName":"设计部"},{"orgId":"itdo02","orgName":"架构部"},{"orgId":"itdo03","orgName":"运维部"},{"orgId":"itdo04","orgName":"研发部"},{"orgId":"itdo05","orgName":"大数据部"}]
查看数据库连接
实战dubbo微服务接入Apollo配置中心
拉取代码并上传到自己的仓库上
git clone -b apollo https://gitee.com/itdotop/dubbo-demo-service.git
cd dubbo-demo-service
rm -rf .git
git init
git add .
git commit -m "apollo commit"
git remote add origin [email protected]:itdotop/dubbo-demo-service.git
git push -u origin master:apollo
配置apollo-portal
创建项目
- 部门
样例部门1（TEST1） - 应用id
dubbo-demo-service - 应用名称
dubbo服务提供者 - 应用负责人
apollo|apollo - 项目管理员
apollo|apollo
提交
进入配置页面
新增配置项1
- Key
dubbo.registry - Value
zookeeper://zk1.itdo.top:2181 - 选择集群
DEV
提交
新增配置项2 - Key
dubbo.port - Value
20880 - 选择集群
DEV
提交
发布配置
点击发布，配置生效
使用jenkins进行CI
参数如下
app_name：dubbo-demo-service
image_name：app/dubbo-demo-service
git_repo：https://gitee.com/itdotop/dubbo-demo-service.git
git_ver:apollo
add_tag:220214_2250
mvn_dir：./
target_dir：./dubbo-server/target
mvn_cmd：mvn clean package -Dmaven.test.skip=true
base_image：base/jre8:8u112
maven：3.6.1-8u221
- 构建失败，提示解析问题
[ERROR] Failed to execute goal on project dubbo-server: Could not resolve dependencies for project com.od.dubbotest:dubbo-server:jar:0.0.1-SNAPSHOT: Failed to collect dependencies at com.ctrip.framework.apollo:apollo-client:jar:1.1.0: Failed to read artifact descriptor for com.ctrip.framework.apollo:apollo-client:jar:1.1.0: Could not transfer artifact com.ctrip.framework.apollo:apollo-client:pom:1.1.0 from/to spring-snapshots (http://repo.spring.io/snapshot): repo.spring.io: Temporary failure in name resolution: Unknown host repo.spring.io: Temporary failure in name resolution -> [Help 1]
检查jenkins解析
jenkins pod解析异常，重启pod后恢复正常
上线新构建的项目
准备资源配置清单
运维主机k8s-7-200.host.top上：
修改/data/k8s-yaml/dubbo-demo-service/deployment-apollo.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-service
namespace: app
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.od.com/app/dubbo-demo-service:apollo_190119_1815 #修改镜像版本
ports:
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=dev -Dapollo.meta=http://config.itdo.top
- name: JAR_BALL
value: dubbo-server.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
注意：增加了env段配置
注意：docker镜像新版的tag
应用资源配置清单
在任意一台k8s运算节点上执行：
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/dubbo-demo-service/deployment-apollo.yaml
deployment.extensions/dubbo-demo-service configured
检查创建成功
观察项目运行情况
http://dubbo-monitor.itdo.top
改造dubbo-demo-web
直接克隆代码
git clone -b apollo https://gitee.com/xiaoyicode/dubbo-demo-web-apollo.git
git clone -b apollo [email protected]:itdotop/dubbo-demo-web.git
cd dubbo-demo-web
rm -rf .git
git init
git add .
git commit -m "apollo commit"
git remote add origin [email protected]:itdotop/dubbo-demo-web.git
git push -u origin master:apollo
配置apollo-portal
创建项目
- 部门
设计部(itdo01) - 应用id
dubbo-demo-web
注：这里的appid一定要跟代码中app.id的值一样 - 应用名称
dubbo服务消费者 - 应用负责人
apollo|apollo - 项目管理员
apollo|apollo
提交
说明：app id配置路径
dubbo-client/src/main/resources/META-INF/app.properties
进入配置页面
新增配置项1 - Key
dubbo.registry - Value
zookeeper://zk1.itdo.top:2181
注:server、consumer、monitor要连接在同一个zk下 - 选择集群
DEV
提交
发布配置
点击发布，配置生效
使用jenkins进行CI
app_name：dubbo-demo-consumer
image_name：app/dubbo-demo-consumer
git_repo：[email protected]:itdotop/dubbo-demo-web.git #因为是私有仓库，这里需要使用ssh方式
git_ver:apollo
add_tag:220215_1720
mvn_dir：./
target_dir：./dubbo-client/target
mvn_cmd：mvn clean package -Dmaven.test.skip=true
base_image：base/jre8:8u112
maven：3.6.1-8u221
上线新构建的项目
准备资源配置清单
运维主机k8s-200.host.top上：
修改配置清单/data/k8s-yaml/dubbo-demo-consumer/deployment-apollo.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: app
labels:
name: dubbo-demo-consumer
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-consumer
template:
metadata:
labels:
app: dubbo-demo-consumer
name: dubbo-demo-consumer
spec:
containers:
- name: dubbo-demo-consumer
image: harbor.itdo.top/app/dubbo-demo-consumer:apllo_190120_1815 #修改镜像版本
ports:
- containerPort: 20880
protocol: TCP
- containerPort: 8080
protocol: TCP
env:
- name: C_OPTS #增加apollo的环境变量，jar启动时使用到
value: -Denv=dev -Dapollo.meta=http://config.itdo.top
- name: JAR_BALL
value: dubbo-client.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
注意：增加了env段配置
注意：docker镜像新版的tag
应用资源配置清单
在任意一台k8s运算节点上执行：
[root@k8s-7-21 ~]# kubectl apply -f http://k8s-yaml.itdo.top/dubbo-demo-consumer/deployment-apollo.yaml
deployment.extensions/dubbo-demo-consumer configured
验证服务正常
通过Apollo配置中心动态维护项目的配置
以dubbo-demo-service项目为例，不用修改代码
- 在http://portal.itdo.top 里修改dubbo.port配置项
- 重启dubbo-demo-service项目
- 配置生效
实战维护多套dubbo微服务环境
生产实践
- 迭代新需求/修复BUG（编码->提GIT）
- 测试环境发版，测试（应用通过编译打包发布至TEST命名空间）
- 测试通过，上线（应用镜像直接发布至PROD命名空间）
系统架构
物理架构
K8S内系统架构
修改/添加域名解析
DNS主机k8s-7-11.host.com上：
vim /var/named/itdo.top.zone
zk-dev 60 IN A 10.4.7.11
zk-test 60 IN A 10.4.7.12
zk-prod 60 IN A 10.4.7.21
config-test 60 IN A 10.4.7.10
config-prod 60 IN A 10.4.7.10
demo-test 60 IN A 10.4.7.10
demo-prod 60 IN A 10.4.7.10
#systemctl restart named
Apollo的k8s应用配置
创建test命名空间，创建prod命名空间
#kubectl create ns test
#kubectl create secret docker-registry harbor --docker-server=harbor.itdo.top --docker-username=admin --docker-password=Harbor.itdo12345 -n test
#kubectl create ns prod
#kubectl create secret docker-registry harbor --docker-server=harbor.itdo.top --docker-username=admin --docker-password=Harbor.itdo12345 -n prod
说明:apollo portal服务在不同环境中可以共用，config/admin服务在每个环境得单独部署一套，分别使用连接不同库
创建ApolloConfigTestDB，创建ApolloConfigProdDB
mysql> drop database ApolloConfigDB; --不删除也可以，不影响
mysql> create database ApolloConfigTestDB;
mysql> use ApolloConfigTestDB;
mysql> source ./apolloconfigdb-test.sql; #注意修改里面的dbname，防止数据库误删除
mysql> update ApolloConfigTestDB.ServerConfig set ServerConfig.Value="http://config-test.itdo.top/eureka" where ServerConfig.Key="eureka.service.url";
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigTestDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";
mysql> create database ApolloConfigProdDB;
mysql> use ApolloConfigProdDB;
mysql> source ./apolloconfigdb-prod.sql; #注意修改里面的dbname，防止数据库误删除
mysql> update ApolloConfigProdDB.ServerConfig set ServerConfig.Value="http://config-prod.itdo.top/eureka" where ServerConfig.Key="eureka.service.url";
mysql> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigProdDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";
准备apollo-config，apollo-admin的资源配置清单（各2套），依次应用，分别发布在test和prod命名空间
# cd /data/k8s-yaml
--新建目录
# mkdir -pv test/{apollo-configservice,apollo-adminservice,dubbo-demo-service,dubbo-demo-consumer}
# mkdir -pv prod/{apollo-configservice,apollo-adminservice,dubbo-demo-service,dubbo-demo-consumer}
--configservice
--复制配置清单文件
# cp -a /data/k8s-yaml/apollo-configservice/*.yaml ./test/apollo-configservice/
# cp -a /data/k8s-yaml/apollo-configservice/*.yaml ./prod/apollo-configservice/
--修改并应用配置清单
# kubectl apply -f http://k8s-yaml.itdo.top/test/apollo-configservice/configmap.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/test/apollo-configservice/deployment.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/test/apollo-configservice/svc.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/test/apollo-configservice/ingress.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/apollo-configservice/configmap.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/apollo-configservice/deployment.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/apollo-configservice/svc.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/apollo-configservice/ingress.yaml
验证访问:
http://config-test.itdo.top
http://config-prod.itdo.top
配置清单修改如下参数：
--adminservice
--复制配置清单文件
# cp -a /data/k8s-yaml/apollo-adminservice/*.yaml ./test/apollo-adminservice/
# cp -a /data/k8s-yaml/apollo-adminservice/*.yaml ./prod/apollo-adminservice/
--应用配置清单
# kubectl apply -f http://k8s-yaml.itdo.top/test/apollo-adminservice/configmap.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/test/apollo-adminservice/deployment.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/apollo-adminservice/configmap.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/apollo-adminservice/deployment.yaml
配置清单修改如下参数：
修改ApolloPortalDB的参数
>update ServerConfig set Value='dev,fat,pro' where Id=1;
修改apollo-portal的configmap 并重启portal
apollo-env.properties: |
dev.meta=http://config.itdo.top
fat.meta=http://config-test.itdo.top
pro.meta=http://config-prod.itdo.top
#kubectl apply -f http://k8s-yaml.itdo.top/apollo-portal/configmap.yaml
Apollo的portal配置
管理员工具
删除应用、集群、AppNamespace，将已配置应用删除
注：规范操作应该在portal界面删除
系统参数
- Key
apollo.portal.envs - Value
dev,fat,pro
查询 - Value
dev,fat,pro
保存
新建dubbo-demo-service和dubbo-demo-web项目
在TEST/PROD环境分别增加配置项并发布(因为我没删除，所以不用重新创建项目)
dubbo-demo-service项目
选择FAT测试环境
dubbo.registry = zookeeper://zk2.itdo.top:2181
dubbo.port = 20880
选PRO生产环境
dubbo.registry = zookeeper://zk-prod.itdo.top:2181
dubbo.port = 20880
dubbo-demo-web项目
选择FAT测试环境
选PRO生产环境
发布dubbo微服务
准备dubbo-demo-service和dubbo-demo-web的资源配置清单（各2套），依次应用，分别发布至app-test和app-prod命名空间
--dubbo-demo-service
--复制配置清单文件
# cp -a /data/k8s-yaml/dubbo-demo-service/*.yaml ./test/dubbo-demo-service/
# cp -a /data/k8s-yaml/dubbo-demo-service/*.yaml ./prod/dubbo-demo-service/
--应用配置清单
# kubectl apply -f http://k8s-yaml.itdo.top/test/dubbo-demo-service/deployment-apollo.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/dubbo-demo-service/deployment-apollo.yaml
配置清单修改如下参数：
--dubbo-demo-consumer
--复制配置清单文件
# cp -a /data/k8s-yaml/dubbo-demo-consumer/*.yaml ./test/dubbo-demo-consumer/
# cp -a /data/k8s-yaml/dubbo-demo-consumer/*.yaml ./prod/dubbo-demo-consumer/
--应用配置清单
# kubectl apply -f http://k8s-yaml.itdo.top/test/dubbo-demo-consumer/deployment-apollo.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/test/dubbo-demo-consumer/service.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/test/dubbo-demo-consumer/ingress.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/dubbo-demo-consumer/deployment-apollo.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/dubbo-demo-consumer/service.yaml
# kubectl apply -f http://k8s-yaml.itdo.top/prod/dubbo-demo-consumer/ingress.yaml
访问验证:
http://demo-test.itdo.top/hello?name=test
http://demo-prod.itdo.top/hello?name=prod
配置清单修改如下参数：
使用dubbo-monitor查验
互联网公司技术部的日常
- 产品经理整理需求，需求评审，出产品原型
- 开发同学夜以继日的开发，提测
- 测试同学使用Jenkins持续集成，并发布至测试环境
- 验证功能，通过->待上线or打回->修改代码
- 提交发版申请，运维同学将测试后的包发往生产环境
项目提测发版流程
1.修改提交代码,告知版本版本；
2.使用jenkins构建新版本镜像；
3.修改测试环境dp对应的镜像地址，重启应用，测试过通过；
4.修改正式环境dp对应的镜像地址，重启应用，上线；