测试环境说明:
harbor1
IP:10.XX.XX.X1
role:harbor、redis、mysql、postgreSQL
harbor2
IP:10.XX.XX.X2
role:harbor、nginx、nfs
功能
redis: harbor_session
mysql: harbor_db
posrgreSQL:clair_db
nfs: 共享存储盘(测试使用)
——————————————
环境需求与文档:
https://docs.docker.com/install/linux/docker-ce/centos/
[harbor ~]# docker –version
Docker version 19.03.6, build 369ce74a3c
python –version
Python 2.7.5
安装docker-compose
https://github.com/docker/compose/releases
docker-compose version 1.25.4, build 8d51620a
离线安装包
https://github.com/goharbor/harbor/releases/download/v1.10.1/harbor-offline-installer-v1.10.1.tgz
harbor-offline-installer-v1.10.1.tgz
v1.10.1初始化postgresql
https://github.com/goharbor/harbor/blob/v1.10.1/make/photon/db/initial-registry.sql
安装redis
https://redis.io/download
注意事项:
docker默认使用https,需要有证书进行配置
这是一个主从复制的案例:
https://www.jianshu.com/p/7374f7481d18
这个一个k8s集成文档
https://my.oschina.net/u/2306127/blog/1819645
参考文档
环境准备:https://www.cnblogs.com/breezey/p/9444231.html
selinux防火墙关闭
host主机解析
搭建nfs
部署外部redis、mysql
环境:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
| [harbor]# getenforce Permissive [harbor ~ ]# vim /etc/sysconfig/selinux [harbor]# systemctl status firewalld ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: man:firewalld(1) [harbor ~ ]# firewall-cmd --state not running [harbor ~ ]# python --version Python 2.7.5 [harbor ~ ]# docker --version Docker version 19.03.3, build a872fc2f86
|
安装nfs
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
| [harbor2 ~]# yum -y install nfs-utils [harbor2 ~]# mkdir /data/images [harbor2 ~]# vim /etc/exports [harbor2 ~]# cat /etc/exports /data/images 10.XX.XX.X2/24(rw,no_root_squash) [harbor2]# systemctl start nfs [harbor2]# systemctl status nfs ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled) Active: active (exited) since 二 2020-02-18 16:33:33 CST; 3s ago Process: 6650 ExecStartPost=/bin/sh -c if systemctl -q is-active gssproxy; then systemctl reload gssproxy ; fi (code=exited, status=0/SUCCESS) Process: 6633 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS) Process: 6631 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS) Main PID: 6633 (code=exited, status=0/SUCCESS) Tasks: 0 Memory: 0B CGroup: /system.slice/nfs-server.service
2月 18 16:33:33 master3 systemd[1]: Starting NFS server and services... 2月 18 16:33:33 master3 systemd[1]: Started NFS server and services.
|
挂载nfs
1 2 3 4 5
| [harbor1 ~]# yum -y install nfs-utils [harbor1 ~]# mkdir /data/images [harbor1 ~]# mount 10.XX.XX.X2:/data/images /data/images [root@master2 ~]# df -Th |grep /data/images 10.XX.XX.X2:/myimages nfs4 148G 15G 128G 10% /data/myimages
|
安装mysql客户端
1 2 3
| yum -y install mysql [harbor] /usr/bin/mysql
|
docker-compose启动redis和mysql
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
| [harbor ~]# cat docker-compose.yml version: '3' services:
redis: hostname: redis-server container_name: redis-server command: redis-server --requirepass your_passwd(reids一定要加密码,不然就会被挖矿) restart: always image: redis:3 volumes: - /database/redis:/data ports: - '6379:6379' postgres: hostname: postgres restart: always container_name: postgres-server image: postgres volumes: - /database/postgres:/data ports: - '5432:5432' environment: POSTGRES_PASSWORD: your_passwd
[root@master2 ~]# docker-compose up -d Creating network "root_default" with the default driver Creating redis-server ... done Creating postgres-server ... done Creating mysql-server ... done [root@master2 ~]# docker-compose ps Name Command State Ports -------------------------------------------------------------------------------------------- mysql-server docker-entrypoint.sh --cha ... Up 0.0.0.0:3306->3306/tcp, 33060/tcp postgres-server docker-entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp redis-server docker-entrypoint.sh redis ... Up 0.0.0.0:6379->6379/tcp
|
Mysql导入表结构
1 2 3 4 5 6 7 8 9 10 11 12 13 14
| [root@master2 ~]# mysql -h10.XX.XX.X1 -uroot -p mysql> CREATE DATABASE registry DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) mysql> use registry Database changed mysql> CREATE TABLE schema_migrations(version bigint not null primary key, dirty boolean not null); Query OK, 0 rows affected (0.05 sec) mysql> show tables; + | Tables_in_registry | + | schema_migrations | + 1 row in set (0.00 sec)
|
postgresql导入表结构
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
| yum install postgresql-server -y [root@master2 ~]# psql -h 10.XX.XX.X1 -p 5432 -U postgres 用户 postgres 的口令: kY#82oYVueDMDA psql (9.2.24, 服务器 12.2 (Debian 12.2-1.pgdg100+1)) 警告:psql 版本9.2, 服务器版本12.0. 一些psql功能可能无法工作. 输入 "help" 来获取帮助信息.
postgres=# \i /root/initial-registry.sql CREATE DATABASE psql (9.2.24, 服务器 12.2 (Debian 12.2-1.pgdg100+1)) 警告:psql 版本9.2, 服务器版本12.0. 一些psql功能可能无法工作. 您现在已经连线到数据库 "registry",用户 "postgres". CREATE TABLE
|
设置http访问
1 2 3 4 5 6 7 8 9
| [harbor1 ~]# find / -name docker.service
$DOCKER_OPTS \ $DOCKER_STORAGE_OPTIONS \ $DOCKER_NETWORK_OPTIONS \ $INSECURE_REGISTRY --insecure-registry 10.XX.XX.X1
|
解压启动harbor
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
| [harbor1 ~]# tar -zxvf harbor-offline-installer-v1.10.1.tgz harbor/harbor.v1.10.1.tar.gz
harbor/prepare harbor/LICENSE harbor/install.sh harbor/common.sh harbor/harbor.yml
[harbor1 ~]# vim harbor.yml 修改hostname、数据库密码、使用了外置数据库打开外置数据库的注释修改IP和密码 [harbor1 ~]# ./prepare 执行prepare检测启动环境 [harbor1 ~]# ./install.sh 启动harbor
|
登录、拉取、上传测试:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
| [root@master2 harbor]# docker login 10.XX.XX.X1 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[harbor1 harbor]# docker tag admin/test:v1 10.XX.XX.X1/library/admin/test [harbor1 harbor]# docker push 10.XX.XX.X1/library/admin/test
[root@master2 ~]# docker push 10.XX.XX.X1/library/postgres:v2 The push refers to repository [10.XX.XX.X1/library/postgres] 0f4c9531c043: Pushed f2296b808d85: Pushed 428585e93601: Pushed 0b14078a7e74: Pushed 2eb0b93970c9: Pushed 9b95b8eade9b: Pushed 8abf7e1726c6: Pushed ad5d7aba6303: Pushed 7a1725b13885: Pushed 1bb65a17f346: Pushed a872403d70e5: Pushed 9129eada97a4: Pushed efb2aa6f2c78: Pushed 488dfecc21b1: Pushed v2: digest: sha256:40ebbae0ce4d29eacb5a0ad3ae450f31f089124acdc2cc68b177872f716d0454 size: 3245
[harbor harbor]# docker pull 10.XX.XX.X2/library/admin/test:v1 v1: Pulling from library/admin/test bc51dd8edc1b: Pull complete d2b355dbb6c6: Pull complete d237363a1a91: Pull complete ff4b9d2fde66: Pull complete 646492d166e7: Pull complete 50eeac6fd5fb: Pull complete 502963de6da8: Pull complete d7263f7627b9: Pull complete d234d8f1a205: Pull complete 1b2c24e5275c: Pull complete 3f7c6dd9a7ae: Pull complete d6d6977a74b3: Pull complete acf1093f8b78: Pull complete 77e1ac8c247a: Pull complete Digest: sha256:40ebbae0ce4d29eacb5a0ad3ae450f31f089124acdc2cc68b177872f716d0454 Status: Downloaded newer image for 10.XX.XX.X2/library/admin/test:v1 10.XX.XX.X2/library/admin/test:v1 [harbor harbor]# docker images |grep test 10.XX.XX.X2/library/admin/test v1 0d2531ee3abd 5 days ago 397MB
[harbor ~]# docker tag goharbor/prepare:v1.10.1 10.XX.XX.X2/library/admin/test:v2 [harbor ~]# docker push 10.XX.XX.X2/library/admin/test:v2 The push refers to repository [10.XX.XX.X2/library/admin/test] bf5ac9b9c61c: Pushed 06e0f1585c01: Pushed 3710d94e58dd: Pushed 5c5501748347: Pushed f59c6315bf8a: Pushed fb3507ff707e: Pushed 93e0577272a9: Pushed v2: digest: sha256:e025a09df19c99e0afb4b116cbd698d5de27fb39e26314934460c7bae7c21afc size: 1787
|
Harbor预演测试过程
目前测试进度:
- 搭建 v1.10.1(harbor最新版本),单节点使用内置postgresql,和本地磁盘存储,测试成功
- 搭建 v1.10.1 使用外置mysql、postgresql、redis(全部docker-compose启动)harbor服务启动后不能登录,发现该版本不支持外置mysql(查阅官方文档https://github.com/goharbor/harbor/blob/master/docs/1.10/install-config/configure-yml-file.md显示目前该版本外置数据库仅支持postgresql),服务启动后redis auth认证有问题
- 搭建 v1.10.1 更改外置数据库为postgresql,redis使用手动搭建方式进行单点测试,搭建、测试拉取、推送镜像成功
- 版本 v1.10.0 和v1.8.0 经过查阅github文档、启动测试,该版本不支持外置mysql,目前仅支持postpresql
- 搭建 v1.10.1 外接数据库 postpres、redis ,2个harbor节点构成集群共享存储,搭建、测试拉取、推送镜像成功
还需继续研究项:
- 研究https认证
- 研究nginx keeplive 分流
- 版本 v1.5.4(发于2018.1.29版本)支持外置mysql,尝试搭建v1.5.4进行测试和调研
- 继续研究最新版harbor集群postgresql容灾方式
常用版本列举对比 https://github.com/goharbor/harbor/releases:
版本:v1.10
发版时间:2019.12.13
新功能对比:
外部数据库仅支持postgresql
支持第三方漏洞扫描程序
新增项目配置规则
管理员和用户增加功能,新增了“受限访客”用户
增强OIDC功能,复制功能增强
问题:
v1.10.0 job中有正在运行的任务,当redis重启后,任务被冻结,需点击重试恢复,后续补丁修复
v1.10.1 修复了1.9.3版本拉取镜像等https://github.com/goharbor/harbor/issues?q=is%3Aissue+label%3Atarget%2F1.10.1+is%3Aclosed
优缺点:
目前最新版,修复了旧版本的bug,增加了新的功能
统一了配置文件
版本:v1.8
发版时间:2019.5.21
新功能对比:
外部数据库仅支持postgresql
支持OIDC,使用外部验证验证身份
扩展了镜像复制功能
新增机器人账户
新增非harbor存储库之间互相复制镜像
问题:
复制策略标签过滤器丢失
不支持清单
某些情况下redis持久化数据巨大问题
优缺点:
查看官方文档和网上博客反馈,存在一些bug
配置文件也是统一的修改配置文件启动方式简单
版本:v1.5
发版时间:2018.10.26
新功能对比:
外部数据库支持mysql
升级Clair
NVD迁移至AWS
添加了镜像标签过滤器
添加LDAP组
用户可以为镜像添加标签
新增仓库只读模式
问题:
1.5.0 存在镜像标签问题,镜像复制签名问题
1.5.4 修复了前面几个版本的bug,不支持在线安装,支持离线安装
https://github.com/goharbor/harbor/releases/tag/v1.5.4
优缺点:
1.5.4支持外置mysql
1.5之后的版本,添加了新的功能,mysql换为postgresql,官方文档显示是将多个mysql迁移为统一的postgresql方便管理