前言
Docker 与 Jenkins 经常会放到一起构建 CI (持续集成)系统这里结合Docker Registry 分享一下在Docker中构建 Jenkins 容器的相关操作，详细可以参阅 官方文档
Tip: 当前的最新版本为 Docker 1.10 Released on January 15, 2016
概要
自定义Jenkins镜像
准备构建环境
在构建环境中准备相应的证书文件和插件信息[root@docker docker]# mkdir build && cd build
[root@docker build]# pwd
/root/docker/build
[root@docker build]# vim plugins
[root@docker build]# cat plugins
role-strategy:2.2.0
[root@docker build]# cp ../../certs/docker.* .
[root@docker build]# ll
total 16
-rw------- 1 root root 1281 Jan 27 13:52 docker.crt
-rw------- 1 root root 1045 Jan 27 13:52 docker.csr
-rw------- 1 root root 1679 Jan 27 13:52 docker.key
-rw-r--r-- 1 root root 20 Jan 27 13:51 plugins
[root@docker build]#
Tip: 这里我使用的自签名证书
创建Dockerfile
[root@docker build]# vim Dockerfile
[root@docker build]# cat Dockerfile
FROM jenkins
#New plugins must be placed in the plugins file
COPY plugins /usr/share/jenkins/plugins
#The plugins.sh script will install new plugins
RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins
#Copy private key and cert to image
COPY docker.crt /var/lib/jenkins/cert
COPY docker.key /var/lib/jenkins/pk
#Configure HTTP off and HTTPS on, using port 1973
ENV JENKINS_OPTS --httpPort=-1 --httpsPort=1973 --httpsCertificate=/var/lib/jenkins/cert --httpsPrivateKey=/var/lib/jenkins/pk
[root@docker build]# ll
total 20
-rw------- 1 root root 1281 Jan 27 13:52 docker.crt
-rw------- 1 root root 1045 Jan 27 13:52 docker.csr
-rw-r--r-- 1 root root 497 Jan 27 14:13 Dockerfile
-rw------- 1 root root 1679 Jan 27 13:52 docker.key
-rw-r--r-- 1 root root 20 Jan 27 13:51 plugins
[root@docker build]#
Note: Dockerfile plugins 还有两个证书文件 (docker.crt and docker.key) 必须在同一个目录里，包含 Dockerfile 的目录叫作构建环境，文件只有放在构建环境中才能在构建过程中被集成进去
构建镜像
[root@docker build]# docker build -t ci-infrastructure/jnkns-img .
Sending build context to Docker daemon 9.728 kB
Step 1 : FROM jenkins
---> fc39417bd5fb
Step 2 : COPY plugins /usr/share/jenkins/plugins
---> 0139ec49d08d
Removing intermediate container a9f6f1ead720
Step 3 : RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins
---> Running in 1cab523a28c9
Downloading role-strategy:2.2.0
---> b65ed720a699
Removing intermediate container 1cab523a28c9
Step 4 : COPY docker.crt /var/lib/jenkins/cert
---> aa2257cc683d
Removing intermediate container 7f72a7d983b1
Step 5 : COPY docker.key /var/lib/jenkins/pk
---> 0e692408d048
Removing intermediate container b0bfe48b921b
Step 6 : ENV JENKINS_OPTS --httpPort=-1 --httpsPort=1973 --httpsCertificate=/var/lib/jenkins/cert --httpsPrivateKey=/var/lib/jenkins/pk
---> Running in 7ae0ddd5277e
---> c384253964b5
Removing intermediate container 7ae0ddd5277e
Successfully built c384253964b5
[root@docker build]# echo $?
0
[root@docker build]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
ci-infrastructure/jnkns-img latest c384253964b5 15 seconds ago 708.2 MB
<none> <none> d34c985f0918 15 hours ago 292.7 MB
<none> <none> 9287fae7a16e 32 hours ago 169.3 MB
ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
docker-registry:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
docker:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
h103:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
localhost:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
192.168.100.104:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
h104:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
localhost:5000/myfirstimage latest 8693db7e8a00 7 days ago 187.9 MB
jenkins latest fc39417bd5fb 2 weeks ago 708.1 MB
registry 2 683f9cd9cf88 3 weeks ago 224.5 MB
hello-world latest 0a6ba66e537a 3 months ago 960 B
[root@docker build]#
推送镜像
[root@docker build]# docker tag ci-infrastructure/jnkns-img docker:5000/ci/jnkns-img
[root@docker build]# docker push docker:5000/ci/jnkns-img
The push refers to a repository [docker:5000/ci/jnkns-img] (len: 1)
unable to ping registry endpoint https://docker:5000/v0/
v2 ping attempt failed with error: Get https://docker:5000/v2/: x509: certificate signed by unknown authority
v1 ping attempt failed with error: Get https://docker:5000/v1/_ping: x509: certificate signed by unknown authority
[root@docker build]# cd /root/certs
[root@docker certs]# ls
docker.crt docker.csr docker.key
[root@docker certs]# cat docker.crt >> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
[root@docker certs]# systemctl stop docker && systemctl start docker
[root@docker certs]# docker push docker:5000/ci/jnkns-img
The push refers to a repository [docker:5000/ci/jnkns-img] (len: 1)
Post https://docker:5000/v2/ci/jnkns-img/blobs/uploads/: no basic auth credentials
[root@docker certs]# docker login docker:5000
Username: testuser
Password:
Email: yyghdfz@163.com
WARNING: login credentials saved in /root/.docker/config.json
Login Succeeded
[root@docker certs]# docker push docker:5000/ci/jnkns-img
The push refers to a repository [docker:5000/ci/jnkns-img] (len: 1)
c384253964b5: Pushed
0e692408d048: Pushed
aa2257cc683d: Pushed
b65ed720a699: Pushed
0139ec49d08d: Pushed
fc39417bd5fb: Pushed
55422ac36eba: Pushed
b48f4074fc73: Pushed
53e20479e6a7: Pushed
585059426ec6: Pushed
6234bb424ca2: Pushed
b31b78b6c124: Pushed
7e844a128314: Pushed
6842d0a24c05: Pushed
9afbe4c3ddc8: Pushed
ff135e80b6aa: Pushed
05e608b5b672: Pushed
b12dfca65359: Pushed
4ee671494b6b: Pushed
ce2b29af7753: Pushed
5c63804eac90: Pushed
523ef1d23f22: Pushed
latest: digest: sha256:613ef35ff2fff0a26bab66dd9213463b034d4e536e9a6d52cbaeacb767fdf828 size: 87506
[root@docker certs]#
- 确保Registry地址没错，如果有问题可以使用
docker tag来调整 - 确保有证书，如果没有，要先导入，然后重启docker
- 确保进行了基础认证，如果没有要进行认证(在没有基础认证的Registry中不必关心这一点)
拉取镜像
可以使用其它的机器通过 docker pull 来测试一下上传的镜像[root@h104 certs]# docker pull docker:5000/ci/jnkns-img
Using default tag: latest
latest: Pulling from ci/jnkns-img
98f6335f9102: Pull complete
efae71df8aca: Pull complete
007c4719623e: Pull complete
0e54d32ff5f2: Pull complete
5b825467fc4f: Pull complete
Digest: sha256:613ef35ff2fff0a26bab66dd9213463b034d4e536e9a6d52cbaeacb767fdf828
Status: Downloaded newer image for docker:5000/ci/jnkns-img:latest
[root@h104 certs]#
[root@h104 certs]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
docker:5000/ci/jnkns-img latest 5b825467fc4f 35 minutes ago 708.2 MB
ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
192.168.100.103:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
docker:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
docker:5002/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
h103:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
localhost:5000/myfirstimage latest 8693db7e8a00 7 days ago 187.9 MB
localhost:5000/ubuntu latest 8693db7e8a00 7 days ago 187.9 MB
jenkins latest fc39417bd5fb 2 weeks ago 708.1 MB
registry 2 683f9cd9cf88 3 weeks ago 224.5 MB
hello-world latest 0a6ba66e537a 3 months ago 960 B
[root@h104 certs]#
通过镜像运行容器
[root@h104 ~]# docker run -p 1973:1973 --name jenkins01 docker:5000/ci/jnkns-img
Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Jan 27, 2016 1:27:39 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Jan 27, 2016 1:27:41 PM winstone.Logger logInternal
INFO: Winstone shutdown successfully
Jan 27, 2016 1:27:41 PM winstone.Logger logInternal
SEVERE: Container startup failed
java.io.IOException: Failed to start a listener: winstone.HttpsConnectorFactory
at winstone.Launcher.spawnListener(Launcher.java:209)
at winstone.Launcher.<init>(Launcher.java:149)
at winstone.Launcher.main(Launcher.java:354)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at Main._main(Main.java:293)
at Main.main(Main.java:98)
Caused by: java.io.FileNotFoundException: /var/lib/jenkins/cert (Permission denied)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at winstone.HttpsConnectorFactory.start(HttpsConnectorFactory.java:88)
at winstone.Launcher.spawnListener(Launcher.java:207)
... 8 more
[root@h104 ~]#
报错
出现了报错通过官方的文档，和docker hub中的说明没有找到根本原因
通过google，有人使用keystore解决了这个bug
暂时不使用https，降级构建Dockerfile (去掉https会丢失安全性，之后再回头慢慢研究原因)
注释掉https的相关配置，然后再构建镜像
[root@docker build]# vim Dockerfile
[root@docker build]# cat Dockerfile
FROM jenkins
#New plugins must be placed in the plugins file
COPY plugins /usr/share/jenkins/plugins
#The plugins.sh script will install new plugins
RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins
#Copy private key and cert to image
#COPY docker.crt /var/lib/jenkins/cert
#COPY docker.key /var/lib/jenkins/pk
#Configure HTTP off and HTTPS on, using port 1973
#ENV JENKINS_OPTS --httpPort=-1 --httpsPort=1973 --httpsCertificate=/var/lib/jenkins/cert --httpsPrivateKey=/var/lib/jenkins/pk
[root@docker build]#
[root@docker build]# docker build -t test/jnkns-img .
Sending build context to Docker daemon 9.728 kB
Step 1 : FROM jenkins
---> fc39417bd5fb
Step 2 : COPY plugins /usr/share/jenkins/plugins
---> Using cache
---> 0139ec49d08d
Step 3 : RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins
---> Using cache
---> b65ed720a699
Successfully built b65ed720a699
[root@docker build]#
[root@docker build]# docker tag test/jnkns-img docker:5000/ci/jnkns-img2
[root@docker build]# docker push docker:5000/ci/jnkns-img2
The push refers to a repository [docker:5000/ci/jnkns-img2] (len: 1)
b65ed720a699: Pushed
0139ec49d08d: Pushed
fc39417bd5fb: Pushed
0c27fdb0b33b: Pushed
55422ac36eba: Pushed
b48f4074fc73: Pushed
53e20479e6a7: Pushed
585059426ec6: Pushed
6234bb424ca2: Pushed
b31b78b6c124: Pushed
7e844a128314: Pushed
6842d0a24c05: Pushed
9afbe4c3ddc8: Pushed
ff135e80b6aa: Pushed
05e608b5b672: Pushed
b12dfca65359: Pushed
4ee671494b6b: Pushed
ce2b29af7753: Pushed
5c63804eac90: Pushed
523ef1d23f22: Pushed
latest: digest: sha256:b0593124c0f6790329649ae4863bb0c1d66b9aa32c9ce260c505a9ccfd185bd2 size: 78740
[root@docker build]#
[root@h104 ~]# docker run -p 8080:8080 --name jenkins01 docker:5000/ci/jnkns-img2
Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Jan 27, 2016 1:45:57 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Jan 27, 2016 1:45:59 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: jetty-winstone-2.8
Jan 27, 2016 1:46:01 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: NO JSP Support for , did not find org.apache.jasper.servlet.JspServlet
Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Jan 27, 2016 1:46:03 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Started SelectChannelConnector@0.0.0.0:8080
Jan 27, 2016 1:46:03 PM winstone.Logger logInternal
INFO: Winstone Servlet Engine v2.0 running: controlPort=disabled
Jan 27, 2016 1:46:04 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
Jan 27, 2016 1:46:20 PM jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
Jan 27, 2016 1:46:20 PM jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
Jan 27, 2016 1:46:20 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
Jan 27, 2016 1:46:20 PM jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
Jan 27, 2016 1:46:27 PM jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
Jan 27, 2016 1:46:28 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Started Download metadata
Jan 27, 2016 1:46:28 PM jenkins.util.groovy.GroovyHookScript execute
INFO: Executing /var/jenkins_home/init.groovy.d/tcp-slave-agent-port.groovy
Jan 27, 2016 1:46:28 PM org.jenkinsci.main.modules.sshd.SSHD start
INFO: Started SSHD at port 47557
Jan 27, 2016 1:46:29 PM jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
Jan 27, 2016 1:46:29 PM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
--> setting agent port for jnlp
--> setting agent port for jnlp... done
Jan 27, 2016 1:46:45 PM hudson.model.UpdateSite updateData
INFO: Obtained the latest update center data file for UpdateSource default
Jan 27, 2016 1:46:46 PM hudson.model.DownloadService$Downloadable load
INFO: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller
Jan 27, 2016 1:46:48 PM hudson.model.DownloadService$Downloadable load
INFO: Obtained the updated data file for hudson.tasks.Ant.AntInstaller
Jan 27, 2016 1:47:16 PM hudson.model.DownloadService$Downloadable load
INFO: Obtained the updated data file for hudson.tools.JDKInstaller
Jan 27, 2016 1:47:16 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Finished Download metadata. 48,013 ms
...
...
[root@h104 ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
236348a3c9ff docker:5000/ci/jnkns-img2 "/bin/tini -- /usr/lo" 7 minutes ago Up 7 minutes 0.0.0.0:8080->8080/tcp, 50000/tcp jenkins01
[root@h104 ~]# netstat -ant | grep 80
tcp6 0 0 :::8080 :::* LISTEN
[root@h104 ~]#
访问Jenkins
通过下面步骤进入已安装插件列表[系统管理]->[管理插件]->[已安装]
可以看到 Role-based Authorization Strategy 插件，版本和我们指定的一样
[root@docker build]# cat plugins
role-strategy:2.2.0
[root@docker build]#
命令汇总
cat pluginsvim Dockerfilecat Dockerfiledocker build -t ci-infrastructure/jnkns-img .docker imagesdocker tag ci-infrastructure/jnkns-img docker:5000/ci/jnkns-imgdocker push docker:5000/ci/jnkns-imgcat docker.crt >> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pemsystemctl stop docker && systemctl start dockerdocker login docker:5000docker push docker:5000/ci/jnkns-imgdocker pull docker:5000/ci/jnkns-imgdocker run -p 1973:1973 --name jenkins01 docker:5000/ci/jnkns-imgvim Dockerfilecat Dockerfiledocker build -t test/jnkns-img .docker tag test/jnkns-img docker:5000/ci/jnkns-img2docker push docker:5000/ci/jnkns-img2docker run -p 8080:8080 --name jenkins01 docker:5000/ci/jnkns-img2
