Jenkins更新検証環境をDockerで作る
Jenkins・pluginの更新をする際に検証できる環境が欲しい!となったので、Docker上に構築したメモ
成果物
やりたかったこと
- できるだけ運用環境と近い状態から以下の検証ができるようにする
- Jenkins自体の更新確認
- pluginの追加・更新確認
運用環境
- CentOS 7
- Jenkins version 2.138.3
- インストール参考: weblabo.oscasierra.net
Docker Hubからjenkins imageを探す
❯ docker pull jenkins/jenkins:2.138.3 2.138.3: Pulling from jenkins/jenkins Status: Downloaded newer image for jenkins/jenkins:2.138.3 docker.io/jenkins/jenkins:2.138.3 # コンテナの中に入る ❯ docker run -it --name jenkinstest jenkins/jenkins:2.138.3 /bin/bash jenkins@88681d8f77c5:/$ jenkins@88681d8f77c5:~$ cat /etc/issue Debian GNU/Linux 9 \n \l
Debian
だったのでこれは🙅
❯ docker pull jenkins/jenkins:2.182-centos 2.182-centos: Pulling from jenkins/jenkins Status: Downloaded newer image for jenkins/jenkins:2.182-centos docker.io/jenkins/jenkins:2.182-centos # 起動してみる ❯ docker run -p 8080:8080 -p 50000:50000 jenkins/jenkins:2.182-centos Running from: /usr/share/jenkins/jenkins.war webroot: EnvVars.masterEnvVars.get("JENKINS_HOME") # warファイルから起動してる?? # コンテナの中に入る ❯ docker run -it --name jenkins-update-test jenkins/jenkins:2.182-centos /bin/bash bash-4.2$ cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) bash-4.2$ echo $HOME /var/jenkins_home
war
ファイルからの起動のようなのでこれは🙅
Centos 7 のimageにJenkinsをyumインストールする方法を試す
rpmを探す
- Jenkins Redhat Packages
- 2.138.3 見つからない
- Jenkins mirrors redhat
- Jenkins mirrors redhat-stable
- http://mirrors.jenkins-ci.org/redhat-stable/jenkins-2.204.2-1.1.noarch.rpm
- ダウンロードできた一番古そうなrpmを使ってみる
Jenkinsインストール
Javaランタイムインストール
[root@b92d081ebab0 /]# yum -y install java-1.8.0-openjdk
Jenkins公式yumリポジトリの追加
[root@b92d081ebab0 /]# curl -o /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo [root@b92d081ebab0 /]# rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key [root@b92d081ebab0 /]# yum info jenkins Loaded plugins: fastestmirror, ovl Loading mirror speeds from cached hostfile * base: ftp.jaist.ac.jp * extras: ftp.jaist.ac.jp * updates: ftp.jaist.ac.jp jenkins | 2.9 kB 00:00:00 jenkins/primary_db | 35 kB 00:00:00 Available Packages Name : jenkins Arch : noarch Version : 2.263.1 Release : 1.1 Size : 64 M Repo : jenkins Summary : Jenkins Automation Server URL : http://jenkins.io/ License : MIT/X License, GPL/CDDL, ASL2 Description : Jenkins is an open source automation server which enables developers around the world to reliably : automate their development lifecycle processes of all kinds, including build, document, test, : package, stage, deployment, static analysis and many?more. : : Jenkins is being widely used in areas of Continous Integration, Continuous Delivery, DevOps, and : other areas. And it is not only about software, the same automation techniques can be applied in : other areas like Hardware Engineering, Embedded Systems, BioTech, etc. : : For information see https://jenkins.io : : : Authors: : -------- : Kohsuke Kawaguchi <kk@kohsuke.org>
rpmインストール
[root@b92d081ebab0 tmp]# curl -OL http://mirrors.jenkins-ci.org/redhat-stable/jenkins-2.204.2-1.1.noarch.rpm [root@b92d081ebab0 tmp]# ls hsperfdata_root jenkins-2.204.2-1.1.noarch.rpm ks-script-DrRL8A yum.log [root@b92d081ebab0 tmp]# rpm -ivh jenkins-2.204.2-1.1.noarch.rpm [root@b92d081ebab0 tmp]# systemctl start jenkins Failed to get D-Bus connection: Operation not permitted
Failed to get D-Bus connection: Operation not permitted
エラー
Dockerのエラーのようなので、以下を参考にして対応
一旦削除してdocker runやり直し 特権モードで起動させる
❯ docker run -d --privileged --name jenkins-test centos:centos7 /sbin/init ❯ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES cd5af186cc01 centos:centos7 "/sbin/init" 28 seconds ago Up 26 seconds jenkins-test ❯ docker exec -it jenkins-test /bin/bash [root@cd5af186cc01 /]# ## ~~java install, repository 登録, rpm install~~ [root@cd5af186cc01 tmp]# systemctl start jenkins Job for jenkins.service failed because the control process exited with error code. See "systemctl status jenkins.service" and "journalctl -xe" for details. [root@cd5af186cc01 tmp]# systemctl status jenkins.service ● jenkins.service - LSB: Jenkins Automation Server Loaded: loaded (/etc/rc.d/init.d/jenkins; bad; vendor preset: disabled) Active: failed (Result: exit-code) since Sun 2021-01-03 14:45:56 UTC; 15s ago Docs: man:systemd-sysv-generator(8) Process: 355 ExecStart=/etc/rc.d/init.d/jenkins start (code=exited, status=1/FAILURE) Jan 03 14:45:56 cd5af186cc01 systemd[1]: Starting LSB: Jenkins Automation Server... Jan 03 14:45:56 cd5af186cc01 jenkins[355]: /etc/rc.d/init.d/jenkins: line 59: /etc/init.d/functions: No s...tory Jan 03 14:45:56 cd5af186cc01 systemd[1]: jenkins.service: control process exited, code=exited status=1 Jan 03 14:45:56 cd5af186cc01 systemd[1]: Failed to start LSB: Jenkins Automation Server. Jan 03 14:45:56 cd5af186cc01 systemd[1]: Unit jenkins.service entered failed state. Jan 03 14:45:56 cd5af186cc01 systemd[1]: jenkins.service failed. Hint: Some lines were ellipsized, use -l to show in full.
/etc/rc.d/init.d/jenkins: line 59: /etc/init.d/functions: No such file or directory
エラー
以下と同じエラーだったので、同様の対応をしてみる
[root@cd5af186cc01 tmp]# yum provides */init.d/functions* [root@cd5af186cc01 tmp]# yum install -y initscripts [root@cd5af186cc01 tmp]# ls /etc/init.d/functions /etc/init.d/functions [root@cd5af186cc01 tmp]# systemctl start jenkins [root@cd5af186cc01 tmp]# systemctl status jenkins ● jenkins.service - LSB: Jenkins Automation Server Loaded: loaded (/etc/rc.d/init.d/jenkins; bad; vendor preset: disabled) Active: active (running) since Sun 2021-01-03 15:04:01 UTC; 7s ago Docs: man:systemd-sysv-generator(8) Process: 464 ExecStart=/etc/rc.d/init.d/jenkins start (code=exited, status=0/SUCCESS) CGroup: /docker/cd5af186cc0158a972d74dc63069144faa3849d05296134356de89d37a8f7a04/system.slice/jenkins.service └─493 /etc/alternatives/java -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_... Jan 03 15:04:00 cd5af186cc01 systemd[1]: Starting LSB: Jenkins Automation Server... Jan 03 15:04:00 cd5af186cc01 runuser[469]: pam_unix(runuser:session): session opened for user jenkins by (uid=0) Jan 03 15:04:01 cd5af186cc01 runuser[469]: pam_unix(runuser:session): session closed for user jenkins Jan 03 15:04:01 cd5af186cc01 jenkins[464]: Starting Jenkins [ OK ] Jan 03 15:04:01 cd5af186cc01 systemd[1]: Started LSB: Jenkins Automation Server. Hint: Some lines were ellipsized, use -l to show in full.
起動🙆
Dockerfileを作る
以下を参考に作成
Jenkins updateを試す
- サービスSTOP → yum install → サービスSTART
[root@localhost jenkins]# systemctl stop jenkins [root@localhost jenkins]# yum install jenkins Loaded plugins: fastestmirror, ovl Loading mirror speeds from cached hostfile * base: ty1.mirror.newmediaexpress.com * extras: ty1.mirror.newmediaexpress.com * updates: ty1.mirror.newmediaexpress.com Resolving Dependencies --> Running transaction check ---> Package jenkins.noarch 0:2.204.2-1.1 will be updated ---> Package jenkins.noarch 0:2.273-1.1 will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================ Package Arch Version Repository Size ================================================================================================================ Updating: jenkins noarch 2.273-1.1 jenkins 66 M Transaction Summary ================================================================================================================ Upgrade 1 Package Total download size: 66 M Is this ok [y/d/N]: y Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. jenkins-2.273-1.1.noarch.rpm | 66 MB 00:00:09 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : jenkins-2.273-1.1.noarch 1/2 Cleanup : jenkins-2.204.2-1.1.noarch 2/2 Verifying : jenkins-2.273-1.1.noarch 1/2 Verifying : jenkins-2.204.2-1.1.noarch 2/2 Updated: jenkins.noarch 0:2.273-1.1 Complete! [root@localhost jenkins]# [root@localhost jenkins]# systemctl start jenkins
workspaceが空だと一瞬で終わる模様
Jenkinsのjob,plugin等のバックアップは
のバックアップスクリプトを利用させてもらっている。 そこのコメントに気になる記述がある。
JenkinsCIををrpmでアップデートすると、$JENKINS_HOMEをchown -Rする糞コマンド入っているので、workspace/以下を綺麗にしておくかexport JENKINS_INSTALL_SKIP_CHOWN=trueしておかないと何時間もupdate終わらんとかになる
実際に更新する際は、 export JENKINS_INSTALL_SKIP_CHOWN=true
をしておくのがよさそう
バックアップデータからの復元でハマったところ
Error migrating users
バックアップファイルを解凍してjenkinsディレクトリにコピーすればOK
Docker imageの作成→コンテナ起動→jenkins起動は問題なかった
http://localhost:8080
にアクセスしてみると以下のエラーが表示され、ログイン画面がでない
java.nio.file.DirectoryNotEmptyException: /var/lib/jenkins/users/hoge at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:498) at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262) at java.nio.file.Files.move(Files.java:1395) at hudson.model.UserIdMigrator.migrateUsers(UserIdMigrator.java:97) at hudson.model.UserIdMapper.load(UserIdMapper.java:184) at hudson.model.UserIdMapper.init(UserIdMapper.java:75)
users
データの読み込みに失敗している?
/var/log/jenkins/jenkins.log
も見ながら解消していく
2021-01-15 02:48:09.783+0000 [id=34] INFO hudson.model.UserIdMigrator#migrateUsers: Migrating user 'hoge' from 'users/hoge/' to 'users/hoge_6878077609320809469/' 2021-01-15 02:48:09.792+0000 [id=34] SEVERE hudson.model.UserIdMapper#load: Error migrating users. java.nio.file.DirectoryNotEmptyException: /var/lib/jenkins/users/hoge at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:498) at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262) at java.nio.file.Files.move(Files.java:1395) at hudson.model.UserIdMigrator.migrateUsers(UserIdMigrator.java:97) at hudson.model.UserIdMapper.load(UserIdMapper.java:184) at hudson.model.UserIdMapper.init(UserIdMapper.java:75) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104) at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1121) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2021-01-15 02:48:09.807+0000 [id=34] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed UserIdMapper.init
エラー内容を調べる
jenkins-ci.361315.n4.nabble.com
In this case, rename the directory $JENKINS_HOME/users/admin_6635238516816951048/ to $JENKINS_HOME/users/admin/ while Jenkins is not running. Repeat this for all other user directories listed in the log. Doing all of the above will undo the partial migration. Jenkins will re-run it on the next startup.
対応する
このコメントを参考
service stop → users/users.xml
の削除 or rename → service start → 古いユーザーcredentialのまま使う
users/users.xml
はバックアップデータに存在しなかったので、以下の方法で対応
- バックアップの
users
ディレクリ名をusers_tmp
に変更 - 空の
users
ディレクトリを作成 - jenkins service startする
- この時点でJenkinsの画面が表示されるか確認。job等は復元されている状態
- ログインユーザー情報がないので、ログインはできない
- jenkins service stopする
users_tmp
ディレクト名をusers
に変更- jenkins service startする
- ログインできるか確認
おわりに
Dockerfileからのimage作成、コンテナ起動などはシェルスクリプトを作成した。
Dockerで更新検証環境を作ったことで、pluginを気軽に試したり更新したりできるようになりそう😋