2023年7月26日水曜日

ホームサーバーの環境移行(4)

気が付いたら5か月経っていた。早くね。
2月にやったことをちゃんとまとめていなかった報いか。

現状

  1. samba
  2. gogs
  3. MariaDB
  4. webmin

今回の対象

  • nextcolud

nextcloudの引っ越し

ここを参考にする
https://docs.nextcloud.com/server/latest/admin_manual/maintenance/index.html
https://hub.docker.com/_/nextcloud
https://blog.seigo2016.com/blog/h-blxsnew_s

事前準備

ユーザー作成

  • ユーザー:nextcloud_docker 1004
  • グループ:nextcloud-rtls-docker 200999

サブ UID/サブ GIDの設定

$ cat /etc/subuid
nextcloud_docker:200000:65536

$ cat /etc/subgid
nextcloud_docker:200000:65536

インストール

nextcloud_docker@blackcore:~$ dockerd-rootless-setuptool.sh install
[INFO] systemd not detected, dockerd-rootless.sh needs to be started manually:

PATH=/usr/bin:/sbin:/usr/sbin:$PATH dockerd-rootless.sh 

[INFO] CLI context "rootless" already exists
[INFO] Use CLI context "rootless"
Current context is now "rootless"

[INFO] Make sure the following environment variables are set (or add them to ~/.bashrc):

# WARNING: systemd not found. You have to remove XDG_RUNTIME_DIR manually on every logout.
export XDG_RUNTIME_DIR=/home/nextcloud_docker/.docker/run
export PATH=/usr/bin:$PATH
Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///home/nextcloud_docker/.docker/run/docker.sock
nextcloud_docker@blackcore:~$ cat .config/systemd/user/docker.service 
[Unit]
Description=Docker Application Container Engine (Rootless)
Documentation=https://docs.docker.com/go/rootless/

[Service]
Environment=PATH=/home/nextcloud_docker/bin:/sbin:/usr/sbin:/home/nextcloud_docker/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
ExecStart=/bin/dockerd-rootless.sh
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
Type=notify
NotifyAccess=all
KillMode=mixed

[Install]
WantedBy=default.target

設定

loginctl enable-linger nextcloud_docker
nextcloud_docker@blackcore:~$ XDG_RUNTIME_DIR=/run/user/$(id -u nextcloud_docker) systemctl --user daemon-reload
nextcloud_docker@blackcore:~$ XDG_RUNTIME_DIR=/run/user/$(id -u nextcloud_docker) systemctl --user start docker
nextcloud_docker@blackcore:~$ XDG_RUNTIME_DIR=/run/user/$(id -u nextcloud_docker) systemctl --user status docker
● docker.service - Docker Application Container Engine (Rootless)
     Loaded: loaded (/home/nextcloud_docker/.config/systemd/user/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-02-22 06:07:22 JST; 3s ago
       Docs: https://docs.docker.com/go/rootless/
   Main PID: 912102 (rootlesskit)
      Tasks: 54
     Memory: 45.7M
        CPU: 205ms
     CGroup: /user.slice/user-1004.slice/user@1004.service/app.slice/docker.service
             ├─912102 rootlesskit --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /bin/dockerd-rootless.sh
             ├─912113 /proc/self/exe --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /bin/dockerd-rootless.sh
             ├─912132 slirp4netns --mtu 65520 -r 3 --disable-host-loopback --enable-sandbox --enable-seccomp 912113 tap0
             ├─912140 dockerd
             └─912166 containerd --config /run/user/1004/docker/containerd/containerd.toml --log-level info

 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.095812655+09:00" level=warning msg="WARNING: No io.max (wbps) support"
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.095815109+09:00" level=warning msg="WARNING: No io.max (riops) support"
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.095817634+09:00" level=warning msg="WARNING: No io.max (wiops) support"
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.095820449+09:00" level=warning msg="WARNING: bridge-nf-call-iptables is disabled"
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.095823155+09:00" level=warning msg="WARNING: bridge-nf-call-ip6tables is disabled"
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.095831981+09:00" level=info msg="Docker daemon" commit=bc3805a graphdriver=overlay2 version=23.0.1
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.095857399+09:00" level=info msg="Daemon has completed initialization"
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.104160647+09:00" level=info msg="[core] [Server #10] Server created" module=grpc
 2月 22 06:07:22 blackcore systemd[892130]: Started Docker Application Container Engine (Rootless).
 2月 22 06:07:22 blackcore dockerd-rootless.sh[912140]: time="2023-02-22T06:07:22.110857925+09:00" level=info msg="API listen on /run/user/1004/docker.sock"

XDG_RUNTIME_DIR=/run/user/$(id -u nextcloud_docker) systemctl --user enable docker

動作確認

hello-worldが動くかでテスト

nextcloud_docker@blackcore:~$ docker -H unix:///run/user/1004/docker.sock run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete 
Digest: sha256:6e8b6f026e0b9c419ea0fd02d3905dd0952ad1feea67543f525c73a0a790fefb
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

nextcloud用のdocker-composeを作成

nextcloud_docker@blackcore:/mnt/backuparea/nextcloud$ cat db.env
MYSQL_ROOT_PASSWORD=****************
MYSQL_PASSWORD=****************
MYSQL_DATABASE=nextcloud
MYSQL_USER=nextcloud

nextcloud_docker@blackcore:/mnt/backuparea/nextcloud$ cat docker-compose.yml 
version: '3'

services:
  db:
    image: mariadb:10.5
    command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW
    restart: always
    volumes:
      - db:/var/lib/mysql
    environment:
      - MARIADB_AUTO_UPGRADE=1
      - MARIADB_DISABLE_UPGRADE_BACKUP=1
    env_file:
      - db.env
    ports:
      - 23306:3306

  redis:
    image: redis:alpine
    restart: always

  app:
    image: nextcloud:fpm-alpine
    restart: always
    volumes:
      - nextcloud:/var/www/html
    environment:
      - MYSQL_HOST=db
      - REDIS_HOST=redis
    env_file:
      - db.env
    depends_on:
      - db
      - redis

  web:
    image: nginx
    restart: always
    ports:
      - 28080:80
    volumes:
      - nextcloud:/var/www/html:ro
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - app

  cron:
    image: nextcloud:fpm-alpine
    restart: always
    volumes:
      - nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db
      - redis

volumes:
  db:
  nextcloud:

あとは

nextcloud_docker@blackcore:/mnt/backuparea/nextcloud$ cat nginx.conf 
worker_processes auto;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    # Prevent nginx HTTP Server Detection
    server_tokens   off;

    keepalive_timeout  65;

    #gzip  on;

    upstream php-handler {
        server app:9000;
    }

    server {
        listen 80;

        # HSTS settings
        # WARNING: Only add the preload option once you read about
        # the consequences in https://hstspreload.org/. This option
        # will add the domain to a hardcoded list that is shipped
        # in all major browsers and getting removed from this list
        # could take several months.
        #add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;" always;

        # set max upload size
        client_max_body_size 512M;
        fastcgi_buffers 64 4K;

        # Enable gzip but do not remove ETag headers
        gzip on;
        gzip_vary on;
        gzip_comp_level 4;
        gzip_min_length 256;
        gzip_proxied expired no-cache no-store private no_last_modified no_etag auth;
        gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;

        # Pagespeed is not supported by Nextcloud, so if your server is built
        # with the `ngx_pagespeed` module, uncomment this line to disable it.
        #pagespeed off;

        # HTTP response headers borrowed from Nextcloud `.htaccess`
        add_header Referrer-Policy                      "no-referrer"   always;
        add_header X-Content-Type-Options               "nosniff"       always;
        add_header X-Download-Options                   "noopen"        always;
        add_header X-Frame-Options                      "SAMEORIGIN"    always;
        add_header X-Permitted-Cross-Domain-Policies    "none"          always;
        add_header X-Robots-Tag                         "none"          always;
        add_header X-XSS-Protection                     "1; mode=block" always;

        # Remove X-Powered-By, which is an information leak
        fastcgi_hide_header X-Powered-By;

        # Path to the root of your installation
        root /var/www/html;

        # Specify how to handle directories -- specifying `/index.php$request_uri`
        # here as the fallback means that Nginx always exhibits the desired behaviour
        # when a client requests a path that corresponds to a directory that exists
        # on the server. In particular, if that directory contains an index.php file,
        # that file is correctly served; if it doesn't, then the request is passed to
        # the front-end controller. This consistent behaviour means that we don't need
        # to specify custom rules for certain paths (e.g. images and other assets,
        # `/updater`, `/ocm-provider`, `/ocs-provider`), and thus
        # `try_files $uri $uri/ /index.php$request_uri`
        # always provides the desired behaviour.
        index index.php index.html /index.php$request_uri;

        # Rule borrowed from `.htaccess` to handle Microsoft DAV clients
        location = / {
            if ( $http_user_agent ~ ^DavClnt ) {
                return 302 /remote.php/webdav/$is_args$args;
            }
        }

        location = /robots.txt {
            allow all;
            log_not_found off;
            access_log off;
        }

        # Make a regex exception for `/.well-known` so that clients can still
        # access it despite the existence of the regex rule
        # `location ~ /(\.|autotest|...)` which would otherwise handle requests
        # for `/.well-known`.
        location ^~ /.well-known {
            # The rules in this block are an adaptation of the rules
            # in `.htaccess` that concern `/.well-known`.

            location = /.well-known/carddav { return 301 /remote.php/dav/; }
            location = /.well-known/caldav  { return 301 /remote.php/dav/; }

            location /.well-known/acme-challenge    { try_files $uri $uri/ =404; }
            location /.well-known/pki-validation    { try_files $uri $uri/ =404; }

            # Let Nextcloud's API for `/.well-known` URIs handle all other
            # requests by passing them to the front-end controller.
            return 301 /index.php$request_uri;
        }

        # Rules borrowed from `.htaccess` to hide certain paths from clients
        location ~ ^/(?:build|tests|config|lib|3rdparty|templates|data)(?:$|/)  { return 404; }
        location ~ ^/(?:\.|autotest|occ|issue|indie|db_|console)                { return 404; }

        # Ensure this block, which passes PHP files to the PHP process, is above the blocks
        # which handle static assets (as seen below). If this block is not declared first,
        # then Nginx will encounter an infinite rewriting loop when it prepends `/index.php`
        # to the URI, resulting in a HTTP 500 error response.
        location ~ \.php(?:$|/) {
            # Required for legacy support
            rewrite ^/(?!index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|oc[ms]-provider\/.+|.+\/richdocumentscode\/proxy) /index.php$request_uri;

            fastcgi_split_path_info ^(.+?\.php)(/.*)$;
            set $path_info $fastcgi_path_info;

            try_files $fastcgi_script_name =404;

            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param PATH_INFO $path_info;
            #fastcgi_param HTTPS on;

            fastcgi_param modHeadersAvailable true;         # Avoid sending the security headers twice
            fastcgi_param front_controller_active true;     # Enable pretty urls
            fastcgi_pass php-handler;

            fastcgi_intercept_errors on;
            fastcgi_request_buffering off;
        }

        location ~ \.(?:css|js|svg|gif)$ {
            try_files $uri /index.php$request_uri;
            expires 6M;         # Cache-Control policy borrowed from `.htaccess`
            access_log off;     # Optional: Don't log access to assets
        }

        location ~ \.woff2?$ {
            try_files $uri /index.php$request_uri;
            expires 7d;         # Cache-Control policy borrowed from `.htaccess`
            access_log off;     # Optional: Don't log access to assets
        }

        # Rule borrowed from `.htaccess`
        location /remote {
            return 301 /remote.php$request_uri;
        }

        location / {
            try_files $uri $uri/ /index.php$request_uri;
        }
    }
}

実行してみる

nextcloud_docker@blackcore:/mnt/backuparea/nextcloud$ docker-compose -H unix:///run/user/1004/docker.sock up
Starting nextcloud_db_1    ... done
Starting nextcloud_redis_1 ... done
Starting nextcloud_cron_1  ... done
Starting nextcloud_app_1   ... done
Starting nextcloud_web_1   ... done
Attaching to nextcloud_db_1, nextcloud_redis_1, nextcloud_app_1, nextcloud_cron_1, nextcloud_web_1
db_1     | 2023-03-16 20:30:42+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.5.19+maria~ubu2004 started.

動いた!


バックアップデータを復元してみる

https://docs.nextcloud.com/server/latest/admin_manual/maintenance/index.html
https://docs.nextcloud.com/server/latest/admin_manual/maintenance/migrating.html#

dbデータ

  • import
mysql -h localhost --port=23306 -u root -p nextcloud < nextcloud-sqlbkp.bak 
  • oc_storagesの変更

dataファイル

  • コピー
sudo cp -r nextcloud/ /mnt/backuparea/nextcloud/data/
sudo chown 200081 -R /mnt/backuparea/nextcloud/data/
sudo chgrp 200081 -R /mnt/backuparea/nextcloud/data/
  • パーミッション
adeno@blackcore:/mnt/backuparea/nextcloud$ ls -l data/nextcloud/data
合計 135988
drwxr-xr-x  7 200081 extcloud-rtls-docker      4096  2月 17  2019 admin
(略)
  • 備忘録
    最終手段でシンボリックリンクで対応したと思ったけど、何だっけ??

微調整

セキュリティ&セットアップ警告

HTTPSの対応

Traefikを試してみる?

https://coders-shelf.com/traefik-intro/
https://qiita.com/adwin/items/ccc34ef5f4c88d8fa02c#おまけ-https-化も楽勝

cat traefik.yml 
------------------------------------------------------------
api:
  insecure: true # WebUI にアクセスできるように設定
  dashboard: true

entryPoints:
  http:
    address: ":80"

  https:
    address: ":20443"

providers:
  docker:
#    network: sample_traefik
    exposedByDefault: false

cat docker-compose.yml 
------------------------------------------------------------
ersion: '3'

services:
  db:
    image: mariadb:10.5
    command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW
    restart: always
    volumes:
      - ./data/db:/var/lib/mysql
    environment:
      - MARIADB_AUTO_UPGRADE=1
      - MARIADB_DISABLE_UPGRADE_BACKUP=1
    env_file:
      - db.env
    ports:
      - 23306:3306

  redis:
    image: redis:alpine
    restart: always

  app:
    image: nextcloud:fpm-alpine
    restart: always
    volumes:
      - ./data/nextcloud:/var/www/html
    environment:
      - MYSQL_HOST=db
      - REDIS_HOST=redis
    env_file:
      - db.env
    depends_on:
      - db
      - redis

  web:
    image: nginx
    restart: always
    ports:
      - 28080:80
    volumes:
      - ./data/nextcloud:/var/www/html:ro
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - app
    labels:
      - traefik.enable=true
      - traefik.http.routers.servicename.rule=Host(`blackcore.local`)
      - traefik.http.routers.servicename.entrypoints=https
      - traefik.http.routers.servicename.tls=true

  cron:
    image: nextcloud:fpm-alpine
    restart: always
    volumes:
      - ./data/nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db
      - redis

  traefik:
    image: traefik
    ports:
      - 28081:8080
      - 28082:80
      - 20443:20443
    volumes:
      - /run/user/1004/docker.sock:/var/run/docker.sock:ro
      - ./traefik.yml:/etc/traefik/traefik.yml

volumes:
  db:
  nextcloud:

httpsでのアクセスで、404 not foundとなってしまう。

nginxでhttpsを

そもそもリバースプロキシではなくて、nginxでhttps対応ができればやりたいことはできる。
nginx.confに以下を追加

cat nginx.conf
---
	# SSL configuration
	#
	listen 443 ssl default_server;
	listen [::]:443 ssl default_server;
 	#ssl_certificate /etc/nginx/server.crt;
	#ssl_certificate_key /etc/nginx/server.key;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers HIGH:!aNULL:!MD5;
	ssl_certificate     /etc/letsencrypt/live/*****.jp/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/*****.jp/privkey.pem;

長くなってきたし、時間も空いてしまってよくわからなくなってきたので、1回整理したい。