ããã¯ããªã«ãããããŠæžãããã®ïŒ
åã ãã1床詊ããŠããããã£ããAmazon S3äºæã®ãªããžã§ã¯ãã¹ãã¬ãŒãžMinIOãæ±ã£ãŠã¿ããããªãšã
MinIOãšã¯
MinIOã¯ãAmazon S3äºæã®ãªããžã§ã¯ãã¹ãã¬ãŒãžã§ãã
MinIO | High Performance, Kubernetes Native Object Storage
Goã§å®è£ ãããŠããŸãã
GitHub - minio/minio: High Performance, Kubernetes Native Object Storage
3ã€ã®ã©ã€ã»ã³ã¹åœ¢æ ãããããã§ããããµããŒãã«å·®ãããã¿ããã§ããã
ããã¥ã¡ã³ãã¯ããã¢ã¡ã¿ã«ç°å¢åããKubernetesç°å¢åããVMware Cloud Fundationç°å¢åãã®ãã®ããããŸããã
ä»åã¯ãã¢ã¡ã¿ã«ç°å¢åãã®ãã®ãåç
§ããŸãã
MinIO High Performance Object Storage — MinIO Baremetal Documentation
MinIO Object Storage for Hybrid Cloud — MinIO Hybrid Cloud Documentation
æ©èœã«ã€ããŠã¯ããã¡ãã
MinIO Server Features — MinIO Baremetal Documentation
NotificationãããŒãžã§ãã³ã°ãèé害æ§ãã¬ããªã±ãŒã·ã§ã³ãåããŸãã
ã¡ãªã¿ã«ããããã®ããã¥ã¡ã³ãã¯çŸåšäœæäžã§ãã¬ã¬ã·ãŒæ±ããšãªã£ãããã¥ã¡ã³ãããããŸããçŸåšã®ããã¥ã¡ã³ãã«
èšèŒããªããã®ã«ã€ããŠã¯ãã¬ã¬ã·ãŒããã¥ã¡ã³ãã®æ¹ãåç
§ããå¿
èŠãããããã§ãã
MinIO | The MinIO Quickstart Guide
ããšãã°ãAWS CLIãåçš®ããã°ã©ãã³ã°èšèªåãã®ã¯ã©ã€ã¢ã³ãã©ã€ãã©ãªãªã©ã¯ã¬ã¬ã·ãŒããã¥ã¡ã³ãã«ãã
ãããŸããã
MinIO | AWS CLI with MinIO - Cookbook/Recipe
MinIO | Java Client Quickstart Guide
Amazon S3äºæãšããããšã§ãAWS CLIãAWS SDKã䜿ããããã§ãããå°çšã®ã¯ã©ã€ã¢ã³ãã³ãã³ãã管çã³ãã³ãã
ã¯ã©ã€ã¢ã³ãã©ã€ãã©ãªãåããŠããŸãã
MinIO Client (mc) — MinIO Baremetal Documentation
MinIO Admin (mc admin) — MinIO Baremetal Documentation
ä»åã¯ããšããããã¹ã¿ã³ãã¢ãã³ã§MinIOãµãŒããŒãç«ãŠãŠãã¯ã©ã€ã¢ã³ãã³ãã³ããAWS CLIã§ã¢ã¯ã»ã¹ããŠ
ã¿ãããšæããŸãã
ç°å¢
ä»åã®ç°å¢ã¯ããã¡ããUbuntu Linux 20.04 LTSã§ãã
$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.3 LTS Release: 20.04 Codename: focal $ uname -srvmpio Linux 5.4.0-84-generic #94-Ubuntu SMP Thu Aug 26 20:27:37 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
AWS CLIãã€ã³ã¹ããŒã«ããŠãããŸããã
$ aws --version aws-cli/2.2.36 Python/3.8.8 Linux/5.4.0-84-generic exe/x86_64.ubuntu.20 prompt/off
MinIOãµãŒããŒãã€ã³ã¹ããŒã«ãã
ãŸãã¯ãMinIOãµãŒããŒãã€ã³ã¹ããŒã«ããŸãã
MinIO | Code and downloads to create high performance object storage
ã€ã³ã¹ããŒã«æ¹æ³ã¯å®è¡ãã€ããªãrpmããã±ãŒãžãdebããã±ãŒãžããããŠDockerã€ã¡ãŒãžããããŸãããä»åã¯
debããã±ãŒãžã®ãã®ã䜿ãããšã«ããŸãã
debããã±ãŒãžãããŠã³ããŒãããŠ
$ curl -O https://dl.min.io/server/minio/release/linux-amd64/minio_20210903035613.0.0_amd64.deb
ã€ã³ã¹ããŒã«ã
$ sudo dpkg -i minio_20210903035613.0.0_amd64.deb
ããŒãžã§ã³ã
$ minio --version minio version RELEASE.2021-09-03T03-56-13Z
å®è¡æ¹æ³ã«ã€ããŠã¯ãã¡ãã«èšèŒããããŸããã
MinIO Server — MinIO Baremetal Documentation
ãã«ãã確èªã
$ minio --help NAME: minio - High Performance Object Storage DESCRIPTION: Build high performance data infrastructure for machine learning, analytics and application data workloads with MinIO USAGE: minio [FLAGS] COMMAND [ARGS...] COMMANDS: server start object storage server gateway start object storage gateway FLAGS: --certs-dir value, -S value path to certs directory (default: "/home/vagrant/.minio/certs") --quiet disable startup information --anonymous hide sensitive information from logging --json output server logs and startup information in json format --help, -h show help --version, -v print the version VERSION: RELEASE.2021-09-03T03-56-13Z
èµ·åæ¹æ³ã«server
ãšgateway
ãããããã§ãã
MinIO Gatewayã¯ãAWSãAzureã®ãªããžã§ã¯ãã¹ãã¬ãŒãžãããã¯ãšã³ãã«ããããã»ã¹ã§ãã
ä»åã¯ãserver
ã®æ¹ã䜿ããŸãã
$ minio server --help NAME: minio server - start object storage server USAGE: minio server [FLAGS] DIR1 [DIR2..] minio server [FLAGS] DIR{1...64} minio server [FLAGS] DIR{1...64} DIR{65...128} DIR: DIR points to a directory on a filesystem. When you want to combine multiple drives into a single large system, pass one directory per filesystem separated by space. You may also use a '...' convention to abbreviate the directory arguments. Remote directories in a distributed setup are encoded as HTTP(s) URIs. FLAGS: --address value bind to a specific ADDRESS:PORT, ADDRESS can be an IP or hostname (default: ":9000") --console-address value bind to a specific ADDRESS:PORT for embedded Console UI, ADDRESS can be an IP or hostname --certs-dir value, -S value path to certs directory (default: "/home/vagrant/.minio/certs") --quiet disable startup information --anonymous hide sensitive information from logging --json output server logs and startup information in json format --help, -h show help EXAMPLES: 1. Start minio server on "/home/shared" directory. $ minio server /home/shared 2. Start single node server with 64 local drives "/mnt/data1" to "/mnt/data64". $ minio server /mnt/data{1...64} 3. Start distributed minio server on an 32 node setup with 32 drives each, run following command on all the nodes $ export MINIO_ROOT_USER=minio $ export MINIO_ROOT_PASSWORD=miniostorage $ minio server http://node{1...32}.example.com/mnt/export{1...32} 4. Start distributed minio server in an expanded setup, run the following command on all the nodes $ export MINIO_ROOT_USER=minio $ export MINIO_ROOT_PASSWORD=miniostorage $ minio server http://node{1...16}.example.com/mnt/export{1...32} \ http://node{17...64}.example.com/mnt/export{1...64}
ããŒã¿ãä¿åããããã®ãã£ã¬ã¯ããªãå¿ èŠãªãããªã®ã§ãäœæããŠ
$ mkdir data
èµ·åã
$ minio server data API: http://192.168.121.136:9000 http://192.168.33.10:9000 http://127.0.0.1:9000 RootUser: minioadmin RootPass: minioadmin Console: http://192.168.121.136:42799 http://192.168.33.10:42799 http://127.0.0.1:42799 RootUser: minioadmin RootPass: minioadmin Command-line: https://docs.min.io/docs/minio-client-quickstart-guide $ mc alias set myminio http://192.168.121.136:9000 minioadmin minioadmin Documentation: https://docs.min.io WARNING: Console endpoint is listening on a dynamic port (42799), please use --console-address ":PORT" to choose a static port. WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables
ç¹ã«æå®ããªããšãWeb Consoleã®ããŒããã©ã³ãã ã«æ±ºãŸããããªã®ã§ãæå®ããŠãããŸããããã¥ã¡ã³ãã«ç¿ã£ãŠ
9001
ã«ããŠãããŸããã
$ minio server data --console-address ":9001" API: http://192.168.121.136:9000 http://192.168.33.10:9000 http://127.0.0.1:9000 RootUser: minioadmin RootPass: minioadmin Console: http://192.168.121.136:9001 http://192.168.33.10:9001 http://127.0.0.1:9001 RootUser: minioadmin RootPass: minioadmin Command-line: https://docs.min.io/docs/minio-client-quickstart-guide $ mc alias set myminio http://192.168.121.136:9000 minioadmin minioadmin Documentation: https://docs.min.io WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables
èµ·åæã«è¡šç€ºãããŠããã®ã¯ãMinIOã®ç®¡çãŠãŒã¶ãŒã®ååãšãã¹ã¯ãŒãã§ããããAWS SDKã®ã¯ã¬ãã³ã·ã£ã«ã§ã
ãããŸãã
ããã¯ãMINIO_ROOT_USER
ããã³MINIO_ROOT_PASSWORD
ç°å¢å€æ°ã§æå®ããããšãã§ããŸãã
$ MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=password minio server data --console-address ":9001" API: http://192.168.121.136:9000 http://192.168.33.10:9000 http://127.0.0.1:9000 RootUser: admin RootPass: password Console: http://192.168.121.136:9001 http://192.168.33.10:9001 http://127.0.0.1:9001 RootUser: admin RootPass: password Command-line: https://docs.min.io/docs/minio-client-quickstart-guide $ mc alias set myminio http://192.168.121.136:9000 admin password Documentation: https://docs.min.io
ä»åã¯ãããã©ã«ãã®minioadmin
/ minioadmin
ããã®ãŸãŸäœ¿ãããšã«ããŸãã
$ minio server data --console-address ":9001"
MinIOã¯ã©ã€ã¢ã³ããã€ã³ã¹ããŒã«ãã
ç¶ããŠã¯ãMinIOã¯ã©ã€ã¢ã³ããã€ã³ã¹ããŒã«ããŸãããã
ãã¡ãããdebããã±ãŒãžã§ã€ã³ã¹ããŒã«ããŸããããŠã³ããŒãããŠ
$ curl -O https://dl.min.io/client/mc/release/linux-amd64/mcli_20210902092127.0.0_amd64.deb
ã€ã³ã¹ããŒã«ã
$ sudo dpkg -i mcli_20210902092127.0.0_amd64.deb
ããŒãžã§ã³ã
$ mcli --version mcli version RELEASE.2021-09-02T09-21-27Z
ã¡ãªã¿ã«ãå®è¡å¯èœãã¡ã€ã«ã§ã€ã³ã¹ããŒã«ãããšmc
ã³ãã³ããšãªããrpmããã±ãŒãžãdebããã±ãŒãžã§ã€ã³ã¹ããŒã«
ãããšmcli
ã³ãã³ãã«ãªããŸãã
MinIO | Code and downloads to create high performance object storage
ã©ã¡ãã«ãããã€ã³ã¹ããŒã«ãããã®ã¯åäžãã€ããªã ãã§ããã
$ dpkg -L mcli /usr /usr/local /usr/local/bin /usr/local/bin/mcli
ããŠã³ããŒãããŒãžã«èšèŒãããŠããã³ãã³ãã«åŸããmyminio
ãšããååã«ããŒã«ã«ã®MinIOããšã€ãªã¢ã¹ãšããŠ
èšå®ããŸãããã®æããŠãŒã¶ãŒåãšãã¹ã¯ãŒããå¿
èŠã§ãã
$ mcli alias set myminio http://localhost:9000 minioadmin minioadmin
MinIOãµãŒããŒã®æ å ±ã確èªã
$ mcli admin info myminio â localhost:9000 Uptime: 6 minutes Version: 2021-09-03T03:56:13Z Network: 1/1 OK
ããã€ãã³ãã³ããè©ŠããŠã¿ãŸãã
ãã±ããã®äœæã
$ mcli mb myminio/my-bucket Bucket created successfully `myminio/my-bucket`.
確èªã
$ mcli ls myminio [2021-09-10 22:58:17 JST] 0B my-bucket/
ããŒã«ã«ã§ãã¡ã€ã«ãäœæããŠ
$ echo 'Hello World' > hello.txt
ã¢ããããŒãã
$ mcli cp hello.txt myminio/my-bucket/hello.txt
ããŠã³ããŒãã
$ mcli cp myminio/my-bucket/hello.txt hello-copy.txt $ cat hello-copy.txt Hello World
ãã¡ã€ã«åé€ã
$ mcli rm myminio/my-bucket/hello.txt Removing `myminio/my-bucket/hello.txt`.
ãã±ããåé€ã
$ mcli rb myminio/my-bucket Removed `myminio/my-bucket` successfully.
ãã1床ãã±ããã®äœæãšãã¡ã€ã«ã¢ããããŒãããããŠãã±ãããããã²ãšã€äœæããŸãã
$ mcli mb myminio/my-bucket $ mcli cp hello.txt myminio/my-bucket/hello.txt $ mcli mb myminio/my-bucket2
èµ·åæã«--console-address ":9001"
ãæå®ããã®ã§ãhttp://[MinIOãµãŒããŒã®IPã¢ãã¬ã¹]:9001
ã§Web Consoleã«
ã¢ã¯ã»ã¹ã§ããŸãã
MinIO Console — MinIO Baremetal Documentation
ãã°ã€ã³ã¢ã«ãŠã³ãã¯ãèµ·åæã«è¡šç€ºããããŠãŒã¶ãŒåãšãã¹ã¯ãŒãã§ãã
Web Consoleã§ã¯ãMinioãµãŒããŒã®æ å ±ãåç §ãããããã±ãããžã®ã¢ã¯ã»ã¹ããŠãŒã¶ãŒç®¡çãªã©ãã§ãããããŸãã
AWS CLIã§ã¢ã¯ã»ã¹ãã
ç¶ããŠãAWS CLIã§ã¢ã¯ã»ã¹ããŠã¿ãŸãããã
ã¯ã¬ãã³ã·ã£ã«ãšãªãŒãžã§ã³ã¯ãç°å¢å€æ°ã§æå®ããããšã«ããŸãã
$ export AWS_ACCESS_KEY_ID=minioadmin $ export AWS_SECRET_ACCESS_KEY=minioadmin $ export AWS_DEFAULT_REGION=ap-northeast-1
AWS CLIã§ã¢ã¯ã»ã¹ããéã¯ã--endpoint-url http://[MinIOãµãŒããŒã®IPã¢ãã¬ã¹]:9000
ãæå®ããŸãã
$ aws --endpoint-url http://localhost:9000 s3 ls 2021-09-10 23:03:54 my-bucket 2021-09-10 23:04:19 my-bucket2
å ã»ã©ãMinIOã®ã¯ã©ã€ã¢ã³ãããŒã«ã§äœæãããã±ãããèŠããŠããŸãã
ãã¡ã€ã«ã®ããŠã³ããŒãã
$ aws --endpoint-url http://localhost:9000 s3 cp s3://my-bucket/hello.txt - Hello World
ãã¡ããOKããã§ããã
systemdã§MinIOãµãŒããŒãèµ·åãã
ãšããã§ãMinIOãµãŒããŒã®ããã±ãŒãžãèŠããšãã©ãããsystemdã®å®çŸ©ãã¡ã€ã«ãå«ãŸããŠããããã§ãã
$ dpkg -L minio /usr /usr/local /usr/local/bin /usr/local/bin/minio /etc /etc/systemd /etc/systemd/system /etc/systemd/system/minio.service
systemdã®ãŠãããå®çŸ©ãã¡ã€ã«ã¯ããããªæãã§ããã
/etc/systemd/system/minio.service
[Unit] Description=MinIO Documentation=https://docs.min.io Wants=network-online.target After=network-online.target AssertFileIsExecutable=/usr/local/bin/minio [Service] WorkingDirectory=/usr/local User=minio-user Group=minio-user ProtectProc=invisible EnvironmentFile=-/etc/default/minio ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi" ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES # Let systemd restart this service always Restart=always # Specifies the maximum file descriptor number that can be opened by this process LimitNOFILE=65536 # Specifies the maximum number of threads this process can create TasksMax=infinity # Disable timeout logic and wait until process is stopped TimeoutStopSec=infinity SendSIGKILL=no [Install] WantedBy=multi-user.target # Built for ${project.name}-${project.version} (${project.name})
æå¹åããŠ
$ sudo systemctl enable minio Created symlink /etc/systemd/system/multi-user.target.wants/minio.service â /etc/systemd/system/minio.service.
èµ·åããããšãããšã倱æããŸãã
$ sudo systemctl start minio Job for minio.service failed because the control process exited with error code. See "systemctl status minio.service" and "journalctl -xe" for details.
ãã°ãèŠããšãã¯ã¬ãã³ã·ã£ã«ã«åé¡ãããããã§ãã
9æ 10 23:18:28 server systemd[1]: Starting MinIO... 9æ 10 23:18:28 server systemd[1989]: minio.service: Failed to determine user credentials: No such process 9æ 10 23:18:28 server systemd[1989]: minio.service: Failed at step USER spawning /bin/bash: No such process
ãŠãããå®çŸ©ã®ãŠãŒã¶ãŒããã³ã°ã«ãŒãæå®ãèŠããšãååšããªããŠãŒã¶ãŒãšã°ã«ãŒããæå®ãããŠããŸãã
User=minio-user Group=minio-user
ãã®ãŸãŸã ãšèµ·åããªãã®ã§ããŠãŒã¶ãŒãè¿œå ããŸãã
$ sudo useradd minio-user
ããã§ããŸã 倱æããŸãã
$ sudo systemctl start minio Job for minio.service failed because the control process exited with error code. See "systemctl status minio.service" and "journalctl -xe" for details.
ãšã©ãŒã¡ãã»ãŒãžã確èªãããšã/etc/default/minio
ãšãããã¡ã€ã«ã«ç°å¢å€æ°ã®å®çŸ©ããªããšèšã£ãŠããŸãã
9æ 10 23:19:44 server systemd[1]: Starting MinIO... 9æ 10 23:19:44 server bash[2030]: Variable MINIO_VOLUMES not set in /etc/default/minio
ããã¯ãExecStartPre
ããã³ExecStart
ãèŠããšçç±ãããããŸãã
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi" ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
èµ·åæã®èšå®ãããªããã°ãããªãããã§ãã
ã§ã¯ãããŒã¿ãä¿åãããã£ã¬ã¯ããªãäœæããŠ
$ sudo mkdir -p /var/lib/minio/data $ sudo chown -R minio-user:minio-user /var/lib/minio
/etc/default/minio
ãã¡ã€ã«ãäœæããŸããããã§ãèµ·åæã®ãªãã·ã§ã³ãšããŒã¿ã®ä¿åå
ãæå®ããŸãã
/etc/default/minio
MINIO_OPTS=--console-address :9001 MINIO_VOLUMES=/var/lib/minio/data
ä»åºŠã¯èµ·åããããã«ãªããŸãã
$ sudo systemctl start minio
ããã©ã«ãã®ã¯ã¬ãã³ã·ã£ã«ã ãšèŠåãåºãã®ã§
9æ 10 23:26:00 server minio[2273]: WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables
奜ã¿ã«å¿ããŠãMINIO_ROOT_USER
ããã³MINIO_ROOT_PASSWORD
ãèšå®ããŠã¯ã¬ãã³ã·ã£ã«ãæå®ãããš
ããã§ãããã
/etc/default/minio
MINIO_OPTS=--console-address :9001 MINIO_VOLUMES=/var/lib/minio/data MINIO_ROOT_USER=root-user MINIO_ROOT_PASSWORD=password
ãŸãšã
Amazon S3äºæã®ãªããžã§ã¯ãã¹ãã¬ãŒãžãMinIOãè©ŠããŠã¿ãŸããã
MinIOã®ã¯ã©ã€ã¢ã³ãã³ãã³ãã¯æåã¯æžæã£ããããŸããããããšã¯ããã»ã©å°ããŸããã§ãããã
ãæ軜ã«äœ¿ããããªã®ã§ãAmazon S3ã®ä»£ããã欲ããæã¯LocalStackãšäœ¿ãåããŠãããããªãšæããŸãã