文章目录[隐藏]
实现效果是每晚三点自动备份数据库和网站等数据并使用 Openssl 加密归档,保留最近一个月的数据,通过开源的 rclone 工具同步到 Google Drive 和 OneDrive 上,这样就万无一失了
1. Shell 脚本
脚本是自己使用,因此比较简单。
#!/bin/bash DATE=`date +%Y%m%d` BACKUP_TEMP_PATH=/backup/tmp # Database DB_NAME= DB_USR= DB_PW= DB_OUT_NAME="db_$DATE" # Website WEB_PATH= WEB_OUT_NAME="web_$DATE" # Archive ARCHIVE_PATH=/backup/archives ARCHIVE_LOG_FILE=/backup/log/backup.log # Backup mysql # mysqldump -u$DB_USR -p$DB_PW --databases $DB_NAME > $BACKUP_TEMP_PATH/$DB_OUT_NAME.sql mysqldump --login-path=$DB_USR --databases $DB_NAME > $BACKUP_TEMP_PATH/$DB_OUT_NAME.sql # Backup webdsite cd $WEB_PATH zip -rq $BACKUP_TEMP_PATH/$WEB_OUT_NAME.zip * # Archive files cd $BACKUP_TEMP_PATH # tar cf - * | openssl enc -e -aes-128-cbc -pbkdf2 -k [password] -out $ARCHIVE_PATH/$DATE.tar.aes tar cf - * | gpg -e -r [recipient] -o $ARCHIVE_PATH/$DATE.tar.gpg # Backup the latest 30 days rm $BACKUP_TEMP_PATH/* find $ARCHIVE_PATH -name "*.tar.gpg" -type f -mtime +30 -exec rm {} \; # Sync to Google Drive rclone sync $ARCHIVE_PATH gdrive:[sync path] # Sync to Microsoft OneDrive rclone sync $ARCHIVE_PATH onedrive:[sync path] # Output log file echo -e "\e[1;32m---------------------------------------------\e[0m" >> $ARCHIVE_LOG_FILE echo -e "Backup date: "$(date +%Y-%m-%d)"\t""Finish time: "$(date +%H:%M:%S) >> $ARCHIVE_LOG_FILE echo -e "\e[1;32m---------------------------------------------\e[0m\n" >> $ARCHIVE_LOG_FILE
这里说明一下,使用mysqldump备份的时候,如果通过账号密码导出会提示使用明文密码不安全的警告,虽然不影响使用,但看着就不爽,因此改用了 MySQL5.7 之后提供的新方式,通过 --login-path=
的方式登录数据库,首先保存一个加密验证信息:
mysql_config_editor set --login-path=[name] --host=localhost --user=[usr] --password
查看所设置的信息:
mysql_config_editor print --all
删除指定信息:
mysql_config_editor remove --login-path=[usr]
值得一提的,我第一次设置的时候毫无疑问的失败了- -,一直无法登录,后来才知道所设置的用户密码中不能包括"#“符号,不然就会报错。
2. 配置 rclone
官方有一键安装脚本,适用于 Linux/macOS/BSD 系统:
curl https://rclone.org/install.sh | sudo bash
之后需要在服务器上配置 Google Dirve 和 OneDrive 的连接,输入 rclone config
命令,根据提示的信息进行操作,下面演示的版本为 v1.57.0,不同版本可能会略有出入,但思路是一样的。
Google Drive
- using defaults No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n # 输入 n,新建远程连接 name> gdrive # 输入自定义名称,用于区分不同连接 Option Storage. Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value. 1 / 1Fichier \ "fichier" 2 / Alias for an existing remote \ "alias" 3 / Amazon Drive \ "amazon cloud drive" 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS \ "s3" 5 / Backblaze B2 \ "b2" 6 / Better checksums for other remotes \ "hasher" 7 / Box \ "box" 8 / Cache a remote \ "cache" 9 / Citrix Sharefile \ "sharefile" 10 / Compress a remote \ "compress" 11 / Dropbox \ "dropbox" 12 / Encrypt/Decrypt a remote \ "crypt" 13 / Enterprise File Fabric \ "filefabric" 14 / FTP Connection \ "ftp" 15 / Google Cloud Storage (this is not Google Drive) \ "google cloud storage" 16 / Google Drive \ "drive" 17 / Google Photos \ "google photos" 18 / Hadoop distributed file system \ "hdfs" 19 / Hubic \ "hubic" 20 / In memory object storage system. \ "memory" 21 / Jottacloud \ "jottacloud" 22 / Koofr \ "koofr" 23 / Local Disk \ "local" 24 / Mail.ru Cloud \ "mailru" 25 / Mega \ "mega" 26 / Microsoft Azure Blob Storage \ "azureblob" 27 / Microsoft OneDrive \ "onedrive" 28 / OpenDrive \ "opendrive" 29 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH) \ "swift" 30 / Pcloud \ "pcloud" 31 / Put.io \ "putio" 32 / QingCloud Object Storage \ "qingstor" 33 / SSH/SFTP Connection \ "sftp" 34 / Sia Decentralized Cloud \ "sia" 35 / Sugarsync \ "sugarsync" 36 / Tardigrade Decentralized Cloud Storage \ "tardigrade" 37 / Transparently chunk/split large files \ "chunker" 38 / Union merges the contents of several upstream fs \ "union" 39 / Uptobox \ "uptobox" 40 / Webdav \ "webdav" 41 / Yandex Disk \ "yandex" 42 / Zoho \ "zoho" 43 / http Connection \ "http" 44 / premiumize.me \ "premiumizeme" 45 / seafile \ "seafile" Storage> 16 # 找到 Google Drive 的序号,我这里的版本是 16 Option client_id. Google Application Client Id Setting your own is recommended. See https://rclone.org/drive/#making-your-own-client-id for how to create your own. If you leave this blank, it will use an internal key which is low performance. Enter a string value. Press Enter for the default (""). client_id> # 一般留空,直接回车,如果你需要同步大量文件可以自己去 Google Cloud 申请一个 client_id 而不使用 rclone 的 api 减少限制 Option client_secret. OAuth Client Secret. Leave blank normally. Enter a string value. Press Enter for the default (""). client_secret> # 一般留空,直接回车 Option scope. Scope that rclone should use when requesting access from drive. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value. 1 / Full access all files, excluding Application Data Folder. \ "drive" 2 / Read-only access to file metadata and file contents. \ "drive.readonly" / Access to files created by rclone only. 3 | These are visible in the drive website. | File authorization is revoked when the user deauthorizes the app. \ "drive.file" / Allows read and write access to the Application Data folder. 4 | This is not visible in the drive website. \ "drive.appfolder" / Allows read-only access to file metadata but 5 | does not allow any access to read or download file content. \ "drive.metadata.readonly" scope> 1 # 根据你的需求选择访问权限,这里选择完全权限 Option root_folder_id. ID of the root folder. Leave blank normally. Fill in to access "Computers" folders (see docs), or for rclone to use a non root folder as its starting point. Enter a string value. Press Enter for the default (""). root_folder_id> # 一般留空,直接回车 Option service_account_file. Service Account Credentials JSON file path. Leave blank normally. Needed only if you want use SA instead of interactive login. Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`. Enter a string value. Press Enter for the default (""). service_account_file> # 一般留空,直接回车 Edit advanced config? y) Yes n) No (default) y/n> n # 输入 n,不进行高级设置 Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes (default) n) No y/n> n # 输入 n Option config_verification_code. Verification code Go to this URL, authenticate then paste the code here. https://accounts.google.com/o/oauth2/xxxxxx Enter a string value. Press Enter for the default (""). config_verification_code> # 输入访问上述网址后的 token Configure this as a Shared Drive (Team Drive)? y) Yes n) No (default) y/n> n # 是否配置为团队盘 -------------------- [gdrive] type = drive scope = drive token = {"access_token":""} team_drive = -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y # 上述配置是否正确,输入 y 确认即可完成配置 Current remotes: Name Type ==== ==== gdrive drive e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q # 输入 q 退出配置
连接 OneDrive 的 token 需要在本地 Windows 电脑上下载 Windows 版 rclone 获取,下载后解压,在 rclone 文件夹中的资源管理器地址栏输入 cmd,打开命令提示符,输入 rclone 配置中提示的命令:
rclone authorize "onedrive"
接下来会弹出浏览器,登录账号进行授权,然后返回 cmd 窗口会出现类似信息:
2021/11/27 21:53:38 NOTICE: Log in and authorize rclone for access 2021/11/27 21:53:38 NOTICE: Waiting for code... 2021/11/27 21:53:42 NOTICE: Got code Paste the following into your remote machine ---> xxxx <---End paste
复制保存对应 “xxxx” 的所有内容,接下来会用到,然后回到服务器上输入 rclone config
进行配置:
- using defaults No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n # 输入 n,新建远程连接 name> onedrive # 输入自定义名称,用于区分不同连接 Option Storage. Type of storage to configure. Enter a string value. Press Enter for the default (""). Choose a number from below, or type in your own value. 1 / 1Fichier \ "fichier" 2 / Alias for an existing remote \ "alias" 3 / Amazon Drive \ "amazon cloud drive" 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS \ "s3" 5 / Backblaze B2 \ "b2" 6 / Better checksums for other remotes \ "hasher" 7 / Box \ "box" 8 / Cache a remote \ "cache" 9 / Citrix Sharefile \ "sharefile" 10 / Compress a remote \ "compress" 11 / Dropbox \ "dropbox" 12 / Encrypt/Decrypt a remote \ "crypt" 13 / Enterprise File Fabric \ "filefabric" 14 / FTP Connection \ "ftp" 15 / Google Cloud Storage (this is not Google Drive) \ "google cloud storage" 16 / Google Drive \ "drive" 17 / Google Photos \ "google photos" 18 / Hadoop distributed file system \ "hdfs" 19 / Hubic \ "hubic" 20 / In memory object storage system. \ "memory" 21 / Jottacloud \ "jottacloud" 22 / Koofr \ "koofr" 23 / Local Disk \ "local" 24 / Mail.ru Cloud \ "mailru" 25 / Mega \ "mega" 26 / Microsoft Azure Blob Storage \ "azureblob" 27 / Microsoft OneDrive \ "onedrive" 28 / OpenDrive \ "opendrive" 29 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH) \ "swift" 30 / Pcloud \ "pcloud" 31 / Put.io \ "putio" 32 / QingCloud Object Storage \ "qingstor" 33 / SSH/SFTP Connection \ "sftp" 34 / Sia Decentralized Cloud \ "sia" 35 / Sugarsync \ "sugarsync" 36 / Tardigrade Decentralized Cloud Storage \ "tardigrade" 37 / Transparently chunk/split large files \ "chunker" 38 / Union merges the contents of several upstream fs \ "union" 39 / Uptobox \ "uptobox" 40 / Webdav \ "webdav" 41 / Yandex Disk \ "yandex" 42 / Zoho \ "zoho" 43 / http Connection \ "http" 44 / premiumize.me \ "premiumizeme" 45 / seafile \ "seafile" Storage> 27 # 找到 OneDrive 的序号,我这里的版本是 27 Option client_id. Google Application Client Id Setting your own is recommended. See https://rclone.org/drive/#making-your-own-client-id for how to create your own. If you leave this blank, it will use an internal key which is low performance. Enter a string value. Press Enter for the default (""). client_id> # 一般留空,直接回车 Option client_secret. OAuth Client Secret. Leave blank normally. Enter a string value. Press Enter for the default (""). client_secret> # 一般留空,直接回车 Option region. Choose national cloud region for OneDrive. Enter a string value. Press Enter for the default ("global"). Choose a number from below, or type in your own value. 1 / Microsoft Cloud Global \ "global" 2 / Microsoft Cloud for US Government \ "us" 3 / Microsoft Cloud Germany \ "de" 4 / Azure and Office 365 operated by 21Vianet in China \ "cn" region> 1 # 选择你的 OneDrive 区域 Edit advanced config? y) Yes n) No (default) y/n> n # 输入 n,不进行高级设置 Use auto config? * Say Y if not sure * Say N if you are working on a remote or headless machine y) Yes (default) n) No y/n> n # 输入 n Option config_token. For this to work, you will need rclone available on a machine that has a web browser available. For more help and alternate methods see: https://rclone.org/remote_setup/ Execute the following on the machine with the web browser (same rclone version recommended): rclone authorize "onedrive" Then paste the result. Enter a string value. Press Enter for the default (""). config_token> # 粘贴前面在 Windows 上获取的内容 Option config_type. Type of connection Enter a string value. Press Enter for the default ("onedrive"). Choose a number from below, or type in an existing value. 1 / OneDrive Personal or Business \ "onedrive" 2 / Root Sharepoint site \ "sharepoint" / Sharepoint site name or URL 3 | E.g. mysite or https://contoso.sharepoint.com/sites/mysite \ "url" 4 / Search for a Sharepoint site \ "search" 5 / Type in driveID (advanced) \ "driveid" 6 / Type in SiteID (advanced) \ "siteid" / Sharepoint server-relative path (advanced) 7 | E.g. /teams/hr \ "path" config_type> 1 # 输入你的 OneDrive 类型 Drive OK? Found drive "root" of type "personal" URL: https://onedrive.live.com/?cid=xxxxx y) Yes (default) n) No y/n> y # 输入 y -------------------- [onedrive] type = onedrive client_id = client_secret = token = {"access_token":"","expiry":""} drive_id = drive_type = personal -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y # 输入 y 确认 Current remotes: Name Type ==== ==== gdrive drive onedrive onedrive e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q # 输入 q 退出配置
配置完成后,你可以同步一个简单的文件进行测试,比如在 /home/rclone-test
目录下新建一个 1.txt
文件,然后使用同步命令:
rclone sync /home/rclone-test onedrive:rclone-test/temp
然后登录你的 OneDrive 查看 rclone-test/temp
目录下是否有 1.txt
文件。
至于 rclone 完整的命令操作,可以查看官方文档。
3. 定时任务
设置好 rclone 之后就该定时执行备份脚本了,这里通过 Crontab 设置定时任务,每晚两点执行备份脚本,记得给脚本可执行权限:
crontab -e 00 2 * * * /backup/backup.sh
使用说明:
分 时 日 月 周 执行命令
第 1 列:分钟 0-59,每分钟用 * 或者 */1 表示,整点分钟数为 00 或 0
第 2 列:小时 0-23(0 表示 0 点)
第 3 列:日 1-31
第 4 列:月 1-12
第 5 列:星期 0-6(0 表示星期天)
第 6 列:运行的命令
Updated 2020-11-21
经研究发现,Openssl 并不适合用于文档加密,因此改用 GnuPG