mongodump和mongorestore
作者：互联网
mongodump和mongorestore
介绍
mongodump能够在Mongodb运行时进行备份，它的工作原理是对运行的Mongodb做查询，然后将所有查到的文档写入磁盘。
但是存在的问题时使用mongodump产生的备份不一定是数据库的实时快照，如果我们在备份时对数据库进行了写入操作，
则备份出来的文件可能不完全和Mongodb实时数据相等。另外在备份时可能会对其它客户端性能产生不利的影响。
全量备份
备份可在RS副本集内任一节点执行，最好在从库
PS:mongodump备份时不需要锁库，可以通过--oplog参数将备份时段的变更记录以日志形式一起备份出来(单实例不支持oplog)
mongodump全量备份会转储admin库下的users和version，所有的用户库和备份时段内的oplog，
转储格式为bson格式的数据文件和json格式的元数据文件
增量备份（oplog日志备份）
只有副本集可以增量备份，单实例没有oplog无法增备
根据全量备份时段备份的的oplog,获取全量备份完成的时间戳
bsondump oplog.bson
可以得出时间戳为（1593309601,1），后期可以通过脚本解析json直接获取
这里说明一下：
MongoDB中的时间戳为(timestamp,i)的形式，其中timestamp为十位时间戳，精确到秒，i表示累增序数，记录着该时间戳内有多少并发操作。
# 基于时间戳的增量备份
mongodump -u admin -p 123456 -h localhost --port 27017 --authenticationDatabase admin -d local -c oplog.rs --query '{ts:{$gt:Timestamp(1593309601,1)}}' -o /var/lib/mongobak/mongodump_oplog_`date +%Y-%m-%d_%H:%M:%S`
全量恢复
mongorestore恢复时，需要服务开启且停止业务写入，防止数据恢复时写入导致数据错乱
同机（副本集）全量恢复
同副本集的恢复，需要指定主库ip,在primary主库上做恢复，通过oplog同步给从库
mongorestore -h 192.168.73.120 --port 27018 -u admin -p 123456 --authenticationDatabase admin --oplogReplay --dir /var/lib/mongobak/mongodump_2020-06-28
PS:mongorestore只针对insert做恢复，不会恢复update。对于同一个_id的文档，mongorestore会认定为重复并跳过。
可以使用--drop,在import前先删除collection
mongodump用法如下
$ mongodump --help
参数说明：
-h:指明数据库宿主机的IP
-u:指明数据库的用户名
-p:指明数据库的密码
-d:指明数据库的名字
-c:指明collection的名字
-o:指明到要导出的文件名
-q:指明导出数据的过滤条件
-j, --numParallelCollections= number of collections to dump in parallel (4 by default)
--oplog 备份的同时备份oplog
创建备份用户
# 选择认证库
use admin
# 创建备份用户
db.createUser(
{
user: "backup",
pwd: "backup",
roles: [
{
role: "backup", //拥有备份权限
db: "admin" //认证库
},
{
role: "restore", //拥有恢复权限
db: "admin" //认证库
}
]
}
)
# 验证用户
db.auth('backup','backup')
use admin
db.createUser(
{
user: "backup",
pwd: "backup",
roles: [
{
role: "read",
db: "local"
},{
role: "backup",
db: "admin"
},
{
role: "restore",
db: "admin"
}
]
}
)
# 创建恢复用户
db.createUser(
{
user: "restore",
pwd: "restore",
roles: [ { role: "restore", db: "admin" } ]
}
)
# 验证用户
db.auth('restore','restore')
mongodump和mongorestore基本使用
全库备份
mkdir -p /mongodb/backup/full
mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup/full
[mongod@mysql-node01 mongodb]$ mongodump -uroot -pw --authenticationDatabase admin -o /mongodb/backup/full/
2020-10-25T10:18:59.040+0800 writing admin.system.users to
2020-10-25T10:18:59.041+0800 done dumping admin.system.users (1 document)
2020-10-25T10:18:59.041+0800 writing admin.system.version to
2020-10-25T10:18:59.041+0800 done dumping admin.system.version (2 documents)
2020-10-25T10:18:59.041+0800 writing simon.log3 to
2020-10-25T10:18:59.041+0800 writing simon.log to
2020-10-25T10:18:59.041+0800 writing simon.log1 to
2020-10-25T10:18:59.041+0800 writing simon.log2 to
2020-10-25T10:18:59.071+0800 done dumping simon.log3 (10000 documents)
2020-10-25T10:18:59.071+0800 writing simon.city to
2020-10-25T10:18:59.072+0800 done dumping simon.log (10000 documents)
2020-10-25T10:18:59.072+0800 writing simon.city2 to
2020-10-25T10:18:59.089+0800 done dumping simon.city (4079 documents)
2020-10-25T10:18:59.089+0800 writing simon.test to
2020-10-25T10:18:59.092+0800 done dumping simon.log2 (10000 documents)
2020-10-25T10:18:59.092+0800 done dumping simon.log1 (10000 documents)
2020-10-25T10:18:59.093+0800 done dumping simon.test (1 document)
2020-10-25T10:18:59.095+0800 done dumping simon.city2 (4079 documents)
[mongod@mysql-node01 mongodb]$
[mongod@mysql-node01 mongodb]$ cd !$
cd /mongodb/backup/full/
[mongod@mysql-node01 full]$ ls
admin simon
[mongod@mysql-node01 full]$ ls simon/
city2.bson log1.bson log3.bson test.bson
city2.metadata.json log1.metadata.json log3.metadata.json test.metadata.json
city.bson log2.bson log.bson
city.metadata.json log2.metadata.json log.metadata.json
[mongod@mysql-node01 full]$
备份simon库
mkdir -p /mongodb/backup/simon
$ mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -o /mongodb/backup/simon
[mongod@mysql-node01 full]$ mongodump -uroot -pw --authenticationDatabase admin -d simon -o /mongodb/backup/simon/
2020-10-25T10:21:22.039+0800 writing simon.log3 to
2020-10-25T10:21:22.039+0800 writing simon.log to
2020-10-25T10:21:22.039+0800 writing simon.log1 to
2020-10-25T10:21:22.039+0800 writing simon.log2 to
2020-10-25T10:21:22.059+0800 done dumping simon.log3 (10000 documents)
2020-10-25T10:21:22.059+0800 writing simon.city to
2020-10-25T10:21:22.065+0800 done dumping simon.log (10000 documents)
2020-10-25T10:21:22.065+0800 writing simon.city2 to
2020-10-25T10:21:22.072+0800 done dumping simon.city (4079 documents)
2020-10-25T10:21:22.072+0800 writing simon.test to
2020-10-25T10:21:22.073+0800 done dumping simon.city2 (4079 documents)
2020-10-25T10:21:22.074+0800 done dumping simon.test (1 document)
2020-10-25T10:21:22.090+0800 done dumping simon.log1 (10000 documents)
2020-10-25T10:21:22.093+0800 done dumping simon.log2 (10000 documents)
[mongod@mysql-node01 full]$
[mongod@mysql-node01 full]$
[mongod@mysql-node01 full]$
[mongod@mysql-node01 full]$ cd /mongodb/backup/simon/
[mongod@mysql-node01 simon]$ ls
simon
[mongod@mysql-node01 simon]$ ls simon/
city2.bson log1.bson log3.bson test.bson
city2.metadata.json log1.metadata.json log3.metadata.json test.metadata.json
city.bson log2.bson log.bson
city.metadata.json log2.metadata.json log.metadata.json
[mongod@mysql-node01 simon]$
备份simon库下的log集合
$ mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -c log -o /mongodb/backup/simon
[mongod@mysql-node01 simon]$ mongodump -uroot -pw --authenticationDatabase admin -d simon -c log3 -o /mongodb/backup/simon/
2020-10-25T10:22:47.637+0800 writing simon.log3 to
2020-10-25T10:22:47.651+0800 done dumping simon.log3 (10000 documents)
[mongod@mysql-node01 simon]$
压缩备份
$ mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -o /mongodb/backup/ --gzip
mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup/ --gzip
$ mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -d app -c vast -o /mongodb/backup/ --gzip
[mongod@mysql-node01 simon]$ mkdir -p /mongodb/backup/full-gzip
[mongod@mysql-node01 simon]$ mongodump -uroot -pw --authenticationDatabase admin -o /mongodb/backup/full-gzip/ --gzip
2020-10-25T10:27:49.023+0800 writing admin.system.users to
2020-10-25T10:27:49.024+0800 done dumping admin.system.users (1 document)
2020-10-25T10:27:49.024+0800 writing admin.system.version to
2020-10-25T10:27:49.024+0800 done dumping admin.system.version (2 documents)
2020-10-25T10:27:49.024+0800 writing simon.log3 to
2020-10-25T10:27:49.024+0800 writing simon.log to
2020-10-25T10:27:49.024+0800 writing simon.log1 to
2020-10-25T10:27:49.025+0800 writing simon.log2 to
2020-10-25T10:27:49.060+0800 done dumping simon.log (10000 documents)
2020-10-25T10:27:49.060+0800 writing simon.city to
2020-10-25T10:27:49.067+0800 done dumping simon.log3 (10000 documents)
2020-10-25T10:27:49.067+0800 writing simon.city2 to
2020-10-25T10:27:49.086+0800 done dumping simon.city (4079 documents)
2020-10-25T10:27:49.086+0800 writing simon.test to
2020-10-25T10:27:49.094+0800 done dumping simon.log2 (10000 documents)
2020-10-25T10:27:49.107+0800 done dumping simon.log1 (10000 documents)
2020-10-25T10:27:49.108+0800 done dumping simon.test (1 document)
2020-10-25T10:27:49.111+0800 done dumping simon.city2 (4079 documents)
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ du -sh /mongodb/backup/full
4.3M /mongodb/backup/full
[mongod@mysql-node01 simon]$ du -sh /mongodb/backup/full-gzip/
424K /mongodb/backup/full-gzip/
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ cd /mongodb/backup/full-gzip/
[mongod@mysql-node01 simon]$ pwd
/mongodb/backup/full-gzip
[mongod@mysql-node01 simon]$ ls
[mongod@mysql-node01 full-gzip]$ ls simon/
city2.bson.gz log1.metadata.json.gz log.bson.gz
city2.metadata.json.gz log2.bson.gz log.metadata.json.gz
city.bson.gz log2.metadata.json.gz test.bson.gz
city.metadata.json.gz log3.bson.gz test.metadata.json.gz
log1.bson.gz log3.metadata.json.gz
[mongod@mysql-node01 full-gzip]$
从全备中恢复指定的库
恢复simon库到 simon_gzip
$ mongorestore -uroot -pw --authenticationDatabase admin -d simon_gzip /mongodb/backup/full-gzip/simon --gzip
[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d simon_gzip /mongodb/backup/full-gzip/simon --gzip
2020-10-25T10:34:28.034+0800 the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-10-25T10:34:28.034+0800 building a list of collections to restore from /mongodb/backup/full-gzip/simon dir
2020-10-25T10:34:28.034+0800 reading metadata for simon_gzip.city from /mongodb/backup/full-gzip/simon/city.metadata.json.gz
2020-10-25T10:34:28.046+0800 reading metadata for simon_gzip.city2 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:34:28.046+0800 reading metadata for simon_gzip.log2 from /mongodb/backup/full-gzip/simon/log2.metadata.json.gz
2020-10-25T10:34:28.046+0800 reading metadata for simon_gzip.log3 from /mongodb/backup/full-gzip/simon/log3.metadata.json.gz
2020-10-25T10:34:28.046+0800 restoring simon_gzip.city from /mongodb/backup/full-gzip/simon/city.bson.gz
2020-10-25T10:34:28.064+0800 restoring simon_gzip.log2 from /mongodb/backup/full-gzip/simon/log2.bson.gz
2020-10-25T10:34:28.070+0800 restoring simon_gzip.city2 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:34:28.087+0800 restoring simon_gzip.log3 from /mongodb/backup/full-gzip/simon/log3.bson.gz
2020-10-25T10:34:28.104+0800 no indexes to restore
2020-10-25T10:34:28.104+0800 finished restoring simon_gzip.city2 (4079 documents)
2020-10-25T10:34:28.104+0800 reading metadata for simon_gzip.log1 from /mongodb/backup/full-gzip/simon/log1.metadata.json.gz
2020-10-25T10:34:28.122+0800 restoring simon_gzip.log1 from /mongodb/backup/full-gzip/simon/log1.bson.gz
2020-10-25T10:34:28.123+0800 no indexes to restore
2020-10-25T10:34:28.123+0800 finished restoring simon_gzip.city (4079 documents)
2020-10-25T10:34:28.123+0800 reading metadata for simon_gzip.log from /mongodb/backup/full-gzip/simon/log.metadata.json.gz
2020-10-25T10:34:28.135+0800 restoring simon_gzip.log from /mongodb/backup/full-gzip/simon/log.bson.gz
2020-10-25T10:34:28.164+0800 no indexes to restore
2020-10-25T10:34:28.164+0800 finished restoring simon_gzip.log2 (10000 documents)
2020-10-25T10:34:28.164+0800 reading metadata for simon_gzip.test from /mongodb/backup/full-gzip/simon/test.metadata.json.gz
2020-10-25T10:34:28.167+0800 no indexes to restore
2020-10-25T10:34:28.167+0800 finished restoring simon_gzip.log3 (10000 documents)
2020-10-25T10:34:28.184+0800 restoring simon_gzip.test from /mongodb/backup/full-gzip/simon/test.bson.gz
2020-10-25T10:34:28.201+0800 no indexes to restore
2020-10-25T10:34:28.201+0800 finished restoring simon_gzip.test (1 document)
2020-10-25T10:34:28.219+0800 no indexes to restore
2020-10-25T10:34:28.219+0800 finished restoring simon_gzip.log1 (10000 documents)
2020-10-25T10:34:28.219+0800 no indexes to restore
2020-10-25T10:34:28.219+0800 finished restoring simon_gzip.log (10000 documents)
2020-10-25T10:34:28.219+0800 done
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password:
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("013a134e-1322-4900-8466-62591614e91d") }
MongoDB server version: 3.6.20
>
> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
simon 0.002GB
simon_gzip 0.000GB
> use simon_gzip
switched to db simon_gzip
> show tables
city
city2
log
log1
log2
log3
test
>
> db.city2.count( { "CountryCode" : "CHN" } )
363
>
从全备中恢复指定的集合/表
恢复simon库下的city2集合
# 恢复指定的集合，需要指定具体的集合文件
mongorestore -uroot -pw --authenticationDatabase admin -d world -c t1 --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c t1 --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:44:38.259+0800 checking for collection data in /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:44:38.259+0800 reading metadata for world.t1 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:44:38.266+0800 restoring world.t1 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:44:38.330+0800 no indexes to restore
2020-10-25T10:44:38.330+0800 finished restoring world.t1 (4079 documents)
2020-10-25T10:44:38.330+0800 done
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password:
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("873b4e0b-d53a-479c-8aba-4fac57b2d1c2") }
MongoDB server version: 3.6.20
>
> use world
switched to db world
> db.t1.count()
4079
>
> db.t1.count( { "CountryCode" : "CHN" } )
363
>
drop表示恢复的时候把之前的集合drop掉(危险)
# Drop 会把原来的集合删除，然后再创建新的集合
$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c city2 --drop --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz
[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c city2 --drop --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.074+0800 checking for collection data in /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.075+0800 reading metadata for world.city2 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:50:57.083+0800 restoring world.city2 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.145+0800 no indexes to restore
2020-10-25T10:50:57.145+0800 finished restoring world.city2 (4079 documents)
2020-10-25T10:50:57.145+0800 done
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password:
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("636470f7-c0a7-4cc8-a358-933e98d08109") }
MongoDB server version: 3.6.20
> use world
switched to db world
> show tables
city2
t1
> db.city2.count( { CountryCode : 'CHN' } )
363
>
标签：10,simon,25T10,mongorestore,mongodump,0800,2020,gzip 来源： https://www.cnblogs.com/oldSimon/p/16290815.html