其他分享
首页 > 其他分享> > mongodump和mongorestore

mongodump和mongorestore

作者:互联网

mongodump和mongorestore

介绍

mongodump能够在Mongodb运行时进行备份,它的工作原理是对运行的Mongodb做查询,然后将所有查到的文档写入磁盘。
但是存在的问题时使用mongodump产生的备份不一定是数据库的实时快照,如果我们在备份时对数据库进行了写入操作,
则备份出来的文件可能不完全和Mongodb实时数据相等。另外在备份时可能会对其它客户端性能产生不利的影响。

全量备份

备份可在RS副本集内任一节点执行,最好在从库

PS:mongodump备份时不需要锁库,可以通过--oplog参数将备份时段的变更记录以日志形式一起备份出来(单实例不支持oplog)

mongodump全量备份会转储admin库下的users和version,所有的用户库和备份时段内的oplog,

转储格式为bson格式的数据文件和json格式的元数据文件

增量备份(oplog日志备份)

只有副本集可以增量备份,单实例没有oplog无法增备
根据全量备份时段备份的的oplog,获取全量备份完成的时间戳

bsondump oplog.bson


可以得出时间戳为(1593309601,1),后期可以通过脚本解析json直接获取

这里说明一下:

MongoDB中的时间戳为(timestamp,i)的形式,其中timestamp为十位时间戳,精确到秒,i表示累增序数,记录着该时间戳内有多少并发操作。
# 基于时间戳的增量备份
mongodump -u admin -p 123456 -h localhost --port 27017 --authenticationDatabase admin -d local -c oplog.rs --query '{ts:{$gt:Timestamp(1593309601,1)}}' -o /var/lib/mongobak/mongodump_oplog_`date +%Y-%m-%d_%H:%M:%S`

全量恢复

mongorestore恢复时,需要服务开启且停止业务写入,防止数据恢复时写入导致数据错乱

同机(副本集)全量恢复
同副本集的恢复,需要指定主库ip,在primary主库上做恢复,通过oplog同步给从库

mongorestore -h 192.168.73.120 --port 27018 -u admin -p 123456 --authenticationDatabase admin --oplogReplay --dir /var/lib/mongobak/mongodump_2020-06-28

PS:mongorestore只针对insert做恢复,不会恢复update。对于同一个_id的文档,mongorestore会认定为重复并跳过。
可以使用--drop,在import前先删除collection

mongodump用法如下

$ mongodump --help
参数说明:
-h:指明数据库宿主机的IP
-u:指明数据库的用户名
-p:指明数据库的密码
-d:指明数据库的名字
-c:指明collection的名字
-o:指明到要导出的文件名
-q:指明导出数据的过滤条件
-j, --numParallelCollections=  number of collections to dump in parallel (4 by default)
--oplog  备份的同时备份oplog

创建备份用户

# 选择认证库
use admin

# 创建备份用户
db.createUser(
{
    user: "backup",
    pwd: "backup",
    roles: [ 
    	{ 
    	role: "backup", 	//拥有备份权限
    	db: "admin" 		//认证库
    	},
        { 
    	role: "restore", 	//拥有恢复权限
    	db: "admin" 		//认证库
    	}
    ]
}
)

# 验证用户
db.auth('backup','backup')

use admin
db.createUser(
{
    user: "backup",
    pwd: "backup",
    roles: [ 
    	        {
    	role: "read",
    	db: "local"
    	},{ 
    	role: "backup",
    	db: "admin"
    	},
        {
    	role: "restore",
    	db: "admin"
    	}
    ]
}
)



# 创建恢复用户
db.createUser(
{
    user: "restore",
    pwd: "restore",
    roles: [ { role: "restore", db: "admin" } ]
}
)
# 验证用户
db.auth('restore','restore')

mongodump和mongorestore基本使用

全库备份

mkdir -p /mongodb/backup/full
mongodump  -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup/full

[mongod@mysql-node01 mongodb]$ mongodump -uroot -pw --authenticationDatabase admin -o /mongodb/backup/full/
2020-10-25T10:18:59.040+0800	writing admin.system.users to 
2020-10-25T10:18:59.041+0800	done dumping admin.system.users (1 document)
2020-10-25T10:18:59.041+0800	writing admin.system.version to 
2020-10-25T10:18:59.041+0800	done dumping admin.system.version (2 documents)
2020-10-25T10:18:59.041+0800	writing simon.log3 to 
2020-10-25T10:18:59.041+0800	writing simon.log to 
2020-10-25T10:18:59.041+0800	writing simon.log1 to 
2020-10-25T10:18:59.041+0800	writing simon.log2 to 
2020-10-25T10:18:59.071+0800	done dumping simon.log3 (10000 documents)
2020-10-25T10:18:59.071+0800	writing simon.city to 
2020-10-25T10:18:59.072+0800	done dumping simon.log (10000 documents)
2020-10-25T10:18:59.072+0800	writing simon.city2 to 
2020-10-25T10:18:59.089+0800	done dumping simon.city (4079 documents)
2020-10-25T10:18:59.089+0800	writing simon.test to 
2020-10-25T10:18:59.092+0800	done dumping simon.log2 (10000 documents)
2020-10-25T10:18:59.092+0800	done dumping simon.log1 (10000 documents)
2020-10-25T10:18:59.093+0800	done dumping simon.test (1 document)
2020-10-25T10:18:59.095+0800	done dumping simon.city2 (4079 documents)
[mongod@mysql-node01 mongodb]$
[mongod@mysql-node01 mongodb]$ cd !$
cd /mongodb/backup/full/
[mongod@mysql-node01 full]$ ls
admin  simon
[mongod@mysql-node01 full]$ ls simon/
city2.bson           log1.bson           log3.bson           test.bson
city2.metadata.json  log1.metadata.json  log3.metadata.json  test.metadata.json
city.bson            log2.bson           log.bson
city.metadata.json   log2.metadata.json  log.metadata.json
[mongod@mysql-node01 full]$ 

备份simon库

mkdir -p /mongodb/backup/simon
$ mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -o /mongodb/backup/simon

[mongod@mysql-node01 full]$ mongodump -uroot -pw --authenticationDatabase admin -d simon -o /mongodb/backup/simon/
2020-10-25T10:21:22.039+0800	writing simon.log3 to 
2020-10-25T10:21:22.039+0800	writing simon.log to 
2020-10-25T10:21:22.039+0800	writing simon.log1 to 
2020-10-25T10:21:22.039+0800	writing simon.log2 to 
2020-10-25T10:21:22.059+0800	done dumping simon.log3 (10000 documents)
2020-10-25T10:21:22.059+0800	writing simon.city to 
2020-10-25T10:21:22.065+0800	done dumping simon.log (10000 documents)
2020-10-25T10:21:22.065+0800	writing simon.city2 to 
2020-10-25T10:21:22.072+0800	done dumping simon.city (4079 documents)
2020-10-25T10:21:22.072+0800	writing simon.test to 
2020-10-25T10:21:22.073+0800	done dumping simon.city2 (4079 documents)
2020-10-25T10:21:22.074+0800	done dumping simon.test (1 document)
2020-10-25T10:21:22.090+0800	done dumping simon.log1 (10000 documents)
2020-10-25T10:21:22.093+0800	done dumping simon.log2 (10000 documents)
[mongod@mysql-node01 full]$ 
[mongod@mysql-node01 full]$ 
[mongod@mysql-node01 full]$ 
[mongod@mysql-node01 full]$ cd /mongodb/backup/simon/
[mongod@mysql-node01 simon]$ ls
simon
[mongod@mysql-node01 simon]$ ls simon/
city2.bson           log1.bson           log3.bson           test.bson
city2.metadata.json  log1.metadata.json  log3.metadata.json  test.metadata.json
city.bson            log2.bson           log.bson
city.metadata.json   log2.metadata.json  log.metadata.json
[mongod@mysql-node01 simon]$

备份simon库下的log集合

$ mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -c log -o /mongodb/backup/simon

[mongod@mysql-node01 simon]$ mongodump -uroot -pw --authenticationDatabase admin -d simon -c log3 -o /mongodb/backup/simon/
2020-10-25T10:22:47.637+0800	writing simon.log3 to 
2020-10-25T10:22:47.651+0800	done dumping simon.log3 (10000 documents)
[mongod@mysql-node01 simon]$

压缩备份

$ mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -o /mongodb/backup/ --gzip
mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup/ --gzip
$ mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -d app -c vast -o /mongodb/backup/ --gzip


[mongod@mysql-node01 simon]$ mkdir -p /mongodb/backup/full-gzip
[mongod@mysql-node01 simon]$ mongodump -uroot -pw --authenticationDatabase admin -o /mongodb/backup/full-gzip/ --gzip
2020-10-25T10:27:49.023+0800	writing admin.system.users to 
2020-10-25T10:27:49.024+0800	done dumping admin.system.users (1 document)
2020-10-25T10:27:49.024+0800	writing admin.system.version to 
2020-10-25T10:27:49.024+0800	done dumping admin.system.version (2 documents)
2020-10-25T10:27:49.024+0800	writing simon.log3 to 
2020-10-25T10:27:49.024+0800	writing simon.log to 
2020-10-25T10:27:49.024+0800	writing simon.log1 to 
2020-10-25T10:27:49.025+0800	writing simon.log2 to 
2020-10-25T10:27:49.060+0800	done dumping simon.log (10000 documents)
2020-10-25T10:27:49.060+0800	writing simon.city to 
2020-10-25T10:27:49.067+0800	done dumping simon.log3 (10000 documents)
2020-10-25T10:27:49.067+0800	writing simon.city2 to 
2020-10-25T10:27:49.086+0800	done dumping simon.city (4079 documents)
2020-10-25T10:27:49.086+0800	writing simon.test to 
2020-10-25T10:27:49.094+0800	done dumping simon.log2 (10000 documents)
2020-10-25T10:27:49.107+0800	done dumping simon.log1 (10000 documents)
2020-10-25T10:27:49.108+0800	done dumping simon.test (1 document)
2020-10-25T10:27:49.111+0800	done dumping simon.city2 (4079 documents)
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ du -sh /mongodb/backup/full
4.3M	/mongodb/backup/full
[mongod@mysql-node01 simon]$ du -sh /mongodb/backup/full-gzip/
424K	/mongodb/backup/full-gzip/
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ cd /mongodb/backup/full-gzip/
[mongod@mysql-node01 simon]$ pwd
/mongodb/backup/full-gzip
[mongod@mysql-node01 simon]$ ls	
[mongod@mysql-node01 full-gzip]$ ls simon/
city2.bson.gz           log1.metadata.json.gz  log.bson.gz
city2.metadata.json.gz  log2.bson.gz           log.metadata.json.gz
city.bson.gz            log2.metadata.json.gz  test.bson.gz
city.metadata.json.gz   log3.bson.gz           test.metadata.json.gz
log1.bson.gz            log3.metadata.json.gz
[mongod@mysql-node01 full-gzip]$

从全备中恢复指定的库

恢复simon库到 simon_gzip

$ mongorestore -uroot -pw --authenticationDatabase admin -d simon_gzip /mongodb/backup/full-gzip/simon --gzip


[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d simon_gzip /mongodb/backup/full-gzip/simon --gzip
2020-10-25T10:34:28.034+0800	the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-10-25T10:34:28.034+0800	building a list of collections to restore from /mongodb/backup/full-gzip/simon dir
2020-10-25T10:34:28.034+0800	reading metadata for simon_gzip.city from /mongodb/backup/full-gzip/simon/city.metadata.json.gz
2020-10-25T10:34:28.046+0800	reading metadata for simon_gzip.city2 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:34:28.046+0800	reading metadata for simon_gzip.log2 from /mongodb/backup/full-gzip/simon/log2.metadata.json.gz
2020-10-25T10:34:28.046+0800	reading metadata for simon_gzip.log3 from /mongodb/backup/full-gzip/simon/log3.metadata.json.gz
2020-10-25T10:34:28.046+0800	restoring simon_gzip.city from /mongodb/backup/full-gzip/simon/city.bson.gz
2020-10-25T10:34:28.064+0800	restoring simon_gzip.log2 from /mongodb/backup/full-gzip/simon/log2.bson.gz
2020-10-25T10:34:28.070+0800	restoring simon_gzip.city2 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:34:28.087+0800	restoring simon_gzip.log3 from /mongodb/backup/full-gzip/simon/log3.bson.gz
2020-10-25T10:34:28.104+0800	no indexes to restore
2020-10-25T10:34:28.104+0800	finished restoring simon_gzip.city2 (4079 documents)
2020-10-25T10:34:28.104+0800	reading metadata for simon_gzip.log1 from /mongodb/backup/full-gzip/simon/log1.metadata.json.gz
2020-10-25T10:34:28.122+0800	restoring simon_gzip.log1 from /mongodb/backup/full-gzip/simon/log1.bson.gz
2020-10-25T10:34:28.123+0800	no indexes to restore
2020-10-25T10:34:28.123+0800	finished restoring simon_gzip.city (4079 documents)
2020-10-25T10:34:28.123+0800	reading metadata for simon_gzip.log from /mongodb/backup/full-gzip/simon/log.metadata.json.gz
2020-10-25T10:34:28.135+0800	restoring simon_gzip.log from /mongodb/backup/full-gzip/simon/log.bson.gz
2020-10-25T10:34:28.164+0800	no indexes to restore
2020-10-25T10:34:28.164+0800	finished restoring simon_gzip.log2 (10000 documents)
2020-10-25T10:34:28.164+0800	reading metadata for simon_gzip.test from /mongodb/backup/full-gzip/simon/test.metadata.json.gz
2020-10-25T10:34:28.167+0800	no indexes to restore
2020-10-25T10:34:28.167+0800	finished restoring simon_gzip.log3 (10000 documents)
2020-10-25T10:34:28.184+0800	restoring simon_gzip.test from /mongodb/backup/full-gzip/simon/test.bson.gz
2020-10-25T10:34:28.201+0800	no indexes to restore
2020-10-25T10:34:28.201+0800	finished restoring simon_gzip.test (1 document)
2020-10-25T10:34:28.219+0800	no indexes to restore
2020-10-25T10:34:28.219+0800	finished restoring simon_gzip.log1 (10000 documents)
2020-10-25T10:34:28.219+0800	no indexes to restore
2020-10-25T10:34:28.219+0800	finished restoring simon_gzip.log (10000 documents)
2020-10-25T10:34:28.219+0800	done
[mongod@mysql-node01 simon]$ 
    
    
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password: 
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("013a134e-1322-4900-8466-62591614e91d") }
MongoDB server version: 3.6.20 
> 
> show dbs
admin       0.000GB
config      0.000GB
local       0.000GB
simon       0.002GB
simon_gzip  0.000GB
> use simon_gzip
switched to db simon_gzip
> show tables
city
city2
log
log1
log2
log3
test
>
> db.city2.count( { "CountryCode" : "CHN" } )
363
> 

从全备中恢复指定的集合/表

恢复simon库下的city2集合

# 恢复指定的集合,需要指定具体的集合文件

mongorestore -uroot -pw --authenticationDatabase admin -d world -c t1 --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz


[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c t1 --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz 
2020-10-25T10:44:38.259+0800	checking for collection data in /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:44:38.259+0800	reading metadata for world.t1 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:44:38.266+0800	restoring world.t1 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:44:38.330+0800	no indexes to restore
2020-10-25T10:44:38.330+0800	finished restoring world.t1 (4079 documents)
2020-10-25T10:44:38.330+0800	done
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password: 
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("873b4e0b-d53a-479c-8aba-4fac57b2d1c2") }
MongoDB server version: 3.6.20
> 
> use world
switched to db world
> db.t1.count()
4079
> 
> db.t1.count( { "CountryCode" : "CHN" } )
363
> 

drop表示恢复的时候把之前的集合drop掉(危险)

# Drop 会把原来的集合删除,然后再创建新的集合

$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c city2 --drop --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz

[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c city2 --drop --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.074+0800	checking for collection data in /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.075+0800	reading metadata for world.city2 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:50:57.083+0800	restoring world.city2 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.145+0800	no indexes to restore
2020-10-25T10:50:57.145+0800	finished restoring world.city2 (4079 documents)
2020-10-25T10:50:57.145+0800	done
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password: 
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("636470f7-c0a7-4cc8-a358-933e98d08109") }
MongoDB server version: 3.6.20
> use world
switched to db world
> show tables
city2
t1
> db.city2.count( { CountryCode : 'CHN' } )
363
> 

标签:10,simon,25T10,mongorestore,mongodump,0800,2020,gzip
来源: https://www.cnblogs.com/oldSimon/p/16290815.html