Symptoms
- VM backup fails randomly.
- VM backup fails during create snapshot.
2013-05-30 21:07:06 avvcbimage Info <14632>: snapshot create completed
2013-05-30 21:07:06 avvcbimage Info <14634>: New snapshot ‘VDP-1369919135de3a2a20f17d33073b17a0fe31acc41fbd2efdfe’, moref is ‘snapshot-569′.
2013-05-30 21:11:13 avvcbimage Error <14690>: No response from Avamar Administrator to retain snapshot request, please check Avamar Administrator log during period in the vicinity of this timestamp for root cause
2013-05-30 21:11:22 avvcbimage Info <0000>: Starting graceful (staged) termination, MC did not respond to retain snapshot request (wrap-up stage)
2013-05-30 21:11:33 avvcbimage Error <9759>: createSnapshot: snapshot creation failed
Cause
- In err.log located /data01/cur/, we notice that the blackout window had initiated during which the mc server won’t take any requests for backup and therefore the request was terminated.
2013/05/30-20:59:39.39573 {0.0} [perfbeat.5:193] WARN: <1060> perfbeat::outoftolerance mask=[] average=2388.18 limit=238.8182 mbpersec=115.90
2013/05/30-21:00:58.37298 {0.0} [perfbeat.0:183] WARN: <0963> server node 0.0 is swapping: check configuration
2013/05/30-21:14:39.59909 {0.0} [perfbeat.5:193] WARN: <1060> perfbeat::outoftolerance mask=[] average=2388.18 limit=238.8182 mbpersec=129.30
2013/05/30-21:15:01.41611 {0.0} [sched.gc:3372] <4200> starting scheduled garbage collection
2013/05/30-21:16:28.47383 {0.0} [perfbeat.0:183] WARN: <0963> server node 0.0 is swapping: check configuration
2013/05/30-21:20:02.10546 {0.0} [perfbeat.1:185] WARN: <1060> perfbeat::outoftolerance mask=[gc] average=33.43 limit=3.3433 mbpersec=0.49
Resolution:
This is as per design.
Backup will stop working and no backup requests will be handled by the mc server during the blackout window.