[MaskRCNN] add a better R50

examples/FasterRCNN/README.md

@@ -98,15 +98,17 @@ Performance in [Detectron](https://github.com/facebookresearch/Detectron/) can b
  | R50-FPN                        | 38.9;35.4 [:arrow_down:][R50FPN2x]                                      | 38.6;34.5                                          | 24h                    | <details><summary>2x</summary>`TRAIN.LR_SCHEDULE=2x` </details>                                                                                                                                                                                                                                                                                                          |
  | R50-FPN-GN                     | 40.4;36.3 [:arrow_down:][R50FPN2xGN]                                    | 40.3;35.7                                          | 29h                    | <details><summary>2x+GN</summary>`FPN.NORM=GN BACKBONE.NORM=GN`<br/>`FPN.FRCNN_HEAD_FUNC=fastrcnn_4conv1fc_gn_head`<br/>`FPN.MRCNN_HEAD_FUNC=maskrcnn_up4conv_gn_head` <br/>`TRAIN.LR_SCHEDULE=2x`                                                                                                                                                                       |
  | R50-FPN                        | 41.7;36.2 [:arrow_down:][R50FPN1xCas]                                   |                                                    | 16h                    | <details><summary>+Cascade</summary>`FPN.CASCADE=True` </details>                                                                                                                                                                                                                                                                                                        |
+ | R50-FPN-GN                     | 46.1;40.1 [:arrow_down:][R50FPN4xGNCasAug]                              |                                                    | 36h (on 16 V100s)      | <details><summary>4x+GN+Cascade+TrainAug</summary>`FPN.CASCADE=True`<br/>`FPN.NORM=GN BACKBONE.NORM=GN`<br/>`FPN.FRCNN_HEAD_FUNC=fastrcnn_4conv1fc_gn_head`<br/>`FPN.MRCNN_HEAD_FUNC=maskrcnn_up4conv_gn_head`<br/>`PREPROC.TRAIN_SHORT_EDGE_SIZE=[640,800]`<br/>`TRAIN.LR_SCHEDULE=4x` </details>                                                                       |
  | R101-C4                        | 40.1;34.6 [:arrow_down:][R101C41x]                                      |                                                    | 27h                    | <details><summary>standard</summary>`MODE_FPN=False`<br/>`BACKBONE.RESNET_NUM_BLOCKS=[3,4,23,3]` </details>                                                                                                                                                                                                                                                              |
  | R101-FPN                       | 40.7;36.8 [:arrow_down:][R101FPN1x] <sup>[2](#ft2)</sup>                | 40.0;35.9                                          | 17h                    | <details><summary>standard</summary>`BACKBONE.RESNET_NUM_BLOCKS=[3,4,23,3]` </details>                                                                                                                                                                                                                                                                                   |
  | R101-FPN                       | 46.6;40.3 [:arrow_down:][R101FPN3xCasAug]                               |                                                    | 64h                    | <details><summary>3x+Cascade+TrainAug</summary>` FPN.CASCADE=True`<br/>`BACKBONE.RESNET_NUM_BLOCKS=[3,4,23,3]`<br/>`TEST.RESULT_SCORE_THRESH=1e-4`<br/>`PREPROC.TRAIN_SHORT_EDGE_SIZE=[640,800]`<br/>`TRAIN.LR_SCHEDULE=3x` </details>                                                                                                                                   |
- | R101-FPN-GN<br/>(From Scratch) | 47.7;41.7 [:arrow_down:][R101FPN9xGNCasAugScratch] <sup>[3](#ft3)</sup> | 47.4;40.5                                          | 28h (on 64 V100s)      | <details><summary>9x+GN+Cascade+TrainAug</summary>` FPN.CASCADE=True`<br/>`BACKBONE.RESNET_NUM_BLOCKS=[3,4,23,3]`<br/>`FPN.NORM=GN BACKBONE.NORM=GN`<br/>`FPN.FRCNN_HEAD_FUNC=fastrcnn_4conv1fc_gn_head`<br/>`FPN.MRCNN_HEAD_FUNC=maskrcnn_up4conv_gn_head`<br/>`PREPROC.TRAIN_SHORT_EDGE_SIZE=[640,800]`<br/>`TRAIN.LR_SCHEDULE=9x`<br/>`BACKBONE.FREEZE_AT=0`</details> |
+ | R101-FPN-GN<br/>(From Scratch) | 47.7;41.7 [:arrow_down:][R101FPN9xGNCasAugScratch] <sup>[3](#ft3)</sup> | 47.4;40.5                                          | 28h (on 64 V100s)      | <details><summary>9x+GN+Cascade+TrainAug</summary>`FPN.CASCADE=True`<br/>`BACKBONE.RESNET_NUM_BLOCKS=[3,4,23,3]`<br/>`FPN.NORM=GN BACKBONE.NORM=GN`<br/>`FPN.FRCNN_HEAD_FUNC=fastrcnn_4conv1fc_gn_head`<br/>`FPN.MRCNN_HEAD_FUNC=maskrcnn_up4conv_gn_head`<br/>`PREPROC.TRAIN_SHORT_EDGE_SIZE=[640,800]`<br/>`TRAIN.LR_SCHEDULE=9x`<br/>`BACKBONE.FREEZE_AT=0`</details> |
 
  [R50C41x]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R50C41x.npz
  [R50FPN2x]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R50FPN2x.npz
  [R50FPN2xGN]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R50FPN2xGN.npz
  [R50FPN1xCas]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R50FPN1xCas.npz
+ [R50FPN4xGNCasAug]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R50FPN4xGNCasAug.npz
  [R101C41x]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R101C41x.npz
  [R101FPN1x]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R101FPN1x.npz
  [R101FPN3xCasAug]: http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R101FPN3xCasAug.npz

tensorpack/train/trainers.py

@@ -159,7 +159,7 @@ class SyncMultiGPUTrainerReplicated(SingleCostTrainer):
             certain numerical issues in practice.
     """
 
-    BROADCAST_EVERY_EPOCH = True
+    BROADCAST_EVERY_EPOCH = False
 
     @map_arg(gpus=_int_to_range)
     def __init__(self, gpus, average=True, mode=None):