send stat

README.md

@@ -29,6 +29,7 @@ Abstract your training task into three components:
 	+ Print some variables of interest
 	+ Run inference on a test dataset
 	+ Run some operations once a while
+	+ Send the accuracy to your phone
 
 With the above components defined, tensorpack trainer will run the training iterations for you.
 Multi-GPU training is ready to use by simply switching the trainer.

examples/OpenAIGym/README.md

@@ -19,9 +19,12 @@ Models are available for the following gym atari environments (click links for v
 + [DoubleDunk-v0](https://gym.openai.com/evaluations/eval_FI1GpF4TlCuf29KccTpQ)
 + [ElevatorAction-v0](https://gym.openai.com/evaluations/eval_SqeAouMvR0icRivx2xprZg)
 + [FishingDerby-v0](https://gym.openai.com/evaluations/eval_pPLCnFXsTVaayrIboDOs0g)
++ [JourneyEscape](https://gym.openai.com/evaluations/eval_S9nQuXLRSu7S5x21Ay6AA)
 + [Pong-v0](https://gym.openai.com/evaluations/eval_8L7SV59nSW6GGbbP3N4G6w)
++ [Qbert-v0](https://gym.openai.com/evaluations/eval_wekCJkrWQm9NrOUzltXg)
 + [Seaquest-v0](https://gym.openai.com/evaluations/eval_N2624y3NSJWrOgoMSpOi4w)
 + [Tennis-v0](https://gym.openai.com/evaluations/eval_gDjJD0MMS1yLm1T0hdqI4g)
++ [UpNDown-v0](https://gym.openai.com/evaluations/eval_KmkvMJkxQFSED20wFUMdIA)
 + [VideoPinball-v0](https://gym.openai.com/evaluations/eval_PWwzNhVFR2CxjYvEsPfT1g)
 
 Note that atari game settings in gym are quite different from DeepMind papers, so the scores are not comparable. The most notable differences are:

tensorpack/callbacks/stat.py

@@ -3,15 +3,14 @@
 # Author: Yuxin Wu <ppwwyyxxc@gmail.com>
 
 import tensorflow as tf
-import re
-import os
+import re, os
 import operator
 import json
 
 from .base import Callback
 from ..utils import *
 
-__all__ = ['StatHolder', 'StatPrinter']
+__all__ = ['StatHolder', 'StatPrinter', 'SendStat']
 
 class StatHolder(object):
     """
@@ -107,3 +106,27 @@ class StatPrinter(Callback):
     def _trigger_epoch(self):
         self.trainer.stat_holder.add_stat('global_step', self.global_step)
         self.trainer.stat_holder.finalize()
+
+class SendStat(Callback):
+    """
+    Execute a command with some specific stats.
+    For example, send the stats to your phone through pushbullet:
+
+        SendStat('curl -u your_id: https://api.pushbullet.com/v2/pushes \
+            -d type=note -d title="validation error" \
+            -d body={validation_error} > /dev/null 2>&1',
+                'validation_error')
+    """
+    def __init__(self, command, stats):
+        self.command = command
+        if not isinstance(stats, list):
+            stats = [stats]
+        self.stats = stats
+
+    def _trigger_epoch(self):
+        holder = self.trainer.stat_holder
+        v = {k: holder.get_stat_now(k) for k in self.stats}
+        cmd = self.command.format(**v)
+        ret = os.system(cmd)
+        if ret != 0:
+            logger.error("Command {} failed with ret={}!".format(cmd, ret))