Commit 7158eede authored by Yuxin Wu's avatar Yuxin Wu

avoid divide-by-zero in GPUUtilizationTracker (#999)

parent e51323c1
......@@ -61,3 +61,13 @@ As long as the type of data is supported, the data will be dispatched to and log
As a result, tensorboard will show not only summaries in the graph, but also your custom data.
For example, a precise validation error often needs to be computed manually, outside the TensorFlow graph.
With a uniform monitor backend, this number will show up in tensorboard as well.
### Textual Logs
```python
from tensorpack.utils import logger
# logger has the methods of Python's logging.Logger
logger.info("Hello World!")
```
See [APIs of utils.logger](../modules/utils.html#module-tensorpack.utils.logger)
......@@ -105,9 +105,10 @@ class GPUUtilizationTracker(Callback):
if stop_evt.is_set(): # or on exit
return
evt.clear()
# Ignore the last datapoint. Usually is zero, makes us underestimate the util.
stats -= data
cnt -= 1
if cnt > 1:
# Ignore the last datapoint. Usually is zero, makes us underestimate the util.
stats -= data
cnt -= 1
rst_queue.put(stats / cnt)
break
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment