Commit f9f1e437 authored by Yuxin Wu's avatar Yuxin Wu

update docs

parent e0c1ee77
...@@ -14,8 +14,10 @@ You'll only need to __select__ what trainer to use. ...@@ -14,8 +14,10 @@ You'll only need to __select__ what trainer to use.
Following the terminology in TensorFlow, Following the terminology in TensorFlow,
a "tower" function is something that takes input tensors and adds __one replicate__ of the model to the graph. a "tower" function is something that takes input tensors and adds __one replicate__ of the model to the graph.
Most types of neural-network training could fall into this category. Most types of neural-network training could fall into this category.
This concept is used mainly to support: All non-base trainers in tensorpack is a subclass of [TowerTrainer](../modules/train.html#tensorpack.train.TowerTrainer).
The concept of tower is used mainly to support:
1. Data-parallel multi-GPU training, where a replicate is built on each GPU. 1. Data-parallel multi-GPU training, where a replicate is built on each GPU.
2. Automatically building the graph for inference, where a replicate is built under inference mode. 2. Automatically building the graph for inference, where a replicate is built under inference mode.
...@@ -24,8 +26,10 @@ This concept is used mainly to support: ...@@ -24,8 +26,10 @@ This concept is used mainly to support:
### MultiGPU Trainers ### MultiGPU Trainers
For data-parallel multi-GPU training, different [multi-GPU trainers](http://tensorpack.readthedocs.io/en/latest/modules/train.html) For data-parallel multi-GPU training, different [multi-GPU trainers](http://tensorpack.readthedocs.io/en/latest/modules/train.html)
implement different parallel logic, all reaching the same performance as the implement different parallel logic.
[official TF benchmark](https://www.tensorflow.org/performance/benchmarks). They take care of device placement, gradient averaging and synchronoization
in the efficient way and all reach the same performance as the
[official TF benchmarks](https://www.tensorflow.org/performance/benchmarks).
It takes only one line of code change to use them. It takes only one line of code change to use them.
Note some common problems when using these trainers: Note some common problems when using these trainers:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment