update docs

README.md

@@ -12,17 +12,18 @@ Tensorpack is a training interface based on TensorFlow.
 It's Yet Another TF high-level API, with __speed__, __readability__ and __flexibility__ built together.
 
 1. Focus on __training speed__.
-	+	Speed comes for free with tensorpack -- it uses TensorFlow in the __efficient way__ with no extra overhead.
+	+ Speed comes for free with tensorpack -- it uses TensorFlow in the __efficient way__ with no extra overhead.
 	  On different CNNs, it runs training [1.2~5x faster](https://github.com/tensorpack/benchmarks/tree/master/other-wrappers) than the equivalent Keras code.
 
-	+ Data-parallel multi-GPU/distributed training is off-the-shelf to use with
-      one line of code. It scales as well as Google's [official benchmark](https://www.tensorflow.org/performance/benchmarks).
+	+ Data-parallel multi-GPU/distributed training strategy is off-the-shelf to use.
+      It scales as well as Google's [official benchmark](https://www.tensorflow.org/performance/benchmarks).
 
-	+ See [tensorpack/benchmarks](https://github.com/tensorpack/benchmarks) for more benchmark scripts.
+	+ See [tensorpack/benchmarks](https://github.com/tensorpack/benchmarks) for
+      some benchmark scripts.
 
 2. Focus on __large datasets__.
 	+ It's unnecessary to read/preprocess data with a new language called TF.
-		Tensorpack helps you load large datasets (e.g. ImageNet) in __pure Python__ with autoparallelization.
+	  Tensorpack helps you load large datasets (e.g. ImageNet) in __pure Python__ with autoparallelization.
 
 3. It's not a model wrapper.
 	+ There are too many symbolic function wrappers in the world. Tensorpack includes only a few common models.

docs/README.md

@@ -11,4 +11,5 @@ will build the docs in `build/html`.
 
 ### Build Dash/Zeal docset
 
-`make docset` produces `tensorpack.docset`.
+1. `pip install doc2dash`
+2. `make docset` produces `tensorpack.docset`.

docs/conf.py

@@ -69,7 +69,7 @@ if ON_RTD:
 else:
     # skip this when building locally
     intersphinx_timeout = 0.1
-intersphinx_mapping = {'python': ('https://docs.python.org/3.4', None)}
+intersphinx_mapping = {'python': ('https://docs.python.org/3.6', None)}
 # -------------------------
 
 # Add any paths that contain templates here, relative to this directory.

docs/index.rst

@@ -10,22 +10,25 @@ It's Yet Another TF wrapper, but different in:
 - Focus on **training speed**.
 
   - Speed comes for free with tensorpack -- it uses TensorFlow in the
-    **efficient way** with no extra overhead. On various CNNs, it runs 1.5~1.7x faster than the equivalent Keras code.
+    **efficient way** with no extra overhead. On various CNNs, it runs 
+    `1.2~5x faster <https://github.com/tensorpack/benchmarks/tree/master/other-wrappers>`_
+    than the equivalent Keras code.
 
-  - Data-parallel multi-GPU training is off-the-shelf to use. It is as fast as Google's
+  - Data-parallel multi-GPU/distributed training strategy is off-the-shelf to use. 
+    It is as fast as Google's
     `official benchmark <https://www.tensorflow.org/performance/benchmarks>`_.
     You cannot beat its speed unless you're a TensorFlow expert.
 
   - See `tensorpack/benchmarks <https://github.com/tensorpack/benchmarks>`_ for some benchmark scripts.
 
-- Focus on large datasets.
+- Focus on **large datasets**.
 
-  - It's painful to read/preprocess data through TF. Tensorpack helps you load large datasets (e.g. ImageNet) in
-    **pure Python** with autoparallelization.
+  - It's unnecessary to read/preprocess data with a new language called TF.
+    Tensorpack helps you load large datasets (e.g. ImageNet) in **pure Python** with autoparallelization.
 
 - It's not a model wrapper.
 
-  - There are already too many symbolic function wrappers.
+  - There are already too many symbolic function wrappers in the world.
     Tensorpack includes only a few common models, but you can use any other wrappers within tensorpack, including sonnet/Keras/slim/tflearn/tensorlayer/....
 
 See :doc:`tutorial/index` to know more about these features:
@@ -36,5 +39,3 @@ See :doc:`tutorial/index` to know more about these features:
 
   tutorial/index
   modules/index
-..  casestudies/index
-

docs/modules/callbacks.rst

 tensorpack.callbacks package
 ============================
 
+__Everything__ other than the training iterations happen in the callbacks.
+Most of the fancy things you want to do will probably end up here.
+See relevant tutorials: :doc:`../tutorial/callback`.
+
 .. container:: custom-index
 
     .. raw:: html

docs/modules/dataflow.rst

 tensorpack.dataflow package
 ===========================
 
+Relevant tutorials: :doc:`../tutorial/dataflow`, :doc:`../tutorial/input-source`.
+
 .. container:: custom-index
 
     .. raw:: html

docs/modules/input_source.rst

 tensorpack.input_source package
 ================================
 
+Relevant tutorials: :doc:`../tutorial/input-source`.
+
 .. automodule:: tensorpack.input_source
     :members:
     :undoc-members:

docs/modules/models.rst

 tensorpack.models package
 =========================
 
+Relevant tutorials: :doc:`../tutorial/symbolic`.
+
 .. container:: custom-index
 
     .. raw:: html

docs/modules/train.rst

 tensorpack.train package
 ========================
 
+Relevant tutorials: :doc:`../tutorial/trainer`, :doc:`../tutorial/training-interface`
+
 .. container:: custom-index
 
     .. raw:: html

docs/tutorial/extend/callback.md

@@ -98,7 +98,7 @@ You can overwrite any of the following methods to define a new callback:
   By default it will get called by `_trigger_epoch`,
   but you can customize the scheduling of this method by
   [`PeriodicTrigger`](../../modules/callbacks.html#tensorpack.callbacks.PeriodicTrigger),
-  to let this method run every k steps (potentially more frequently) or every k epochs.
+  to let this method run every k steps or every k epochs.
 
 ### What you can do in the callback
 

docs/tutorial/extend/model.md

@@ -3,7 +3,7 @@
 
 The first thing to note: __you never have to write a layer__.
 Tensorpack layers are nothing but wrappers of symbolic functions.
-In tensorpack, you can use any symbolic functions you have written or seen elsewhere with or without tensorpack layers.
+In tensorpack, you can use __any__ symbolic functions you have written or seen elsewhere with or without tensorpack layers.
 
 If you would like, you can make a symbolic function become a "layer" by following some simple rules, and then gain benefits from the framework.
 

docs/tutorial/extend/trainer.md

@@ -2,7 +2,7 @@
 
 ### Role of Trainer
 
-Tensorpack follows the "define-and-run" paradigm. A training has two steps:
+Tensorpack follows the "define-and-run" paradigm. Therefore a training script has two steps:
 
 1. __Define__: Build graph for the model.
 	Users can call whatever tensorflow functions to setup the graph.
@@ -59,19 +59,20 @@ In practice you'll not use this method directly, but use [high-level interface](
 
 ### Write a Trainer
 
-The existing trainers should be enough for single-tower single-cost optimization tasks.
+The existing trainers should be enough for data-parallel single-cost optimization tasks.
 If you just want to do some extra work during training, first consider writing it as a callback,
 or write an issue to see if there is a better solution than creating new trainers.
 If your task is fundamentally different from single-cost optimization, you will need to write a trainer.
 
-You can do customize training by either using or inheriting the base `Trainer` class.
-You will need to define two things for a new Trainer:
+You can customize the trainer by either using or inheriting the base `Trainer` class.
+You will need to do two things for a new Trainer:
 
-1. Define the graph.
-	Add any tensors and ops you like, either before creating the trainer or inside `Trainer.__init__`.
+1. Define the graph. There are 2 ways you can do this:
+    1. Create any tensors and ops you like, before creating the trainer.
+    2. Create them inside `Trainer.__init__`.
 
-2. What is the iteration. There are 2 ways to define the iteration:
-	1. Set `Trainer.train_op`. This op will be run by default.
+2. Define what is the iteration. There are 2 ways to define the iteration:
+	1. Set `Trainer.train_op` to a TensorFlow operation. This op will be run by default.
 	2. Subclass `Trainer` and override the `run_step()` method. This way you can do something more than running an op.
 
 There are several different [GAN trainers](../../examples/GAN/GAN.py) for reference.

docs/tutorial/input-source.md

@@ -36,8 +36,8 @@ This is the major reason why tensorpack is [faster](https://github.com/tensorpac
 ## Python Reader or TF Reader ?
 
 The above discussion is valid regardless of what you use to load/preprocess data,
-either Python code or TensorFlow operators.
-Both are supported in tensorpack, while we recommend using Python.
+either Python code or TensorFlow operators, or a mix of two.
+Both are supported in tensorpack, while we recommend using Python. 
 
 ### TensorFlow Reader: Pros
 * Faster read/preprocessing.
@@ -63,8 +63,8 @@ Both are supported in tensorpack, while we recommend using Python.
 The disadvantage of TF reader is obvious and it's huge: it's __too complicated__.
 
 Unlike running a mathematical model, reading data is a complicated and poorly-structured task.
-You need to handle different formats, handle corner cases, noisy data,
-which all require condition operations, loops, sometimes even exception handling. These operations
+You need to handle different formats, handle corner cases, noisy data, combination of data,
+which require condition operations, loops, data structures, sometimes even exception handling. These operations
 are __naturally not suitable__ for a symbolic graph.
 
 Let's take a look at what users are asking for `tf.data`:
@@ -75,7 +75,7 @@ Let's take a look at what users are asking for `tf.data`:
 * [Sort/skip some data](https://github.com/tensorflow/tensorflow/issues/14250)
 * [Write data to files](https://github.com/tensorflow/tensorflow/issues/15014)
 
-To support all these features which could've been done with __3 lines of code in Python__, you need either a new TF
+To support all these features which could've been done with __3 lines of code in Python __, you need either a new TF
 API, or ask [Dataset.from_generator](https://www.tensorflow.org/versions/r1.4/api_docs/python/tf/contrib/data/Dataset#from_generator)
 (i.e. Python again) to the rescue.
 

tensorpack/graph_builder/training.py

@@ -179,8 +179,8 @@ class SyncMultiGPUReplicatedBuilder(DataParallelBuilder):
     `tensorflow/benchmarks <https://github.com/tensorflow/benchmarks>`_.
 
     Attribute:
-        grads: #GPU number of lists of (g, v). Synchronized gradients on each device, available after build()
-            Though on different devices, they should contain the same value.
+        grads: #GPU number of lists of (g, v). Synchronized gradients on each device,
+        available after build() Though on different devices, they should contain the same value.
     """
 
     def __init__(self, towers, average, mode):