update docs

docs/tutorial/faq.md

@@ -27,7 +27,7 @@ Then it is a good time to open an issue.
 3. The [ProgressBar](../modules/callbacks.html#tensorpack.callbacks.ProgressBar)
 	 callback can print some scalar statistics, though not enabled by default.
 
-4. Read [Summary and Loggin](summary.html) for more options on logging.
+4. Read [Summary and Logging](summary.html) for more options on logging.
 
 ## How to freeze some variables in training
 

docs/tutorial/intro.rst

@@ -9,14 +9,20 @@ Why tensorpack?
 ~~~~~~~~~~~~~~~~~~~
 
 TensorFlow is powerful, but at the same time too complicated for a lot of people, especially when **speed** is a concern.
-Users can often write slow code with low-level APIs or other high-level APIs.
+Users can often write slow code with low-level APIs or other existing high-level wrappers.
 Even a lot of official TensorFlow examples are written for simplicity rather than efficiency,
 which as a result makes people think TensorFlow is slow.
 
-Tensorpack uses TensorFlow efficiently, and hides these details under its APIs.
-You no longer need to learn about
-multi-GPU model replication, device placement, variables synchronization, queues -- anything that's unrelated to the model itself.
-You still need to learn to write models with TF, but everything else is taken care of by tensorpack, in the efficient way.
+The `official TensorFlow benchmark <https://github.com/tensorflow/benchmarks/tree/master/scripts/tf_cnn_benchmarks>`_ said this in their README:
+
+  These models are designed for performance. For models that have clean and easy-to-read implementations, see the TensorFlow Official Models.
+
+which seems to suggest that you cannot have performance and ease-of-use together.
+However you can have them both in tensorpack.
+Tensorpack uses TensorFlow efficiently, and hides performance details under its APIs.
+You no longer need to write
+data prefetch, multi-GPU replication, device placement, variables synchronization -- anything that's unrelated to the model itself.
+You still need to learn to write models with TF, but performance is all taken care of by tensorpack.
 
 A High Level Glance
 ~~~~~~~~~~~~~~~~~~~

examples/ResNet/cifar10-preact18-mixup.py

@@ -106,7 +106,8 @@ def get_data(train_or_test, isMixup, alpha):
         if not isTrain or not isMixup:
             return [images, one_hot_labels]
 
-        # mixup:
+        # mixup implementation:
+        # Note that for larger images, it's more efficient to do mixup on GPUs (i.e. in the graph)
         weight = np.random.beta(alpha, alpha, BATCH_SIZE)
         x_weight = weight.reshape(BATCH_SIZE, 1, 1, 1)
         y_weight = weight.reshape(BATCH_SIZE, 1)

tensorpack/dataflow/dataset/ilsvrc.py

@@ -217,6 +217,10 @@ class ILSVRC12(ILSVRC12Files):
         super(ILSVRC12, self).__init__(
             dir, name, meta_dir, shuffle, dir_structure)
 
+    """
+    There are some CMYK / png images, but cv2 seems robust to them.
+    https://github.com/tensorflow/models/blob/c0cd713f59cfe44fa049b3120c417cc4079c17e3/research/inception/inception/data/build_imagenet_data.py#L264-L300
+    """
     def get_data(self):
         for fname, label in super(ILSVRC12, self).get_data():
             im = cv2.imread(fname, cv2.IMREAD_COLOR)

tensorpack/dataflow/prefetch.py

@@ -194,28 +194,29 @@ class PrefetchDataZMQ(_MultiProcessZMQDataFlow):
     Prefetch data from a DataFlow using multiple processes, with ZeroMQ for
     communication.
     It will fork the calling process of :meth:`reset_state()`,
-    and collect datapoints from `ds` in each process by ZeroMQ IPC pipe.
+    and collect datapoints from the given dataflow in each process by ZeroMQ IPC pipe.
 
     Note:
         1. An iterator cannot run faster automatically -- what's happenning is
            that the underlying dataflow will be forked ``nr_proc`` times.
            As a result, we have the following guarantee on the dataflow correctness:
 
-           a. When ``nr_proc=1``, the dataflow produces the same data as ``ds`` in the same order.
-           b. When ``nr_proc>1``, the dataflow produces the same distribution
-              of data as ``ds`` if each sample from ``ds`` is i.i.d. (e.g. fully shuffled).
+           a. When ``nr_proc=1``, this dataflow produces the same data as the
+                given dataflow in the same order.
+           b. When ``nr_proc>1``, if each sample from the given dataflow is i.i.d. (e.g. fully shuffled),
+                then this dataflow produces the **same distribution** of data as the given dataflow.
+                This implies that there will be duplication, reordering, etc.
                 You probably only want to use it for training.
-        2. The fork of processes happened in the `reset_state()` method.
+                If the samples are not i.i.d., the behavior is undefined.
+        2. `reset_state()` of the given dataflow will be called **once and only once** in the worker processes.
+        3. The fork of processes happened in this dataflow's `reset_state()` method.
            Please note that forking a TensorFlow GPU session may be unsafe.
            If you're managing this dataflow on your own,
            it's better to fork before creating the session.
-        3. After the fork has happened, this dataflow becomes not fork-safe.
+        4. After the fork has happened, this dataflow becomes not fork-safe.
            i.e., if you fork an already reset instance of this dataflow,
            it won't be usable in the forked process.
-        4. Calling `reset_state()` more than once is a no-op, i.e. the worker processes won't get called.
-        5. When nesting like this: ``PrefetchDataZMQ(PrefetchDataZMQ(df, nr_proc=a), nr_proc=b)``.
-           A total of ``a * b`` instances of ``df`` worker processes will be created.
-           Also in this case, some zmq pipes cannot be cleaned at exit.
+        5. Do not nest two `PrefetchDataZMQ`.
         6. By default, a UNIX named pipe will be created in the current directory.
            However, certain non-local filesystem such as NFS/GlusterFS/AFS doesn't always support pipes.
            You can change the directory by ``export TENSORPACK_PIPEDIR=/other/dir``.