Commit 762e4dcc authored by Yuxin Wu's avatar Yuxin Wu

some doc change

parent 8f056dc1
......@@ -8,6 +8,8 @@ so you won't need to look at here very often.
Here are a list of things that were changed, starting from an early version.
TensorFlow itself also changes API and those are not listed here.
+ [2017/04/09](https://github.com/ppwwyyxx/tensorpack/commit/5beab907895aec36bdcaed62e25b976aad7979b8).
`ParamRestore` was renamed to `DictRestore`.
+ [2017/03/16](https://github.com/ppwwyyxx/tensorpack/commit/ccae46f4a3ca89dc3df901a338eef8447d19a730).
`session_config` option in `TrainConfig` and `PredictConfig` is deprecated.
Use `session_creator` to define how to create session instead.
......
### Write an image augmentor
First thing to note: an augmentor is a part of the DataFlow, so you can always
[write a DataFlow](http://tensorpack.readthedocs.io/en/latest/tutorial/extend/dataflow.html)
to do whatever operations to your data, rather than writing an augmentor.
Augmentors just sometimes make things easier.
An augmentor maps images to images.
If you have such a mapping function `f` already, you can simply use `imgaug.MapImage(f)` as the
augmentor, or use `MapDataComponent(df, f, index)` as the DataFlow.
In other words, for simple mapping you don't need to write an augmentor.
An augmentor does something more than applying the mapping. The interface you'll need to implement
is:
```python
class MyAug(imgaug.ImageAugmentor):
def _get_augment_params(self, img):
# generated random params with self.rng
return params
def _augment(self, img, params):
return augmented_img
```
It does the following extra things for you:
1. `self.rng` is a `np.random.RandomState` object,
guranteed to have different seeds when you use multiprocess prefetch.
In multiprocess settings, you'll always need it to generate random numbers.
2. Random parameters and the actual augmentation is separated. This allows you to apply the
same random transformation to several images (with `AugmentImageComponents`),
which is important to tasks such as segmentation.
......@@ -3,14 +3,14 @@
## Does it support data format X / augmentation Y / layer Z?
The library tries to support everything, but it couldn't really include everything.
The library tries to __support__ everything, but it couldn't really __include__ everything.
For the XYZ you need, you can either implement them, or use any existing code and wrap it
For your XYZ, you can either implement them, or use any existing python code and wrap it
with tensorpack interface. See [Extend Tensorpack](http://tensorpack.readthedocs.io/en/latest/tutorial/index.html#extend-tensorpack)
for more details.
It you think:
1. The framework has limitation so your XYZ cannot be supported, OR
If you think:
1. The framework has limitation in its interface so your XYZ cannot be supported, OR
2. Your XYZ is very common, or very well-defined, so it would be nice to include it.
Then it's a good time to open an issue.
......@@ -21,26 +21,27 @@ When you enable `ModelSaver` as a callback,
trained models will be stored in TensorFlow checkpoint format, which typically includes a
`.data-xxxxx` file and a `.index` file. Both are necessary.
To inspect a checkpoint, the easiest way is `tf.train.NewCheckpointReader`. Please note that it
expects a path without the extension.
To inspect a checkpoint, the easiest tool is `tf.train.NewCheckpointReader`. Please note that it
expects a model path without the extension.
You can dump a cleaner version of the model (with only model/trainable variables), with
You can dump a cleaner version of the model (without unnecessary variables), using
`scripts/dump-model-params.py`, as a simple `var-name: value` dict saved in npy format.
The script expects a metagraph file which is also saved by `ModelSaver`.
## How to load a model / do transfer learning
All model loading (in either training or testing) is through the `session_init` interface
All model loading (in either training or testing) is through the `session_init` option
in `TrainConfig` or `PredictConfig`.
It accepts a `SessionInit` instance, where the common options are `SaverRestore` which restores
TF checkpoint, or `DictRestore` which restores a dict. `get_model_loader` is a small helper to
decide which one to use from file name.
Doing transfer learning is painless. Variable restoring is completely based on name match between
Doing transfer learning is straightforward. Variable restoring is completely based on name match between
the current graph and the `SessionInit` initializer.
Therefore, if you want to re-train some layer, just rename it.
And unmatched variables on both side will be printed as warning.
Therefore, if you want to load some model, just use the same name.
If you want to re-train some layer, just rename it.
Unmatched variables on both side will be printed as warning.
To freeze some variables, there are [different ways](https://github.com/ppwwyyxx/tensorpack/issues/87#issuecomment-270545291)
with pros and cons.
......@@ -48,6 +48,7 @@ Extend Tensorpack
:maxdepth: 1
extend/dataflow
extend/augmentor
extend/model
extend/trainer
extend/callback
......@@ -18,13 +18,8 @@ See the docstring in DCGAN.py for usage.
"""
# Don't want to mix two examples together, but want to reuse the code.
# So here just import stuff from DCGAN-CelebA, and change the batch size & model
import imp
DCGAN = imp.load_source(
'DCGAN',
os.path.join(os.path.dirname(__file__), 'DCGAN.py'))
# So here just import stuff from DCGAN, and change the batch size & model
import DCGAN
G.BATCH = 64
......
......@@ -62,20 +62,6 @@ class Augmentor(object):
class ImageAugmentor(Augmentor):
def augment(self, img):
"""
Perform augmentation on the image (possibly) in-place.
Args:
img (np.ndarray): an [h,w] or [h,w,c] image.
Returns:
np.ndarray: the augmented image, always of type float32.
"""
img, params = self._augment_return_params(img)
return img
def _fprop_coord(self, coord, param):
return coord
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment