Commit 762e4dcc authored by Yuxin Wu's avatar Yuxin Wu

some doc change

parent 8f056dc1
...@@ -8,6 +8,8 @@ so you won't need to look at here very often. ...@@ -8,6 +8,8 @@ so you won't need to look at here very often.
Here are a list of things that were changed, starting from an early version. Here are a list of things that were changed, starting from an early version.
TensorFlow itself also changes API and those are not listed here. TensorFlow itself also changes API and those are not listed here.
+ [2017/04/09](https://github.com/ppwwyyxx/tensorpack/commit/5beab907895aec36bdcaed62e25b976aad7979b8).
`ParamRestore` was renamed to `DictRestore`.
+ [2017/03/16](https://github.com/ppwwyyxx/tensorpack/commit/ccae46f4a3ca89dc3df901a338eef8447d19a730). + [2017/03/16](https://github.com/ppwwyyxx/tensorpack/commit/ccae46f4a3ca89dc3df901a338eef8447d19a730).
`session_config` option in `TrainConfig` and `PredictConfig` is deprecated. `session_config` option in `TrainConfig` and `PredictConfig` is deprecated.
Use `session_creator` to define how to create session instead. Use `session_creator` to define how to create session instead.
......
### Write an image augmentor
First thing to note: an augmentor is a part of the DataFlow, so you can always
[write a DataFlow](http://tensorpack.readthedocs.io/en/latest/tutorial/extend/dataflow.html)
to do whatever operations to your data, rather than writing an augmentor.
Augmentors just sometimes make things easier.
An augmentor maps images to images.
If you have such a mapping function `f` already, you can simply use `imgaug.MapImage(f)` as the
augmentor, or use `MapDataComponent(df, f, index)` as the DataFlow.
In other words, for simple mapping you don't need to write an augmentor.
An augmentor does something more than applying the mapping. The interface you'll need to implement
is:
```python
class MyAug(imgaug.ImageAugmentor):
def _get_augment_params(self, img):
# generated random params with self.rng
return params
def _augment(self, img, params):
return augmented_img
```
It does the following extra things for you:
1. `self.rng` is a `np.random.RandomState` object,
guranteed to have different seeds when you use multiprocess prefetch.
In multiprocess settings, you'll always need it to generate random numbers.
2. Random parameters and the actual augmentation is separated. This allows you to apply the
same random transformation to several images (with `AugmentImageComponents`),
which is important to tasks such as segmentation.
...@@ -3,14 +3,14 @@ ...@@ -3,14 +3,14 @@
## Does it support data format X / augmentation Y / layer Z? ## Does it support data format X / augmentation Y / layer Z?
The library tries to support everything, but it couldn't really include everything. The library tries to __support__ everything, but it couldn't really __include__ everything.
For the XYZ you need, you can either implement them, or use any existing code and wrap it For your XYZ, you can either implement them, or use any existing python code and wrap it
with tensorpack interface. See [Extend Tensorpack](http://tensorpack.readthedocs.io/en/latest/tutorial/index.html#extend-tensorpack) with tensorpack interface. See [Extend Tensorpack](http://tensorpack.readthedocs.io/en/latest/tutorial/index.html#extend-tensorpack)
for more details. for more details.
It you think: If you think:
1. The framework has limitation so your XYZ cannot be supported, OR 1. The framework has limitation in its interface so your XYZ cannot be supported, OR
2. Your XYZ is very common, or very well-defined, so it would be nice to include it. 2. Your XYZ is very common, or very well-defined, so it would be nice to include it.
Then it's a good time to open an issue. Then it's a good time to open an issue.
...@@ -21,26 +21,27 @@ When you enable `ModelSaver` as a callback, ...@@ -21,26 +21,27 @@ When you enable `ModelSaver` as a callback,
trained models will be stored in TensorFlow checkpoint format, which typically includes a trained models will be stored in TensorFlow checkpoint format, which typically includes a
`.data-xxxxx` file and a `.index` file. Both are necessary. `.data-xxxxx` file and a `.index` file. Both are necessary.
To inspect a checkpoint, the easiest way is `tf.train.NewCheckpointReader`. Please note that it To inspect a checkpoint, the easiest tool is `tf.train.NewCheckpointReader`. Please note that it
expects a path without the extension. expects a model path without the extension.
You can dump a cleaner version of the model (with only model/trainable variables), with You can dump a cleaner version of the model (without unnecessary variables), using
`scripts/dump-model-params.py`, as a simple `var-name: value` dict saved in npy format. `scripts/dump-model-params.py`, as a simple `var-name: value` dict saved in npy format.
The script expects a metagraph file which is also saved by `ModelSaver`. The script expects a metagraph file which is also saved by `ModelSaver`.
## How to load a model / do transfer learning ## How to load a model / do transfer learning
All model loading (in either training or testing) is through the `session_init` interface All model loading (in either training or testing) is through the `session_init` option
in `TrainConfig` or `PredictConfig`. in `TrainConfig` or `PredictConfig`.
It accepts a `SessionInit` instance, where the common options are `SaverRestore` which restores It accepts a `SessionInit` instance, where the common options are `SaverRestore` which restores
TF checkpoint, or `DictRestore` which restores a dict. `get_model_loader` is a small helper to TF checkpoint, or `DictRestore` which restores a dict. `get_model_loader` is a small helper to
decide which one to use from file name. decide which one to use from file name.
Doing transfer learning is painless. Variable restoring is completely based on name match between Doing transfer learning is straightforward. Variable restoring is completely based on name match between
the current graph and the `SessionInit` initializer. the current graph and the `SessionInit` initializer.
Therefore, if you want to re-train some layer, just rename it. Therefore, if you want to load some model, just use the same name.
And unmatched variables on both side will be printed as warning. If you want to re-train some layer, just rename it.
Unmatched variables on both side will be printed as warning.
To freeze some variables, there are [different ways](https://github.com/ppwwyyxx/tensorpack/issues/87#issuecomment-270545291) To freeze some variables, there are [different ways](https://github.com/ppwwyyxx/tensorpack/issues/87#issuecomment-270545291)
with pros and cons. with pros and cons.
...@@ -48,6 +48,7 @@ Extend Tensorpack ...@@ -48,6 +48,7 @@ Extend Tensorpack
:maxdepth: 1 :maxdepth: 1
extend/dataflow extend/dataflow
extend/augmentor
extend/model extend/model
extend/trainer extend/trainer
extend/callback extend/callback
...@@ -18,13 +18,8 @@ See the docstring in DCGAN.py for usage. ...@@ -18,13 +18,8 @@ See the docstring in DCGAN.py for usage.
""" """
# Don't want to mix two examples together, but want to reuse the code. # Don't want to mix two examples together, but want to reuse the code.
# So here just import stuff from DCGAN-CelebA, and change the batch size & model # So here just import stuff from DCGAN, and change the batch size & model
import imp import DCGAN
DCGAN = imp.load_source(
'DCGAN',
os.path.join(os.path.dirname(__file__), 'DCGAN.py'))
G.BATCH = 64 G.BATCH = 64
......
...@@ -62,20 +62,6 @@ class Augmentor(object): ...@@ -62,20 +62,6 @@ class Augmentor(object):
class ImageAugmentor(Augmentor): class ImageAugmentor(Augmentor):
def augment(self, img):
"""
Perform augmentation on the image (possibly) in-place.
Args:
img (np.ndarray): an [h,w] or [h,w,c] image.
Returns:
np.ndarray: the augmented image, always of type float32.
"""
img, params = self._augment_return_params(img)
return img
def _fprop_coord(self, coord, param): def _fprop_coord(self, coord, param):
return coord return coord
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment