Commit 7965f28c authored by Yuxin Wu's avatar Yuxin Wu

fix GoogleNetRandomCropResize interpolation

parent 931640c5
...@@ -12,92 +12,53 @@ There are two ways to do inference during training. ...@@ -12,92 +12,53 @@ There are two ways to do inference during training.
2. If your inference follows the paradigm of: 2. If your inference follows the paradigm of:
"evaluate some tensors for each input, and aggregate the results in the end". "evaluate some tensors for each input, and aggregate the results in the end".
You can use the `InferenceRunner` interface with some `Inferencer**. You can use the `InferenceRunner` interface with some `Inferencer`.
This will further support prefetch & data-parallel inference. This will further support prefetch & data-parallel inference.
More details to come.
Currently this lacks documentation, but you can refer to examples
that uses `InferenceRunner` or custom `Inferencer` to learn more.
In both methods, your tower function will be called again, with `TowerContext.is_training==False`. In both methods, your tower function will be called again, with `TowerContext.is_training==False`.
You can use this predicate to choose a different code path in inference mode. You can use this predicate to choose a different code path in inference mode.
## Inference After Training
Tensorpack is a training interface -- __it doesn't care what happened after training__. ## Inference After Training: What Tensorpack Does
You already have everything you need for inference or model diagnosis after
training:
1. The model (the graph): you've already written it yourself with TF symbolic functions.
2. The trained parameters: tensorpack saves them in standard TF checkpoint format.
### Step 1: build the model
You can build a graph however you like, with pure TensorFlow. If your model is written with
tensorpack's `ModelDesc`, you can also build it like this:
```python
a, b = tf.placeholder(...), tf.placeholder(...)
# call ANY symbolic functions on a, b. e.g.:
with TowerContext('', is_training=False):
model.build_graph(a, b)
```
```eval_rst
.. note:: **Do not use metagraph for inference!**.
Metagraph is the wrong abstraction for a "model".
It stores the entire graph which contains not only the mathematical model, but also all the
training settings (queues, iterators, summaries, evaluations, multi-gpu replications).
Therefore it is usually wrong to import a training metagraph for inference.
It's especially error-prone to load a metagraph on top of a non-empty graph. Tensorpack provides some small tools to do the most basic types of inference for demo purposes.
The potential name conflicts between the current graph and the nodes in the You can use them but
metagraph can lead to esoteric bugs or sometimes completely ruin the model. __these approaches are often suboptimal and may fail__.
They may often be inefficient or lack functionalities you need.
It's also very common to change the graph for inference.
For example, you may need a different data layout for CPU inference,
or you may need placeholders in the inference graph (which may not even exist in
the training graph). However metagraph is not designed to be easily modified at all.
Due to the above reasons, to do inference, it's best to recreate a clean graph (and save it if needed) by yourself.
```
### Step 2: load the checkpoint
You can just use `tf.train.Saver` for all the work.
Alternatively, use tensorpack's `SaverRestore(path).init(tf.get_default_session())`
Now, you've already built a graph for inference, and the checkpoint is loaded.
You can then apply any graph processing or use deployment tools TensorFlow supports.
These are unrelated to tensorpack, and you'll need to read TF docs and __do it on your own__.
If you need anything more complicated, please
learn what TensorFlow can do, and __do it on your own__ because Tensorpack
is a training interface and doesn't care what happened after training.
### OfflinePredictor ### OfflinePredictor
Tensorpack provides one tool [OfflinePredictor](../modules/predict.html#tensorpack.predict.OfflinePredictor), Tensorpack provides [OfflinePredictor](../modules/predict.html#tensorpack.predict.OfflinePredictor),
to merge the above two steps together. for inference demo after training.
It has simple functionailities to build the graph, load the checkpoint, and It has functionailities to build the graph, load the checkpoint, and
return a callable for you for simple prediction. return a callable for you for simple prediction. Refer to its docs for details.
OfflinePredictor is only for quick demo purposes. OfflinePredictor is only for quick demo purposes.
It runs inference on numpy arrays, therefore may not be the most efficient way. It runs inference on numpy arrays, therefore may not be the most efficient way.
It also has very limited functionalities. It also has very limited functionalities.
If you need anything more complicated, please __do it on your own__ because Tensorpack
doesn't care what happened after training.
A simple explanation of how it works: A simple example of how it works:
```python ```python
pred_config = PredictConfig( pred_config = PredictConfig(
session_init=get_model_loader(model_path), session_init=get_model_loader(model_path),
model=YourModel(), model=YourModel(),
input_names=['input1', 'input2'], input_names=['input1', 'input2'], # tensor names in the graph, or name of the declared inputs
output_names=['output1', 'output2']) output_names=['output1', 'output2']) # tensor names in the graph
predictor = OfflinePredictor(pred_config) predictor = OfflinePredictor(pred_config)
outputs = predictor(input1_array, input2_array) output1_array, output2_array = predictor(input1_array, input2_array)
``` ```
As mentioned before, you might want to use a different graph for inference, It's __common to use a different graph for inference__,
e.g., use NHWC format, support base64-encoded images. e.g., use NHWC format, support encoded image format, etc.
You can make these changes in the `model` or `tower_func` in your `PredictConfig`. You can make these changes inside the `model` or `tower_func` in your `PredictConfig`.
The example in [examples/basic/export-model.py](../examples/basic/export-model.py) demonstrates such an altered inference graph. The example in [examples/basics/export-model.py](../examples/basics/export-model.py) demonstrates such an altered inference graph.
### Exporter ### Exporter
...@@ -114,7 +75,7 @@ you can also save your models into other formats so it may be more friendly for ...@@ -114,7 +75,7 @@ you can also save your models into other formats so it may be more friendly for
This format contains both the graph and the variables. Refer to TensorFlow This format contains both the graph and the variables. Refer to TensorFlow
serving documentation on how to use it. serving documentation on how to use it.
2. Export to a frozen and pruned graph: 2. Export to a frozen and pruned graph for TensorFlow's builtin tools such as TOCO:
```python ```python
ModelExporter(pred_config).export_compact('/path/to/compact_graph.pb', toco_compatible=True) ModelExporter(pred_config).export_compact('/path/to/compact_graph.pb', toco_compatible=True)
...@@ -133,9 +94,71 @@ you can also save your models into other formats so it may be more friendly for ...@@ -133,9 +94,71 @@ you can also save your models into other formats so it may be more friendly for
tf.import_graph_def(graph_def) tf.import_graph_def(graph_def)
``` ```
[examples/basic/export-model.py](../examples/basic/export-model.py) demonstrates the usage of such a frozen/pruned graph. [examples/basics/export-model.py](../examples/basics/export-model.py)
demonstrates the usage of such a frozen/pruned graph.
Again, you may often want to use a different graph for inference and you can
do so by the arguments of `PredictConfig`.
## Inference After Training: Do It Yourself
Tensorpack is a training interface -- __it doesn't care what happened after training__.
It already provides everything you need for inference or model diagnosis after
training:
1. The model (the graph): you've already written it yourself with TF symbolic functions.
Nothing about it is related to the tensorpack interface.
If you use tensorpack layers, they are mainly just wrappers around `tf.layers`.
2. The trained parameters: tensorpack saves them in standard TF checkpoint format.
Nothing about it is related to tensorpack.
With the model and the weights, you can do inference with whatever approaches
TensorFlow supports. Usually it involves the following steps:
### Step 1: build the model (graph)
You can build a graph however you like, with pure TensorFlow. If your model is written with
tensorpack's `ModelDesc`, you can also build it like this:
```python
a, b = tf.placeholder(...), tf.placeholder(...)
# call ANY symbolic functions on a, b. e.g.:
with TowerContext('', is_training=False):
model.build_graph(a, b)
```
```eval_rst
.. note:: **Do not use metagraph for inference!**.
Metagraph is the wrong abstraction for a "model".
It stores the entire graph which contains not only the mathematical model, but also all the
training settings (queues, iterators, summaries, evaluations, multi-gpu replications).
Therefore it is usually wrong to import a training metagraph for inference.
It's especially error-prone to load a metagraph on top of a non-empty graph.
The potential name conflicts between the current graph and the nodes in the
metagraph can lead to esoteric bugs or sometimes completely ruin the model.
It's also very common to change the graph for inference.
For example, you may need a different data layout for CPU inference,
or you may need placeholders in the inference graph (which may not even exist in
the training graph). However metagraph is not designed to be easily modified at all.
Due to the above reasons, to do inference, it's best to recreate a clean graph (and save it if needed) by yourself.
```
### Step 2: load the checkpoint
You can just use `tf.train.Saver` for all the work.
Alternatively, use tensorpack's `SaverRestore(path).init(tf.get_default_session())`
Now, you've already built a graph for inference, and the checkpoint is loaded.
You may now:
1. use `sess.run` to do inference
2. save the grpah to some formats for further processing
3. apply graph transformation for efficient inference
Note that these steps are not the optimal way for inference. They may very likely These steps are unrelated to tensorpack, and you'll need to learn TensorFlow and
produce an inefficent graph. do it yourself.
To do efficient inference, understand what TensorFlow can do and do it on your
own, because inference after training is unrelated to tensorpack.
...@@ -94,11 +94,11 @@ class GoogleNetRandomCropAndResize(ImageAugmentor): ...@@ -94,11 +94,11 @@ class GoogleNetRandomCropAndResize(ImageAugmentor):
It attempts to crop a random rectangle with 8%~100% area of the original image, It attempts to crop a random rectangle with 8%~100% area of the original image,
and keep the aspect ratio between 3/4 to 4/3. Then it resize this crop to the target shape. and keep the aspect ratio between 3/4 to 4/3. Then it resize this crop to the target shape.
If such crop cannot be found in 10 iterations, it will to a ResizeShortestEdge + CenterCrop. If such crop cannot be found in 10 iterations, it will do a ResizeShortestEdge + CenterCrop.
""" """
def __init__(self, crop_area_fraction=(0.08, 1.), def __init__(self, crop_area_fraction=(0.08, 1.),
aspect_ratio_range=(0.75, 1.333), aspect_ratio_range=(0.75, 1.333),
target_shape=224, interp=cv2.INTER_LINEAR): target_shape=224, interp=cv2.INTER_CUBIC):
""" """
Args: Args:
crop_area_fraction (tuple(float)): Defaults to crop 8%-100% area. crop_area_fraction (tuple(float)): Defaults to crop 8%-100% area.
...@@ -123,7 +123,7 @@ class GoogleNetRandomCropAndResize(ImageAugmentor): ...@@ -123,7 +123,7 @@ class GoogleNetRandomCropAndResize(ImageAugmentor):
out = img[y1:y1 + hh, x1:x1 + ww] out = img[y1:y1 + hh, x1:x1 + ww]
out = cv2.resize(out, (self.target_shape, self.target_shape), interpolation=self.interp) out = cv2.resize(out, (self.target_shape, self.target_shape), interpolation=self.interp)
return out return out
out = ResizeShortestEdge(self.target_shape, interp=cv2.INTER_CUBIC).augment(img) out = ResizeShortestEdge(self.target_shape, interp=self.interp).augment(img)
out = CenterCrop(self.target_shape).augment(out) out = CenterCrop(self.target_shape).augment(out)
return out return out
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment