Commit d4f925a9 authored by Yuxin Wu's avatar Yuxin Wu

fix recommonmark version

parent 3975ae05
...@@ -2,4 +2,4 @@ termcolor ...@@ -2,4 +2,4 @@ termcolor
tqdm tqdm
nltk nltk
decorator decorator
recommonmark recommonmark==0.4.0
## Dataflow ## Dataflow
Dataflow uses Python generator to produce data in an efficient way. Dataflow uses Python generator to produce data.
A Dataflow has to implement the `get_data()` generator method, which generates `datapoints` when called. A Dataflow has to implement the `get_data()` generator method, which generates a `datapoint` when called.
A datapoint must be a list of picklable Python object. A datapoint must be a list of picklable Python object, each is called a `component` of the datapoint.
For example to train on MNIST dataset, you can define a Dataflow that produces datapoints of shape `[(BATCH, 28, 28), (BATCH,)]`.
Then, multiple Dataflows can be composed together to build a complex data-preprocessing pipeline, Then, multiple Dataflows can be composed together to build a complex data-preprocessing pipeline,
including __reading from disk, batching, augmentations, prefetching__, etc. These components written in Python including __reading from disk, batching, augmentations, prefetching__, etc. These components written in Python
can provide a much more flexible data pipeline than with TensorFlow operators. can provide a more flexible data pipeline than with TensorFlow operators.
Take a look at [common Dataflow](../../tensorpack/dataflow/common.py) and a [example of use](../../examples/ResNet/cifar10-resnet.py#L125). Take a look at [common Dataflow](../../tensorpack/dataflow/common.py) and a [example of use](../../examples/ResNet/cifar10-resnet.py#L125).
Optionally, Dataflow can implement the following two methods: Optionally, Dataflow can implement the following two methods:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment