Skip to content

S
seminar-breakout
Commit 675436ed authored by Yuxin Wu's avatar Yuxin Wu
Browse files
Options

trigger doc rebuild

parent 61a5960c
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
docs/tutorial/efficient-dataflow.md
View file @ 675436ed
......@@ -200,8 +200,9 @@ So DataFlow won't be a serious bottleneck if configured properly.
## More Efficient DataFlow
To work with larger datasets (or smaller networks, or more GPUS) you could be seriously bounded by CPU or disk speed of a single machine.
Then it's best to run DataFlow distributely and collect them on the
To work with larger datasets (or smaller networks, or more GPUs) you could be seriously bounded by CPU or disk speed of a single machine.
One way is to optimize the preprocessing routine (e.g. write something in C++ or use TF reading operators).
Another way to scale is to run DataFlow distributely and collect them on the
training machine. E.g.:
```python
# Data Machine #1, process 1-20:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment