Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
S
seminar-breakout
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Shashank Suhas
seminar-breakout
Commits
eccba14e
Commit
eccba14e
authored
May 25, 2019
by
Yuxin Wu
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
update docs
parent
6c185f3b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
10 deletions
+5
-10
docs/tutorial/dataflow.md
docs/tutorial/dataflow.md
+5
-10
No files found.
docs/tutorial/dataflow.md
View file @
eccba14e
...
@@ -18,6 +18,7 @@ DataFlow is __independent of TensorFlow__ since it produces any python objects
...
@@ -18,6 +18,7 @@ DataFlow is __independent of TensorFlow__ since it produces any python objects
(usually numpy arrays).
(usually numpy arrays).
To
`import tensorpack.dataflow`
, you don't even have to install TensorFlow.
To
`import tensorpack.dataflow`
, you don't even have to install TensorFlow.
You can simply use DataFlow as a data processing pipeline and plug it into any other frameworks.
You can simply use DataFlow as a data processing pipeline and plug it into any other frameworks.
And we plan to make it installable as a separate project.
### Load Raw Data
### Load Raw Data
We do not make any assumptions about your data format.
We do not make any assumptions about your data format.
...
@@ -72,17 +73,11 @@ for dp in df:
...
@@ -72,17 +73,11 @@ for dp in df:
# dp is now a list. do whatever
# dp is now a list. do whatever
```
```
### Why DataFlow
### Why DataFlow
?
1.
It's easy: write everything in pure Python, and reuse existing utilities.
It's easy and fast. For more discussions, see
[
Why DataFlow?
](
/tutorial/philosophy/dataflow.html
)
On the contrary, writing data loaders in TF operators is usually painful, and performance is hard to tune.
Nevertheless, using DataFlow is not required.
See more discussions in
[
Python Reader or TF Reader
](
extend/input-source.html#python-reader-or-tf-reader
)
.
Tensorpack supports data loading with native TF operators / TF datasets as well.
2.
It's fast: see
[
Efficient DataFlow
](
efficient-dataflow.html
)
on how to build a fast DataFlow with parallelism.
If you're using DataFlow with tensorpack, also see
[
Input Pipeline tutorial
](
extend/input-source.html
)
on how tensorpack further accelerates data loading in the graph.
Nevertheless, tensorpack supports data loading with native TF operators / TF datasets as well.
Read the
[
API documentation
](
../../modules/dataflow.html
)
Read the
[
API documentation
](
../../modules/dataflow.html
)
to see API details of DataFlow and a complete list of built-in DataFlow.
to see API details of DataFlow and a complete list of built-in DataFlow.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment