Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
S
seminar-breakout
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Shashank Suhas
seminar-breakout
Commits
ca06ba07
Commit
ca06ba07
authored
Jan 12, 2018
by
Yuxin Wu
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
rename 'prefetch.py' to 'parallel.py'
parent
efaf9d7b
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
5 deletions
+4
-5
tensorpack/dataflow/parallel.py
tensorpack/dataflow/parallel.py
+4
-5
No files found.
tensorpack/dataflow/p
refetch
.py
→
tensorpack/dataflow/p
arallel
.py
View file @
ca06ba07
# -*- coding: UTF-8 -*-
# File: p
refetch
.py
# File: p
arallel
.py
from
__future__
import
print_function
...
...
@@ -130,11 +130,10 @@ class PrefetchData(ProxyDataFlow):
b. When ``nr_proc>1``, the dataflow produces the same distribution
of data as ``ds`` if each sample from ``ds`` is i.i.d. (e.g. fully shuffled).
You probably only want to use it for training.
2. This
is significantly slower
than :class:`PrefetchDataZMQ` when data is large.
3.
When nesting
like this: ``PrefetchDataZMQ(PrefetchData(df, nr_proc=a), nr_proc=b)``.
2. This
has more serialization overhead
than :class:`PrefetchDataZMQ` when data is large.
3.
You can nest
like this: ``PrefetchDataZMQ(PrefetchData(df, nr_proc=a), nr_proc=b)``.
A total of ``a`` instances of ``df`` worker processes will be created.
This is different from the behavior of :class:`PrefetchDataZMQ`
4. `reset_state()` is a no-op. The worker processes won't get called.
4. fork happens in `__init__`. `reset_state()` is a no-op. The worker processes won't get called.
"""
class
_Worker
(
mp
.
Process
):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment