Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
S
seminar-breakout
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Shashank Suhas
seminar-breakout
Commits
9fac1a6c
Commit
9fac1a6c
authored
Nov 26, 2019
by
Yuxin Wu
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
use yield from in dataflow; update logger name in dataflow.
parent
0641618d
Changes
8
Show whitespace changes
Inline
Side-by-side
Showing
8 changed files
with
19 additions
and
31 deletions
+19
-31
.github/ISSUE_TEMPLATE/unexpected-problems---bugs.md
.github/ISSUE_TEMPLATE/unexpected-problems---bugs.md
+2
-2
CHANGES.md
CHANGES.md
+1
-0
README.md
README.md
+0
-1
tensorpack/dataflow/common.py
tensorpack/dataflow/common.py
+5
-10
tensorpack/dataflow/parallel.py
tensorpack/dataflow/parallel.py
+1
-2
tensorpack/dataflow/parallel_map.py
tensorpack/dataflow/parallel_map.py
+4
-8
tensorpack/dataflow/raw.py
tensorpack/dataflow/raw.py
+3
-7
tensorpack/utils/logger.py
tensorpack/utils/logger.py
+3
-1
No files found.
.github/ISSUE_TEMPLATE/unexpected-problems---bugs.md
View file @
9fac1a6c
...
...
@@ -60,7 +60,7 @@ If this command failed, tell us your version of Python/TF/tensorpack.
Note that:
+
You can install Tensorpack master by
`pip install -U git+https://github.com/
ppwwyyxx
/tensorpack.git`
+
You can install Tensorpack master by
`pip install -U git+https://github.com/
tensorpack
/tensorpack.git`
and see if your issue is already solved.
+
If you're not using tensorpack under a normal command line shell (e.g.,
using an IDE or jupyter notebook), please retry under a normal command line shell.
...
...
CHANGES.md
View file @
9fac1a6c
...
...
@@ -8,6 +8,7 @@ so you don't need to look at here very often.
Here are a list of things that were changed, starting from an early version.
TensorFlow itself also changes API and those are not listed here.
+
2019/11/10. Drop Python 2 support.
+
[
2019/03/20
](
https://github.com/tensorpack/tensorpack/commit/b8a50d72a7c655b6dc6facb17efd74069ba7f86c
)
.
The concept of
`InputDesc`
was replaced by its equivalent in TF:
`tf.TensorSpec`
. This may be a breaking change if you have customized
...
...
README.md
View file @
9fac1a6c
...
...
@@ -2,7 +2,6 @@
Tensorpack is a neural network training interface based on TensorFlow.
[

](https://travis-ci.org/tensorpack/tensorpack)
[

](http://tensorpack.readthedocs.io)
[

](https://gitter.im/tensorpack/users)
[

](http://models.tensorpack.com)
...
...
tensorpack/dataflow/common.py
View file @
9fac1a6c
...
...
@@ -48,8 +48,7 @@ class TestDataSpeed(ProxyDataFlow):
def
__iter__
(
self
):
""" Will run testing at the beginning, then produce data normally. """
self
.
start
()
for
dp
in
self
.
ds
:
yield
dp
yield
from
self
.
ds
def
start
(
self
):
"""
...
...
@@ -387,12 +386,10 @@ class RepeatedData(ProxyDataFlow):
def
__iter__
(
self
):
if
self
.
num
==
-
1
:
while
True
:
for
dp
in
self
.
ds
:
yield
dp
yield
from
self
.
ds
else
:
for
_
in
range
(
self
.
num
):
for
dp
in
self
.
ds
:
yield
dp
yield
from
self
.
ds
class
RepeatedDataPoint
(
ProxyDataFlow
):
...
...
@@ -519,8 +516,7 @@ class ConcatData(DataFlow):
def
__iter__
(
self
):
for
d
in
self
.
df_lists
:
for
dp
in
d
.
__iter__
():
yield
dp
yield
from
d
class
JoinData
(
DataFlow
):
...
...
@@ -702,8 +698,7 @@ class CacheData(ProxyDataFlow):
if
len
(
self
.
buffer
):
if
self
.
shuffle
:
self
.
rng
.
shuffle
(
self
.
buffer
)
for
dp
in
self
.
buffer
:
yield
dp
yield
from
self
.
buffer
else
:
for
dp
in
self
.
ds
:
yield
dp
...
...
tensorpack/dataflow/parallel.py
View file @
9fac1a6c
...
...
@@ -48,8 +48,7 @@ class _ExceptionWrapper:
def
_repeat_iter
(
get_itr
):
while
True
:
for
x
in
get_itr
():
yield
x
yield
from
get_itr
()
def
_bind_guard
(
sock
,
name
):
...
...
tensorpack/dataflow/parallel_map.py
View file @
9fac1a6c
...
...
@@ -85,11 +85,9 @@ class _ParallelMapData(ProxyDataFlow):
def
__iter__
(
self
):
if
self
.
_strict
:
for
dp
in
self
.
get_data_strict
():
yield
dp
yield
from
self
.
get_data_strict
()
else
:
for
dp
in
self
.
get_data_non_strict
():
yield
dp
yield
from
self
.
get_data_non_strict
()
class
MultiThreadMapData
(
_ParallelMapData
):
...
...
@@ -205,8 +203,7 @@ class MultiThreadMapData(_ParallelMapData):
def
__iter__
(
self
):
with
self
.
_guard
:
for
dp
in
super
(
MultiThreadMapData
,
self
)
.
__iter__
():
yield
dp
yield
from
super
(
MultiThreadMapData
,
self
)
.
__iter__
()
def
__del__
(
self
):
if
self
.
_evt
is
not
None
:
...
...
@@ -320,8 +317,7 @@ class MultiProcessMapDataZMQ(_ParallelMapData, _MultiProcessZMQDataFlow):
def
__iter__
(
self
):
with
self
.
_guard
,
_zmq_catch_error
(
type
(
self
)
.
__name__
):
for
dp
in
super
(
MultiProcessMapDataZMQ
,
self
)
.
__iter__
():
yield
dp
yield
from
super
(
MultiProcessMapDataZMQ
,
self
)
.
__iter__
()
class
MultiProcessMapAndBatchDataZMQ
(
_MultiProcessZMQDataFlow
):
...
...
tensorpack/dataflow/raw.py
View file @
9fac1a6c
...
...
@@ -85,8 +85,7 @@ class DataFromList(RNGDataFlow):
def
__iter__
(
self
):
if
not
self
.
shuffle
:
for
k
in
self
.
lst
:
yield
k
yield
from
self
.
lst
else
:
idxs
=
np
.
arange
(
len
(
self
.
lst
))
self
.
rng
.
shuffle
(
idxs
)
...
...
@@ -110,9 +109,7 @@ class DataFromGenerator(DataFlow):
self
.
_gen
=
gen
def
__iter__
(
self
):
# yield from
for
dp
in
self
.
_gen
():
yield
dp
yield
from
self
.
_gen
()
class
DataFromIterable
(
DataFlow
):
...
...
@@ -129,5 +126,4 @@ class DataFromIterable(DataFlow):
return
self
.
_len
def
__iter__
(
self
):
for
dp
in
self
.
_itr
:
yield
dp
yield
from
self
.
_itr
tensorpack/utils/logger.py
View file @
9fac1a6c
...
...
@@ -46,7 +46,9 @@ class _MyFormatter(logging.Formatter):
def
_getlogger
():
logger
=
logging
.
getLogger
(
'tensorpack'
)
# this file is synced to "dataflow" package as well
package_name
=
"dataflow"
if
__name__
.
startswith
(
"dataflow"
)
else
"tensorpack"
logger
=
logging
.
getLogger
(
package_name
)
logger
.
propagate
=
False
logger
.
setLevel
(
logging
.
INFO
)
handler
=
logging
.
StreamHandler
(
sys
.
stdout
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment