Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
S
seminar-breakout
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Shashank Suhas
seminar-breakout
Commits
6926d22a
Commit
6926d22a
authored
May 09, 2019
by
Yuxin Wu
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
MultiGPU version of "mnist-keras.py"
parent
f7fecef9
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
34 additions
and
18 deletions
+34
-18
examples/keras/README.md
examples/keras/README.md
+8
-3
examples/keras/mnist-keras.py
examples/keras/mnist-keras.py
+22
-13
tensorpack/contrib/keras.py
tensorpack/contrib/keras.py
+4
-2
No files found.
examples/keras/README.md
View file @
6926d22a
...
@@ -15,9 +15,14 @@ are the only two tools I know that can scale the training of a large Keras model
...
@@ -15,9 +15,14 @@ are the only two tools I know that can scale the training of a large Keras model
### Simple Examples:
### Simple Examples:
[
mnist-keras.py
](
mnist-keras.py
)
: a simple MNIST model written mostly in tensorpack style, but use Keras model as symbolic functions.
There are two flavors where you can use a Keras model inside tensorpack:
[
mnist-keras-v2.py
](
mnist-keras-v2.py
)
: the same MNIST model written in Keras style.
1.
Write the tower function similar to a standard tensorpack program, but use some Keras layers in
between. See
[
mnist-keras.py
](
mnist-keras.py
)
on how to do this.
It does not support all tensorpack trainers.
2.
The entire model to train is a Keras model (and there will be no
`ModelDesc`
, etc).
See
[
mnist-keras-v2.py
](
mnist-keras-v2.py
)
.
### ImageNet Example:
### ImageNet Example:
...
@@ -36,7 +41,7 @@ It has:
...
@@ -36,7 +41,7 @@ It has:
Keras does not respect variable scopes or variable
Keras does not respect variable scopes or variable
collections, which contradicts with tensorpack trainers.
collections, which contradicts with tensorpack trainers.
Therefore Keras support is __experimental__.
Therefore Keras support is __experimental__.
These simple examples can run within tensorpack smoothly, but note that a future
These simple examples can run within tensorpack smoothly, but note that a future
version of Keras or a complicated model may break them (unlikely, though).
version of Keras or a complicated model may break them (unlikely, though).
examples/keras/mnist-keras.py
View file @
6926d22a
...
@@ -10,6 +10,7 @@ from tensorpack import *
...
@@ -10,6 +10,7 @@ from tensorpack import *
from
tensorpack.contrib.keras
import
KerasPhaseCallback
from
tensorpack.contrib.keras
import
KerasPhaseCallback
from
tensorpack.dataflow
import
dataset
from
tensorpack.dataflow
import
dataset
from
tensorpack.utils.argtools
import
memoized
from
tensorpack.utils.argtools
import
memoized
from
tensorpack.utils.gpu
import
get_num_gpu
KL
=
keras
.
layers
KL
=
keras
.
layers
...
@@ -23,19 +24,20 @@ Note: this example does not work for replicated-style data-parallel trainers.
...
@@ -23,19 +24,20 @@ Note: this example does not work for replicated-style data-parallel trainers.
IMAGE_SIZE
=
28
IMAGE_SIZE
=
28
@
memoized
# this is necessary for sonnet/K
eras to work under tensorpack
@
memoized
# this is necessary for sonnet/k
eras to work under tensorpack
def
get_keras_model
():
def
get_keras_model
():
M
=
keras
.
models
.
Sequential
()
with
tf
.
name_scope
(
'/'
):
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
activation
=
'relu'
,
input_shape
=
[
IMAGE_SIZE
,
IMAGE_SIZE
,
1
],
padding
=
'same'
))
M
=
keras
.
models
.
Sequential
()
M
.
add
(
KL
.
MaxPooling2D
())
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
activation
=
'relu'
,
input_shape
=
[
IMAGE_SIZE
,
IMAGE_SIZE
,
1
],
padding
=
'same'
))
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
activation
=
'relu'
,
padding
=
'same'
))
M
.
add
(
KL
.
MaxPooling2D
())
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
activation
=
'relu'
,
padding
=
'same'
))
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
activation
=
'relu'
,
padding
=
'same'
))
M
.
add
(
KL
.
MaxPooling2D
())
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
activation
=
'relu'
,
padding
=
'same'
))
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
padding
=
'same'
,
activation
=
'relu'
))
M
.
add
(
KL
.
MaxPooling2D
())
M
.
add
(
KL
.
Flatten
())
M
.
add
(
KL
.
Conv2D
(
32
,
3
,
padding
=
'same'
,
activation
=
'relu'
))
M
.
add
(
KL
.
Dense
(
512
,
activation
=
'relu'
,
kernel_regularizer
=
keras
.
regularizers
.
l2
(
1e-5
)))
M
.
add
(
KL
.
Flatten
())
M
.
add
(
KL
.
Dropout
(
0.5
))
M
.
add
(
KL
.
Dense
(
512
,
activation
=
'relu'
,
kernel_regularizer
=
keras
.
regularizers
.
l2
(
1e-5
)))
M
.
add
(
KL
.
Dense
(
10
,
activation
=
None
,
kernel_regularizer
=
keras
.
regularizers
.
l2
(
1e-5
)))
M
.
add
(
KL
.
Dropout
(
0.5
))
M
.
add
(
KL
.
Dense
(
10
,
activation
=
None
,
kernel_regularizer
=
keras
.
regularizers
.
l2
(
1e-5
)))
return
M
return
M
...
@@ -96,4 +98,11 @@ if __name__ == '__main__':
...
@@ -96,4 +98,11 @@ if __name__ == '__main__':
max_epoch
=
100
,
max_epoch
=
100
,
)
)
launch_train_with_config
(
cfg
,
QueueInputTrainer
())
if
get_num_gpu
()
<=
1
:
# single GPU:
launch_train_with_config
(
cfg
,
QueueInputTrainer
())
else
:
# multi GPU:
launch_train_with_config
(
cfg
,
SyncMultiGPUTrainerParameterServer
(
2
))
# "Replicated" multi-gpu trainer is not supported for Keras model
# since Keras does not respect variable scopes.
tensorpack/contrib/keras.py
View file @
6926d22a
...
@@ -40,7 +40,7 @@ class KerasModelCaller(object):
...
@@ -40,7 +40,7 @@ class KerasModelCaller(object):
self
.
get_model
=
get_model
self
.
get_model
=
get_model
self
.
cached_model
=
None
self
.
cached_model
=
None
def
__call__
(
self
,
input_tensors
):
def
__call__
(
self
,
*
input_tensors
):
"""
"""
Args:
Args:
input_tensors ([tf.Tensor])
input_tensors ([tf.Tensor])
...
@@ -106,6 +106,8 @@ class KerasModelCaller(object):
...
@@ -106,6 +106,8 @@ class KerasModelCaller(object):
post_process_model
(
model
)
post_process_model
(
model
)
if
isinstance
(
outputs
,
list
)
and
len
(
outputs
)
==
1
:
return
outputs
[
0
]
return
outputs
return
outputs
...
@@ -167,7 +169,7 @@ def setup_keras_trainer(
...
@@ -167,7 +169,7 @@ def setup_keras_trainer(
target_tensors
=
list
(
inputs
[
nr_inputs
:])
target_tensors
=
list
(
inputs
[
nr_inputs
:])
# TODO mapping between target tensors & output tensors
# TODO mapping between target tensors & output tensors
outputs
=
model_caller
(
input_tensors
)
outputs
=
model_caller
(
*
input_tensors
)
if
isinstance
(
outputs
,
tf
.
Tensor
):
if
isinstance
(
outputs
,
tf
.
Tensor
):
outputs
=
[
outputs
]
outputs
=
[
outputs
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment