Commit e4953ecf authored by Yuxin Wu's avatar Yuxin Wu

update docs

parent 54eebb3b
...@@ -10,9 +10,10 @@ See [Unawareness of Deep Learning Mistakes](https://medium.com/@ppwwyyxx/unaware ...@@ -10,9 +10,10 @@ See [Unawareness of Deep Learning Mistakes](https://medium.com/@ppwwyyxx/unaware
## Getting Started: ## Getting Started:
+ [An illustrative mnist example with explanation of the framework](mnist-convnet.py) + [An illustrative mnist example with explanation of the framework](mnist-convnet.py)
+ The same mnist example using [tf-slim](mnist-tfslim.py), [Keras layers](mnist-keras.py), [Higher-level Keras](mnist-keras-v2.py) and [with weights visualizations](mnist-visualizations.py) + The same mnist example using [tf-slim](mnist-tfslim.py), and [with weights visualizations](mnist-visualizations.py)
+ A tiny [Cifar ConvNet](cifar-convnet.py) and [SVHN ConvNet](svhn-digit-convnet.py) + A tiny [Cifar ConvNet](cifar-convnet.py) and [SVHN ConvNet](svhn-digit-convnet.py)
+ [A boilerplate file to start with, for your own tasks](boilerplate.py) + [A boilerplate file to start with, for your own tasks](boilerplate.py)
+ If you've used Keras, check out [Keras examples](keras).
## Vision: ## Vision:
| Name | Performance | | Name | Performance |
......
...@@ -3,6 +3,12 @@ ...@@ -3,6 +3,12 @@
Use Keras to define a model a train it with efficient tensorpack trainers. Use Keras to define a model a train it with efficient tensorpack trainers.
### Why?
Keras alone has various overhead. In particular, it is not efficient when working on large models.
The article [Towards Efficient Multi-GPU Training in Keras with TensorFlow](https://medium.com/rossum/towards-efficient-multi-gpu-training-in-keras-with-tensorflow-8a0091074fb2)
has mentioned some of it.
Tensorpack and [horovod](https://github.com/uber/horovod/blob/master/examples/keras_imagenet_resnet50.py)
are the only two tools I know that can scale the training of a large Keras model.
### Simple Examples: ### Simple Examples:
...@@ -21,7 +27,3 @@ It has: ...@@ -21,7 +27,3 @@ It has:
+ With 8 V100s, still has >90% GPU utilization and finished 100 epochs in 19.5 hours + With 8 V100s, still has >90% GPU utilization and finished 100 epochs in 19.5 hours
+ Good accuracy (same as [tensorpack ResNet example](../ResNet)) + Good accuracy (same as [tensorpack ResNet example](../ResNet))
Keras alone is not efficient enough to work on large models like this.
In addition to tensorpack, [horovod](https://github.com/uber/horovod/blob/master/examples/keras_imagenet_resnet50.py)
can also help you to train large models with Keras.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment