Difference between revisions of "Getting started with Deep Learning"

From DeepSense Docs
Jump to: navigation, search
(→‎3. Request an interactive session on a GPU compute node)
 
(10 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
<div class="noautonum">
 
<div class="noautonum">
 +
  
 
== 1. Get started with DeepSense ==
 
== 1. Get started with DeepSense ==
  
−
Follow all the steps from [[Getting started]]. This tutorial assumes you can log on to the DeepSense compute platform and have a version of Anaconda python on your path.
+
Follow all the steps from [[Getting started]]. This tutorial assumes you can log on to the DeepSense compute platform and have a version of Anaconda python on your path. We recommend installing Anaconda in your home directory before starting this tutorial (See [[Installing Software]]).
−
 
 
−
== 2. Download Caffe samples to your home directory ==
 
  
−
<code>/opt/DL/caffe/bin/caffe-install-samples</code>
+
== 2. Prepare Caffe and download Caffe samples to your home directory ==
  
−
== 3. Request an interactive session on a GPU compute node ==
+
(New method)
  
−
<!-- TODO: still need to set up queues to fairly share GPUs -->
+
Activate your anaconda environment. See [[Installing Software]] for how to create an environment. We will assume you have created one called "caffe".
−
<!-- TODO: write instructions doing this with regular LSF without an interactive session. We don't want to encourage everyone to use interactive sessions -->
+
conda activate caffe
−
<code>bsub -Is -gpu - bash</code>
+
Add the IBM-AI anaconda channel if you have not done so already
 +
conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
 +
Install caffe if you have not done so already
 +
conda install caffe
 +
Install the caffe samples
 +
caffe-install-samples
  
−
== 4. Start a python2 Jupyter notebook ==
+
(Old method)
  
−
=== Source the Caffe deep learning toolkit ===
+
<code>/opt/DL/caffe/bin/caffe-install-samples</code>
−
 
 
−
<code>source /opt/DL/caffe/bin/caffe-activate</code>
 
−
 
 
−
=== Start the notebook ===
 
−
 
 
−
<code>jupyter notebook --no-browser --ip=0.0.0.0</code>
 
−
 
 
−
=== Sample output ===
 
−
<pre>[I 13:32:23.937 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
 
−
[C 13:32:23.937 NotebookApp]
 
−
   
 
−
    Copy/paste this URL into your browser when you connect for the first time,
 
−
    to login with a token:
 
−
        http://ds-cmgpu-04:8888/?token=68042f40a10b500f3747ae0a232ee209fa4bf1aa384d29ba&token=68042f40a10b500f3747ae0a232ee209fa4bf1aa384d29ba
 
−
</pre>
 
−
 
 
−
=== Copy the URL, host, and port ===
 
−
 
 
−
Copy the URL but don’t paste it in your browser yet.
 
−
 
 
−
Make a note of which compute host and port the notebook is running on (e.g. host ds-cmgpu-04 and port 8888 in this case)
 
−
 
 
−
== 5. Port Forwarding ==
 
−
 
 
−
In a separate terminal window from your local computer, forward your local port to the remote host:
 
−
 
 
−
<code> ssh -l <username> login1.deepsense.ca -L <port>:<remote_host>:<port></code>
 
−
 
 
−
for example, <code>ssh -l user1 login1.deepsense.ca -L 8888:ds-cmgpu-04:8888</code>
 
  
  
−
Enter the copied URL in your web browser but change the remote host name to “localhost” before pressing enter.
+
== 3. More information ==
−
 
 
−
e.g <code>http://localhost:8888/?token=68042f40a10b500f3747ae0a232ee209fa4bf1aa384d29ba&token=68042f40a10b500f3747ae0a232ee209fa4bf1aa384d29ba</code>
 
−
 
 
−
== 6. Open the desired sample notebook ==
 
−
 
 
−
Be sure to enter the location of the “caffe-samples” directory in your home directory as your caffe-root in the Caffe example notebooks.
 
−
 
 
−
== 7. Enjoy Deep Learning on DeepSense! ==
 
−
 
 
−
== 8. More information ==
 
  
 
Go to Caffe's [http://caffe.berkeleyvision.org/ website] for tutorials and example programs that you can run to get started.
 
Go to Caffe's [http://caffe.berkeleyvision.org/ website] for tutorials and example programs that you can run to get started.
Line 68: Line 33:
 
[http://caffe.berkeleyvision.org/gathered/examples/cifar10.html CIFAR-10 tutorial] - Train a convolutional neural network to classify small images.
 
[http://caffe.berkeleyvision.org/gathered/examples/cifar10.html CIFAR-10 tutorial] - Train a convolutional neural network to classify small images.
  
−
== 9. Using another deep learning toolkit such as Tensorflow ==
+
== 4. Using another deep learning toolkit such as Tensorflow ==
 +
 
  
 +
(New Method)
 +
* Ensure any Anaconda dependencies are installed
 +
** for tensorflow, create a new environment and <code>conda install tensorflow</conda>
 +
* Download example notebooks for the deep learning toolkit to your home directory,
 +
** e.g. <code> git clone https://github.com/aymericdamien/TensorFlow-Examples.git</code>
 +
 +
(Old Method)
 
* Ensure any Anaconda dependencies are installed
 
* Ensure any Anaconda dependencies are installed
 
** for tensorflow, run <code>/opt/DL/tensorflow/bin/install_dependencies</code>
 
** for tensorflow, run <code>/opt/DL/tensorflow/bin/install_dependencies</code>
Line 84: Line 57:
  
 
   https://github.com/aymericdamien/TensorFlow-Examples
 
   https://github.com/aymericdamien/TensorFlow-Examples
−
 
−
 
−
 
−
 
−
 
−
 
  
 
</div> <!-- autonum -->
 
</div> <!-- autonum -->

Latest revision as of 14:26, 3 December 2020


1. Get started with DeepSense

Follow all the steps from Getting started. This tutorial assumes you can log on to the DeepSense compute platform and have a version of Anaconda python on your path. We recommend installing Anaconda in your home directory before starting this tutorial (See Installing Software).

2. Prepare Caffe and download Caffe samples to your home directory

(New method)

Activate your anaconda environment. See Installing Software for how to create an environment. We will assume you have created one called "caffe". 

conda activate caffe

Add the IBM-AI anaconda channel if you have not done so already 

conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/

Install caffe if you have not done so already 

conda install caffe

Install the caffe samples 

caffe-install-samples

(Old method)

/opt/DL/caffe/bin/caffe-install-samples


3. More information

Go to Caffe's website for tutorials and example programs that you can run to get started. See the following links to a couple of the example programs:

LeNet MNIST Tutorial - Train a neural network to understand handwritten digits.

CIFAR-10 tutorial - Train a convolutional neural network to classify small images.

4. Using another deep learning toolkit such as Tensorflow

(New Method)

(Old Method)

  • Ensure any Anaconda dependencies are installed
    • for tensorflow, run /opt/DL/tensorflow/bin/install_dependencies
  • Source the appropriate toolkit instead of caffe-activate
    • e.g. source /opt/DL/tensorflow/bin/tensorflow-activate
  • Download example notebooks for the deep learning toolkit to your home directory,

The TensorFlow home page has various information, including Tutorials, How-To documents, and a Getting Started guide.

Additional tutorials and examples are available from the community, for example: 

 https://github.com/nlintz/TensorFlow-Tutorials

 https://github.com/aymericdamien/TensorFlow-Examples

Retrieved from "https://docs.deepsense.ca/index.php?title=Getting_started_with_Deep_Learning&oldid=327"