Installing Software - Revision history

Rdickson: /* 3.1 Using IBM-AI Deep Learning Anaconda Channel */ URL change on advice from CWhidden

2022-03-08T15:39:39Z

‎3.1 Using IBM-AI Deep Learning Anaconda Channel: URL change on advice from CWhidden

Bgeetika at 12:43, 6 April 2021

2021-04-06T12:43:19Z

Bgeetika at 15:41, 22 December 2020

2020-12-22T15:41:38Z

Bgeetika at 15:38, 22 December 2020

2020-12-22T15:38:50Z

Lyang: /* 3. Installation of Deep Learning packages */

2020-12-22T13:54:10Z

‎3. Installation of Deep Learning packages

Lyang: /* 3.1 Using IBM-AI Deep Learning Anaconda Channel */

2020-12-22T13:51:19Z

‎3.1 Using IBM-AI Deep Learning Anaconda Channel

Bgeetika: /* 3.1 Using IBM-AI Deep Learning Anaconda Channel */

2020-12-04T13:14:32Z

‎3.1 Using IBM-AI Deep Learning Anaconda Channel

Bgeetika: /* 3.2 Install PyTorch 1.6.0 in a user's home directory on DeepSense */

2020-12-04T13:14:05Z

‎3.2 Install PyTorch 1.6.0 in a user's home directory on DeepSense

Bgeetika: /* Systemwide python (managed by DeepSense) */

2020-12-04T13:10:04Z

‎Systemwide python (managed by DeepSense)

Bgeetika: Bgeetika moved page Installing ML/DL Frameworks to Installing Software

2020-12-02T15:26:09Z

Bgeetika moved page Installing ML/DL Frameworks to Installing Software

@@ Line 101: / Line 101: @@
   conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
-  conda config --prepend channels https://ftp.osuosl.org/pub/open-ce/current/linux-ppc64le/
+  conda config --prepend channels https://ftp.osuosl.org/pub/open-ce/1.2.2/
 We suggest creating a new environment for each deep learning package you want to use. For example for Tensorflow:

@@ Line 98: / Line 98: @@
 ===3.1 Using IBM-AI Deep Learning Anaconda Channel===
-To use deep learning packages like Tensorflow on DeepSense you need to add the IBM-AI anaconda channel to your list of available software channels.
+To use deep learning packages like Tensorflow or Pytorch on DeepSense you need to add the anaconda channel to your list of available software channels. You can add either from the following two channels depends upon the version you want. We suggest you to first check the version under these channels and then add.
   conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
 We suggest creating a new environment for each deep learning package you want to use. For example for Tensorflow:

@@ Line 161: / Line 161: @@
   >>>
-==4. Install Visualization Software==
+==4. Installation of Visualization Software==
 Users can easily install Dash in their home directory on DeepSense. The way to install it is very similar to installing a package using conda. Here we provide how to install and use Dash on DeepSense. The install process only takes about 10 minutes if you have already set up your conda environments in your home directory on DeepSense.
-=== Installing Dash on DeepSense ===
+===4.1 Installing Dash on DeepSense ===
 Activate the conda environment you would install Dash<br>
 Example:

@@ Line 161: / Line 161: @@
   >>>
-==4. Install other dependencies ==
+==4. Install Visualization Software==
 If you need additional python libraries then you can install them in your python environment.
@@ Line 177: / Line 195: @@
 A list of recommended packages follows in the next section.
-==5. Recommended packages ==
+==6. Recommended packages ==
-===5.1 Jupyter Notebooks for deep learning ===
+===6.1 Jupyter Notebooks for deep learning ===
   conda install jupyter
-===5.2 (Old Method) Testing Deep Learning packages on the login nodes or non-GPU nodes ===
+===6.2 (Old Method) Testing Deep Learning packages on the login nodes or non-GPU nodes ===
 You may wish to run PowerAI software on the login nodes for testing on the CPU-only nodes for some workflows.
@@ Line 199: / Line 217: @@
 Keep in mind you need to activate the GPU drivers and deep learning package in each browser shell before you are able to use the package in your code or LSF jobs.
-==6. Compiling Software for DeepSense ==
+==7. Compiling Software for DeepSense ==
 DeepSense uses IBM Power8 systems running RedHat Enterprise Linux. Code must be compiled for <code>ppc64le</code> which is PowerPC 64 bit Little Endian.
@@ Line 209: / Line 227: @@
 For example, one DeepSense user attempted to compile the rdkit software package from https://www.rdkit.org/ . This compilation failed when it attempted to use the gcc x86 optimization <code>-mpopcnt</code>. After replacing the optimization with the <code>ppc64le</code> equivalent <code>-mpopcntb</code> the software compiled successfully.
-== 7. Technical and research support ==
+== 8. Technical and research support ==
 DeepSense has a dedicated support team of research scientists ready to help you with technical questions, installing software, or even research questions.

@@ Line 110: / Line 110: @@
 You can then use tensorflow or other deep learning packages as needed by simply activating that anaconda environment. Unlike the old method, you do not need to specifically activate tensorflow or other deep learning methods.
-The above channel is still working, but the supported software version will not be updated any more. For example, you can only install TensorFlow 2.1.2 and PyTorch 1.3.1 using this channel.You can directly visit the IBM-AI anaconda channel URL to see a list of available software (https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/). If you would like to install higher versions of software, you can refer to the next section.
+The above channel is still working, but the supported software version will not be updated any more. For example, you can only install TensorFlow 2.1.2 and PyTorch 1.3.1 using this channel. You can directly visit the IBM-AI anaconda channel URL to see a list of available software and their versions (https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/). If you would like to install higher versions of software, you can refer to the next section.
 ===3.2 Install PyTorch 1.6.0 in a user's home directory on DeepSense ===
@@ Line 160: / Line 160: @@
   >>> import cv2
   >>>
-−
 ==4. Install other dependencies ==

@@ Line 119: / Line 119: @@
 <b>1. Source the conda environment you would like to use. For example:</b>
-	<code>source ~/anaconda3/etc/profile.d/conda.sh</code></br>
+ source ~/anaconda3/etc/profile.d/conda.sh</br>
 <b>2. Activate the environment you would use to install PyTorch. If the environment hasn't been created, a user can create it and install PyTorch in one command line. For example, if you would create an environment with name "my-environment" (This is just an example. Please choose a meaningful name for yourself.) and install PyTorch, you would run the following command:</b>
-	<code>conda create -y -n my-environment python=3.6 pytorch -c file:////software/PyTorch-1.6.0-Build/condabuild</code></br>
+ conda create -y -n my-environment python=3.6 pytorch -c file:////software/PyTorch-1.6.0-Build/condabuild</br>
 <b>3. If the environment has been created, say the name of the environment is "my-environment", you would need to activate the environment first and then install PyTorch. For example:</b>
-	<code>conda activate my-environment</br>
+ conda activate my-environment</br>
-	conda install pytorch -c  file:////software/PyTorch-1.6.0-Build/condabuild</code></br>
+ conda install pytorch -c  file:////software/PyTorch-1.6.0-Build/condabuild</br>
 	This should take about 2 minutes to install.</br>
 <b>4. To test if your install is successful, issue python from the environment where PyTorch is installed. Then run "import torch" to see if there are any errors. For example:</b></br>
-<code>[luy@ds-lg-01 ~]$ conda activate my-environment</br>
+ [luy@ds-lg-01 ~]$ conda activate my-environment</br>
-(my-environment) [luy@ds-lg-01 ~]$ python</br>
+ (my-environment) [luy@ds-lg-01 ~]$ python</br>
-Python 3.6.12 |Anaconda, Inc.| (default, Sep  9 2020, 00:40:10) </br>
+ Python 3.6.12 |Anaconda, Inc.| (default, Sep  9 2020, 00:40:10) </br>
-[GCC 7.3.0] on linux</br>
+ [GCC 7.3.0] on linux</br>
-Type "help", "copyright", "credits" or "license" for more information.</br>
+ Type "help", "copyright", "credits" or "license" for more information.</br>
->>> import torch</br>
+ >>> import torch</br>
->>> </code></br>
+ >>> </br>
 ===3.3 Install Opencv 3.4.10 in a user's home directory on DeepSense ===

@@ Line 35: / Line 35: @@
 First one is anaconda2 installed in /opt/anaconda2 will provide you python 2.7.5. To use this systemwide python add a parameter to your .bashrc file in your home directory:
-<code>echo ". /opt/anaconda2/etc/profile.d/conda.sh" >> ~/.bashrc</code>
+ echo ". /opt/anaconda2/etc/profile.d/conda.sh" >> ~/.bashrc
 Second is anaconda3 installed in /software/WMLA/anaconda3 will provide you python 3.7.4. To use this systemwide python add a parameter to your .bashrc file in your home directory:
-<code>echo ". /software/WMLA/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc</code>
+ echo ". /software/WMLA/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc
 Then source your .bashrc file:
-<code>source ~/.bashrc</code>
+ source ~/.bashrc
 To load the python environment run <code>conda activate</code>

← Older revision	Revision as of 15:26, 2 December 2020
(No difference)