Difference between revisions of "Visualization"

From DeepSense Docs
Jump to: navigation, search
(Created page with "In this section, we would not introduce those popular plotting libraries like Matplotlib, Pandas, Seaborn, ggplot, Plotly, and so on. We would try to introduce and provide inf...")
 
(Using Dash on DeepSense)
 
(4 intermediate revisions by the same user not shown)
Line 13: Line 13:
 
Users can easily install Dash in their home directory on DeepSense. The way to install it is very similar to installing a package using conda. Here we provide how to install and use Dash on DeepSense. The install process only takes about 10 minutes if you have already set up your conda environments in your home directory on DeepSense.
 
Users can easily install Dash in their home directory on DeepSense. The way to install it is very similar to installing a package using conda. Here we provide how to install and use Dash on DeepSense. The install process only takes about 10 minutes if you have already set up your conda environments in your home directory on DeepSense.
  
=== Installing Dash ===
+
=== Installing Dash on DeepSense ===
Activate the conda environment you would install Dash
+
Activate the conda environment you would install Dash<br>
 
Example:
 
Example:
 
  [luy@ds-lg-01 ~]$ conda activate py36-pytorch
 
  [luy@ds-lg-01 ~]$ conda activate py36-pytorch
Line 28: Line 28:
  
 
=== Learning Dash ===
 
=== Learning Dash ===
Users can learn the features of Dash on its [https://dash.plotly.com/basic-callbacks| official webpage]. There are also a lot of Dash tutorials on youtube. You would be able to see how powerful Dash can do for visualizing your data.  
+
Users can learn Dash on its [https://dash.plotly.com/basic-callbacks| official webpage]. There are also a lot of Dash tutorials on youtube. You would be able to see how powerful Dash can do for visualizing your data.
  
 
=== Using Dash on DeepSense ===
 
=== Using Dash on DeepSense ===
 
1. Creating a sample Python script using Dash<br>
 
1. Creating a sample Python script using Dash<br>
Now you can develop your projects using Dash. Below is simple example script showing how Dash works on DeepSense platform. Users can copy and paste the contents of the script and create a Python script in your home directory on DeepSense. Let's name it 'dashTest.py'.
+
Now you can develop your projects using Dash. Below is a simple example script showing how Dash works on DeepSense platform. Users can copy and paste the contents of the script and create a Python script in your home directory on DeepSense. Let's name it 'dashTest.py'.
 
  import torch
 
  import torch
 
  import torchvision
 
  import torchvision
Line 53: Line 53:
  
 
2. Getting an interactive session via LSF <br>
 
2. Getting an interactive session via LSF <br>
Running the following command to submit a new LSF interactive job:
+
If you will be using cpus for your jobs, please run the following command to submit a new LSF interactive job:
 +
(py36-pytorch) [luy@ds-lg-01 ~]$ bsub -Is bash
 +
Above is just an example command. We would use the command for gpu jobs to show more details of the command.
 +
If you will be using gpus for your jobs, please run the following command to submit a new LSF interactive job:
 
  (py36-pytorch) [luy@ds-lg-01 ~]$ bsub -Is -gpu - bash
 
  (py36-pytorch) [luy@ds-lg-01 ~]$ bsub -Is -gpu - bash
 
  Job <6008> is submitted to queue <gpu>.
 
  Job <6008> is submitted to queue <gpu>.
Line 69: Line 72:
 
  * Running on http://0.0.0.0:8050/ (Press CTRL+C to quit)
 
  * Running on http://0.0.0.0:8050/ (Press CTRL+C to quit)
  
3. See the result on a webpage<br.
+
3. See the output on a webpage<br>
 
Keep the session open. This means you should not press CTRL+C. You would need to do port forwarding to open the webpage on the web browser on your local machine. This is very similar to how to use notebook on DeepSense. Below is an example: <br>
 
Keep the session open. This means you should not press CTRL+C. You would need to do port forwarding to open the webpage on the web browser on your local machine. This is very similar to how to use notebook on DeepSense. Below is an example: <br>
 
  Lus-MacBook-Pro:~ lyang$ ssh -l luy login1.deepsense.ca -L 8050:ds-cmgpu-01:8050
 
  Lus-MacBook-Pro:~ lyang$ ssh -l luy login1.deepsense.ca -L 8050:ds-cmgpu-01:8050
Line 81: Line 84:
 
  **********************************************************************
 
  **********************************************************************
 
  [luy@ds-lg-01 ~]$  
 
  [luy@ds-lg-01 ~]$  
Be careful that we obtain the compute host name 'ds-cmgpu-01' and port number '8050' from step 2. The host name and port number may change when your submit your jobs. After the port forwarding is finished, you can copy the url 'http://0.0.0.0:8050/' found at the last line of step 2 and paste it to your local web browser. The above example script would output the following contents on a HTML web page:<br.
+
Be careful that we obtain the compute host name 'ds-cmgpu-01' and port number '8050' from step 2. The host name and port number may change when you submit your jobs. After the port forwarding is finished, you can copy the url 'http://0.0.0.0:8050/' found at the last line of step 2 and paste it to your local web browser. The above example script would output the following contents on a HTML web page:<br.
 
  Hello world!
 
  Hello world!
  

Latest revision as of 16:57, 18 December 2020

In this section, we would not introduce those popular plotting libraries like Matplotlib, Pandas, Seaborn, ggplot, Plotly, and so on. We would try to introduce and provide information about some visualization frameworks and libraries that have more features. Using the features in these frameworks and libraries, you may make your data more understandable and interactive.

Dash

Introduction

We would make use of the introduction of Dash directly from Dash's official webpage. Users can obtain more details about Dash accessing the official webpage.
"Dash is a productive Python framework for building web analytic applications.
Written on top of Flask, Plotly.js, and React.js, Dash is ideal for building data visualization apps with highly custom user interfaces in pure Python. It's particularly suited for anyone who works with data in Python.
Through a couple of simple patterns, Dash abstracts away all of the technologies and protocols that are required to build an interactive web-based application. Dash is simple enough that you can bind a user interface around your Python code in an afternoon.
Dash apps are rendered in the web browser. You can deploy your apps to servers and then share them through URLs. Since Dash apps are viewed in the web browser, Dash is inherently cross-platform and mobile ready.
There is a lot behind the framework. To learn more about how it is built and what motivated Dash, watch our talk from Plotcon below or read our announcement letter.
Dash is an open source library, released under the permissive MIT license. Plotly develops Dash and offers a platform for managing Dash apps in an enterprise environment.
"

Users can easily install Dash in their home directory on DeepSense. The way to install it is very similar to installing a package using conda. Here we provide how to install and use Dash on DeepSense. The install process only takes about 10 minutes if you have already set up your conda environments in your home directory on DeepSense.

Installing Dash on DeepSense

Activate the conda environment you would install Dash
Example:

[luy@ds-lg-01 ~]$ conda activate py36-pytorch

Running the following command to install Dash:

(py36-pytorch) [luy@ds-lg-01 ~]$ conda install -c conda-forge dash

After the installation is done, issue the following command to check if Dash is installed:

(py36-pytorch) [luy@ds-lg-01 ~]$ conda list |grep dash
dash                      1.18.1             pyhd8ed1ab_0    conda-forge
dash-core-components      1.14.1             pyhd8ed1ab_0    conda-forge
dash-html-components      1.1.1              pyh9f0ad1d_0    conda-forge
dash-renderer             1.8.3              pyhd8ed1ab_0    conda-forge
dash-table                4.11.1             pyhd8ed1ab_0    conda-forge

Learning Dash

Users can learn Dash on its official webpage. There are also a lot of Dash tutorials on youtube. You would be able to see how powerful Dash can do for visualizing your data.

Using Dash on DeepSense

1. Creating a sample Python script using Dash
Now you can develop your projects using Dash. Below is a simple example script showing how Dash works on DeepSense platform. Users can copy and paste the contents of the script and create a Python script in your home directory on DeepSense. Let's name it 'dashTest.py'.

import torch
import torchvision
from platform import python_version
import dash
import dash_html_components as html
import subprocess
app = dash.Dash(__name__)
gpu_detail=str(subprocess.run(['nvidia-smi'], stdout=subprocess.PIPE).stdout.decode('utf-8'))
gpu_name=' '.join(gpu_detail.split('\n')[7].split()[2:4])
content=[
   html.H6(children='Hello world!'),
   html.H6(children="Python Version: " + str(python_version())),
   html.H6(children="PyTorch Version: " + str(torch.__version__)),
   html.H6(children="Torchvision Version: " + str(torchvision.__version__)),
   html.H6(children="GPU name: " + gpu_name)]
app.layout = html.Div(children=content)
if __name__ == '__main__':
   app.run_server(host='0.0.0.0')

2. Getting an interactive session via LSF
If you will be using cpus for your jobs, please run the following command to submit a new LSF interactive job:

(py36-pytorch) [luy@ds-lg-01 ~]$ bsub -Is bash

Above is just an example command. We would use the command for gpu jobs to show more details of the command. If you will be using gpus for your jobs, please run the following command to submit a new LSF interactive job:

(py36-pytorch) [luy@ds-lg-01 ~]$ bsub -Is -gpu - bash
Job <6008> is submitted to queue <gpu>.
<<Waiting for dispatch ...>>
<<Starting on ds-cmgpu-01>>
[luy@ds-cmgpu-01 ~]$

Now you have an interactive session opened on the compute host ds-cmgpu-01. You can run the sample script on the command line:

[luy@ds-cmgpu-01 ~]$ python dashTest.py 
Dash is running on http://0.0.0.0:8050/
* Serving Flask app "dashTest" (lazy loading)
* Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
* Debug mode: off
* Running on http://0.0.0.0:8050/ (Press CTRL+C to quit)

3. See the output on a webpage
Keep the session open. This means you should not press CTRL+C. You would need to do port forwarding to open the webpage on the web browser on your local machine. This is very similar to how to use notebook on DeepSense. Below is an example:

Lus-MacBook-Pro:~ lyang$ ssh -l luy login1.deepsense.ca -L 8050:ds-cmgpu-01:8050
luy@login1.deepsense.ca's password: 
Last login: Fri Dec 18 08:41:15 2020 from vpn.deepsense.cs.dal.ca
**********************************************************************
IMPORTANT: Dear DeepSense users, jobs running on the head nodes (login 
node 1 and login node 2) must be less than 10 minutes and have small 
memory and CPU requirements. Jobs violating this will be terminated 
without warning.
**********************************************************************
[luy@ds-lg-01 ~]$ 

Be careful that we obtain the compute host name 'ds-cmgpu-01' and port number '8050' from step 2. The host name and port number may change when you submit your jobs. After the port forwarding is finished, you can copy the url 'http://0.0.0.0:8050/' found at the last line of step 2 and paste it to your local web browser. The above example script would output the following contents on a HTML web page:<br.

Hello world!
Python Version: 3.6.12
PyTorch Version: 1.6.0a0+445c276
Torchvision Version: 0.7.0a0
GPU name: Tesla P100-SXM2...

Please feel free to contact DeepSense at support@deepsense.ca for any questions.