Center for Cognitive Neuroscience - User contributions [en]

Hoffman2:Accessing the Cluster

2016-04-18T21:45:10Z

Acho: /* NX Client - GUI */

[[Hoffman2|Back to all things Hoffman2]]

Here are some of our favorite ways to access the Hoffman2 Cluster login nodes.

==SSH - Command Line==
SSH stands for ''Secure Shell'' and is a method of remotely logging into a computer using an encrypted connection. It is a command line tool and is available on most *nix-based operating systems with ports available for Windows.

===Mac/Linux/Unix===
====Simple SSH====
Use the ssh command from a terminal:
ssh login_id@hoffman2.idre.ucla.edu
where login_id is replaced by your cluster user name.

====GUI-Enabled SSH [Recommended]====
Macs (post - Snow Leopard 10.6.x) no longer come with a X Window System Server pre-installed.

'''Before doing the following steps, please install [http://xquartz.macosforge.org/ XQuartz] and restart your computer.'''
For more information about XQuartz, read [http://support.apple.com/kb/ht5293 here].
***WARNING*** MacOSX10 Yosemite needs to add "export Display=:0.0" to the local user Profile in order for it to work.
# Open up X11/XQuartz or Terminal. Both are under ''Applications > Utilities'' on Macs.
# Type the command
#: <pre>$ ssh -X [USERNAME]@hoffman2.idre.ucla.edu</pre>
#: filling in your Hoffman2 username.
#: The <code>-X</code> is for X11 Forwarding so that any graphics that are rendered on Hoffman2 get forwarded to the screen of your computer.
# Press enter and type in your password when it asks for it. No characters or asterisks will show up while you type.
# Provided your typing was good, you will be greeted by the Hoffman2 login message and have successfully SSH into a login node.

===Windows===
# Go [http://www.hoffman2.idre.ucla.edu/access/login/ here] and follow the instructions under ''Windows''. We recommend [http://www.hoffman2.idre.ucla.edu/access/putty/ PuTTY] or Cgywin.
(If you use putty, please install [ xming http://sourceforge.net/projects/xming/ ] for GUI access.
# Once you have that setup, the process is the same as if you were on a Mac or Linux/Unix machine

==NX Client - GUI==
TODO: FIX NXClient Instructions

: ''The official description of how to do this is found [http://hpc.ucla.edu/hoffman2/access/nx.php here]''
The NX Client program allows you to set up a Virtual Network Computing (VNC)-like session with Hoffman2. This session will keep running even if your Internet connection drops in and out (much like [[Using Screen|screen]] on the command line).

===Mac OS X 10.7+ / Windows / Linux===
==== What You Need====
# Go to the No Machine website ([https://www.nomachine.com/download No Machine]) and download/install the No Machine for Mac OS X/Windows/Linux.
# Hoffman2 NX Client Public Key
#* To get the NX Client Public Key, follow the steps below or email support@ccn.ucla.edu
#** (OSX/Linux) Open up a Terminal and run the following command (replacing USERNAME with your Hoffman2 username)
#**:<code>$ scp USERNAME@hoffman2.idre.ucla.edu:/etc/nxserver/client.id_dsa.key ~/Documents/</code>
#** (Windows) Use a sftp program and download the file /etc/nxserver/client.id_dsa.key on Hoffman2 (hoffman2.idre.ucla.edu)

====Setup====
# Open up No Machine in Application/Desktop and Click Continue.
# A window titled "New Connection" will appear. Fill out the fields accordingly
#* Protocol -- SSH
#* Host -- "hoffman2.idre.ucla.edu"
#* Port -- 22

#* Select "Use the NoMachine login"
#* Select Alternate Server Key and (...) - and find the file (client.id_dsa.key) you downloaded earlier (in your Documents folder).
#* Don't use a proxy
#* Name -- Something like "Hoffman2"

# Double click on the connection you just created (it should be the only one in the list).
# Enter your Hoffman2 username and password and click "OK" (You may also check the box labeled "Save this setting in the configuration file" to avoid retyping this in the future)
# Select "Create a new session". or "New Virtual Desktop"
# In the next menu, select Create new '''GNOME''' virtual desktop.
# A virtual desktop should appear!

Reconnections in this client are not currently supported for Hoffman2, so please make sure to logout and close your connections properly. [http://hpc.ucla.edu/hoffman2/access/nx.php#logout]

====Troubleshooting====
If your NX Client session freezes and you are unable to close it properly, open ''NX Session Administrator'' and disconnect your session from there. This freezing often occurs when your Internet connection is lost abruptly. Another possible cause for freezing is scrolling on certain Windows touchpads.

For more Information [[http://hpc.ucla.edu/hoffman2/access/nx.php Hoffman2 NX Client]]

If you are unable to open Firefox ("Firefox is already running, but is not responding. To open a new window, you must first close the existing Firefox process, or restart your system."), deleting ~/.mozilla might fix the problem. ''Be warned:'' this will erase your profile, including bookmarks, history, saved passwords, etc! For instructions on backing up and restoring profile information, see [https://support.mozilla.org/en-US/kb/back-and-restore-information-firefox-profiles Mozilla Support]. Make sure to perform these actions within No Machine, and not on your local system.

== Change Passwords ==
Once you've logged on and made sure its works, you can change your password to something more rememberable
To change passwords, logon and type:
passwd
It should ask you for your old password and then new ones.

==External Links==
*[http://www.hoffman2.idre.ucla.edu/access/ Hoffman2 Access]
*[[Hoffman2:Accessing_the_Cluster-Historical_Notes | Historical Notes]]

Hoffman2:Accessing the Cluster

2016-04-18T21:44:43Z

Acho: /* External Links */

[[Hoffman2|Back to all things Hoffman2]]

Here are some of our favorite ways to access the Hoffman2 Cluster login nodes.

==SSH - Command Line==
SSH stands for ''Secure Shell'' and is a method of remotely logging into a computer using an encrypted connection. It is a command line tool and is available on most *nix-based operating systems with ports available for Windows.

===Mac/Linux/Unix===
====Simple SSH====
Use the ssh command from a terminal:
ssh login_id@hoffman2.idre.ucla.edu
where login_id is replaced by your cluster user name.

====GUI-Enabled SSH [Recommended]====
Macs (post - Snow Leopard 10.6.x) no longer come with a X Window System Server pre-installed.

'''Before doing the following steps, please install [http://xquartz.macosforge.org/ XQuartz] and restart your computer.'''
For more information about XQuartz, read [http://support.apple.com/kb/ht5293 here].
***WARNING*** MacOSX10 Yosemite needs to add "export Display=:0.0" to the local user Profile in order for it to work.
# Open up X11/XQuartz or Terminal. Both are under ''Applications > Utilities'' on Macs.
# Type the command
#: <pre>$ ssh -X [USERNAME]@hoffman2.idre.ucla.edu</pre>
#: filling in your Hoffman2 username.
#: The <code>-X</code> is for X11 Forwarding so that any graphics that are rendered on Hoffman2 get forwarded to the screen of your computer.
# Press enter and type in your password when it asks for it. No characters or asterisks will show up while you type.
# Provided your typing was good, you will be greeted by the Hoffman2 login message and have successfully SSH into a login node.

===Windows===
# Go [http://www.hoffman2.idre.ucla.edu/access/login/ here] and follow the instructions under ''Windows''. We recommend [http://www.hoffman2.idre.ucla.edu/access/putty/ PuTTY] or Cgywin.
(If you use putty, please install [ xming http://sourceforge.net/projects/xming/ ] for GUI access.
# Once you have that setup, the process is the same as if you were on a Mac or Linux/Unix machine

==NX Client - GUI==
: ''The official description of how to do this is found [http://hpc.ucla.edu/hoffman2/access/nx.php here]''
The NX Client program allows you to set up a Virtual Network Computing (VNC)-like session with Hoffman2. This session will keep running even if your Internet connection drops in and out (much like [[Using Screen|screen]] on the command line).

===Mac OS X 10.7+ / Windows / Linux===
==== What You Need====
# Go to the No Machine website ([https://www.nomachine.com/download No Machine]) and download/install the No Machine for Mac OS X/Windows/Linux.
# Hoffman2 NX Client Public Key
#* To get the NX Client Public Key, follow the steps below or email support@ccn.ucla.edu
#** (OSX/Linux) Open up a Terminal and run the following command (replacing USERNAME with your Hoffman2 username)
#**:<code>$ scp USERNAME@hoffman2.idre.ucla.edu:/etc/nxserver/client.id_dsa.key ~/Documents/</code>
#** (Windows) Use a sftp program and download the file /etc/nxserver/client.id_dsa.key on Hoffman2 (hoffman2.idre.ucla.edu)

====Setup====
# Open up No Machine in Application/Desktop and Click Continue.
# A window titled "New Connection" will appear. Fill out the fields accordingly
#* Protocol -- SSH
#* Host -- "hoffman2.idre.ucla.edu"
#* Port -- 22

#* Select "Use the NoMachine login"
#* Select Alternate Server Key and (...) - and find the file (client.id_dsa.key) you downloaded earlier (in your Documents folder).
#* Don't use a proxy
#* Name -- Something like "Hoffman2"

# Double click on the connection you just created (it should be the only one in the list).
# Enter your Hoffman2 username and password and click "OK" (You may also check the box labeled "Save this setting in the configuration file" to avoid retyping this in the future)
# Select "Create a new session". or "New Virtual Desktop"
# In the next menu, select Create new '''GNOME''' virtual desktop.
# A virtual desktop should appear!

Reconnections in this client are not currently supported for Hoffman2, so please make sure to logout and close your connections properly. [http://hpc.ucla.edu/hoffman2/access/nx.php#logout]

====Troubleshooting====
If your NX Client session freezes and you are unable to close it properly, open ''NX Session Administrator'' and disconnect your session from there. This freezing often occurs when your Internet connection is lost abruptly. Another possible cause for freezing is scrolling on certain Windows touchpads.

For more Information [[http://hpc.ucla.edu/hoffman2/access/nx.php Hoffman2 NX Client]]

If you are unable to open Firefox ("Firefox is already running, but is not responding. To open a new window, you must first close the existing Firefox process, or restart your system."), deleting ~/.mozilla might fix the problem. ''Be warned:'' this will erase your profile, including bookmarks, history, saved passwords, etc! For instructions on backing up and restoring profile information, see [https://support.mozilla.org/en-US/kb/back-and-restore-information-firefox-profiles Mozilla Support]. Make sure to perform these actions within No Machine, and not on your local system.

== Change Passwords ==
Once you've logged on and made sure its works, you can change your password to something more rememberable
To change passwords, logon and type:
passwd
It should ask you for your old password and then new ones.

==External Links==
*[http://www.hoffman2.idre.ucla.edu/access/ Hoffman2 Access]
*[[Hoffman2:Accessing_the_Cluster-Historical_Notes | Historical Notes]]

Hoffman2:Accessing the Cluster

2016-04-18T21:43:33Z

Acho: /* Windows */

[[Hoffman2|Back to all things Hoffman2]]

Here are some of our favorite ways to access the Hoffman2 Cluster login nodes.

==SSH - Command Line==
SSH stands for ''Secure Shell'' and is a method of remotely logging into a computer using an encrypted connection. It is a command line tool and is available on most *nix-based operating systems with ports available for Windows.

===Mac/Linux/Unix===
====Simple SSH====
Use the ssh command from a terminal:
ssh login_id@hoffman2.idre.ucla.edu
where login_id is replaced by your cluster user name.

====GUI-Enabled SSH [Recommended]====
Macs (post - Snow Leopard 10.6.x) no longer come with a X Window System Server pre-installed.

'''Before doing the following steps, please install [http://xquartz.macosforge.org/ XQuartz] and restart your computer.'''
For more information about XQuartz, read [http://support.apple.com/kb/ht5293 here].
***WARNING*** MacOSX10 Yosemite needs to add "export Display=:0.0" to the local user Profile in order for it to work.
# Open up X11/XQuartz or Terminal. Both are under ''Applications > Utilities'' on Macs.
# Type the command
#: <pre>$ ssh -X [USERNAME]@hoffman2.idre.ucla.edu</pre>
#: filling in your Hoffman2 username.
#: The <code>-X</code> is for X11 Forwarding so that any graphics that are rendered on Hoffman2 get forwarded to the screen of your computer.
# Press enter and type in your password when it asks for it. No characters or asterisks will show up while you type.
# Provided your typing was good, you will be greeted by the Hoffman2 login message and have successfully SSH into a login node.

===Windows===
# Go [http://www.hoffman2.idre.ucla.edu/access/login/ here] and follow the instructions under ''Windows''. We recommend [http://www.hoffman2.idre.ucla.edu/access/putty/ PuTTY] or Cgywin.
(If you use putty, please install [ xming http://sourceforge.net/projects/xming/ ] for GUI access.
# Once you have that setup, the process is the same as if you were on a Mac or Linux/Unix machine

==NX Client - GUI==
: ''The official description of how to do this is found [http://hpc.ucla.edu/hoffman2/access/nx.php here]''
The NX Client program allows you to set up a Virtual Network Computing (VNC)-like session with Hoffman2. This session will keep running even if your Internet connection drops in and out (much like [[Using Screen|screen]] on the command line).

===Mac OS X 10.7+ / Windows / Linux===
==== What You Need====
# Go to the No Machine website ([https://www.nomachine.com/download No Machine]) and download/install the No Machine for Mac OS X/Windows/Linux.
# Hoffman2 NX Client Public Key
#* To get the NX Client Public Key, follow the steps below or email support@ccn.ucla.edu
#** (OSX/Linux) Open up a Terminal and run the following command (replacing USERNAME with your Hoffman2 username)
#**:<code>$ scp USERNAME@hoffman2.idre.ucla.edu:/etc/nxserver/client.id_dsa.key ~/Documents/</code>
#** (Windows) Use a sftp program and download the file /etc/nxserver/client.id_dsa.key on Hoffman2 (hoffman2.idre.ucla.edu)

====Setup====
# Open up No Machine in Application/Desktop and Click Continue.
# A window titled "New Connection" will appear. Fill out the fields accordingly
#* Protocol -- SSH
#* Host -- "hoffman2.idre.ucla.edu"
#* Port -- 22

#* Select "Use the NoMachine login"
#* Select Alternate Server Key and (...) - and find the file (client.id_dsa.key) you downloaded earlier (in your Documents folder).
#* Don't use a proxy
#* Name -- Something like "Hoffman2"

# Double click on the connection you just created (it should be the only one in the list).
# Enter your Hoffman2 username and password and click "OK" (You may also check the box labeled "Save this setting in the configuration file" to avoid retyping this in the future)
# Select "Create a new session". or "New Virtual Desktop"
# In the next menu, select Create new '''GNOME''' virtual desktop.
# A virtual desktop should appear!

Reconnections in this client are not currently supported for Hoffman2, so please make sure to logout and close your connections properly. [http://hpc.ucla.edu/hoffman2/access/nx.php#logout]

====Troubleshooting====
If your NX Client session freezes and you are unable to close it properly, open ''NX Session Administrator'' and disconnect your session from there. This freezing often occurs when your Internet connection is lost abruptly. Another possible cause for freezing is scrolling on certain Windows touchpads.

For more Information [[http://hpc.ucla.edu/hoffman2/access/nx.php Hoffman2 NX Client]]

If you are unable to open Firefox ("Firefox is already running, but is not responding. To open a new window, you must first close the existing Firefox process, or restart your system."), deleting ~/.mozilla might fix the problem. ''Be warned:'' this will erase your profile, including bookmarks, history, saved passwords, etc! For instructions on backing up and restoring profile information, see [https://support.mozilla.org/en-US/kb/back-and-restore-information-firefox-profiles Mozilla Support]. Make sure to perform these actions within No Machine, and not on your local system.

== Change Passwords ==
Once you've logged on and made sure its works, you can change your password to something more rememberable
To change passwords, logon and type:
passwd
It should ask you for your old password and then new ones.

==External Links==
*[http://hpc.ucla.edu/hoffman2/access/access.php Hoffman2 Access]
*[http://hpc.ucla.edu/hoffman2/access/nx.php Hoffman2 via NX Client]
*[[Hoffman2:Accessing_the_Cluster-Historical_Notes | Historical Notes]]

Hoffman2:Accessing the Cluster

2016-04-18T21:43:18Z

Acho: /* Windows */

[[Hoffman2|Back to all things Hoffman2]]

Here are some of our favorite ways to access the Hoffman2 Cluster login nodes.

==SSH - Command Line==
SSH stands for ''Secure Shell'' and is a method of remotely logging into a computer using an encrypted connection. It is a command line tool and is available on most *nix-based operating systems with ports available for Windows.

===Mac/Linux/Unix===
====Simple SSH====
Use the ssh command from a terminal:
ssh login_id@hoffman2.idre.ucla.edu
where login_id is replaced by your cluster user name.

====GUI-Enabled SSH [Recommended]====
Macs (post - Snow Leopard 10.6.x) no longer come with a X Window System Server pre-installed.

'''Before doing the following steps, please install [http://xquartz.macosforge.org/ XQuartz] and restart your computer.'''
For more information about XQuartz, read [http://support.apple.com/kb/ht5293 here].
***WARNING*** MacOSX10 Yosemite needs to add "export Display=:0.0" to the local user Profile in order for it to work.
# Open up X11/XQuartz or Terminal. Both are under ''Applications > Utilities'' on Macs.
# Type the command
#: <pre>$ ssh -X [USERNAME]@hoffman2.idre.ucla.edu</pre>
#: filling in your Hoffman2 username.
#: The <code>-X</code> is for X11 Forwarding so that any graphics that are rendered on Hoffman2 get forwarded to the screen of your computer.
# Press enter and type in your password when it asks for it. No characters or asterisks will show up while you type.
# Provided your typing was good, you will be greeted by the Hoffman2 login message and have successfully SSH into a login node.

===Windows===
# Go [http://www.hoffman2.idre.ucla.edu/access/login/ here] and follow the instructions under ''Windows''. We recommend [http://www.hoffman2.idre.ucla.edu/access/putty/ PuTTY] or Cgywin.
(If you use putty, please install [ xming http://sourceforge.net/projects/xming/ ] for GUI access.
# Once you have that setup, the process is the same as if you were on a Mac or Linux/Unix machine
# TODO: [More information Coming]

==NX Client - GUI==
: ''The official description of how to do this is found [http://hpc.ucla.edu/hoffman2/access/nx.php here]''
The NX Client program allows you to set up a Virtual Network Computing (VNC)-like session with Hoffman2. This session will keep running even if your Internet connection drops in and out (much like [[Using Screen|screen]] on the command line).

===Mac OS X 10.7+ / Windows / Linux===
==== What You Need====
# Go to the No Machine website ([https://www.nomachine.com/download No Machine]) and download/install the No Machine for Mac OS X/Windows/Linux.
# Hoffman2 NX Client Public Key
#* To get the NX Client Public Key, follow the steps below or email support@ccn.ucla.edu
#** (OSX/Linux) Open up a Terminal and run the following command (replacing USERNAME with your Hoffman2 username)
#**:<code>$ scp USERNAME@hoffman2.idre.ucla.edu:/etc/nxserver/client.id_dsa.key ~/Documents/</code>
#** (Windows) Use a sftp program and download the file /etc/nxserver/client.id_dsa.key on Hoffman2 (hoffman2.idre.ucla.edu)

====Setup====
# Open up No Machine in Application/Desktop and Click Continue.
# A window titled "New Connection" will appear. Fill out the fields accordingly
#* Protocol -- SSH
#* Host -- "hoffman2.idre.ucla.edu"
#* Port -- 22

#* Select "Use the NoMachine login"
#* Select Alternate Server Key and (...) - and find the file (client.id_dsa.key) you downloaded earlier (in your Documents folder).
#* Don't use a proxy
#* Name -- Something like "Hoffman2"

# Double click on the connection you just created (it should be the only one in the list).
# Enter your Hoffman2 username and password and click "OK" (You may also check the box labeled "Save this setting in the configuration file" to avoid retyping this in the future)
# Select "Create a new session". or "New Virtual Desktop"
# In the next menu, select Create new '''GNOME''' virtual desktop.
# A virtual desktop should appear!

Reconnections in this client are not currently supported for Hoffman2, so please make sure to logout and close your connections properly. [http://hpc.ucla.edu/hoffman2/access/nx.php#logout]

====Troubleshooting====
If your NX Client session freezes and you are unable to close it properly, open ''NX Session Administrator'' and disconnect your session from there. This freezing often occurs when your Internet connection is lost abruptly. Another possible cause for freezing is scrolling on certain Windows touchpads.

For more Information [[http://hpc.ucla.edu/hoffman2/access/nx.php Hoffman2 NX Client]]

If you are unable to open Firefox ("Firefox is already running, but is not responding. To open a new window, you must first close the existing Firefox process, or restart your system."), deleting ~/.mozilla might fix the problem. ''Be warned:'' this will erase your profile, including bookmarks, history, saved passwords, etc! For instructions on backing up and restoring profile information, see [https://support.mozilla.org/en-US/kb/back-and-restore-information-firefox-profiles Mozilla Support]. Make sure to perform these actions within No Machine, and not on your local system.

== Change Passwords ==
Once you've logged on and made sure its works, you can change your password to something more rememberable
To change passwords, logon and type:
passwd
It should ask you for your old password and then new ones.

==External Links==
*[http://hpc.ucla.edu/hoffman2/access/access.php Hoffman2 Access]
*[http://hpc.ucla.edu/hoffman2/access/nx.php Hoffman2 via NX Client]
*[[Hoffman2:Accessing_the_Cluster-Historical_Notes | Historical Notes]]

Hoffman2:Getting an Account

2016-04-18T21:41:26Z

Acho: /* External Links */

[[Hoffman2|Back to all things Hoffman2]]

==Requesting Hoffman2 Account==
===What You Need===
A UCLA BOL account, available for free to any UCLA staff, student, or faculty member. If you do not have a BOL account, head to the [https://logon.ucla.edu UCLA Logon] services page. Click on "Create UCLA Logon ID".

===Applying for the Account===
ATTENTION: If you are a PI interested in Hoffman2, please see the section
[[#Becoming A Faculty Sponsor | Becoming a Faculty Sponsor]] below.
#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "New User Registration"
#Authenticate using your UCLA BOL credentials
#Fill out the form with appropriate information. For Hoffman2, your Faculty Sponsor should be Mark Cohen, Alison Burggren (for Susan Bookheimer's lab), or your respective PI if they are a Faculty Sponsor on Hoffman.
;Proposed UserName
:This will be the username you use to sign into the cluster with.
;Select a Resource
:For the Mark Cohen/Susan Bookheimer labs, choose "Hoffman2". However, you can request access to any cluster that is a member of the Grid Portal.

Click Submit.
You will receive an email with a link to a temporary password. '''PLEASE WRITE IT DOWN.''' The link expires after 72 hours. If you missed the link or it expired, go back to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Application Page] and click Forgot Your Cluster Password? It will take about a day for the cluster to resend you a new password.

You can change your password once you've logged in by using [[Hoffman2:Accessing_the_Cluster#Change_Passwords | passwd.]]

==Becoming A Faculty Sponsor==
If you are a PI or Lab Manager interested in the Hoffman2 Cluster, you will want to create a Faculty Sponsor account first. Also, if you are a member of another lab collaborating with the Cohen or Bookheimer labs, you may want to forward this information to your PI or Lab Manager. Faculty Sponsors can approve (or deny) applications for membership to their group. They also receive a group folder and a unique group id so their users can work and share data easily with each other.

#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "Request to become faculty sponsor" (On the Bottom)
#Fill out the form with appropriate information.

Under 'Reason', about any generic reason is appropriate for faculty members. For example, "To perform fMRI analysis." will likely suffice.

==External Links==
*[https://grid.ucla.edu:9443/gridsphere/gridsphere?cid=home&JavaScript=enabled UCLA Grid Portal]
*[https://logon.ucla.edu UCLA BOL Home Page]
*[http://www.hoffman2.idre.ucla.edu/getting-started/ Hoffman2 Account Page]
*[[Hoffman2:Getting_an_Account-Historical_Notes | Historical Notes]]

Hoffman2:Getting an Account

2016-04-18T21:40:57Z

Acho: /* Becoming A Faculty Sponsor */

[[Hoffman2|Back to all things Hoffman2]]

==Requesting Hoffman2 Account==
===What You Need===
A UCLA BOL account, available for free to any UCLA staff, student, or faculty member. If you do not have a BOL account, head to the [https://logon.ucla.edu UCLA Logon] services page. Click on "Create UCLA Logon ID".

===Applying for the Account===
ATTENTION: If you are a PI interested in Hoffman2, please see the section
[[#Becoming A Faculty Sponsor | Becoming a Faculty Sponsor]] below.
#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "New User Registration"
#Authenticate using your UCLA BOL credentials
#Fill out the form with appropriate information. For Hoffman2, your Faculty Sponsor should be Mark Cohen, Alison Burggren (for Susan Bookheimer's lab), or your respective PI if they are a Faculty Sponsor on Hoffman.
;Proposed UserName
:This will be the username you use to sign into the cluster with.
;Select a Resource
:For the Mark Cohen/Susan Bookheimer labs, choose "Hoffman2". However, you can request access to any cluster that is a member of the Grid Portal.

Click Submit.
You will receive an email with a link to a temporary password. '''PLEASE WRITE IT DOWN.''' The link expires after 72 hours. If you missed the link or it expired, go back to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Application Page] and click Forgot Your Cluster Password? It will take about a day for the cluster to resend you a new password.

You can change your password once you've logged in by using [[Hoffman2:Accessing_the_Cluster#Change_Passwords | passwd.]]

==Becoming A Faculty Sponsor==
If you are a PI or Lab Manager interested in the Hoffman2 Cluster, you will want to create a Faculty Sponsor account first. Also, if you are a member of another lab collaborating with the Cohen or Bookheimer labs, you may want to forward this information to your PI or Lab Manager. Faculty Sponsors can approve (or deny) applications for membership to their group. They also receive a group folder and a unique group id so their users can work and share data easily with each other.

#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "Request to become faculty sponsor" (On the Bottom)
#Fill out the form with appropriate information.

Under 'Reason', about any generic reason is appropriate for faculty members. For example, "To perform fMRI analysis." will likely suffice.

==External Links==
*[https://grid.ucla.edu:9443/gridsphere/gridsphere?cid=home&JavaScript=enabled UCLA Grid Portal]
*[https://logon.ucla.edu UCLA BOL Home Page]
*[http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Hoffman2 Account Page]
*[[Hoffman2:Getting_an_Account-Historical_Notes | Historical Notes]]

Hoffman2:Getting an Account

2016-04-18T21:40:03Z

Acho: /* Applying for the Account */

[[Hoffman2|Back to all things Hoffman2]]

==Requesting Hoffman2 Account==
===What You Need===
A UCLA BOL account, available for free to any UCLA staff, student, or faculty member. If you do not have a BOL account, head to the [https://logon.ucla.edu UCLA Logon] services page. Click on "Create UCLA Logon ID".

===Applying for the Account===
ATTENTION: If you are a PI interested in Hoffman2, please see the section
[[#Becoming A Faculty Sponsor | Becoming a Faculty Sponsor]] below.
#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "New User Registration"
#Authenticate using your UCLA BOL credentials
#Fill out the form with appropriate information. For Hoffman2, your Faculty Sponsor should be Mark Cohen, Alison Burggren (for Susan Bookheimer's lab), or your respective PI if they are a Faculty Sponsor on Hoffman.
;Proposed UserName
:This will be the username you use to sign into the cluster with.
;Select a Resource
:For the Mark Cohen/Susan Bookheimer labs, choose "Hoffman2". However, you can request access to any cluster that is a member of the Grid Portal.

Click Submit.
You will receive an email with a link to a temporary password. '''PLEASE WRITE IT DOWN.''' The link expires after 72 hours. If you missed the link or it expired, go back to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Application Page] and click Forgot Your Cluster Password? It will take about a day for the cluster to resend you a new password.

You can change your password once you've logged in by using [[Hoffman2:Accessing_the_Cluster#Change_Passwords | passwd.]]

==Becoming A Faculty Sponsor==
If you are a PI or Lab Manager interested in the Hoffman2 Cluster, you will want to create a Faculty Sponsor account first. Also, if you are a member of another lab collaborating with the Cohen or Bookheimer labs, you may want to forward this information to your PI or Lab Manager. Faculty Sponsors can approve (or deny) applications for membership to their group. They also receive a group folder and a unique group id so their users can work and share data easily with each other.

#Navigate to the [http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Account Applications] page.
#Read over the application summary
#Click "Request to become faculty sponsor" (On the Bottom)
#Fill out the form with appropriate information.

Under 'Reason', about any generic reason is appropriate for faculty members. For example, "To perform fMRI analysis." will likely suffice.

==External Links==
*[https://grid.ucla.edu:9443/gridsphere/gridsphere?cid=home&JavaScript=enabled UCLA Grid Portal]
*[https://logon.ucla.edu UCLA BOL Home Page]
*[http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Hoffman2 Account Page]
*[[Hoffman2:Getting_an_Account-Historical_Notes | Historical Notes]]

Hoffman2:Getting an Account

2016-04-18T21:39:34Z

Acho: /* Applying for the Account */

[[Hoffman2|Back to all things Hoffman2]]

==Requesting Hoffman2 Account==
===What You Need===
A UCLA BOL account, available for free to any UCLA staff, student, or faculty member. If you do not have a BOL account, head to the [https://logon.ucla.edu UCLA Logon] services page. Click on "Create UCLA Logon ID".

===Applying for the Account===
ATTENTION: If you are a member of another lab or are a PI interested, please see the section
[[#Becoming A Faculty Sponsor | Becoming a Faculty Sponsor]] below.
#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "New User Registration"
#Authenticate using your UCLA BOL credentials
#Fill out the form with appropriate information. For Hoffman2, your Faculty Sponsor should be Mark Cohen, Alison Burggren (for Susan Bookheimer's lab), or your respective PI if they are a Faculty Sponsor on Hoffman.
;Proposed UserName
:This will be the username you use to sign into the cluster with.
;Select a Resource
:For the Mark Cohen/Susan Bookheimer labs, choose "Hoffman2". However, you can request access to any cluster that is a member of the Grid Portal.

Click Submit.
You will receive an email with a link to a temporary password. '''PLEASE WRITE IT DOWN.''' The link expires after 72 hours. If you missed the link or it expired, go back to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Application Page] and click Forgot Your Cluster Password? It will take about a day for the cluster to resend you a new password.

You can change your password once you've logged in by using [[Hoffman2:Accessing_the_Cluster#Change_Passwords | passwd.]]

==Becoming A Faculty Sponsor==
If you are a PI or Lab Manager interested in the Hoffman2 Cluster, you will want to create a Faculty Sponsor account first. Also, if you are a member of another lab collaborating with the Cohen or Bookheimer labs, you may want to forward this information to your PI or Lab Manager. Faculty Sponsors can approve (or deny) applications for membership to their group. They also receive a group folder and a unique group id so their users can work and share data easily with each other.

#Navigate to the [http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Account Applications] page.
#Read over the application summary
#Click "Request to become faculty sponsor" (On the Bottom)
#Fill out the form with appropriate information.

Under 'Reason', about any generic reason is appropriate for faculty members. For example, "To perform fMRI analysis." will likely suffice.

==External Links==
*[https://grid.ucla.edu:9443/gridsphere/gridsphere?cid=home&JavaScript=enabled UCLA Grid Portal]
*[https://logon.ucla.edu UCLA BOL Home Page]
*[http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Hoffman2 Account Page]
*[[Hoffman2:Getting_an_Account-Historical_Notes | Historical Notes]]

Hoffman2:Introduction-Historical Notes

2016-04-18T21:38:05Z

Acho: /* Hoffman2 Storage */

[[Hoffman2:Introduction | Back to Hoffman2:Introduction]]
====Historical Notes====
===== Hoffman2 Storage =====
======April 2016======
Express Queue removed from Information (No longer in use by Hoffman)

======June 2013======
: ''Before July 2013, for users that were part of groups that purchased storage, their home directories were the same as their personal group directories. e.g.''
::<code> /u/home/j/jbruin</code>
:''did not exist, but''
::<code> /u/home/mscohen/jbruin</code>
:''did exist and was the home directory (and personal group directory) for the user jbruin. IDRE changed this behavior after the Summer Maintenance restart in 2013 to better separate users from their groups. This separation more cleanly allows users to be part of multiple storage groups (e.g. belonging to sbook and mscohen groups), or switch between single groups over time, while retaining their own personal space on the cluster. A symlink named '''project''' was placed in the new home directories pointing to the old home directories. e.g.''
::<code> /u/home/j/jbruin/project -> /u/home/mscohen/jbruin</code>

======June 2011======
: ''Before July 2011, there was a symlink pointing from /u/home9 to /u/home as a legacy support mechanism. This symlink was finally removed after the Summer Maintenance of 2011 and some adjustments had to be made by anyone still using home9 references.''

'''CHANGES MAKE ME GO LIKE THIS!'''
[[File:Hoffman2-Pro_Status.jpg|none|none|none|none|500px]]

Hoffman2:Introduction

2016-04-18T21:36:10Z

Acho: /* External Links / Notes */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://www.hoffman2.idre.ucla.edu/computing/gpuq/ here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://www.hoffman2.idre.ucla.edu/computing/policies/ up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://www.hoffman2.idre.ucla.edu/data-storage/ click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative backups.

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://www.hoffman2.idre.ucla.edu/data-storage/ here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://www.hoffman2.idre.ucla.edu/computing/ here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://www.hoffman2.idre.ucla.edu/ Hoffman2 Webpage]
*[http://www.hoffman2.idre.ucla.edu/usage-status/ Hoffman2 Statistics]
*[http://www.hoffman2.idre.ucla.edu/data-storage Hoffman2 Data Storage]
*[http://www.hoffman2.idre.ucla.edu/computing Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:34:04Z

Acho: /* Queues */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://www.hoffman2.idre.ucla.edu/computing/gpuq/ here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://www.hoffman2.idre.ucla.edu/computing/policies/ up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://www.hoffman2.idre.ucla.edu/data-storage/ click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative backups.

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://www.hoffman2.idre.ucla.edu/data-storage/ here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://www.hoffman2.idre.ucla.edu/computing/ here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:33:23Z

Acho: /* Queues */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://www.hoffman2.idre.ucla.edu/computing/gpuq/ here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://www.hoffman2.idre.ucla.edu/computing/policies/ up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://www.hoffman2.idre.ucla.edu/data-storage/ click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative backups.

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://www.hoffman2.idre.ucla.edu/data-storage/ here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:33:04Z

Acho: /* Temporary Storage */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://www.hoffman2.idre.ucla.edu/computing/gpuq/ here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://www.hoffman2.idre.ucla.edu/computing/policies/ up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://www.hoffman2.idre.ucla.edu/data-storage/ click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative backups.

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://www.hoffman2.idre.ucla.edu/data-storage/ here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:26:22Z

Acho: /* Storage Space */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://www.hoffman2.idre.ucla.edu/computing/gpuq/ here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://www.hoffman2.idre.ucla.edu/computing/policies/ up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://www.hoffman2.idre.ucla.edu/data-storage/ click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative backups.

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:16:52Z

Acho: /* Computing Nodes */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://www.hoffman2.idre.ucla.edu/computing/gpuq/ here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://www.hoffman2.idre.ucla.edu/computing/policies/ up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:15:58Z

Acho: /* Computing Nodes */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://hpc.ucla.edu/hoffman2/computing/gpuq.php here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://hpc.ucla.edu/hoffman2/computing/policies.php#highp up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:10:03Z

Acho: /* What is Hoffman2? */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/ here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Stats.] The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://hpc.ucla.edu/hoffman2/computing/gpuq.php here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://hpc.ucla.edu/hoffman2/computing/policies.php#highp up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:09:27Z

Acho: /* What is Hoffman2? */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://www.hoffman2.idre.ucla.edu/usage-status/ here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Stats.] The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://hpc.ucla.edu/hoffman2/computing/gpuq.php here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://hpc.ucla.edu/hoffman2/computing/policies.php#highp up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:09:00Z

Acho: /* What is Hoffman2? */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://www.hoffman2.idre.ucla.edu/]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Stats.] The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://hpc.ucla.edu/hoffman2/computing/gpuq.php here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://hpc.ucla.edu/hoffman2/computing/policies.php#highp up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2016-04-18T21:07:19Z

Acho: /* What is Hoffman2? */

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [https://idre.ucla.edu/hoffman2]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Stats.] The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://hpc.ucla.edu/hoffman2/computing/gpuq.php here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://hpc.ucla.edu/hoffman2/computing/policies.php#highp up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

FNIRS

2015-11-18T19:28:33Z

Acho: Blanked the page

Encryption Policy

2015-10-15T22:41:23Z

Acho: /* Macs */

== Encryption Policy ==
Outlined here:

• All faculty, fellows, residents, students, volunteers and staff
• All mobile devices – laptops; tablets, mobile phones
• All removable media -- external hard drives; USB flash drives
• Non-University-owned devices
• Devices issued by UCLA Health or DGSOM
• All devices WHETHER OR NOT they are used to access restricted information
• Any mobile device used for any University business

For more information, go [http://employee.uclahealth.org/device-security/ Here]

== Devices ==
=== Laptops ===
====Windows====
Recommend using '''Bitlocker Drive Encryption'''
(Built into Windows Vista and later+)

[http://www.pcworld.com/article/2308725/a-beginners-guide-to-bitlocker-windows-built-in-encryption-tool.html Instructions Here]. (It is very important to save the backup-code/key at the end!)

It is very important to save the backup-code/key at the end!

====Macs ====
Recommend using '''File Vault Encryption''' (Built into Mac OSX)

[https://support.apple.com/en-us/HT204837 Instructions Here] (It is very important to save the backup-code/key at the end!)

====Others====
Other encryption programs are suggested [http://employee.uclahealth.org/device-security-toolkit/ here].

===USB/External Device ===
''Highly recommend going to Encryption Fair or your local IT for encryption''

[http://www.tomsguide.com/faq/id-2318737/encrypt-usb-flash-drive.html Windows Instructions]

[http://www.theinstructional.com/guides/encrypt-an-external-disk-or-usb-stick-with-a-password Mac Instructions]

=== Mobile Device ===
Mobile phone need to have AirWatch installed, if eligible.

To check if you are eligible:

<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">If the user is non-exempt (e.g. hourly pay status/bi-weekly pay), then they are not eligible for Airwatch unless approved by HR.

If the user is exempt (paid monthly), then they are eligible and should go to the encryption fair starting on Monday for help on downloading and installing Airwatch. Users also MUST have an AD account. </pre>

Pin/Pattern Code Screen Lock is not enough (though highly recommended!)
Please Install AirWatch at the [http://employee.uclahealth.org/encryption-fairs/ Encryption Fairs]

== Encryption Fair ==
Fore more information and dates : [http://employee.uclahealth.org/encryption-fairs/ Go Here]/

Encryption Policy

2015-10-15T22:41:07Z

Acho: /* Windows */

== Encryption Policy ==
Outlined here:

• All faculty, fellows, residents, students, volunteers and staff
• All mobile devices – laptops; tablets, mobile phones
• All removable media -- external hard drives; USB flash drives
• Non-University-owned devices
• Devices issued by UCLA Health or DGSOM
• All devices WHETHER OR NOT they are used to access restricted information
• Any mobile device used for any University business

For more information, go [http://employee.uclahealth.org/device-security/ Here]

== Devices ==
=== Laptops ===
====Windows====
Recommend using '''Bitlocker Drive Encryption'''
(Built into Windows Vista and later+)

[http://www.pcworld.com/article/2308725/a-beginners-guide-to-bitlocker-windows-built-in-encryption-tool.html Instructions Here]. (It is very important to save the backup-code/key at the end!)

It is very important to save the backup-code/key at the end!

====Macs ====
Recommend using '''File Vault Encryption''' (Built into Mac OSX)

[https://support.apple.com/en-us/HT204837 Instructions Here]

====Others====
Other encryption programs are suggested [http://employee.uclahealth.org/device-security-toolkit/ here].

===USB/External Device ===
''Highly recommend going to Encryption Fair or your local IT for encryption''

[http://www.tomsguide.com/faq/id-2318737/encrypt-usb-flash-drive.html Windows Instructions]

[http://www.theinstructional.com/guides/encrypt-an-external-disk-or-usb-stick-with-a-password Mac Instructions]

=== Mobile Device ===
Mobile phone need to have AirWatch installed, if eligible.

To check if you are eligible:

<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">If the user is non-exempt (e.g. hourly pay status/bi-weekly pay), then they are not eligible for Airwatch unless approved by HR.

If the user is exempt (paid monthly), then they are eligible and should go to the encryption fair starting on Monday for help on downloading and installing Airwatch. Users also MUST have an AD account. </pre>

Pin/Pattern Code Screen Lock is not enough (though highly recommended!)
Please Install AirWatch at the [http://employee.uclahealth.org/encryption-fairs/ Encryption Fairs]

== Encryption Fair ==
Fore more information and dates : [http://employee.uclahealth.org/encryption-fairs/ Go Here]/

Encryption Policy

2015-10-15T22:40:48Z

Acho: /* Windows */

== Encryption Policy ==
Outlined here:

• All faculty, fellows, residents, students, volunteers and staff
• All mobile devices – laptops; tablets, mobile phones
• All removable media -- external hard drives; USB flash drives
• Non-University-owned devices
• Devices issued by UCLA Health or DGSOM
• All devices WHETHER OR NOT they are used to access restricted information
• Any mobile device used for any University business

For more information, go [http://employee.uclahealth.org/device-security/ Here]

== Devices ==
=== Laptops ===
====Windows====
Recommend using '''Bitlocker Drive Encryption'''
(Built into Windows Vista and later+)

[http://www.pcworld.com/article/2308725/a-beginners-guide-to-bitlocker-windows-built-in-encryption-tool.html Instructions Here]

It is very important to save the backup-code/key at the end!

====Macs ====
Recommend using '''File Vault Encryption''' (Built into Mac OSX)

[https://support.apple.com/en-us/HT204837 Instructions Here]

====Others====
Other encryption programs are suggested [http://employee.uclahealth.org/device-security-toolkit/ here].

===USB/External Device ===
''Highly recommend going to Encryption Fair or your local IT for encryption''

[http://www.tomsguide.com/faq/id-2318737/encrypt-usb-flash-drive.html Windows Instructions]

[http://www.theinstructional.com/guides/encrypt-an-external-disk-or-usb-stick-with-a-password Mac Instructions]

=== Mobile Device ===
Mobile phone need to have AirWatch installed, if eligible.

To check if you are eligible:

<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">If the user is non-exempt (e.g. hourly pay status/bi-weekly pay), then they are not eligible for Airwatch unless approved by HR.

If the user is exempt (paid monthly), then they are eligible and should go to the encryption fair starting on Monday for help on downloading and installing Airwatch. Users also MUST have an AD account. </pre>

Pin/Pattern Code Screen Lock is not enough (though highly recommended!)
Please Install AirWatch at the [http://employee.uclahealth.org/encryption-fairs/ Encryption Fairs]

== Encryption Fair ==
Fore more information and dates : [http://employee.uclahealth.org/encryption-fairs/ Go Here]/

Encryption Policy

2015-10-15T22:39:43Z

Acho: Created page with "== Encryption Policy == Outlined here: • All faculty, fellows, residents, students, volunteers and staff • All mobile devices – laptops; tablets, mobile phones • A..."

== Encryption Policy ==
Outlined here:

• All faculty, fellows, residents, students, volunteers and staff
• All mobile devices – laptops; tablets, mobile phones
• All removable media -- external hard drives; USB flash drives
• Non-University-owned devices
• Devices issued by UCLA Health or DGSOM
• All devices WHETHER OR NOT they are used to access restricted information
• Any mobile device used for any University business

For more information, go [http://employee.uclahealth.org/device-security/ Here]

== Devices ==
=== Laptops ===
====Windows====
Recommend using '''Bitlocker Drive Encryption'''
(Built into Windows Vista and later+)

[http://www.pcworld.com/article/2308725/a-beginners-guide-to-bitlocker-windows-built-in-encryption-tool.html Instructions Here]

====Macs ====
Recommend using '''File Vault Encryption''' (Built into Mac OSX)

[https://support.apple.com/en-us/HT204837 Instructions Here]

====Others====
Other encryption programs are suggested [http://employee.uclahealth.org/device-security-toolkit/ here].

===USB/External Device ===
''Highly recommend going to Encryption Fair or your local IT for encryption''

[http://www.tomsguide.com/faq/id-2318737/encrypt-usb-flash-drive.html Windows Instructions]

[http://www.theinstructional.com/guides/encrypt-an-external-disk-or-usb-stick-with-a-password Mac Instructions]

=== Mobile Device ===
Mobile phone need to have AirWatch installed, if eligible.

To check if you are eligible:

<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">If the user is non-exempt (e.g. hourly pay status/bi-weekly pay), then they are not eligible for Airwatch unless approved by HR.

If the user is exempt (paid monthly), then they are eligible and should go to the encryption fair starting on Monday for help on downloading and installing Airwatch. Users also MUST have an AD account. </pre>

Pin/Pattern Code Screen Lock is not enough (though highly recommended!)
Please Install AirWatch at the [http://employee.uclahealth.org/encryption-fairs/ Encryption Fairs]

== Encryption Fair ==
Fore more information and dates : [http://employee.uclahealth.org/encryption-fairs/ Go Here]/

Hoffman2:Getting an Account

2015-09-28T16:48:06Z

Acho: /* Applying for the Account */

[[Hoffman2|Back to all things Hoffman2]]

==Requesting Hoffman2 Account==
===What You Need===
A UCLA BOL account, available for free to any UCLA staff, student, or faculty member. If you do not have a BOL account, head to the [https://logon.ucla.edu UCLA Logon] services page. Click on "Create UCLA Logon ID".

===Applying for the Account===
ATTENTION: If you are a member of another lab or are a PI interested in obtaining Hoffman access, please see the section
[[#Becoming A Faculty Sponsor | Becoming a Faculty Sponsor]]
#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "New User Registration"
#Authenticate using your UCLA BOL credentials
#Fill out the form with appropriate information. For Hoffman2, your Faculty Sponsor should be Mark Cohen, Alison Burggren (for Susan Bookheimer's lab), or your respective PI if they are a Faculty Sponsor on Hoffman.
;Proposed UserName
:This will be the username you use to sign into the cluster with.
;Select a Resource
:For the Mark Cohen/Susan Bookheimer labs, choose "Hoffman2". However, you can request access to any cluster that is a member of the Grid Portal.

Click Submit.
You will receive an email with a link to a temporary password. '''PLEASE WRITE IT DOWN.''' The link expires after 72 hours. If you missed the link or it expired, go back to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Application Page] and click Forgot Your Cluster Password? It will take about a day for the cluster to resend you a new password.

You can change your password once you've logged in by using [[Hoffman2:Accessing_the_Cluster#Change_Passwords | passwd.]]

==Becoming A Faculty Sponsor==
If you are a PI or Lab Manager interested in the Hoffman2 Cluster, you will want to create a Faculty Sponsor account first. Also, if you are a member of another lab collaborating with the Cohen or Bookheimer labs, you may want to forward this information to your PI or Lab Manager. Faculty Sponsors can approve (or deny) applications for membership to their group. They also receive a group folder and a unique group id so their users can work and share data easily with each other.

#Navigate to the [http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Account Applications] page.
#Read over the application summary
#Click "Request to become faculty sponsor" (On the Bottom)
#Fill out the form with appropriate information.

Under 'Reason', about any generic reason is appropriate for faculty members. For example, "To perform fMRI analysis." will likely suffice.

==External Links==
*[https://grid.ucla.edu:9443/gridsphere/gridsphere?cid=home&JavaScript=enabled UCLA Grid Portal]
*[https://logon.ucla.edu UCLA BOL Home Page]
*[http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Hoffman2 Account Page]
*[[Hoffman2:Getting_an_Account-Historical_Notes | Historical Notes]]

Hoffman2:Getting an Account

2015-09-28T16:47:42Z

Acho: /* Applying for the Account */

[[Hoffman2|Back to all things Hoffman2]]

==Requesting Hoffman2 Account==
===What You Need===
A UCLA BOL account, available for free to any UCLA staff, student, or faculty member. If you do not have a BOL account, head to the [https://logon.ucla.edu UCLA Logon] services page. Click on "Create UCLA Logon ID".

===Applying for the Account===
ATTENTION: If you are a member of another lab or are a PI interested in obtaining Hoffman access, please see the section
[[#Becoming A Faculty Sponsor | Becoming a Faculty Sponsor]]
#Navigate to the [http://www.hoffman2.idre.ucla.edu/getting-started/ Account Applications] page.
#Read over the application summary
#Click "New User Registration"
#Authenticate using your UCLA BOL credentials
#Fill out the form with appropriate information. For Hoffman2, your Faculty Sponsor should be Mark Cohen, Alison Burggren (for Susan Bookheimer's lab), or your respective PI if they are a Faculty Sponsor on Hoffman.
;Proposed UserName
:This will be the username you use to sign into the cluster with.
;Select a Resource
:For the Mark Cohen/Susan Bookheimer labs, choose "Hoffman2". However, you can request access to any cluster that is a member of the Grid Portal.

Click Submit.
You will receive an email with a link to a temporary password. '''PLEASE WRITE IT DOWN.''' The link expires after 72 hours. If you missed the link or it expired, go back to the [http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Application Page] and click Forgot Your Cluster Password? It will take about a day for the cluster to resend you a new password.

You can change your password once you've logged in by using [[Hoffman2:Accessing_the_Cluster#Change_Passwords | passwd.]]

==Becoming A Faculty Sponsor==
If you are a PI or Lab Manager interested in the Hoffman2 Cluster, you will want to create a Faculty Sponsor account first. Also, if you are a member of another lab collaborating with the Cohen or Bookheimer labs, you may want to forward this information to your PI or Lab Manager. Faculty Sponsors can approve (or deny) applications for membership to their group. They also receive a group folder and a unique group id so their users can work and share data easily with each other.

#Navigate to the [http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Account Applications] page.
#Read over the application summary
#Click "Request to become faculty sponsor" (On the Bottom)
#Fill out the form with appropriate information.

Under 'Reason', about any generic reason is appropriate for faculty members. For example, "To perform fMRI analysis." will likely suffice.

==External Links==
*[https://grid.ucla.edu:9443/gridsphere/gridsphere?cid=home&JavaScript=enabled UCLA Grid Portal]
*[https://logon.ucla.edu UCLA BOL Home Page]
*[http://hpc.ucla.edu/hoffman2/getting-started/getting-started.php Hoffman2 Account Page]
*[[Hoffman2:Getting_an_Account-Historical_Notes | Historical Notes]]

Hoffman2:Submitting Jobs

2015-06-25T23:22:50Z

Acho: /* Examples */

[[Hoffman2|Back to all things Hoffman2]]

If you remember from [[Hoffman2:Introduction#Sun Grid Engine|Anatomy of the Computing Cluster]], the Sun Grid Engine on Hoffman2 is the scheduler for all computing jobs. It takes your computing job request, considers what resources you are asking for and then puts your job in a line waiting for those resources to become available.

Ask for a simple 1GB of memory and a single computing core with a short time window, and your job will likely get placed at the front of the line and start running soon if not immediately. And for the vast majority of people, this will be the case.

Ask for a lot of memory or many computing cores, and your job will get put further back in the line because it will have to wait for more things to become available. If your job needs these types of resources, you are probably at a level where reading this tutorial isn't very helpful.

Ask for too little RAM or too little time and your job will be killed or end prematurely leaving you with no results to examine.

Choose wisely.

So how does one submit a computing job request? You've got some options:
# '''job.q'''
#: Use a simple tool that ATS wrote. It has a menu and walks you through submitting things but has been known to possibly forget certain necessary flags.
# '''qsub'''
#: Get under the hood and do it yourself. It can get messy but it can also be faster and you have more flexibility with options.
# '''command files'''
#: You've graduated to a higher level of operations, but we can help you get there with examples of our own command files.
# '''job arrays'''
#: You've got a lot of repetitive tasks to run, these will be your friend.

==Aggregating Output Files==
By default, whenever you submit a job, the standard output and error files get created in whichever directory you submitted the job from unless you tell qsub differently with the "-o" and "-e" arguments. '''This can be very annoying when trying to reduce your file count as output files can be everywhere.'''

This is how you can avoid running around looking for these files:
# [[Hoffman2:Accessing the Cluster#SSH - Command Line|SSH into Hoffman2]]
# Use your favorite [[Text Editors|text editor]] to edit the file <code>~/.sge_request</code>
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <pre>$ vim ~/.sge_request</pre>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs]]
#:* <pre>$ emacs ~/.sge_request</pre>
#: [[Text Editors#NEdit (H2) (OSX)|NEdit]]
#:* <pre>$ nedit ~/.sge_request</pre>
# Insert this line into the file
#:* <pre>-o $HOME/job-output-files/</pre>
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* Type <code>A</code> - capital A - to go to the end of the line and enter insert mode
#:* Type or paste in the specified lines.
#: [[Text Editors#Emacs (H2)(OSX)|Emacs]]
#:* Type or paste in the specified lines.
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Type or paste in the specified lines.
# Save the file
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <code>ESC + ":wq" + ENTER</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs command line]]
#:* <code>CTRL+x, CTRL+c</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs GUI]]
#:* <code>CTRL+x, CTRL+c, y</code>
#:* or use the menu system
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Use the menu.
# Now use the following command to create the special directory that will receive all of the output and error files for the jobs you run.
#: <pre>mkdir ~/job-output-files</pre>
# Make an edit to your ~/.bash_profile so that you can run [[Hoffman2:Interactive Sessions]] without a problem
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <pre>$ vim ~/.bash_profile</pre>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs]]
#:* <pre>$ emacs ~/.bash_profile</pre>
#: [[Text Editors#NEdit (H2) (OSX)|NEdit]]
#:* <pre>$ nedit ~/.bash_profile</pre>
# Insert this line at the '''bottom''' of the file
#:* <pre>alias qrsh='qrsh -o /dev/null'</pre>
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* Type <code>G</code> - capital G - to go to the end of the file
#:* Type <code>A</code> - capital A - to go to the end of the line and enter insert mode
#:* Type <code>ENTER</code> - to insert a newline
#:* Type or paste in the specified lines.
#: [[Text Editors#Emacs (H2)(OSX)|Emacs]]
#:* Use the arrow keys to scroll the cursor down to the bottom of the document and add a newline.
#:* Type or paste in the specified lines.
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Use the arrow keys to scroll the cursor down to the bottom of the document and add a newline.
#:* Type or paste in the specified lines.
# Save the file
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <code>ESC + ":wq" + ENTER</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs command line]]
#:* <code>CTRL+x, CTRL+c</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs GUI]]
#:* <code>CTRL+x, CTRL+c, y</code>
#:* or use the menu system
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Use the menu.

==job.q==
Once you've identified or written a script you'd like to run, [[Hoffman2:Accessing the Cluster#SSH - Command Line|SSH into Hoffman2]] and enter <code>job.q</code>. Then it is just a matter of following its step-by-step instructions.

From the tool's main menu, you can type ''Info'' to read up about how to use it and we highly encourage you to do so.

But we know patience is a virtue that most of us aren't blessed with. So we'll walk you through submitting a basic job so you can hit the ground running.

===Example===
# Once on Hoffman2, you'll need to edit one file so pull out your favorite [[Text Editors|text editor]] and edit the file
#: <pre>~/.queuerc</pre>
# Add the line
#: <pre>set qqodir = ~/job-output</pre>
# You've just set the default directory where your job command files will be created. Save the configuration file and close your text editor.
# Make that directory using the command
#: <pre>$ mkdir ~/job-output</pre>
# Now execute
#:<pre>$ job.q</pre>
# Press enter to acknowledge the message about some files that get created (READ IT FIRST THOUGH).
# Type ''Build <ENTER>'' to begin creating an SGE command file.
# The program now asks you which script you'd like to run, enter the following text to use our example script
#: <pre>/u/home/FMRI/apps/examples/qsub/gather.sh</pre>
# The program now asks how much memory the job will need (in [http://en.wikipedia.org/wiki/Megabyte Megabytes]). This script is really simple, so let's go with the minimum and enter ''64''.
# The program now asks how long will the job take (in hours). Go with the minimum 1 hour; it will complete in much less than this.
# The program now asks if your job should be limited to only your resource group's cores. Answer ''n'' because you do not need to be limiting yourself here and the job is not going to be running for more than 24 hours.
# Soon, the program will tell you that ''gather.sh.cmd'' has been built and saved.
# When it asks you if you would like to submit your job, say no. Then type ''Quit <ENTER>'' to leave the program.
# Now you should be able to run
#: <pre>ls ~/job-output</pre>
#: and see ''gather.sh.cmd''. This file will stay there until you delete it and can be run over and over again. Making a command file like this is especially useful if there is a task you'll be running repeatedly on Hoffman2. But if this is something you only need to run once, you should delete the file so you don't needlessly approach your [[Hoffman2:Quotas|quota]].
# The time has come to actually run the program (thought we'd never get to that, didn't you?). Type
#: <pre>qsub job-output/gather.sh.cmd</pre>
#: and after hitting enter, a message similar to this will pop up:
#: <pre>Your job 1882940 ("gather.sh.cmd") has been submitted</pre>
#: where the number is your JobID, a unique numerical identifier for the computer job you have submitted to the queue.
# Now you can check if the job has finished running by doing
#: <pre>ls ~/job-output</pre>
# When two files named ''gather.sh.output.[JOBID]'' and ''gather.sh.joblog.[JOBID]'' (where JOBID is your job's unique identifier) appear, your job has run.
#: ''gather.sh.output.[JOBID]''
#:: This file has all the standard output generated by your script. In this case it will just have the line
#::: ''Standard output would appear here.''
#: ''gather.sh.joblog.[JOBID]''
#:: This file has all the details about when, where, and how your job was processed. Useful information if you are going to be running this job over and over and need to fine tune the resources it uses.
# Better ways of checking on your job can be found [[Hoffman2:Monitoring Jobs|here]].
# The script you ran is an aggregator. It looks in a list of directories, each assumed to contain a specifically named file, and gathers the contents of each of those files into one central file in your home directory. This file is named ''gather-[TIMESTAMP].txt'' where TIMESTAMP is when the script was run and follows [http://en.wikipedia.org/wiki/ISO_8601 ISO 8601] style encoding. You are encouraged to type
#: <pre>/u/home/FMRI/apps/examples/qsub/gather.sh -h</pre>
#: or
#: <pre>/u/home/FMRI/apps/examples/qsub/gather.sh --help</pre>
#: to see how this script works.
# Finally, go check the inbox of the email you used to sign up for your Hoffman2 account. There will be two emails from "root@mail.hoffman2.idre.ucla.edu" that indicate when the job was started and when the job was completed. This is one of the neat features of the queue so that you can be alerted about the progress of your job without having to stay logged into Hoffman2 and checking on it constantly.

==qsub==
Everything that job.q did can be done on the command line. And it can be done better.

===Example===
Run the command:
$ qsub -cwd -V -N J1 -l h_data=64M,express,h_rt=00:05:00 -M eplau -m bea /u/home/FMRI/apps/examples/qsub/gather.sh

And something like the following will be printed out:
Your job 1875395 ("J1") has been submitted

Where the number is your JOBID, a unique numerical identifier for your job.

Let's break down the arguments in that command.

;<code>-cwd</code>
: Change working directory
: When your script runs, change the working directory to where you currently are in the filesystem.
:: e.g. If you were in the director /u/home/mscohen/data/ when you ran the command, the queue will change directories to that location and then execute the script you gave it. This means output and error directories will be placed here for that job.

;<code>-V</code>
: Export environment variables
: Exports all the environment variables to the context of the job. Useful if you have extra environment variables that are needed in your script.
:: e.g. If you had defined the variable SUBJECT_ID in your session on Hoffman2 (<code>export SUBJECT_ID=42</code>) before submitting a job and that variable was called on by your script, then you would need to use this flag. Tools like FreeSurfer look for certain environment variables to be set.

;<code>-N J1</code>
: Name my job
: Names your job "J1." When you [[Hoffman2:Monitoring Jobs#qstat|look at the queue]], this will be the text that shows up in the "name" column. This will also be the beginning of the output (<code>J1.o[JOBID]</code>) and error (<code>J1.e[JOBID]</code>) files for your job.

;<code>-l h_data=64M,express,time=00:05:00</code>
: Resource allocation (that's a lower case "elle")
: This is the resources flag meaning that the text immediately after it will ask for things like:
:* certain amount of memory, in [http://en.wikipedia.org/wiki/Megabyte Megabytes], or [http://en.wikipedia.org/wiki/Gigabyte Gigabytes]
:** h_data=64M (64 MB RAM) or h_data=1G (1 GB RAM)
:** "mem" no longer works
: In this case, our demands for RAM are really low, so we are requesting only 64MB.
: '''Edit (2013.09)''' - If your job uses more RAM than it requested, your job WILL be killed in order to avoid it hurting other jobs running on the same node. It is imperative that you set this RAM request properly.
:* certain length of computing time, in the form HH:MM:SS
:** h_rt=00:05:00 or
:** time=00:05:00
: In this case the script will complete its task rapidly, hence we are only asking for 5 minutes of computing time.
:* queue type, only a few options here
:** [http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm#express express]
:**: Time limit of 2 hours, and it tends to be overloaded so it isn't recommended
:** [http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm#highp highp]
:**: Job length maximum of 14 days but can only be run on nodes belonging to your resource group (type <code>mygroup</code> to see what type of resources you have available). If you are in the mscohen or sbook usergroups on Hoffman2, you have access to some of these highp nodes.
:** [blank] (nothing, nada, zilch)
:**: [http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm#day Standard queue], which has a maximum job length of 24 hours
: In this case, we are asking to be put on the express queue since this is such a short job, but the standard queue would have worked just as well if not better.

;<code>-M eplau</code>
: Define mailing list
: This defines the list of users that will be mailed if email updates are requested. The default address is that of the job-owner, but multiple emails can be specified using a comma separated list.
:: e.g. In this case, the email will be sent to the address on file for the user "eplau"

;<code>-m bea</code>
: Define mailing rules
: This defines when Hoffman2 should email you about your job. There are five options here
:* b - when the job begins
:* e - when the job ends
:* a - when the job is aborted
:* s - when the job is suspended
:* n - never
: The first four can be used in any combination, but the last obviously nullifies the others.

There are many other flags that you could use, but these are the basics that will get you through most of your computing. Feel free to explore the others in the [http://www.ats.ucla.edu/clusters/common/computing/batch/man_submit.htm qsub Man page].

==Command Files==
Typing accurately can be difficult at times, so why put yourself through the trouble of having to retype the same arguments over and over if you will always be using about the same values? Enter command files.

You already have experience making a command file (~/job-output/gather.sh.cmd) from when you used the tool <code>job.q</code>. But did you know that you can edit that command file to make changes to how it runs, or write your own?

The command files generated by <code>job.q</code> are fairly well commented, so if you take a look at them with your favorite [Text Editors|text editor] you should be able to change their behavior. For instance, if you go into the command file from the job.q example, find the lines that say
# Notify at beginning and end of job
#$ -m bea
You recognize that this is the flag about when to send email messages. Go ahead and change this to
# Notify at the end and on abort
#$ -m ae
And you should only receive one email when your job finishes.

===q.sh===
You could make a generic command file that contains all the basic flags that you care about. We've even got an example ready and available for you at
/u/home/FMRI/apps/examples/qsub/q.sh
The script contents are shown below:
qsub <<CMD
#!/bin/bash
# Use current working directory
#$ -cwd
# Error stream is merged with the standard output
#$ -j y
# Use the bash shell for job execution
#$ -S /bin/bash
# Use your normal environment variables in the job
#$ -V
# Use 1GB of RAM and the main queue, with a maximum of 2 hours computing time
#$ -l h_data=1024M,h_rt=2:00:00
$@
CMD
To use this command file to submit the ''gather.sh'' example script, you would execute the command:
$ q.sh gather.sh
You can do this because if you have [[Hoffman2:Software Tools#Setting Up Your Account to Access the Tools|set up your Bash profile correctly]], they are in your [[Hoffman2:UNIX Tutorial#PATH|Unix PATH variable]]. You can replace ''gather.sh'' with any script you want executed and it will be submitted as a job on the cluster. We recommend that you make your own copy of ''q.sh'' and keep it in your local ''bin'' directory (~/bin) so that you can edit it to suit your needs.

==Job Arrays==
There is an SGE qsub argument that allows you to submit multiple jobs in parallel that use the same script. It is
-t lower-upper:interval
where
;<code>lower</code>
: is replaced with the starting number
;<code>upper</code>
: is replaced with the ending number
;<code>interval</code>
: is replaced with the step interval
So adding the argument
-t 10-100:5
will step through the numbers 10, 15, 20, 25, ..., 100 submitting a job for each one.

In jobs that are called with this flag, there will be an [[Hoffman2:UNIX Tutorial#Environment Variables|environment variable]] called <code>SGE_TASK_ID</code> whose value will be incremented over the range you specified. Each possible value of SGE_TASK_ID will be submitted as its own job, so your work will be parallelized.

===Examples===
Why would anyone use this? Here are some examples

====Lots of numbers====
Let's say you have a script, '''myFunc.sh''', that takes one numerical input and computes a bunch of values based on that input. But you need to run <code>myFunc.sh</code> for input values 1 to 100. One solution would be to write a wrapper script '''myFuncSlowWrapper.sh''' as
#!/bin/bash
# myFuncSlowWrapper.sh
for i in {1..100};
do
myFunc.sh $i;
done

The only drawback is that this will take quite a while since all 100 iterations will be done on a single processor. With job arrays, the computations will be split among many processors and can finish much more quickly. You would instead write a wrapper script called '''myFuncFastWrapper.sh''' as
#!/bin/bash
# myFuncFastWrapper.sh
echo $SGE_TASK_ID
myFunc.sh $SGE_TASK_ID

And submit it with
qsub -cwd -V -N PJ -l h_data=1024M,h_rt=01:00:00 -M eplau -m bea -t 1-100:1 myFuncWrapper.sh

====Lots of files====
Let's say you have a script, '''myFunc2.sh''', that takes the name of a file as input and opens that file and runs a bunch of computations on its contents. But you have 100 such files to process. One solution would be to write a wrapper script '''myFunc2SlowWrapper.sh''' as
#!/bin/bash
# myFunc2SlowWrapper.sh
for FILE in `ls dir/of/files`;
do
myFunc2.sh $FILE
done

But this will take quite a while since all 100 iterations will be done on a single processor. With job arrays, the computations will be split among many processors since they are submitted as their own jobs and can finish much more quickly. You could instead create a file that contains a list of all 100 files that need to be processed and call it '''filesToProcess'''. Then write a wrapper script called '''myFunc2FastWrapper.sh''' as
#!/bin/bash
# myFunc2FastWrapper.sh
echo $SGE_TASK_ID
myFunc2.sh `sed -n ${SGE_TASK_ID}p /path/to/list/of/files`

where you replace ''/path/to/list/of/file'' with the path to '''fileToProcess'''. The code
`sed -n ${SGE_TASK_ID}p /path/to/list/of/files`
uses <code>sed</code> to grab the ${SGE_TASK_ID}'th line from the file '''/path/to/list/of/files''' and returns it (thanks to the tick marks, <code>SHIFT + ~</code>).

Then you'd submit it with
qsub -cwd -V -N PJ -l h_data=1024M,express,h_rt=01:00:00 -M eplau -m bea -t 1-100:1 myFunc2Wrapper.sh

If your files were named regularly with a '-number' at the end (e.g. 'file-1', 'file-2', 'file-3', ... 'file-n'), you could just make '''myFunc2FastWrapperB.sh''' as
#!/bin/bash
# myFunc2FastWrapperB.sh
echo $SGE_TASK_ID
myFunc2.sh file-${SGE_TASK_ID}
and submit it the same way.

==External Links==
*[http://www.ats.ucla.edu/clusters/common/computing/batch/man_submit.htm qsub Man page]
*[http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm Hoffman2 Types of Queues]

Hoffman2:Submitting Jobs

2015-06-25T23:21:00Z

Acho: /* Examples */

[[Hoffman2|Back to all things Hoffman2]]

If you remember from [[Hoffman2:Introduction#Sun Grid Engine|Anatomy of the Computing Cluster]], the Sun Grid Engine on Hoffman2 is the scheduler for all computing jobs. It takes your computing job request, considers what resources you are asking for and then puts your job in a line waiting for those resources to become available.

Ask for a simple 1GB of memory and a single computing core with a short time window, and your job will likely get placed at the front of the line and start running soon if not immediately. And for the vast majority of people, this will be the case.

Ask for a lot of memory or many computing cores, and your job will get put further back in the line because it will have to wait for more things to become available. If your job needs these types of resources, you are probably at a level where reading this tutorial isn't very helpful.

Ask for too little RAM or too little time and your job will be killed or end prematurely leaving you with no results to examine.

Choose wisely.

So how does one submit a computing job request? You've got some options:
# '''job.q'''
#: Use a simple tool that ATS wrote. It has a menu and walks you through submitting things but has been known to possibly forget certain necessary flags.
# '''qsub'''
#: Get under the hood and do it yourself. It can get messy but it can also be faster and you have more flexibility with options.
# '''command files'''
#: You've graduated to a higher level of operations, but we can help you get there with examples of our own command files.
# '''job arrays'''
#: You've got a lot of repetitive tasks to run, these will be your friend.

==Aggregating Output Files==
By default, whenever you submit a job, the standard output and error files get created in whichever directory you submitted the job from unless you tell qsub differently with the "-o" and "-e" arguments. '''This can be very annoying when trying to reduce your file count as output files can be everywhere.'''

This is how you can avoid running around looking for these files:
# [[Hoffman2:Accessing the Cluster#SSH - Command Line|SSH into Hoffman2]]
# Use your favorite [[Text Editors|text editor]] to edit the file <code>~/.sge_request</code>
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <pre>$ vim ~/.sge_request</pre>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs]]
#:* <pre>$ emacs ~/.sge_request</pre>
#: [[Text Editors#NEdit (H2) (OSX)|NEdit]]
#:* <pre>$ nedit ~/.sge_request</pre>
# Insert this line into the file
#:* <pre>-o $HOME/job-output-files/</pre>
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* Type <code>A</code> - capital A - to go to the end of the line and enter insert mode
#:* Type or paste in the specified lines.
#: [[Text Editors#Emacs (H2)(OSX)|Emacs]]
#:* Type or paste in the specified lines.
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Type or paste in the specified lines.
# Save the file
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <code>ESC + ":wq" + ENTER</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs command line]]
#:* <code>CTRL+x, CTRL+c</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs GUI]]
#:* <code>CTRL+x, CTRL+c, y</code>
#:* or use the menu system
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Use the menu.
# Now use the following command to create the special directory that will receive all of the output and error files for the jobs you run.
#: <pre>mkdir ~/job-output-files</pre>
# Make an edit to your ~/.bash_profile so that you can run [[Hoffman2:Interactive Sessions]] without a problem
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <pre>$ vim ~/.bash_profile</pre>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs]]
#:* <pre>$ emacs ~/.bash_profile</pre>
#: [[Text Editors#NEdit (H2) (OSX)|NEdit]]
#:* <pre>$ nedit ~/.bash_profile</pre>
# Insert this line at the '''bottom''' of the file
#:* <pre>alias qrsh='qrsh -o /dev/null'</pre>
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* Type <code>G</code> - capital G - to go to the end of the file
#:* Type <code>A</code> - capital A - to go to the end of the line and enter insert mode
#:* Type <code>ENTER</code> - to insert a newline
#:* Type or paste in the specified lines.
#: [[Text Editors#Emacs (H2)(OSX)|Emacs]]
#:* Use the arrow keys to scroll the cursor down to the bottom of the document and add a newline.
#:* Type or paste in the specified lines.
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Use the arrow keys to scroll the cursor down to the bottom of the document and add a newline.
#:* Type or paste in the specified lines.
# Save the file
#: [[Text Editors#Vim (H2) (OSX)|VIM]]
#:* <code>ESC + ":wq" + ENTER</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs command line]]
#:* <code>CTRL+x, CTRL+c</code>
#: [[Text Editors#Emacs (H2) (OSX)|Emacs GUI]]
#:* <code>CTRL+x, CTRL+c, y</code>
#:* or use the menu system
#: [[Text Editors#NEdit (H2)|NEdit]]
#:* Use the menu.

==job.q==
Once you've identified or written a script you'd like to run, [[Hoffman2:Accessing the Cluster#SSH - Command Line|SSH into Hoffman2]] and enter <code>job.q</code>. Then it is just a matter of following its step-by-step instructions.

From the tool's main menu, you can type ''Info'' to read up about how to use it and we highly encourage you to do so.

But we know patience is a virtue that most of us aren't blessed with. So we'll walk you through submitting a basic job so you can hit the ground running.

===Example===
# Once on Hoffman2, you'll need to edit one file so pull out your favorite [[Text Editors|text editor]] and edit the file
#: <pre>~/.queuerc</pre>
# Add the line
#: <pre>set qqodir = ~/job-output</pre>
# You've just set the default directory where your job command files will be created. Save the configuration file and close your text editor.
# Make that directory using the command
#: <pre>$ mkdir ~/job-output</pre>
# Now execute
#:<pre>$ job.q</pre>
# Press enter to acknowledge the message about some files that get created (READ IT FIRST THOUGH).
# Type ''Build <ENTER>'' to begin creating an SGE command file.
# The program now asks you which script you'd like to run, enter the following text to use our example script
#: <pre>/u/home/FMRI/apps/examples/qsub/gather.sh</pre>
# The program now asks how much memory the job will need (in [http://en.wikipedia.org/wiki/Megabyte Megabytes]). This script is really simple, so let's go with the minimum and enter ''64''.
# The program now asks how long will the job take (in hours). Go with the minimum 1 hour; it will complete in much less than this.
# The program now asks if your job should be limited to only your resource group's cores. Answer ''n'' because you do not need to be limiting yourself here and the job is not going to be running for more than 24 hours.
# Soon, the program will tell you that ''gather.sh.cmd'' has been built and saved.
# When it asks you if you would like to submit your job, say no. Then type ''Quit <ENTER>'' to leave the program.
# Now you should be able to run
#: <pre>ls ~/job-output</pre>
#: and see ''gather.sh.cmd''. This file will stay there until you delete it and can be run over and over again. Making a command file like this is especially useful if there is a task you'll be running repeatedly on Hoffman2. But if this is something you only need to run once, you should delete the file so you don't needlessly approach your [[Hoffman2:Quotas|quota]].
# The time has come to actually run the program (thought we'd never get to that, didn't you?). Type
#: <pre>qsub job-output/gather.sh.cmd</pre>
#: and after hitting enter, a message similar to this will pop up:
#: <pre>Your job 1882940 ("gather.sh.cmd") has been submitted</pre>
#: where the number is your JobID, a unique numerical identifier for the computer job you have submitted to the queue.
# Now you can check if the job has finished running by doing
#: <pre>ls ~/job-output</pre>
# When two files named ''gather.sh.output.[JOBID]'' and ''gather.sh.joblog.[JOBID]'' (where JOBID is your job's unique identifier) appear, your job has run.
#: ''gather.sh.output.[JOBID]''
#:: This file has all the standard output generated by your script. In this case it will just have the line
#::: ''Standard output would appear here.''
#: ''gather.sh.joblog.[JOBID]''
#:: This file has all the details about when, where, and how your job was processed. Useful information if you are going to be running this job over and over and need to fine tune the resources it uses.
# Better ways of checking on your job can be found [[Hoffman2:Monitoring Jobs|here]].
# The script you ran is an aggregator. It looks in a list of directories, each assumed to contain a specifically named file, and gathers the contents of each of those files into one central file in your home directory. This file is named ''gather-[TIMESTAMP].txt'' where TIMESTAMP is when the script was run and follows [http://en.wikipedia.org/wiki/ISO_8601 ISO 8601] style encoding. You are encouraged to type
#: <pre>/u/home/FMRI/apps/examples/qsub/gather.sh -h</pre>
#: or
#: <pre>/u/home/FMRI/apps/examples/qsub/gather.sh --help</pre>
#: to see how this script works.
# Finally, go check the inbox of the email you used to sign up for your Hoffman2 account. There will be two emails from "root@mail.hoffman2.idre.ucla.edu" that indicate when the job was started and when the job was completed. This is one of the neat features of the queue so that you can be alerted about the progress of your job without having to stay logged into Hoffman2 and checking on it constantly.

==qsub==
Everything that job.q did can be done on the command line. And it can be done better.

===Example===
Run the command:
$ qsub -cwd -V -N J1 -l h_data=64M,express,h_rt=00:05:00 -M eplau -m bea /u/home/FMRI/apps/examples/qsub/gather.sh

And something like the following will be printed out:
Your job 1875395 ("J1") has been submitted

Where the number is your JOBID, a unique numerical identifier for your job.

Let's break down the arguments in that command.

;<code>-cwd</code>
: Change working directory
: When your script runs, change the working directory to where you currently are in the filesystem.
:: e.g. If you were in the director /u/home/mscohen/data/ when you ran the command, the queue will change directories to that location and then execute the script you gave it. This means output and error directories will be placed here for that job.

;<code>-V</code>
: Export environment variables
: Exports all the environment variables to the context of the job. Useful if you have extra environment variables that are needed in your script.
:: e.g. If you had defined the variable SUBJECT_ID in your session on Hoffman2 (<code>export SUBJECT_ID=42</code>) before submitting a job and that variable was called on by your script, then you would need to use this flag. Tools like FreeSurfer look for certain environment variables to be set.

;<code>-N J1</code>
: Name my job
: Names your job "J1." When you [[Hoffman2:Monitoring Jobs#qstat|look at the queue]], this will be the text that shows up in the "name" column. This will also be the beginning of the output (<code>J1.o[JOBID]</code>) and error (<code>J1.e[JOBID]</code>) files for your job.

;<code>-l h_data=64M,express,time=00:05:00</code>
: Resource allocation (that's a lower case "elle")
: This is the resources flag meaning that the text immediately after it will ask for things like:
:* certain amount of memory, in [http://en.wikipedia.org/wiki/Megabyte Megabytes], or [http://en.wikipedia.org/wiki/Gigabyte Gigabytes]
:** h_data=64M (64 MB RAM) or h_data=1G (1 GB RAM)
:** "mem" no longer works
: In this case, our demands for RAM are really low, so we are requesting only 64MB.
: '''Edit (2013.09)''' - If your job uses more RAM than it requested, your job WILL be killed in order to avoid it hurting other jobs running on the same node. It is imperative that you set this RAM request properly.
:* certain length of computing time, in the form HH:MM:SS
:** h_rt=00:05:00 or
:** time=00:05:00
: In this case the script will complete its task rapidly, hence we are only asking for 5 minutes of computing time.
:* queue type, only a few options here
:** [http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm#express express]
:**: Time limit of 2 hours, and it tends to be overloaded so it isn't recommended
:** [http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm#highp highp]
:**: Job length maximum of 14 days but can only be run on nodes belonging to your resource group (type <code>mygroup</code> to see what type of resources you have available). If you are in the mscohen or sbook usergroups on Hoffman2, you have access to some of these highp nodes.
:** [blank] (nothing, nada, zilch)
:**: [http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm#day Standard queue], which has a maximum job length of 24 hours
: In this case, we are asking to be put on the express queue since this is such a short job, but the standard queue would have worked just as well if not better.

;<code>-M eplau</code>
: Define mailing list
: This defines the list of users that will be mailed if email updates are requested. The default address is that of the job-owner, but multiple emails can be specified using a comma separated list.
:: e.g. In this case, the email will be sent to the address on file for the user "eplau"

;<code>-m bea</code>
: Define mailing rules
: This defines when Hoffman2 should email you about your job. There are five options here
:* b - when the job begins
:* e - when the job ends
:* a - when the job is aborted
:* s - when the job is suspended
:* n - never
: The first four can be used in any combination, but the last obviously nullifies the others.

There are many other flags that you could use, but these are the basics that will get you through most of your computing. Feel free to explore the others in the [http://www.ats.ucla.edu/clusters/common/computing/batch/man_submit.htm qsub Man page].

==Command Files==
Typing accurately can be difficult at times, so why put yourself through the trouble of having to retype the same arguments over and over if you will always be using about the same values? Enter command files.

You already have experience making a command file (~/job-output/gather.sh.cmd) from when you used the tool <code>job.q</code>. But did you know that you can edit that command file to make changes to how it runs, or write your own?

The command files generated by <code>job.q</code> are fairly well commented, so if you take a look at them with your favorite [Text Editors|text editor] you should be able to change their behavior. For instance, if you go into the command file from the job.q example, find the lines that say
# Notify at beginning and end of job
#$ -m bea
You recognize that this is the flag about when to send email messages. Go ahead and change this to
# Notify at the end and on abort
#$ -m ae
And you should only receive one email when your job finishes.

===q.sh===
You could make a generic command file that contains all the basic flags that you care about. We've even got an example ready and available for you at
/u/home/FMRI/apps/examples/qsub/q.sh
The script contents are shown below:
qsub <<CMD
#!/bin/bash
# Use current working directory
#$ -cwd
# Error stream is merged with the standard output
#$ -j y
# Use the bash shell for job execution
#$ -S /bin/bash
# Use your normal environment variables in the job
#$ -V
# Use 1GB of RAM and the main queue, with a maximum of 2 hours computing time
#$ -l h_data=1024M,h_rt=2:00:00
$@
CMD
To use this command file to submit the ''gather.sh'' example script, you would execute the command:
$ q.sh gather.sh
You can do this because if you have [[Hoffman2:Software Tools#Setting Up Your Account to Access the Tools|set up your Bash profile correctly]], they are in your [[Hoffman2:UNIX Tutorial#PATH|Unix PATH variable]]. You can replace ''gather.sh'' with any script you want executed and it will be submitted as a job on the cluster. We recommend that you make your own copy of ''q.sh'' and keep it in your local ''bin'' directory (~/bin) so that you can edit it to suit your needs.

==Job Arrays==
There is an SGE qsub argument that allows you to submit multiple jobs in parallel that use the same script. It is
-t lower-upper:interval
where
;<code>lower</code>
: is replaced with the starting number
;<code>upper</code>
: is replaced with the ending number
;<code>interval</code>
: is replaced with the step interval
So adding the argument
-t 10-100:5
will step through the numbers 10, 15, 20, 25, ..., 100 submitting a job for each one.

In jobs that are called with this flag, there will be an [[Hoffman2:UNIX Tutorial#Environment Variables|environment variable]] called <code>SGE_TASK_ID</code> whose value will be incremented over the range you specified. Each possible value of SGE_TASK_ID will be submitted as its own job, so your work will be parallelized.

===Examples===
Why would anyone use this? Here are some examples

====Lots of numbers====
Let's say you have a script, '''myFunc.sh''', that takes one numerical input and computes a bunch of values based on that input. But you need to run <code>myFunc.sh</code> for input values 1 to 100. One solution would be to write a wrapper script '''myFuncSlowWrapper.sh''' as
#!/bin/bash
# myFuncSlowWrapper.sh
for i in {1..100};
do
myFunc.sh $i;
done

The only drawback is that this will take quite a while since all 100 iterations will be done on a single processor. With job arrays, the computations will be split among many processors and can finish much more quickly. You would instead write a wrapper script called '''myFuncFastWrapper.sh''' as
#!/bin/bash
# myFuncFastWrapper.sh
echo $SGE_TASK_ID
myFunc.sh $SGE_TASK_ID

And submit it with
qsub -cwd -V -N PJ -l h_data=1024M,express,h_rt=01:00:00 -M eplau -m bea -t 1-100:1 myFuncWrapper.sh

====Lots of files====
Let's say you have a script, '''myFunc2.sh''', that takes the name of a file as input and opens that file and runs a bunch of computations on its contents. But you have 100 such files to process. One solution would be to write a wrapper script '''myFunc2SlowWrapper.sh''' as
#!/bin/bash
# myFunc2SlowWrapper.sh
for FILE in `ls dir/of/files`;
do
myFunc2.sh $FILE
done

But this will take quite a while since all 100 iterations will be done on a single processor. With job arrays, the computations will be split among many processors since they are submitted as their own jobs and can finish much more quickly. You could instead create a file that contains a list of all 100 files that need to be processed and call it '''filesToProcess'''. Then write a wrapper script called '''myFunc2FastWrapper.sh''' as
#!/bin/bash
# myFunc2FastWrapper.sh
echo $SGE_TASK_ID
myFunc2.sh `sed -n ${SGE_TASK_ID}p /path/to/list/of/files`

where you replace ''/path/to/list/of/file'' with the path to '''fileToProcess'''. The code
`sed -n ${SGE_TASK_ID}p /path/to/list/of/files`
uses <code>sed</code> to grab the ${SGE_TASK_ID}'th line from the file '''/path/to/list/of/files''' and returns it (thanks to the tick marks, <code>SHIFT + ~</code>).

Then you'd submit it with
qsub -cwd -V -N PJ -l h_data=1024M,express,h_rt=01:00:00 -M eplau -m bea -t 1-100:1 myFunc2Wrapper.sh

If your files were named regularly with a '-number' at the end (e.g. 'file-1', 'file-2', 'file-3', ... 'file-n'), you could just make '''myFunc2FastWrapperB.sh''' as
#!/bin/bash
# myFunc2FastWrapperB.sh
echo $SGE_TASK_ID
myFunc2.sh file-${SGE_TASK_ID}
and submit it the same way.

==External Links==
*[http://www.ats.ucla.edu/clusters/common/computing/batch/man_submit.htm qsub Man page]
*[http://www.ats.ucla.edu/clusters/hoffman2/computing/batch/policies.htm Hoffman2 Types of Queues]

Staglin Panic Button

2015-04-01T18:37:45Z

Acho: /* Staglin Panic Button */

=== Prowl Install Instructions ===
(IPHONES ONLY)

(1) You'll need to install the "Prowl" app from the App Store. It runs $3, but it's a worthy price for the notification system. The icon is a thin black cat with yellow eyes.

(2) Once that is installed, you'll need a Prowl account. Available for free here: https://www.prowlapp.com/register.php

(3) Navigate to the "API Keys" tab of the Prowl website and generate a new API key. Send this key value to andrew.y.cho@ucla.edu and it'll be added to the list.

(4) For the application preferences of Prowl, make sure that notifications are turned on. There are other settings like "Quiet Hours" that Prowl allows you to configure.

Staglin Panic Button

2015-04-01T18:37:35Z

Acho: Created page with "== Staglin Panic Button == === Prowl Install Instructions === (IPHONES ONLY) (1) You'll need to install the "Prowl" app from the App Store. It runs $3, but it's a worthy p..."

== Staglin Panic Button ==

=== Prowl Install Instructions ===
(IPHONES ONLY)

(1) You'll need to install the "Prowl" app from the App Store. It runs $3, but it's a worthy price for the notification system. The icon is a thin black cat with yellow eyes.

(2) Once that is installed, you'll need a Prowl account. Available for free here: https://www.prowlapp.com/register.php

(3) Navigate to the "API Keys" tab of the Prowl website and generate a new API key. Send this key value to andrew.y.cho@ucla.edu and it'll be added to the list.

(4) For the application preferences of Prowl, make sure that notifications are turned on. There are other settings like "Quiet Hours" that Prowl allows you to configure.

Hoffman2:MATLAB Licenses

2014-10-31T01:35:37Z

Acho:

[[Hoffman2|Back to all things Hoffman2]]

This page describes how to setup pulling licenses from Hoffman2 onto your local machine.

== Windows ==
1. Install Cwygin with the instructions [http://docs.oracle.com/cd/E24628_01/install.121/e22624/preinstall_req_cygwin_ssh.htm#EMBSC152 here].
ONLY FOLLOW SECTION 5.3 - Installing Cygwin

2. Move win-matlab.sh [Ask your adminstrator for this file] into C:/cygwin64/usr/local/bin

3. Make sure Matlab is installed in Programs (Default Settings)

4. Open up Cywgin and type [ win-matlab.sh ] and it should ask for your hoffman2 credentials.
- You will be entering your password several times. So no, you didn't type it incorrectly.

== Macs/Linux ==
1. Make sure Matlab is installed in your Applications Folder (Mac OSX) or bin folder (Unix)

2. Make sure you have the matlab script on your desktop, or somewhere easily accessible [Ask your administrator for this file].

3. Open up Terminal, go to the directory you have your script, and run matlab script [ sh mac-matlab.sh].
- You will be entering your password several times. So don't worry, you type it correctly the first time.

== Hoffman2 ==
1. Log onto Hoffman2 using SSH. Please make sure you have GUI X11-window installed. [[Hoffman2:Accessing_the_Cluster | More Information Here]]

2. Type in [ matlab ]. It will ask how long you want your session to last. Enter in however many hours you will use Matlab for. Be warned! This is a hard limit and is ENFORCED strictly. You will be kicked off of matlab when your time runs out.

Hoffman2:MATLAB Licenses

2014-10-31T01:18:40Z

Acho:

[[Hoffman2|Back to all things Hoffman2]]

This page describes how to setup pulling licenses from Hoffman2 onto your local machine.

== Windows ==
1. Install Cwygin with the instructions [http://docs.oracle.com/cd/E24628_01/install.121/e22624/preinstall_req_cygwin_ssh.htm#EMBSC152 here].
ONLY FOLLOW SECTION 5.3 - Installing Cygwin

2. Move win-matlab.sh [Ask your adminstrator for this file] into C:/cygwin64/usr/local/bin

3. Make sure Matlab is installed in Programs (Default Settings)

4. Open up Cywgin and type [ win-matlab.sh ] and it should ask for your hoffman2 credentials.
- You will be entering your password several times. So no, you didn't type it incorrectly.

== Macs/Linux ==
1. Make sure Matlab is installed in your Applications Folder (Mac OSX) or bin folder (Unix)

2. Open up Terminal and Run matlab.sh (from anywhere).
- You will be entering your password several times. So don't worry, you type it correctly the first time.

== Hoffman2 ==
1. Log onto Hoffman2 using SSH. Please make sure you have GUI X11-window installed. [[Hoffman2:Accessing_the_Cluster | More Information Here]]

2. Type in [ matlab ]. It will ask how long you want your session to last. Enter in however many hours you will use Matlab for. Be warned! This is a hard limit and is ENFORCED strictly. You will be kicked off of matlab when your time runs out.

Hoffman2:MATLAB Licenses

2014-10-31T00:36:36Z

Acho: Created page with "Back to all things Hoffman2 This page describes how to setup pulling licenses from Hoffman2 onto your local machine. == Windows == 1. Install Cwygin with the in..."

Hoffman2:Quotas

2014-10-14T21:50:01Z

Acho:

[[Hoffman2|Back to all things Hoffman2]]

Users and groups of users on Hoffman2 only have access to a predefined amount of disk space and number of files.

'''After quota is reached, your account (or all the accounts of a usergroup) will have reduced capabilities since you won't be able to make any new files.'''

<pre style="color: red">'''UNDER CONSTRUCTION!!!!!'''
This page is under construction. Will have an update soon. SORRY!</pre>

==Stay In The Know==
Keep yourself apprised of how much data you are using with these tools.

===Personal Quotas===
myquota
:Returns information about how much disk space you are using and how many files you have. An example output is shown below.
$ myquota
User quotas for eplau (UID 8693) (in GBs):
Filesystem Usage (in GB) Quota File Count File Quota
/home/mscohen 141 5120 58575 8000000
Filesystem /home/mscohen usage: 3857 of 5120 GBs (75.3%) and 7024000 of 8000000 files (87.8%)
/home/sbook 1 2048 4 5000000
Filesystem /home/sbook usage: 5901 of 6144 GBs (96.1%) and 5881211 of 6000000 files (98.0%)

===Group Quotas===
myquota -g [GROUPNAME]
:Returns information about how much space you and everyone in your resource group are using on Hoffman2. An example output is shown below.
$ myquota -g mscohen
Group mscohen Report (/home/mscohen):
Username UID Usage (in GB) Quota File Count File Quota
aarontre 9307 164 6144 81477 8000000
aburggre 8223 0 6144 10 8000000
akshaan 10094 0 6144 120 8000000
alenarto 8800 254 6144 97615 8000000
alhead 9612 100 6144 1001 8000000
ariana 8186 293 6144 420801 8000000
ayc 8955 160 6144 1180551 8000000
cdrodrig 9545 3 6144 52 8000000
dcmoyer 9397 921 6144 173754 8000000
diannaha 8134 0 6144 171 8000000
eddieyan 10322 0 6144 16 8000000
eplau 8693 523 6144 123964 8000000
eshwang1 9811 0 6144 54 8000000
fbiessma 10212 0 6144 67 8000000
fmri 1901 0 6144 479 8000000
jbramen 8369 632 6144 462428 8000000
jbrown 8187 0 6144 148 8000000
jianwen 9921 0 6144 19 8000000
kaavyara 9295 0 6144 1064 8000000
kerr 8555 487 6144 34095 8000000
kesslers 9815 25 6144 60996 8000000
mhussien 9922 0 6144 10 8000000
mlschaef 9447 11 6144 15783 8000000
mowyong 10329 0 6144 201 8000000
mscohen 4004 18 6144 8458 8000000
mundaeru 9542 0 6144 10 8000000
mwollner 9696 35 6144 2721 8000000
pamelita 8557 1280 6144 946730 8000000
root 0 0 6144 1 8000000
sanjayra 9846 0 6144 10 8000000
xiahongj 9047 666 6144 289695 8000000
Filesystem mscohen usage: 5656 of 6144 GBs (92.1%) and 3921309 of 8000000 files (49.0%)

==Clean Up After Yourself==
Every so often, some spring cleaning is useful. We have an app for that.
=====clean_me.py=====
Available to everyone in the FMRI usergroup on Hoffman2, this script was designed to find certain types of pesky files that have been known to build up over time but aren't actually necessary:
* ''.DS_store'' files
* Empty directories
* tsplots
* Empty files
* Extended Attributes
* mat files
and it gives you the option of deleting them.

To run it, just change into the directory you wish to clean and use the command:
$ clean_me.py

==When it Hits the Fan==
[[File:Hoffman2-ButtonPress.gif]]

You tried to remember to clean up your files, you even kept a large dataset on your computer last weekend instead of working with it on Hoffman2. But still your quota, or your group's quota was reached. How does one fix this?

# First things first, run <code>clean_me.py</code>
# Start identifying what you can delete. This is a wonderful opportunity to audit your home directory to see what really is in the directory called ''temp-567'' and what you actually put in the file ''subjects-az-list''.
# [[Tar Tutorial|Tar]] things up. That is to say, take that huge collection of DICOM files and turn them into a single file to cut down on overhead (remember it is about both file count and file size on Hoffman2). Unfamiliar with what tar is? Check out the [[Tar Tutorial]].

Hoffman2:Introduction

2014-10-14T05:39:03Z

Acho:

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://hpc.ucla.edu/hoffman2/hoffman2.php here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12,000 processors across three data centers and this number continues to grow as the cluster is expanded. [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Stats.] The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://hpc.ucla.edu/hoffman2/computing/gpuq.php here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://hpc.ucla.edu/hoffman2/computing/policies.php#highp up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<pre>/u/home/[u]/[username]</pre>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin # Common home directory</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/project/mscohen # Common group directory</pre>
::<pre>/u/project/mscohen/data # Common group "data" directory, create subdirectories for specific shared projects</pre>
::<pre>/u/project/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/project/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Univa Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the UGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The UGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2:Introduction

2014-10-14T02:57:35Z

Acho:

[[Hoffman2|Back to all things Hoffman2]]

==What is Hoffman2?==
The Hoffman2 Cluster is a campus computing resource at UCLA and is named for Paul Hoffman (1947-2003). It is maintained by the [https://idre.ucla.edu/ IDRE] at UCLA and the main official webage is [http://hpc.ucla.edu/hoffman2/hoffman2.php here]. With many high end processor, data storage, and backup technologies, it is a useful tool for executing research computations especially when working with large datasets. More than 1000 users are currently registered and the cluster sees tremendous usage. Click [[Hoffman2:Getting an Account|here]] to find out how to join. In September 2014 alone, there were more than 5.5 million compute hours logged. See more usage statistics [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php here].

==Anatomy of the Computing Cluster==
What does Hoffman2 consist of?
* Login Nodes
* Computing Nodes
* Storage Space
* Sun Grid Engine (a brain of sorts)

[[File:Hoffman2-layout.png]]
 
''**Image taken from a previous ATS "Using Hoffman2 Cluster" slide deck and modified for our point.**''

===Login Nodes===
There are four login nodes which allow you to access and interact with the Hoffman2 Cluster. These are essentially four dedicated computers that you can [[Hoffman2:Accessing the Cluster#SSH|SSH]] into and use to look at and edit your files or submit computing jobs to the queue (more on what the queue is in a bit). It is important to remember that these are four computers being shared by ALL the Hoffman2 users. Doing ANY type of heavy computing on these nodes is frowned upon. If you are:
*moving lots of files
*calculating the inverse solution to an EEG signal, or
*running a bunch of python scripts to extract tractography of a brain
You should NOT be doing this on a login node. If the sysadmins at ATS find any process that is taking up too many resources on the login nodes, they reserve the right to terminate the process immediately.

===Computing Nodes===
As of April 2014, Hoffman2 is made up of more than 12000 processors across three data centers and this number continues to grow as the cluster is expanded. [http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Stats.] The individual cores of the processors are where your programs gets executed when you submit a job to the cluster. There are ways to request different amount of resources, such as how much RAM or CPU cores your program/job needs.

There is also a GPU cluster that has more than 300 nodes, but access to this must be requested separately from a normal Hoffman2 account. For more information, go [http://hpc.ucla.edu/hoffman2/computing/gpuq.php here]

The reason the number of computing cores continues to grow is because more resource groups (like individual research labs) join Hoffman2 and buy nodes to be integrated into the cluster. Nodes contributed by a resource group are guaranteed to that resource group and can be used to run longer jobs ([http://hpc.ucla.edu/hoffman2/computing/policies.php#highp up to 14 days]). As of June 2013, the Cohen and Bookheimer groups on Hoffman2 have 96 cores:
* 6 nodes (installed pre 2010) each with
** 8 cores
** 8GB RAM
* 3 nodes (installed Fall 2012) each with
** 16 cores
** 48GB RAM
Use the command <code>mygroup</code> to see what resources you have available.

===Storage Space===
For official and up-to-date information about storage space, [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php click here]. If you want a quick overview, see below.

====Long Term Storage====
IDRE maintains high end storage systems (BlueArc and Panasas) for Hoffman2 disk space. There have built in redundancies and are fault tolerant. Redundant Backups are also available.

If all of that sounded Greek to you, the important thing to understand is that there is a lot of disk space on Hoffman2 and IDRE takes great pains to make sure your data is safe. If you are paranoid, there is alternative [ backups ].

=====Home Directories=====
:When you login to Hoffman2, you get dropped into your home directory immediately. Home directory locations follow the pattern
::<code>/u/home/[u]/[username]</code>
:Where <code>[u]</code> is the first letter of the username, e.g.
::<code>/u/home/j/jbruin</code>
::<code> /u/home/t/ttrojan</code>

:Your home directory is where you can keep your personal files (papers, correspondences, notes, etc.) and files you frequently change (source code, configuration files, job command files). '''It is not the place for your large datasets for computing.''' Data in your home directory is accessible from all login and computing nodes.

:Every user is allowed to store up to 20GB of data files in their home directory. If you are part of a cluster contributing group, you can also store data files in that group's common space described in the next section...

:[[Hoffman2:Quotas|Find out how much space your group is using on Hoffman2.]]

=====Group Directory=====
:Group directories are given to groups that purchase extended storage space (in 1TB/1million file increments for three year periods, as of Summer 2013). This is common space designed for collaboration and is where your datasets should mainly be stored. Individual users are given directories under the main group directory to help organize data ownership. For example:
::<pre>/u/home/mscohen # Common group directory</pre>
::<pre>/u/home/mscohen/data # Common group "data" directory, create subdirectories within this for specific projects or uses</pre>
::<pre>/u/home/mscohen/aaronab # mscohen group directory for the user aaronab, different from their /u/home/a/aaronab home directory</pre>
::<pre>/u/home/mscohen/kerr # mscohen group directory for the user kerr, different from their /u/home/k/kerr home directory</pre>

:and these directories are accessible from all login and computing nodes.

:'''These directories have limits to how many files can be put in them and how large those files can be.'''
:*When a group buys in for 1TB/1million files, their quota is considered met when they have EITHER
:** 1TB worth of files, OR
:** 1 million files
:*When a group buys in for 4TB/4million files, their quota is considered met when they have EITHER
:** 4TB worth of files, OR
:** 4 million files
:'''Once a group's quota has been reached, everyone in that group is immediately prevented from creating any more files in the group directory automatically.''' This means any computing jobs you are running may fail due to an inability to write out their results. You may also have trouble starting GUI sessions due to an inability to create temporary files.
:Read about how to monitor your disk quota [[Hoffman2:Quotas|here]].

====Temporary Storage====
When running a computing job on Hoffman2, reading and writing a bunch of files in your home directory can be slow. So faster temporary storage is available to use for ongoing jobs. Read the official description [http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php here].

=====work=====
:'''/work'''
:Each computing node has its own unique "work" directory. This is only accessible by jobs on that specific node. '''Files in /work more than 24 hours old become eligible for automatic deletion.''' There is at least 200GB of this space on each node, but you may only use a portion proportional to the number of cores you are using on that node (you have to share).

:Every job is given a unique subdirectory on ''work'' where it can read and write files rapidly. The [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$TMPDIR</code> points to this directory.

:If your job reads from or writes to a file repeatedly, you may save time by keeping that file in this temporary directory and then moving it to your home directory at job completion so it is not deleted.

=====scratch=====
:'''/u/scratch/[u]/[username]'''
:Where ''[username]'' is replaced with your Hoffman2 username and ''[u]'' is replaced with the first letter of your username. Data here is accessible on all login and computing nodes. You can use up to 2TB of space here, but '''Any files older than 7 days may be automatically deleted by the system'''. Use the [[Hoffman2:UNIX Tutorial#Environment Variables|UNIX environment variable]] <code>$SCRATCH</code> to reliably access your personal scratch directory.

====Cold Storage====
Cold storage is the idea of storing things away for a long time. Disk space on hoffman2 can be very expensive.
IDRE has a cloud archival storage service [http://www.cass.idre.ucla.edu/ here]

===Sun Grid Engine===
The Sun Grid Engine is the brains behind how jobs get executed on the cluster. When you request that a script be run on Hoffman2, the SGE looks at the resources you requested (how much memory, how many computing cores, how many computing hours, etc) and puts your job in a queue (a waiting line for those not familiar with British English) based on your requirements. Less demanding jobs generally get front loaded while more demanding ones must wait for adequate resources to free up. The SGE tries to schedule jobs on computing nodes in order to make the most efficient use of the resources available.

====Queues====
There is more than one queue on Hoffman2. Each is for a slightly different purpose:
; express
: For jobs requesting at most 2 hours of computing time.
; interactive
: For jobs requesting at most 24 hours of computing time and requiring the ability for users to interact with the program running.
; highp
: For jobs requesting at most 14 days of computing time. These are required to run on nodes owned by your group.
And there are others. Read about them [http://hpc.ucla.edu/hoffman2/computing/computing.php here].

Find out how to submit computing jobs to the [[Hoffman2:Submitting Jobs| Hoffman2 Cluster.]]

==External Links / Notes==
*[http://hpc.ucla.edu/hoffman2/hoffman2.php Hoffman2 Webpage]
*[http://hpc.ucla.edu/hoffman2/h2stat/h2stat.php Hoffman2 Statistics]
*[http://hpc.ucla.edu/hoffman2/data-storage/data-storage.php Hoffman2 Data Storage]
*[http://hpc.ucla.edu/hoffman2/computing/computing.php Hoffman2 Computing]
*[[Hoffman2:Introduction-Historical_Notes | Historical Notes]]

Hoffman2

2014-10-11T01:44:29Z

Acho:

A compilation of lab know-how regarding the Hoffman2 Computing Cluster.

Anyone new to the lab and using Hoffman2 NEEDS to read the first section to have adequate working knowledge of the system.

== Getting Started ==
=== Introduction ===
Hoffman2 is a Computing Cluster at UCLA, find out how it generally works so you know how to use it.
: [[Hoffman2:Introduction]]

=== Getting an Account ===
You know what it is, now you want to use it. First you need an account.
: [[Hoffman2:Getting an Account]]

=== Accessing the Cluster ===
Now how do you use that account to access the cluster?
: [[Hoffman2:Accessing the Cluster]]

=== Working in a UNIX Environment ===
Never heard of a command line before? Vaguely know what "permissions" are and have no idea how to navigate a filesystem? This page is meant to take the scare out of the words "command line" so you can actually use Hoffman2, because no matter how many GUIs there are, command line is king.
: [[Hoffman2:UNIX Tutorial]]

=== Quotas ===
Resources are not infinite, and disk space is a resource. Find out how to manage your disk space usage to stay under quota.
: [[Hoffman2:Quotas]]

=== Profile ===
You have an account, know how to get there, and now you need to make one last step for you account to be fully usable. Time to get access to all the fun computation tools.
: [[Hoffman2:Profile]]

== Computing ==
You can find your way through Hoffman2, now it is time to start making things happen.

=== Software Tools ===
You've got your account, you are logged on, now how do you get to using a real software tool?
: [[Hoffman2:Software Tools]]

=== Submitting Jobs ===
Now you have the tools, but how do you ask Hoffman2 to run them for you as a job? Since you aren't supposed to be running them on a login node...
: [[Hoffman2:Submitting Jobs]]

=== Monitoring Jobs ===
Right after they zap their robot monster to life, every mad scientist wishes they had the tools to check on or stop their creation. Now that you can submit jobs, you need to be able to check on them and stop them if they start terrorizing downtown Tokyo.
: [[Hoffman2:Monitoring Jobs]]

=== Interactive Sessions ===
Some software tools need you to interact with them while they work. Other times you just need to be able to run your script over and over while you work to eradicate all of its bugs. Enter ''Interactive'' Sessions.
: [[Hoffman2:Interactive Sessions]]

== Software ==
=== MATLAB ===
How to use MATLAB on the cluster. It is easier than you think. (Well, maybe...)
: [[Hoffman2:MATLAB]]

==== Compiling MATLAB ====
So you have a MATLAB script, but you don't need to GUI open all night to have it process your data. How to submit MATLAB jobs to Hoffman2.
: [[Hoffman2:Compiling MATLAB]]

==== EEGLAB ====
We try to maintain the three most recent versions of EEGLAB for your convenience. Make sure to add it to your MATLAB path.
: [[Hoffman2:MATLAB:EEGLAB]]

===== EEGLAB Jobs =====
Processing multiple subjects through EEGLAB can be tiring and inconvenient if you do it by hand. Learn how to make scripts that run as jobs leveraging the power of Hoffman2.
: [[Hoffman2:MATLAB:EEGLAB:Jobs]]

==== SPM Compiled (Batch) ====
Maybe FSL isn't your cup of tea for neuroimaging work. SPM is a capable alternative and, even though it is MATLAB based, it has a compiled version that will let you leverage the power of the cluster.
: [[Hoffman2:MATLAB:SPM]]

=== R ===
You are probably a statistician, or you just prefer open source software. Here's how to run R on Hoffman2.
: [[Hoffman2:R]]

=== WEKA ===
If machine learning is your thing, maybe you've heard of WEKA. If not, maybe it will be your new best friend.
: [[Hoffman2:WEKA]]

=== FSL ===
FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data.
: [[Hoffman2:FSL]]

== Productivity ==
How about streamlining some of those tasks, or getting more things done.

=== Scripts ===
All of the difficulties you are experiencing now have probably been experienced before by someone else. And for that reason we already have scripts to simplify your life.
: [[Hoffman2:Scripts]]

=== Data Transfer ===
All dressed up with no where to go? That's how Hoffman2 feels if you don't give it any data to work with. Find out how to avoid hurting the Cluster's feelings.
: [[Hoffman2:Data Transfer]]

=== Sharing Filesystems ===
All you want to do is be able to look at your precious data. But it is locked up on Hoffman2 and you want to use tools on your computer to look at it. There's an app for that.
: [[Hoffman2:Sharing Filesystems]]

=== Tools ===
Simple tools that will help your productivity.
: [[Hoffman2:Tools]]

== FAQ ==
Wesley's Usage, so you can plan around it and ask him to stop beating the cluster up.
: [[Hoffman2:WTK Usage]]

Delete/Old Hoffman2 Pages
: [[Hoffman2:Archive]]

Hoffman2:Archive

2014-10-11T01:43:14Z

Acho: Created page with "Back to all things Hoffman2 Old Stuff. You must be pretty desperate to be here. === LONI Pipeline === A Workflow application to make things easier. : Hoffma..."

[[Hoffman2|Back to all things Hoffman2]]

Old Stuff. You must be pretty desperate to be here.

=== LONI Pipeline ===
A Workflow application to make things easier.
: [[Hoffman2:LONI]]

Hoffman2

2014-10-11T01:38:39Z

Acho:

File:MathematicalTools2014.pdf

2014-10-06T07:17:00Z

Acho: Acho uploaded a new version of "File:MathematicalTools2014.pdf"

File:MathematicalTools2014.pdf

2014-10-06T07:14:28Z

Acho:

Hoffman2:Interactive Sessions

2014-09-18T20:04:24Z

Acho:

[[Hoffman2|Back to all things Hoffman2]]

Interactive sessions on Hoffman2 let you have access to a computing node for up to 24 hours. This is ideal for:
* running a intensive program like MATLAB (in fact that's how it [[Hoffman2:MATLAB|works]]), [[Hoffman2:WEKA|WEKA]], [[Hoffman2:R|R]] or FSLView
* debugging a script you will be submitting to the queue later
* moving/tar'ing/untar'ing lots of files
* any other computing or graphics intensive operations
since you aren't supposed to use the login nodes for such heavy lifting.

==Basic Command==
To get one, you need to use the <code>qrsh</code> command with the <code>-l i</code> flag.

For example
$ qrsh -l i
will try to get you an interactive node. That dash-elle flag followed by the "i" is specifying that you want an interactive resource.

Because you didn't specify a time limit this session will only last two hours after which you will be kicked off of the interactive node back to a login node.

And because you didn't specify a memory limit, as of September 2013 job memory enforcement is strict so you will be kicked off if you cross ATS's default memory limit. As of 2013.09.09 this default was 1GB.

And if all the interactive nodes are busy (there are only so many of them), then you will be told it was unable to secure one for you.

If you successfully get a node, your prompt will change from something like
[joebruin@login4 ~] $
to something like
[joebruin@n1234 ~] $
indicating you are on node 1234.

==Longer Time==
If you wanted to specify a a time limit for your interactive session (anything less than 24 hours), use the resource flag again and specify time in the HH:MM:SS format.

For example
$ qrsh -l i,h_rt=4:00:00
will try securing an interactive node for four hours with the default amount of RAM, but if they are all taken you will be kindly told you are out of luck.

==More Memory==
Doing something memory intensive? Like working with a lot of visualizations or multiple datasets? Use the resource flag again and specify a data request.

For example
$ qrsh -l i,h_rt=4:00:00,h_data=4G
will try securing an interactive node for four hours with four gigabytes of RAM, but if no such node is available the cluster will deny your request.

==Now!==
If you absolutely need an interactive session now and can't take no for an answer, use a special flag <code>-now no</code>.

For example
$ qrsh -l i,h_rt=4:00:00 -now no
will try securing an interactive node for four hours with the default amount of RAM. But if all of the interactive nodes are used up, it will put you in a queue waiting for one until you get it.

==Tips==
Sometimes inactivity on your computer will result in Hoffman2 connection break [ Broken Pipe ] (even while computing).

To prevent this from happening:
For Macs - in your /etc/ssh_config -add this line to the bottom
ServerAliveInterval 180

This will tell your ssh to ping the server every 180 seconds to prevent it from timing out.

==External Links==
*[http://www.ats.ucla.edu/clusters/hoffman2/computing/sge_qrsh.htm Getting an interactive node]

Hoffman2:Using Globus

2014-05-25T07:34:55Z

Acho: /* Transfer - Cluster to CASS */

[[Hoffman2:Data_Transfer|Back to Hoffman2:Data_Transfer]]

This page contains many information on how to use Globus to move around files.

==Globus Connect Software (Local Desktop)==
If you want to transfer files to or from your local desktop machine, you need to download and install the Globus Connect software (one-time). You will need to do this step on each of your desktop machines whose files you want to transfer using Globus.

#Point your browser at http://www.globus.org and click Globus Connect. You will see a popup window in a web page. If you don’t see the popup window, click the Get Globus Connect link on that page.
#In Step One, click the button corresponding to your local platform (Mac OS, Linux or Windows) to download to your local desktop machine.
#In Step Two, enter an Endpoint Name to identify your local machine in the Endpoint Name field, (you can ignore the Description field), and click Generate Setup Key.
#Copy the setup key to some file and save it, since you will not see it again.
#Install the downloaded Globus Connect software on your local desktop.
#Run Globus Connect Installation (Windows: globus_connect_install.exe) for the initial setup, and paste in the setup key when prompted.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Local_to_Cluster|directions below.]]

==Adding a CASS Endpoint==
If you have a UCLA cloud archival storage system, you need to go through this process before moving files between your CASS and other storage systems.

#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#On the Transfer Files Page, Click on Manage Endpoints (on the top right).
#Click on Add an Endpoint (a drop down menu should appear)
#Fill out the Information
#*Basics
#**Endpoint Name (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Description = '''[Any Description Will Do - Ex. MSCOHEN-CASS]'''
#**Visible To: '''Private - Visible only to you'''
#**Default Directory: '''/~/'''
#*Identity Providers
#**(In the dropdown menu): '''MyProxy OAuth'''
#*Servers
#**Server Type: '''GridFTP'''
#**Server Domain (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Server Port: '''2811'''
#**Subject DN: '''[Your Subject DN - Ex. /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=1234567...]'''
# Click Create Endpoint.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Cluster_to_CASS|directions below.]]

==Transfer - Local to Cluster==
How to transfer files between the Hoffman2 Cluster and your local computer

#If you want to transfer files to or from your local desktop machine, start Globus Connect (Windows: gc.exe, Mac: Applications/Globus) on your local machine. A small status window will appear. When the connection has been made, the dot at the left of the connection will turn green. (MAKE SURE THE KEY IS ADDED - [[Hoffman2:Using_Globus#Globus_Connect_Software|Instructions Above.]]
#Point your browser at http://www.globus.org, click Sign In, and click Transfer Files. A web page with two Endpoint fields will display.
#In one Endpoint field, pull down the expand menu and select a site. If you are running Globus Connect, the first name in the list is your local desktop machine. When you select your local desktop machine, a list of your home directory files and directories will be displayed.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#When you select hoffman2#ucla a popup window will ask you for your myproxy server username and passphrase. Enter your Hoffman2 username and its password. Leave the Server DN field as it is. The default Lifetime value is 12 hours. If you are transferring a large amount of data, you may need to increase the Lifetime value. Click the Authenticate button.
#Your Hoffman2 Cluster home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.
You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

==Transfer - Cluster to CASS==
How to transfer files between the Hoffman2 Cluster and your Cloud Archival Storage Service
#Make Sure You have setup the CASS Endpoint. If you have not, go to the [[Hoffman2:Using_Globus#Adding_a_CASS_Endpoint|section above.]]
#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#In one Endpoint field, type in '''cass.idre.ucla.edu''' and you should see your '''[username#groupname].cass.idre.ucla.edu''' drop down - click on it.
#Authenticate with your CASS credentials and Click the Authenticate button.
#Your CASS home directory and its contents will display.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#Authenticate with your Hoffman2 credentials and Click the Authenticate button.
#Your Hoffman2 home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

Hoffman2:Using Globus

2014-05-25T07:34:35Z

Acho: /* Transfer - Cluster to CASS */

[[Hoffman2:Data_Transfer|Back to Hoffman2:Data_Transfer]]

This page contains many information on how to use Globus to move around files.

==Globus Connect Software (Local Desktop)==
If you want to transfer files to or from your local desktop machine, you need to download and install the Globus Connect software (one-time). You will need to do this step on each of your desktop machines whose files you want to transfer using Globus.

#Point your browser at http://www.globus.org and click Globus Connect. You will see a popup window in a web page. If you don’t see the popup window, click the Get Globus Connect link on that page.
#In Step One, click the button corresponding to your local platform (Mac OS, Linux or Windows) to download to your local desktop machine.
#In Step Two, enter an Endpoint Name to identify your local machine in the Endpoint Name field, (you can ignore the Description field), and click Generate Setup Key.
#Copy the setup key to some file and save it, since you will not see it again.
#Install the downloaded Globus Connect software on your local desktop.
#Run Globus Connect Installation (Windows: globus_connect_install.exe) for the initial setup, and paste in the setup key when prompted.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Local_to_Cluster|directions below.]]

==Adding a CASS Endpoint==
If you have a UCLA cloud archival storage system, you need to go through this process before moving files between your CASS and other storage systems.

#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#On the Transfer Files Page, Click on Manage Endpoints (on the top right).
#Click on Add an Endpoint (a drop down menu should appear)
#Fill out the Information
#*Basics
#**Endpoint Name (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Description = '''[Any Description Will Do - Ex. MSCOHEN-CASS]'''
#**Visible To: '''Private - Visible only to you'''
#**Default Directory: '''/~/'''
#*Identity Providers
#**(In the dropdown menu): '''MyProxy OAuth'''
#*Servers
#**Server Type: '''GridFTP'''
#**Server Domain (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Server Port: '''2811'''
#**Subject DN: '''[Your Subject DN - Ex. /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=1234567...]'''
# Click Create Endpoint.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Cluster_to_CASS|directions below.]]

==Transfer - Local to Cluster==
How to transfer files between the Hoffman2 Cluster and your local computer

#If you want to transfer files to or from your local desktop machine, start Globus Connect (Windows: gc.exe, Mac: Applications/Globus) on your local machine. A small status window will appear. When the connection has been made, the dot at the left of the connection will turn green. (MAKE SURE THE KEY IS ADDED - [[Hoffman2:Using_Globus#Globus_Connect_Software|Instructions Above.]]
#Point your browser at http://www.globus.org, click Sign In, and click Transfer Files. A web page with two Endpoint fields will display.
#In one Endpoint field, pull down the expand menu and select a site. If you are running Globus Connect, the first name in the list is your local desktop machine. When you select your local desktop machine, a list of your home directory files and directories will be displayed.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#When you select hoffman2#ucla a popup window will ask you for your myproxy server username and passphrase. Enter your Hoffman2 username and its password. Leave the Server DN field as it is. The default Lifetime value is 12 hours. If you are transferring a large amount of data, you may need to increase the Lifetime value. Click the Authenticate button.
#Your Hoffman2 Cluster home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.
You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

==Transfer - Cluster to CASS==
How to transfer files between the Hoffman2 Cluster and your Cloud Archival Storage Service
#Make Sure You have setup the CASS Endpoint. If you have not, go to the [[Hoffman2:Using_Globus#Adding_CASS_Endpoint|section above.]]
#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#In one Endpoint field, type in '''cass.idre.ucla.edu''' and you should see your '''[username#groupname].cass.idre.ucla.edu''' drop down - click on it.
#Authenticate with your CASS credentials and Click the Authenticate button.
#Your CASS home directory and its contents will display.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#Authenticate with your Hoffman2 credentials and Click the Authenticate button.
#Your Hoffman2 home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

Hoffman2:Using Globus

2014-05-25T07:34:11Z

Acho: Undo revision 2619 by Acho (talk)

[[Hoffman2:Data_Transfer|Back to Hoffman2:Data_Transfer]]

This page contains many information on how to use Globus to move around files.

==Globus Connect Software (Local Desktop)==
If you want to transfer files to or from your local desktop machine, you need to download and install the Globus Connect software (one-time). You will need to do this step on each of your desktop machines whose files you want to transfer using Globus.

#Point your browser at http://www.globus.org and click Globus Connect. You will see a popup window in a web page. If you don’t see the popup window, click the Get Globus Connect link on that page.
#In Step One, click the button corresponding to your local platform (Mac OS, Linux or Windows) to download to your local desktop machine.
#In Step Two, enter an Endpoint Name to identify your local machine in the Endpoint Name field, (you can ignore the Description field), and click Generate Setup Key.
#Copy the setup key to some file and save it, since you will not see it again.
#Install the downloaded Globus Connect software on your local desktop.
#Run Globus Connect Installation (Windows: globus_connect_install.exe) for the initial setup, and paste in the setup key when prompted.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Local_to_Cluster|directions below.]]

==Adding a CASS Endpoint==
If you have a UCLA cloud archival storage system, you need to go through this process before moving files between your CASS and other storage systems.

#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#On the Transfer Files Page, Click on Manage Endpoints (on the top right).
#Click on Add an Endpoint (a drop down menu should appear)
#Fill out the Information
#*Basics
#**Endpoint Name (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Description = '''[Any Description Will Do - Ex. MSCOHEN-CASS]'''
#**Visible To: '''Private - Visible only to you'''
#**Default Directory: '''/~/'''
#*Identity Providers
#**(In the dropdown menu): '''MyProxy OAuth'''
#*Servers
#**Server Type: '''GridFTP'''
#**Server Domain (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Server Port: '''2811'''
#**Subject DN: '''[Your Subject DN - Ex. /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=1234567...]'''
# Click Create Endpoint.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Cluster_to_CASS|directions below.]]

==Transfer - Local to Cluster==
How to transfer files between the Hoffman2 Cluster and your local computer

#If you want to transfer files to or from your local desktop machine, start Globus Connect (Windows: gc.exe, Mac: Applications/Globus) on your local machine. A small status window will appear. When the connection has been made, the dot at the left of the connection will turn green. (MAKE SURE THE KEY IS ADDED - [[Hoffman2:Using_Globus#Globus_Connect_Software|Instructions Above.]]
#Point your browser at http://www.globus.org, click Sign In, and click Transfer Files. A web page with two Endpoint fields will display.
#In one Endpoint field, pull down the expand menu and select a site. If you are running Globus Connect, the first name in the list is your local desktop machine. When you select your local desktop machine, a list of your home directory files and directories will be displayed.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#When you select hoffman2#ucla a popup window will ask you for your myproxy server username and passphrase. Enter your Hoffman2 username and its password. Leave the Server DN field as it is. The default Lifetime value is 12 hours. If you are transferring a large amount of data, you may need to increase the Lifetime value. Click the Authenticate button.
#Your Hoffman2 Cluster home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.
You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

==Transfer - Cluster to CASS==
How to transfer files between the Hoffman2 Cluster and your Cloud Archival Storage Service
#Make Sure You have setup the CASS Endpoint. If you have not, go to the [[Hoffman2:Using_Globus# Adding_CASS_Endpoint|section above.]]
#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#In one Endpoint field, type in '''cass.idre.ucla.edu''' and you should see your '''[username#groupname].cass.idre.ucla.edu''' drop down - click on it.
#Authenticate with your CASS credentials and Click the Authenticate button.
#Your CASS home directory and its contents will display.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#Authenticate with your Hoffman2 credentials and Click the Authenticate button.
#Your Hoffman2 home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

Hoffman2:Using Globus

2014-05-25T07:32:27Z

Acho:

[[Hoffman2:Data_Transfer|Back to Hoffman2:Data_Transfer]]

This page contains many information on how to use Globus to move around files.

==Globus Connect Software (Local Desktop)==
If you want to transfer files to or from your local desktop machine, you need to download and install the Globus Connect software (one-time). You will need to do this step on each of your desktop machines whose files you want to transfer using Globus.

#Point your browser at http://www.globus.org and click Globus Connect. You will see a popup window in a web page. If you don’t see the popup window, click the Get Globus Connect link on that page.
#In Step One, click the button corresponding to your local platform (Mac OS, Linux or Windows) to download to your local desktop machine.
#In Step Two, enter an Endpoint Name to identify your local machine in the Endpoint Name field, (you can ignore the Description field), and click Generate Setup Key.
#Copy the setup key to some file and save it, since you will not see it again.
#Install the downloaded Globus Connect software on your local desktop.
#Run Globus Connect Installation (Windows: globus_connect_install.exe) for the initial setup, and paste in the setup key when prompted.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Local_to_Cluster|directions below.]]

==Adding a CASS Endpoint==
If you have a UCLA cloud archival storage system, you need to go through this process before moving files between your CASS and other storage systems.

#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#On the Transfer Files Page, Click on Manage Endpoints (on the top right).
#Click on Add an Endpoint (a drop down menu should appear)
#Fill out the Information
#*Basics
#**Endpoint Name (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Description = '''[Any Description Will Do - Ex. MSCOHEN-CASS]'''
#**Visible To: '''Private - Visible only to you'''
#**Default Directory: '''/~/'''
#*Identity Providers
#**(In the dropdown menu): '''MyProxy OAuth'''
#*Servers
#**Server Type: '''GridFTP'''
#**Server Domain (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Server Port: '''2811'''
#**Subject DN: '''[Your Subject DN - Ex. /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=1234567...]'''
# Click Create Endpoint.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-Cluster_to_Cass|directions below.]]

==Transfer - Local to Cluster==
How to transfer files between the Hoffman2 Cluster and your local computer

#If you want to transfer files to or from your local desktop machine, start Globus Connect (Windows: gc.exe, Mac: Applications/Globus) on your local machine. A small status window will appear. When the connection has been made, the dot at the left of the connection will turn green. (MAKE SURE THE KEY IS ADDED ([[Hoffman2:Data_Transfer#Globus_Connect_software|Instructions Here.]])
#Point your browser at http://www.globus.org, click Sign In, and click Transfer Files. A web page with two Endpoint fields will display.
#In one Endpoint field, pull down the expand menu and select a site. If you are running Globus Connect, the first name in the list is your local desktop machine. When you select your local desktop machine, a list of your home directory files and directories will be displayed.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#When you select hoffman2#ucla a popup window will ask you for your myproxy server username and passphrase. Enter your Hoffman2 username and its password. Leave the Server DN field as it is. The default Lifetime value is 12 hours. If you are transferring a large amount of data, you may need to increase the Lifetime value. Click the Authenticate button.
#Your Hoffman2 Cluster home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.
You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

==Transfer - Cluster to CASS==
How to transfer files between the Hoffman2 Cluster and your Cloud Archival Storage Service
#Make Sure You have setup the CASS Endpoint. If you have not, go to the [[Hoffman2:Using_Globus# Adding_CASS_Endpoint|section below.]]
#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#In one Endpoint field, type in '''cass.idre.ucla.edu''' and you should see your '''[username#groupname].cass.idre.ucla.edu''' drop down - click on it.
#Authenticate with your CASS credentials and Click the Authenticate button.
#Your CASS home directory and its contents will display.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#Authenticate with your Hoffman2 credentials and Click the Authenticate button.
#Your Hoffman2 home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

Hoffman2:Data Transfer

2014-05-25T07:20:28Z

Acho:

[[Hoffman2|Back to all things Hoffman2]]

So you got a whole bunch of data that's sitting on your hard drive or somewhere else, and you need it on hoffman2. How do you get that gigs and gigs of data onto the cluster? Sorry there's no magical genie that instantly does it for you. You're going to have to slowly transfer it through your network on to Hoffman2. PLEASE NOTE: TRANSFER IS ONLY AS FAST AS YOUR INTERNET CONNECTION!

Here's several ways of pushing that cow up the hill.

==Globus Online==
Globus Online is a tool that abstracts a lot of complexity from the data transfer process. It is a GUI system that is capable of starting a transfer, having the internet connection broken and re-established, and then continuing and finishing the transfer automatically.

'''It is the fastest way to transfer data to Hoffman2''' because it has a faster connection than other Hoffman2 nodes.

====Create a Globus Account====
First you need to create a free, Globus account (one-time):
*Point your browser at http://www.globus.org and click Sign Up.
*On the Create an Account page, fill in the information (your name, email address, username, password, etc.) and read the terms, then click Register.
You will receive an email with a link which you need to follow to confirm your new Globus account.

====General Usage====
After creating an account, you need to setup endpoints to transfer from/to.
*Hoffman2 Endpoint
Hoffman2 Endpoint is already setup on Globus, and you just need to type '''hoffman2#ucla''' into the endpoint field.
*CASS Endpoint
[http://www.cass.idre.ucla.edu/ (Cloud Archival Storage Service)] You Need to Set this up on Globus before you can use it.
[[Hoffman2:Using_Globus#Adding_a_CASS_Endpoint | Directions Here.]]
*Local Desktop Endpoint
If you want to transfer files between your local desktop, you need to download the Globus Desktop Client and set it up on Globus before you can transfer any files.
[[Hoffman2:Using_Globus#Globus_Connect_Software | Directions Here.]]

====Transfer Files====
*[[Hoffman2:Using_Globus#Transfer_-_Cluster_to_CASS| Transfer -> Cluster to CASS]]

*[[Hoffman2:Using_Globus#Transfer_-_Local_to_Cluster| Transfer -> Local to Cluster]]

For more information, see the IDRE instructions here
[http://hpc.ucla.edu/hoffman2/file-transfer/gol.php Globus Online]

==scp==
A command line tool for copying files using secure encrypted channels.

====General Usage====
scp username@FROM:location username@TO:location

====Hoffman2 Examples====
Copy a file from local computer to Hoffman2
scp /path/to/local/file username@dtn2.hoffman2.idre.ucla.edu:/path/to/destination/for/copy

Copy a file from Hoffman2 to local computer
scp username@dtn2.hoffman2.idre.ucla.edu:/path/to/file /path/to/destination/for/copy

Copy a directory from local computer to Hoffman2
scp -r /path/to/local/directory username@dtn2.hoffman2.idre.ucla.edu:/path/to/destination/for/copy

Copy a directory from Hoffman2 to local computer
scp -r username@dtn2.hoffman2.idre.ucla.edu:/path/to/directory /path/to/destination/for/copy

==sftp==
Another command line tool that uses secure encrypted channels. There are also GUIs that use this protocol (like [http://cyberduck.ch/ Cyberduck] or [http://filezilla-project.org/ Filezilla]).

====General Usage====
Log in to the server, then punch in your password when prompted and you'll be logged in.
sftp USERNAME@SERVERADDRESS
Pull down a file from the server
get /path/to/server/file /path/to/destination/for/copy
Push a file to the server
put /path/to/local/file /path/to/destination/for/copy/on/server
Log out.
bye

====Hoffman2 Example====
sftp USERNAME@dtn2.hoffman2.idre.ucla.edu
get /server/file /local/file
put /local/file /server/file
bye

==rsync==
Implied in the name, syncs folder/files between filesystems. It also makes use of secure encrypted channels.

====General Usage====
rsync [OPTION] … SRC [SRC] … [USER@]HOST:DEST

====Hoffman2 Example====
We recommend using something like
rsync -av /PATH/TO/SRC/FILES/HERE USERNAME@dtn2.hoffman2.idre.ucla.edu:~
to upload ''/PATH/TO/SRC/FILES/HERE'' from your local machine to your home directory on Hoffman2

==External Links==
*[http://linux.die.net/man/1/scp Man page for scp]
*[http://linux.die.net/man/1/sftp Man page for sftp]
*[http://linux.die.net/man/1/rsync Man page for rsync]
*[http://www.ats.ucla.edu/clusters/hoffman2/gol.htm Globus Online]

Hoffman2:Using Globus

2014-05-25T07:17:07Z

Acho:

[[Hoffman2:Data_Transfer|Back to Hoffman2:Data_Transfer]]

This page contains many information on how to use Globus to move around files.

==Globus Connect Software (Local Desktop)==
If you want to transfer files to or from your local desktop machine, you need to download and install the Globus Connect software (one-time). You will need to do this step on each of your desktop machines whose files you want to transfer using Globus.

#Point your browser at http://www.globus.org and click Globus Connect. You will see a popup window in a web page. If you don’t see the popup window, click the Get Globus Connect link on that page.
#In Step One, click the button corresponding to your local platform (Mac OS, Linux or Windows) to download to your local desktop machine.
#In Step Two, enter an Endpoint Name to identify your local machine in the Endpoint Name field, (you can ignore the Description field), and click Generate Setup Key.
#Copy the setup key to some file and save it, since you will not see it again.
#Install the downloaded Globus Connect software on your local desktop.
#Run Globus Connect Installation (Windows: globus_connect_install.exe) for the initial setup, and paste in the setup key when prompted.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Local_to_Cluster|directions below.]]

==Adding a CASS Endpoint==
If you have a UCLA cloud archival storage system, you need to go through this process before moving files between your CASS and other storage systems.

#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#On the Transfer Files Page, Click on Manage Endpoints (on the top right).
#Click on Add an Endpoint (a drop down menu should appear)
#Fill out the Information
#*Basics
#**Endpoint Name (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Description = '''[Any Description Will Do - Ex. MSCOHEN-CASS]'''
#**Visible To: '''Private - Visible only to you'''
#**Default Directory: '''/~/'''
#*Identity Providers
#**(In the dropdown menu): '''MyProxy OAuth'''
#*Servers
#**Server Type: '''GridFTP'''
#**Server Domain (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Server Port: '''2811'''
#**Subject DN: '''[Your Subject DN - Ex. /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=1234567...]'''
# Click Create Endpoint.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Cluster_to_CASS|directions below.]]

==Transfer - Local to Cluster==
How to transfer files between the Hoffman2 Cluster and your local computer

#If you want to transfer files to or from your local desktop machine, start Globus Connect (Windows: gc.exe, Mac: Applications/Globus) on your local machine. A small status window will appear. When the connection has been made, the dot at the left of the connection will turn green. (MAKE SURE THE KEY IS ADDED - [[Hoffman2:Using_Globus#Globus_Connect_Software|Instructions Above.]]
#Point your browser at http://www.globus.org, click Sign In, and click Transfer Files. A web page with two Endpoint fields will display.
#In one Endpoint field, pull down the expand menu and select a site. If you are running Globus Connect, the first name in the list is your local desktop machine. When you select your local desktop machine, a list of your home directory files and directories will be displayed.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#When you select hoffman2#ucla a popup window will ask you for your myproxy server username and passphrase. Enter your Hoffman2 username and its password. Leave the Server DN field as it is. The default Lifetime value is 12 hours. If you are transferring a large amount of data, you may need to increase the Lifetime value. Click the Authenticate button.
#Your Hoffman2 Cluster home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.
You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

==Transfer - Cluster to CASS==
How to transfer files between the Hoffman2 Cluster and your Cloud Archival Storage Service
#Make Sure You have setup the CASS Endpoint. If you have not, go to the [[Hoffman2:Using_Globus# Adding_CASS_Endpoint|section above.]]
#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#In one Endpoint field, type in '''cass.idre.ucla.edu''' and you should see your '''[username#groupname].cass.idre.ucla.edu''' drop down - click on it.
#Authenticate with your CASS credentials and Click the Authenticate button.
#Your CASS home directory and its contents will display.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#Authenticate with your Hoffman2 credentials and Click the Authenticate button.
#Your Hoffman2 home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

Hoffman2:Using Globus

2014-05-25T06:49:26Z

Acho:

[[Hoffman2:Data_Transfer|Back to Hoffman2:Data_Transfer]]

This page contains many information on how to use Globus to move around files.

==Globus Connect Software (Local Desktop)==
If you want to transfer files to or from your local desktop machine, you need to download and install the Globus Connect software (one-time). You will need to do this step on each of your desktop machines whose files you want to transfer using Globus.

#Point your browser at http://www.globus.org and click Globus Connect. You will see a popup window in a web page. If you don’t see the popup window, click the Get Globus Connect link on that page.
#In Step One, click the button corresponding to your local platform (Mac OS, Linux or Windows) to download to your local desktop machine.
#In Step Two, enter an Endpoint Name to identify your local machine in the Endpoint Name field, (you can ignore the Description field), and click Generate Setup Key.
#Copy the setup key to some file and save it, since you will not see it again.
#Install the downloaded Globus Connect software on your local desktop.
#Run Globus Connect Installation (Windows: globus_connect_install.exe) for the initial setup, and paste in the setup key when prompted.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-_Local_to_Cluster|directions below.]]

==Adding a CASS Endpoint==
If you have a UCLA cloud archival storage system, you need to go through this process before moving files between your CASS and other storage systems.

#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#On the Transfer Files Page, Click on Manage Endpoints (on the top right).
#Click on Add an Endpoint (a drop down menu should appear)
#Fill out the Information
#*Basics
#**Endpoint Name (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Description = '''[Any Description Will Do - Ex. MSCOHEN-CASS]'''
#**Visible To: '''Private - Visible only to you'''
#**Default Directory: '''/~/'''
#*Identity Providers
#**(In the dropdown menu): '''MyProxy OAuth'''
#*Servers
#**Server Type: '''GridFTP'''
#**Server Domain (Substitute GROUP with your own Groupname) = '''[GROUP].cass.idre.ucla.edu'''
#**Server Port: '''2811'''
#**Subject DN: '''[Your Subject DN - Ex. /C=US/O=Globus Consortium/OU=Globus Connect Service/CN=1234567...]'''
# Click Create Endpoint.
#You should now be able to transfer files by following the [[Hoffman2:Using_Globus#Transfer_-Cluster_to_Cass|directions below.]]

==Transfer - Local to Cluster==
How to transfer files between the Hoffman2 Cluster and your local computer

#If you want to transfer files to or from your local desktop machine, start Globus Connect (Windows: gc.exe, Mac: Applications/Globus) on your local machine. A small status window will appear. When the connection has been made, the dot at the left of the connection will turn green. (MAKE SURE THE KEY IS ADDED ([[Hoffman2:Data_Transfer#Globus_Connect_software|Instructions Here.]])
#Point your browser at http://www.globus.org, click Sign In, and click Transfer Files. A web page with two Endpoint fields will display.
#In one Endpoint field, pull down the expand menu and select a site. If you are running Globus Connect, the first name in the list is your local desktop machine. When you select your local desktop machine, a list of your home directory files and directories will be displayed.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#When you select hoffman2#ucla a popup window will ask you for your myproxy server username and passphrase. Enter your Hoffman2 username and its password. Leave the Server DN field as it is. The default Lifetime value is 12 hours. If you are transferring a large amount of data, you may need to increase the Lifetime value. Click the Authenticate button.
#Your Hoffman2 Cluster home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.
You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

==Transfer - Cluster to CASS==
How to transfer files between the Hoffman2 Cluster and your Cloud Archival Storage Service
#Make Sure You have setup the CASS Endpoint. If you have not, go to the [[Hoffman2:Using_Globus# Adding_CASS_Endpoint|section below.]]
#Point your browser to http://www.globus.org and sign in.
#On the top right, Click on the Quick Links -> Transfer Files
#In one Endpoint field, type in '''cass.idre.ucla.edu''' and you should see your '''[username#groupname].cass.idre.ucla.edu''' drop down - click on it.
#Authenticate with your CASS credentials and Click the Authenticate button.
#Your CASS home directory and its contents will display.
#In the other Endpoint field, either pull down the expand menu and select hoffman2#ucla, or type in hoffman2#ucla.
#Authenticate with your Hoffman2 credentials and Click the Authenticate button.
#Your Hoffman2 home directory and its contents will display.
#To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.