Configuration on the new HPC infrastructure¶
In order to better support multiple computing groups around campus as well as to support a heterogenous compute cluster
with multiple CPU architectures, we have put the newly-installed software in a new location (/local/cqls/software),
see more information in the Software section.
We've also added some tools to make managing your environments and software a bit easier to remove barriers for your research projects.
Updating your shell configs (dotfiles)¶
The old dotfiles that your shell is sourcing are found in /local/cluster/etc, and the updated dotfiles are in
/local/cqls/etc. You can manually point to these files yourself, or you can run:
This will get your dotfiles updated so that the new software is preferred over the old software.
Allowing hpcman to help you manage your environment¶
The hpcman software was updated so that it can help you manage certain types of software installs, including R,
conda enviroments (using pixi), and python versions/environments (using uv). To get set up using this system:
Check out a node using salloc and then run these commands:
hpcman user enable
exec $SHELL
hpcman user setup prefix /nfs/dept/lab/home/user/opt # Change this to something appropriate for your lab and user
hpcman user setup pixi
hpcman user setup uv
hpcman user setup R_LIBS_USER
Log out and back in to see these changes.
Getting conda set up¶
For legacy conda envs that do not work well using pixi, you can set up conda by running:
This will get conda and mamba set up for you in bash. The new conda is a
miniforge install and does not use the
defaults channel from Anaconda. An activate.sh script for each CPU architecture
will be in /local/cqls/conda/$ARCH/scripts, and the CONDARC environment variable will point to a condarc file for
you for each architecture as well. users will have to run the hpcman user setup conda once per CPU arch to get the
proper settings files built (for now).
As of the writing of this page, only x86_64 is supported, but others will be supported soon.
Disabling hpcman from updating your environment¶
If you want to disable hpcman from setting environment variables for your environment, you can delete the
source $HOME/.config/hpcman/env.sh lines from your ~/.bashrc and ~/.cshrc.
Manually editing the hpcman-managed variables¶
The config files for hpcman are stored in $HOME/.config/hpcman by default. You can examine the env.sh for bash and
env.csh for tcsh, respectively. As of now, these files will remain synced, with the bash file env.sh taking
precedence, so any changes made to the env.csh file could be overwritten at any time.
Warning
If you edit the hpcman-managed files and then re-run 'hpcman user setup ...' your changes could be over-written at any time
Why can't I connect to the web from a node?¶
Your proxy settings may not be up-to-date. Please make sure your config files are updated and then check your proxy settings:
➜ printenv | grep -i 'https\?_proxy'
http_proxy=http://10.224.64.23:3128
https_proxy=http://10.224.64.23:3128
HTTP_PROXY=http://10.224.64.23:3128
HTTPS_PROXY=http://10.224.64.23:3128
And my Java tool can't connect to the web, either¶
You can set this environment variable before you run your Java code so it picks up the proxy: