CFX

Please note that this application page is referring to the old CentOS software stack which is obsolete and does not work any more with the new Ubuntu setup. You can find an overview on the Ubuntu software stack on this wiki page.

Category

Engineering, Fluid dynamics

Description

ANSYS CFX software is a high-performance, general-purpose fluid dynamics program for solving wide-ranging fluid flow problems, e.g., related rotating machinery such as pumps, fans, compressors, and gas and hydraulic turbines.

Available versions (Euler, old software stack)

Legacy versions	Supported versions	New versions
14.5, 15.0	16.2, 17.0	18.0, 18.2

Please note that this page refers to installations from the old software stack. There are two software stacks on Euler. Newer versions of software are found in the new software stack.

Environment modules (Euler, old software stack)

Version	Module load command	Additional modules loaded automatically
14.5	module load legacy cfx/14.5	centos_cruft/6
15.0	module load legacy cfx/15.0	centos_cruft/6
16.2	module load cfx/16.2	legacy centos_cruft/6
17.0	module load cfx/17.0	legacy centos_cruft/6
18.0	module load new cfx/18.0
18.2	module load new cfx/18.2
19.5	module load new cfx/19.5

Please note that this page refers to installations from the old software stack. There are two software stacks on Euler. Newer versions of software are found in the new software stack.

How to submit a job

You can either submit your CFX job from the command line, using the cfx_submit_job script, or you can directly submit your jobs to the batch system from the CFX graphical user interface (GUI).

Submitting jobs from the command line

CFX jobs can be submitted from the command line, using the script cfx_submit_job.

[leonhard@euler01 ~]$ cfx_submit_job

USAGE: cfx_submit_job [-h|--help] [-n NCPUS] [-W HH:MM] [-M MBytes] [-J job_name] ...

   -h | --help : show usage information and exit
   -b          : batch system, can be either "Slurm" or "LSF"
   -n NCPUS    : number of cores for the job (default 1, maximum 96)
   -W HH:MM    : job run-time limit in hours and minutes (default 1 hour)
   -M MBytes   : memory required per core (default 2 GB)
   -J job_name : name of the job (jobs with the same name will be chained)
   -ptile N    : use N cores per node
   -wrap X     : wrap up at given time point or percent (X = HH:MM or %)
   -lsf "opts" : pass additional bsub options to the batch system (LSF)
   ...         : all subsequent arguments are passed varbatim to cfx5solve

The number of cores (NCPUS) and run-time limit (HH:MM) are optional. The default values are 1 core for one hour. The options -h or --help can also be used to print a brief description of the command's arguments. All other arguments are passed verbatim to cfx5solve. You can get a complete list of all supported options using the command

cfx5solve -h

The most important options are

-def {definition or results file}

-ini {initial values file}

Most other cfx5solve options — especially those relative to partitioning, parallel execution, hosts, etc. — should be avoided due to potential conflicts with the batch system on our clusters.

By default, the cfx_submit_job script will set the parameter -max-elapse-time for the cfx5solve to 80% of the runtime limit that is specified with the -W option to give the solver time to clean up before the batch system will terminate the CFX processes to avoid corrupted files. This value can be changed with the -wrap option.

Submitting jobs from the GUI

It is possible to directly submit CFX jobs out of the GUI to the batch system. When you would like to use this option, then you first need to login with X11 forwarding enabled, in order to start the CFX GUI.

ssh -Y username@hostname

where username is your ETH username and hostname is the hostname of the cluster you would like to use. Afterwards, you need to load the CFX module. This will set your PATH, MANPATH and other variables needed by CFX. At this point, you can start either CFX Launcher or CFX Solver using respectively

cfx5          ← start CFX Launcher, then select CFX Solver in the Tool menu

cfx5solve     ← start CFX Solver directly

When submitting job through the CFX GUI by choosing file->define run in the solver manager, one has the possibility to choose a run mode from a drop-down menu, that will define the resource requirements as the job length and the memory. For the job length, there are three options.

• 4 hours
• 24 hours
• 120 hours

For the memory, you can choose among three options

• M = 1999 MB/core
• L = 3999 MB/core
• XL = 6999 MB/core

For many jobs, M or L should be sufficient. Please only choose the XL option, when you are running a large simulation that requires a large amount of memory per core.

Example

As an example for a parallel CFX job, we are using the StaticMixer, provided by ansys.

[leonhard@euler01 ~]$ module load cfx/17.0
[leonhard@euler01 ~]$ ls -ltr
total 3728
-rw-r--r-- 1 leonhard T0000  212192 Sep 14 07:33 StaticMixer.def
-rw-r--r-- 1 leonhard T0000   20248 Sep 14 07:33 StaticMixer.pre 
[leonhard@euler01 ~]$ cfx_submit_job -n 4 -W 4:00 -M 2000 -def StaticMixer.def
CFX job.
Job <27426727> is submitted to queue <normal.4h>.
[leonhard@euler01 ~]$ bjobs
JOBID      USER        STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
27426727   leonhard    PEND  normal.4h  euler01                 cfx.leo    Sep 14 07:36
[leonhard@euler01 ~]$ bjobs
JOBID      USER        STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
27426727   leonhard    RUN   normal.4h  euler01     4*e1303     cfx.leo    Sep 14 07:36
[leonhard@euler01 ~]$ bjobs
No unfinished job found
[leonhard@euler01 ~]$ ls -ltr
total 3728
-rw-r--r-- 1 leonhard T0000  212192 Sep 14 07:33 StaticMixer.def
-rw-r--r-- 1 leonhard T0000   20248 Sep 14 07:33 StaticMixer.pre
-rw-r--r-- 1 leonhard T0000 3477687 Sep 14 07:38 StaticMixer_001.res
-rw-r--r-- 1 leonhard T0000   73530 Sep 14 07:38 StaticMixer_001.out
-rw-r--r-- 1 leonhard T0000    1218 Sep 14 07:38 cfx.o27426727
[leonhard@euler01 ~]$ grep "has finished" StaticMixer_001.out 
This run of the ANSYS CFX Solver has finished.

The log information of the batch system for this job is stored in the file cfx.o27426727.

License information

Commercial (centrally provided by IT shop)

Links

http://www.ansys.com/Products/Fluids/ANSYS-CFX

https://www.youtube.com/user/ansysinc/videos
https://en.wikipedia.org/wiki/Ansys#Fluid_Dynamics