Comsol

From ScientificComputing
Jump to: navigation, search

Category

Physics

Description

COMSOL Multiphysics is a finite element analysis, solver and simulation software / FEA software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. In addition to conventional physics-based user interfaces, COMSOL Multiphysics also allows entering coupled systems of partial differential equations (PDEs). The PDEs can be entered directly or using the so-called weak form.

Available versions (Euler, old software stack)

Legacy versions Supported versions New versions
4.4, 5.0, 5.1 5.2, 5.2.0.200, 5.2a, 5.3 5.3a, 5.4, 5.5

Please note that this page refers to installations from the old software stack. There are two software stacks on Euler. Newer versions of software are found in the new software stack.

Environment modules (Euler, old software stack)

Version Module load command Additional modules loaded automatically
4.4 module load legacy comsol/4.4
5.0 module load legacy comsol/5.0
5.1 module load legacy comsol/4.4
5.2 module load comsol/5.2
5.2.0.200 module load comsol/5.2.0.200
5.2a module load comsol/5.2a
5.3 module load comsol/5.3
5.3a module load new comsol/5.3a
5.4 module load new comsol/5.4
5.5 module load new comsol/5.5

Please note that this page refers to installations from the old software stack. There are two software stacks on Euler. Newer versions of software are found in the new software stack.

Interactive session

You can start the graphical user interface (GUI) of COMSOL in a batch interactive job with X11 forwarding. This can be useful to change your COMSOL preferences or to prepare a model or check the result of a COMSOL job. You can start an interactive job with the command
srun --pty --x11 [Slurm options] bash

Afterwards, you can load one of the COMSOL modules

module load comsol/5.2

and finally start the GUI with the comsol command.

comsol -np 1
If the COMSOL GUI runs slow through X11, then please have a look at the troubleshooting section of this wiki page. Don't start the COMSOL GUI on a login node, as login nodes only have few resources and are shared by many users. Following the proposed workflow for starting the GUI above prevents from starting COMSOL on a login node.

How to submit a job

You need to submit your COMSOL jobs through the batch system. For a single processor calculation a typical command could look like
 sbatch [Slurm options] --wrap="comsol batch -inputfile infile.mph -outputfile outfile.mph"

Here you need to replace [Slurm options] with Slurm parameters for the resource requirements of the job. Please find a documentation about the parameters of sbatch on the wiki page about the batch system.

Please note that if you don't specify the output file, then the results are written to the .mph file specified in the -inputfile parameter of Comsol.

For using the COMSOLBATCH licenses, you would need to add the -usebatchlic option:

 sbatch [Slurm options] --wrap="comsol batch -usebatchlic -inputfile infile.mph -outputfile outfile.mph"

Parallel jobs using shared memory

For parallel jobs using shared memory, you can specify the number of cores to be used with the -np option of COMSOL:

 sbatch --ntasks=1 --cpus-per-task=4 [Slurm options] --wrap="comsol batch -np 4 -inputfile infile.mph -outputfile outfile.mph"

Please make sure that the value of the -np option of COSMOL always has the same value as the product of --ntasks and --cpus-per-task. Please be careful to not use the -clustersimple option for shared memory jobs as this would make COMSOL start MPI processes instead of using threads.

If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).

Parallel jobs using distributed memory

If you would like to run COMSOL in distributed memory mode, then you need to add the -mpibootstrap slurm option to your COMSOL command:

 sbatch --ntasks=4 [Slurm options] --wrap="comsol batch -mpibootstrap slurm -inputfile infile.mph -outputfile outfile.mph"

Please be careful to not mix the -mpibootstrap slurm option with the -np option as this will start too many threads. The -mpibootstrap slurm option will tell COMSOL to get the required MPI information like hostnames etc. directly from the batch system.

If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).

Example

As an example for a Comsol job, we use the Plasmonic Wire Grating Analyzer which can be downloaded from the Comsol web site. It computes the coefficients of refraction, specular reflection, and first-order diffraction as functions of the angle of incidence for a plasmonic wire grating on a dielectric substrate.
[sfux@eu-login-03 comsol]$ ls
plasmonic_wire_grating.mph
[sfux@eu-login-03 comsol]$ module load comsol/6.1
[sfux@eu-login-03 comsol]$ sbatch --ntasks=1 --cpus-per-task=4 --mem-per-cpu=2g --tmp=8g --wrap="comsol batch -np 4 -autosave off -tmpdir \$TMPDIR -inputfile plasmonic_wire_grating.mph -outputfile output.mph"
Submitted batch job 39996633
[sfux@eu-login-03 comsol]$ squeue
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          39996633 normal.4h     wrap     sfux  R       0:10      1 eu-g5-003-2
[sfux@eu-login-03 comsol]$ squeue
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
[sfux@eu-login-03 comsol]$ ls
output.mph  output.mph.recovery  output.mph.status  plasmonic_wire_grating.mph  slurm-39996633.out
[sfux@eu-login-03 comsol]$ grep -A8 "Current Progress: 100 %" slurm-39996633.out
---------- Current Progress: 100 % - Solving linear system
Memory: 894/894 7336/7336
Solution time: 5 s.
Physical memory: 895 MB
Virtual memory: 7336 MB
Ended at Dec 14, 2023, 8:42:41 AM.
----- Stationary Solver 1 in Study 1/Solver 1 (sol1) -------------------------->
Running: Study 2
<---- Compile Equations: Frequency Domain in Study 2/Solver 2 (sol2) -----------
--
---------- Current Progress: 100 % - Solving linear system
Solution time: 5 s.
Physical memory: 1.08 GB
Virtual memory: 7.35 GB
Ended at Dec 14, 2023, 8:42:50 AM.
----- Stationary Solver 1 in Study 2/Solver 2 (sol2) -------------------------->
Run time: 20 s.
Saving model: /cluster/home/sfux/testrun/comsol/comsol/output.mph
Save time: 0 s.
--
---------- Current Progress: 100 % - Done
Memory: 935/1083 7345/7345
Comsollauncher: Cannot open display:
[sfux@eu-login-03 comsol]$
Comsol can create a large number of files in your home directory, therefore we recommend to have a look at the Comsol tutorial about redirecting files created by Comsol from your home directory to the local scratch.

Benchmark study

For evaluating the parallel efficiency of COMSOL, we performed a scaling study with a model having 150K mesh elements. The result of the study shows (at least for this model that requires a large amount of memory to run), that COMSOL simulations can be heavily memory bound and for certain cases going above 4 cores can be counterproductive.

For this study, we performed the simulation on 1, 2, 4, 6, 8, 12 and 24 cores. All runs were performed twice. The table below shows the run times and the speedup.


Cores runtime 1 [s] speedup 1 runtime 2 [s] speedup 2
1 26779 - 26811 -
2 15237 1.76 15193 1.76
4 6743 3.97 6719 3.99
6 11815 2.27 11752 2.27
8 13435 1.99 13421 2.00
12 19369 1.38 19365 1.38
24 18761 1.43 18667 1.44


In this case, the scaling is proportional to the number of memory channels of the CPU (and not to the number of CPU cores as one would expect). When using 4 cores, we see an almost linear scaling which is very close the best case. Using more than 4 cores then increases the run time substantially. Therefore, it is very important to perform a small scaling study (you could try 1, 2, 4 and 8 cores) to find the sweet spot in terms of number of cores for your COMSOL job.

License information

Commercial software. You need to purchase a license in the IT shop of ETH to use Comsol on the cluster.

Troubleshooting

In case of a problem with COMSOL on one of our clusters, please first contact the cluster support. We have very good relations to the COMSOL support and if we cannot solve a problem, we will open a ticket at the COMSOL support and add the user to this ticket, such that he also gets the answers from COMSOL support.

Problems with the COMSOL GUI

We noticed, that in some cases, the COMSOL 5.2 GUI updates very slowly when staring it on Euler or sometimes it even crashes. If you experience this kind of problem, then you can try to start the COMSOL GUI with the following option:

comsol -np 1 -3drend sw

If the problem still persists, the reason could be that you are on a login node, which is overloaded. We recommend in this case to log out and log in to the cluster again.

Finding the results of a simulation

When you run simulations solely through the Comsol GUI the results show up in the Model Builder Tab under Results and Datasets. When you run the simulation on Euler this will not be the case and the results can be called by clicking on Results in the upper menu bar and Press the More Datasets button and choose the Solution button under Base Datasets and the results should appear.

Notes

Please, do NOT use the comsol parameters -mpmode -nnhost and -mpi ...

Theoretically you can combine the -np and -nn parameters, but unless you

REALLY know what you are doing, we suggest that you avoid it.

Links

Using Comsol BATCH licenses

Comsol files and direcories in the users home directory
Comsol MATLAB LiveLink
Comsol workshop 10.12.2015
https://www.comsol.com/
https://www.comsol.com/support/knowledgebase/1001
https://www.comsol.com/blogs/how-to-run-simulations-in-batch-mode-from-the-command-line
https://www.comsol.com/blogs/how-to-run-on-clusters-from-the-comsol-desktop-environment
https://en.wikipedia.org/wiki/COMSOL_Multiphysics
https://www.youtube.com/user/COMSOLmultiphysics