Comsol

From ScientificComputing
Jump to: navigation, search

Category

Physics

Description

COMSOL Multiphysics is a finite element analysis, solver and simulation software / FEA software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. In addition to conventional physics-based user interfaces, COMSOL Multiphysics also allows entering coupled systems of partial differential equations (PDEs). The PDEs can be entered directly or using the so-called weak form.

Available versions

Legacy versions Supported versions New versions
4.4, 5.0, 5.1 5.2, 5.2.0.200, 5.2a, 5.3 5.3a, 5.4

Environment modules

Version Module load command Additional modules loaded automatically
4.4 module load legacy comsol/4.4
5.0 module load legacy comsol/5.0
5.1 module load legacy comsol/4.4
5.2 module load comsol/5.2
5.2.0.200 module load comsol/5.2.0.200
5.2a module load comsol/5.2a
5.3 module load comsol/5.3
5.3a module load new comsol/5.3a
5.4 module load new comsol/5.4

Interactive session

You can start the graphical user interface (GUI) of COMSOL on one of the login nodes of Euler. This can be useful to change your COMSOL preferences or to prepare a model or check the result of a COMSOL job. In order to start the GUI of COMSOL on a login node, you first need to establish an SSH connection with X11 forwarding enabled.
ssh -Y username@hostname

With username being your NETHZ username. Afterwards, you can load one of the COMSOL modules

module load comsol/5.2

and finally start the GUI with the comsol command.

comsol
If the COMSOL GUI runs slow through X11, then please have a look at the troubleshooting section of this wiki page.

How to submit a job

You need to submit your COMSOL jobs through the batch system. For a single processor calculation a typical command could look like
 bsub [LSF options] comsol batch -inputfile infile.mph -outputfile outfile.mph 

Here you need to replace [LSF options] with LSF parameters for the resource requirements of the job. Please find a documentation about the parameters of bsub on the wiki page about the batch system.

Parallel jobs using shared memory

For parallel jobs using shared memory, you can specify the number of cores to be used with the -np option of COMSOL:

 bsub -n 4 [LSF options] comsol batch -np 4 -inputfile infile.mph -outputfile outfile.mph 

Please make sure that the -n option ob bsub and the -np option of COSMOL always have the same value. Please be careful to not use the -clustersimple option for shared memory jobs as this would make COMSOL start MPI processes instead of using threads.

If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).

Parallel jobs using distributed memory

If you would like to run COMSOL in distributed memory mode, then you need to add the -clustersimple option to your COMSOL command:

 bsub -n 4 [LSF options] comsol batch -clustersimple -inputfile infile.mph -outputfile outfile.mph

Please be careful to not mix the -clustersimple option with the -np option as this will start too many threads.

If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).

Example

As an example for a Comsol job, we use the Plasmonic Wire Grating Analyzer which can be downloaded from the Comsol web site. It computes the coefficients of refraction, specular reflection, and first-order diffraction as functions of the angle of incidence for a plasmonic wire grating on a dielectric substrate.
[leonhard@euler02 comsol]$ ls
plasmonic_wire_grating.mph
[leonhard@euler02 comsol]$ module load comsol/5.2
[leonhard@euler02 comsol]$ bsub -n 4 -W 0:30 -R "rusage[mem=2048, scratch=2000]" "comsol batch -np 4 -autosave off -tmpdir \$TMPDIR -inputfile   plasmonic_wire_grating.mph -outputfile output.mph"
Generic job.
Job <26018628> is submitted to queue <normal.4h>.
[leonhard@euler02 comsol]$ bjobs
JOBID      USER        STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
26018628   leonhard    PEND  normal.4h  euler02                 *utput.mph Aug 29 09:47
[leonhard@euler02 comsol]$ bjobs
JOBID      USER        STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
26018628   leonhard    RUN   normal.4h  euler02     4*e1441     *utput.mph Aug 29 09:47
[leonhard@euler02 comsol]$ bjobs
No unfinished job found
[leonhard@euler02 comsol]$ ls
lsf.o26018628  output.mph  output.mph.bin  output.mph.status  plasmonic_wire_grating.mph
[leonhard@euler02 comsol]$ grep -A8 "Current Progress: 100 %" lsf.o26018628 
---------- Current Progress: 100 % - Solving linear system
Memory: 1062/1069 5666/5666
Stationary Solver 1 in Study 1/Solver 1 (sol1): Solution time: 0 s
                                                Physical memory: 1.06 GB
                                                Virtual memory: 5.67 GB
Run time: 42 s.
Saving model: /cluster/home/leonhard/comsol/output.mph
Save time: 0 s.
Total time: 55 s.
Comsol can create a large number of files in your home directory, therefore we recommend to have a look at the Comsol tutorial about redirecting files created by Comsol from your home directory to the local scratch.

Benchmark study

For evaluating the parallel efficiency of COMSOL, we performed a scaling study with a model having 150K mesh elements. The result of the study shows (at least for this model that requires a large amount of memory to run), that COMSOL simulations can be heavily memory bound and for certain cases going above 4 cores can be counterproductive.

For this study, we performed the simulation on 1, 2, 4, 6, 8, 12 and 24 cores. All runs were performed twice. The table below shows the run times and the speedup.


Cores runtime 1 [s] speedup 1 runtime 2 [s] speedup 2
1 26779 - 26811 -
2 15237 1.76 15193 1.76
4 6743 3.97 6719 3.99
6 11815 2.27 11752 2.27
8 13435 1.99 13421 2.00
12 19369 1.38 19365 1.38
24 18761 1.43 18667 1.44


In this case, the scaling is proportional to the number of memory channels of the CPU (and not to the number of CPU cores as one would expect). When using 4 cores, we see an almost linear scaling which is very close the best case. Using more than 4 cores then increases the run time substantially. Therefore, it is very important to perform a small scaling study (you could try 1, 2, 4 and 8 cores) to find the sweet spot in terms of number of cores for your COMSOL job.

License information

Commercial (centrally provided by IT shop)
https://www.comsol.de/products/licensing

Troubleshooting

In case of a problem with COMSOL on one of our clusters, please first contact the cluster support. We have very good relations to the COMSOL support and if we cannot solve a problem, we will open a ticket at the COMSOL support and add the user to this ticket, such that he also gets the answers from COMSOL support.

Problems with the COMSOL GUI

We noticed, that in some cases, the COMSOL 5.2 GUI updates very slowly when staring it on Euler or sometimes it even crashes. If you experience this kind of problem, then you can try to start the COMSOL GUI with the following option:

comsol -3drend sw
If the problem still persists, the reason could be that you are on a login node, which is overloaded. We recommend in this case to log out and log in to the cluster again.

Notes

Please, do NOT use the comsol parameters -mpmode -nnhost and -mpi ...

Theoretically you can combine the -np and -nn parameters, but unless you

REALLY know what you are doing, we suggest that you avoid it.

Links

Comso files and direcories in the users home directory

Comsol MATLAB LiveLink
Comsol workshop 10.12.2015
https://www.comsol.com/
https://en.wikipedia.org/wiki/COMSOL_Multiphysics
https://www.youtube.com/user/COMSOLmultiphysics