- 1 Category
- 2 Description
- 3 Available versions
- 4 Environment modules
- 5 Interactive session
- 6 How to submit a job
- 7 Example
- 8 Benchmark study
- 9 License information
- 10 Troubleshooting
- 11 Notes
- 12 Links
DescriptionCOMSOL Multiphysics is a finite element analysis, solver and simulation software / FEA software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. In addition to conventional physics-based user interfaces, COMSOL Multiphysics also allows entering coupled systems of partial differential equations (PDEs). The PDEs can be entered directly or using the so-called weak form.
|Legacy versions||Supported versions||New versions|
|4.4, 5.0, 5.1||5.2, 184.108.40.206, 5.2a, 5.3||5.3a, 5.4|
|Version||Module load command||Additional modules loaded automatically|
|4.4||module load legacy comsol/4.4|
|5.0||module load legacy comsol/5.0|
|5.1||module load legacy comsol/4.4|
|5.2||module load comsol/5.2|
|220.127.116.11||module load comsol/18.104.22.168|
|5.2a||module load comsol/5.2a|
|5.3||module load comsol/5.3|
|5.3a||module load new comsol/5.3a|
|5.4||module load new comsol/5.4|
Interactive sessionYou can start the graphical user interface (GUI) of COMSOL on one of the login nodes of Euler. This can be useful to change your COMSOL preferences or to prepare a model or check the result of a COMSOL job. In order to start the GUI of COMSOL on a login node, you first need to establish an SSH connection with X11 forwarding enabled.
ssh -Y username@hostname
With username being your NETHZ username. Afterwards, you can load one of the COMSOL modules
module load comsol/5.2
and finally start the GUI with the comsol command.
comsolIf the COMSOL GUI runs slow through X11, then please have a look at the troubleshooting section of this wiki page.
How to submit a jobYou need to submit your COMSOL jobs through the batch system. For a single processor calculation a typical command could look like
bsub [LSF options] comsol batch -inputfile infile.mph -outputfile outfile.mph
Here you need to replace [LSF options] with LSF parameters for the resource requirements of the job. Please find a documentation about the parameters of bsub on the wiki page about the batch system.
For parallel jobs using shared memory, you can specify the number of cores to be used with the -np option of COMSOL:
bsub -n 4 [LSF options] comsol batch -np 4 -inputfile infile.mph -outputfile outfile.mph
Please make sure that the -n option ob bsub and the -np option of COSMOL always have the same value. Please be careful to not use the -clustersimple option for shared memory jobs as this would make COMSOL start MPI processes instead of using threads.
If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).
Parallel jobs using distributed memory
If you would like to run COMSOL in distributed memory mode, then you need to add the -clustersimple option to your COMSOL command:
bsub -n 4 [LSF options] comsol batch -clustersimple -inputfile infile.mph -outputfile outfile.mph
Please be careful to not mix the -clustersimple option with the -np option as this will start too many threads.If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).
ExampleAs an example for a Comsol job, we use the Plasmonic Wire Grating Analyzer which can be downloaded from the Comsol web site. It computes the coefficients of refraction, specular reflection, and first-order diffraction as functions of the angle of incidence for a plasmonic wire grating on a dielectric substrate.
[leonhard@euler02 comsol]$ ls plasmonic_wire_grating.mph [leonhard@euler02 comsol]$ module load comsol/5.2 [leonhard@euler02 comsol]$ bsub -n 4 -W 0:30 -R "rusage[mem=2048, scratch=2000]" "comsol batch -np 4 -autosave off -tmpdir \$TMPDIR -inputfile plasmonic_wire_grating.mph -outputfile output.mph" Generic job. Job <26018628> is submitted to queue <normal.4h>. [leonhard@euler02 comsol]$ bjobs JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME 26018628 leonhard PEND normal.4h euler02 *utput.mph Aug 29 09:47 [leonhard@euler02 comsol]$ bjobs JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME 26018628 leonhard RUN normal.4h euler02 4*e1441 *utput.mph Aug 29 09:47 [leonhard@euler02 comsol]$ bjobs No unfinished job found [leonhard@euler02 comsol]$ ls lsf.o26018628 output.mph output.mph.bin output.mph.status plasmonic_wire_grating.mph [leonhard@euler02 comsol]$ grep -A8 "Current Progress: 100 %" lsf.o26018628 ---------- Current Progress: 100 % - Solving linear system Memory: 1062/1069 5666/5666 Stationary Solver 1 in Study 1/Solver 1 (sol1): Solution time: 0 s Physical memory: 1.06 GB Virtual memory: 5.67 GB Run time: 42 s. Saving model: /cluster/home/leonhard/comsol/output.mph Save time: 0 s. Total time: 55 s.Comsol can create a large number of files in your home directory, therefore we recommend to have a look at the Comsol tutorial about redirecting files created by Comsol from your home directory to the local scratch.
Benchmark studyFor evaluating the parallel efficiency of COMSOL, we performed a scaling study with a model having 150K mesh elements. The result of the study shows (at least for this model that requires a large amount of memory to run), that COMSOL simulations can be heavily memory bound and for certain cases going above 4 cores can be counterproductive.
For this study, we performed the simulation on 1, 2, 4, 6, 8, 12 and 24 cores. All runs were performed twice. The table below shows the run times and the speedup.
|Cores||runtime 1 [s]||speedup 1||runtime 2 [s]||speedup 2|
In this case, the scaling is proportional to the number of memory channels of the CPU (and not to the number of CPU cores as one would expect). When using 4 cores, we see an almost linear scaling which is very close the best case. Using more than 4 cores then increases the run time substantially. Therefore, it is very important to perform a small scaling study (you could try 1, 2, 4 and 8 cores) to find the sweet spot in terms of number of cores for your COMSOL job.
License informationCommercial (centrally provided by IT shop)
TroubleshootingIn case of a problem with COMSOL on one of our clusters, please first contact the cluster support. We have very good relations to the COMSOL support and if we cannot solve a problem, we will open a ticket at the COMSOL support and add the user to this ticket, such that he also gets the answers from COMSOL support.
Problems with the COMSOL GUI
We noticed, that in some cases, the COMSOL 5.2 GUI updates very slowly when staring it on Euler or sometimes it even crashes. If you experience this kind of problem, then you can try to start the COMSOL GUI with the following option:
comsol -3drend swIf the problem still persists, the reason could be that you are on a login node, which is overloaded. We recommend in this case to log out and log in to the cluster again.
NotesPlease, do NOT use the comsol parameters -mpmode -nnhost and -mpi ...
Theoretically you can combine the -np and -nn parameters, but unless youREALLY know what you are doing, we suggest that you avoid it.