Comsol

Description

Comsol is a multiphysics simulation software.

Available versions on Euler

On Euler the following versions are available through modules:

Version	Module command
6.2	module load comsol/6.2

License

Comsol licenses can be obtained through the IT shop of ETH

Interactive session

You can start the graphical user interface (GUI) of COMSOL in a interactive job with X11 forwarding with

module load comsol/6.2
srun --pty --x11 [Slurm options] bash

Then you can load a COMSOL modules

comsol -np 1

If the COMSOL GUI runs slowly, please have a look at the troubleshooting section of this wiki page. Don't start the COMSOL GUI on a login node, as login nodes have few resources and are shared by many users.

How to submit a serial/parallel job

You need to submit your COMSOL jobs through the batch system. For a single processor calculation a typical command could look like

 sbatch [Slurm options] --wrap="comsol batch -inputfile infile.mph -outputfile outfile.mph"

Here you need to replace [Slurm options] with Slurm parameters for the resource requirements of the job. Please find a documentation about the parameters of sbatch on the wiki page about the batch system.

Please note that if you don't specify the output file, then the results are written to the .mph file specified in the -inputfile parameter of Comsol.

For every COMSOL license, ETH also has access to a COMSOLBATCH licence. For using the COMSOLBATCH licenses, you would need to add the -usebatchlic option:

 sbatch [Slurm options] --wrap="comsol batch -usebatchlic -inputfile infile.mph -outputfile outfile.mph"

Parallel jobs using shared memory

For parallel jobs using shared memory, you can specify the number of cores to be used with the -np option of COMSOL:

 sbatch --ntasks=1 --cpus-per-task=4 [Slurm options] --wrap="comsol batch -np 4 -inputfile infile.mph -outputfile outfile.mph"

Please make sure that the value of the -np option of COSMOL always has the same value as the product of --ntasks and --cpus-per-task. Please be careful to not use the -clustersimple option for shared memory jobs as this would make COMSOL start MPI processes instead of using threads.

If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).

Parallel jobs using distributed memory

If you would like to run COMSOL in distributed memory mode, then you need to add the -mpibootstrap slurm option to your COMSOL command:

 sbatch --ntasks=4 [Slurm options] --wrap="comsol batch -mpibootstrap slurm -inputfile infile.mph -outputfile outfile.mph"

Please be careful to not mix the -mpibootstrap slurm option with the -np option as this will start too many threads. The -mpibootstrap slurm option will tell COMSOL to get the required MPI information like hostnames etc. directly from the batch system.

If you plan to run parallel COMSOL jobs, then we strongly recommend to first do a small scaling study to find out the sweet spot for the parallel efficiency of your model. Since COMSOL can be strongly memory bound, the sweet spot might be to use 4 cores (see scaling study below).

Parallel scaling

For evaluating the parallel efficiency of COMSOL, we performed a scaling study with a model having 150K mesh elements. The result of the study shows (at least for this model that requires a large amount of memory to run), that COMSOL simulations can be heavily memory bound and for certain cases going above 4 cores can be counterproductive.

For this study, we performed the simulation on 1, 2, 4, 6, 8, 12 and 24 cores. All runs were performed twice. The table below shows the run times and the speedup.

Cores	runtime 1 [s]	speedup 1	runtime 2 [s]	speedup 2
1	26779	-	26811	-
2	15237	1.76	15193	1.76
4	6743	3.97	6719	3.99
6	11815	2.27	11752	2.27
8	13435	1.99	13421	2.00
12	19369	1.38	19365	1.38
24	18761	1.43	18667	1.44

In this case, the scaling is proportional to the number of memory channels of the CPU (and not to the number of CPU cores as one would expect). When using 4 cores, we see an almost linear scaling which is very close the best case. Using more than 4 cores then increases the run time substantially. Therefore, it is very important to perform a small scaling study (you could try 1, 2, 4 and 8 cores) to find the sweet spot in terms of number of cores for your COMSOL job.

Troubleshooting

Opening a support request

Euler users can create an account on the Comsol support website and mention that they are from ETH, then the account will be connected to the Comsol license from ETH and you receive direct support from Comsol

Problems with the COMSOL GUI

We noticed, that in some cases, the COMSOL GUI updates very slowly when staring it on Euler or sometimes it even crashes. If you experience this kind of problem, then you can try to start the COMSOL GUI with the following option:

comsol -np 1 -3drend sw

Parameters for parallel jobs

Please, do NOT use the comsol parameters -mpmode -nnhost and -mpi ...

Theoretically you can combine the -np and -nn parameters, but unless you REALLY know what you are doing, we suggest that you avoid it.

Finding the results of a simulation

When you run simulations solely through the Comsol GUI the results show up in the Model Builder Tab under Results and Datasets. When you run the simulation on Euler this will not be the case and the results can be called by clicking on Results in the upper menu bar and Press the More Datasets button and choose the Solution button under Base Datasets and the results should appear.

Links

Comsol files and direcories in the users home directory
Comsol MATLAB LiveLink
https://www.comsol.com/
https://www.comsol.com/support/knowledgebase/1001
https://www.comsol.com/blogs/how-to-run-simulations-in-batch-mode-from-the-command-line
https://www.comsol.com/blogs/how-to-run-on-clusters-from-the-comsol-desktop-environment
https://www.youtube.com/user/COMSOLmultiphysics