Bifrost¶

The massively parallel numerical code Bifrost for simulating stellar atmospheres from the convection zone to the corona.

Bifrost Article (arxiv)

Documentation pages¶

You can find guides and parameters list here: DOCS

Which branch to use?¶

If you inted to just run Bifrost, use & clone only master (or develop) branch with:

git clone --single-branch --branch master git@github.com:ITA-Solar/Bifrost.git

Tip

Rember to add your $HOME/.ssh/[???].pub key to GitHub to be able to clone repositories via ssh. You can find more about it here

If you plan to co-develop, or you are after the latest updates, default branch (develop) is for you, thus proceed with default:

git clone git@github.com:ITA-Solar/Bifrost.git

If you after some specific project ask your collaborator and she/he will point you to right branch.

Looking for specific version?¶

If you after some specific version (results in major papers has been produced with tagged version), check list of version with:

git tag -n

Then you can checkout to specific version with:

git checkout **tag**

e.g.,:

git checkout v2.2

Compilation¶

Compiling Bifrost is a straightforward process, and you can customize the compilation using the following commands:

make EXPERIMENT=experiment_name MACHINE=compiler_option

Here, experiment_name should be the name of the experiment setup file from $BIFROST/EXPERIMENTS, and compiler_option is a compiler setup file specific to your host, located in $BIFROST/OPSYS/MACHINES. The MACHINE argument is optional (but recommended), and if not specified, the Makefile will automatically look for the file compiler_option == hostname in the MACHINES sub-folder.

To print the name of the default compiler_option file for the current host, use:

make machine

For additional information about the compiler setup or machine specifics for a selected compiler_option, you can use:

make info MACHINE=`compiler_option`

Generic MACHINE steup¶

Bifrost supports various machine setups for well-known compilers. Below are examples for two common setups: intel for the oneAPI HPC package (Intel MPI + Intel ifort) and GNU for OpenMPI or MPICH compiled with gfortran.

make EXPERIMENT=experiment_name MACHINE=intel

This command uses the intel machine setup for the oneAPI HPC package.

make EXPERIMENT=experiment_name MACHINE=gnu

This command uses the gnu machine setup, suitable for OpenMPI or MPICH compiled with GCC.

For each compiler, three different versions of machine setups are available:

Full Optimized Executables:

make EXPERIMENT=experiment_name MACHINE=compiler_option

Debugging Version:

make EXPERIMENT=experiment_name MACHINE=compiler_option.dbg

Fast Executable with essential debugging flags:

make EXPERIMENT=experiment_name MACHINE=compiler_option.dbg.fast

Example: make EXPERIMENT=test MACHINE=gnu.dbg.fast (only essential dbg)

Choose the appropriate version based on your requirements for optimization and debugging.

Cleaning¶

To clean all temporary files created during compilation (such as *.o and *.mod files), use:

make clean

or

make deep_clean

which, executes make clean and additionally sets in all *.f90 files default value for $Id$ string.// Remember to call it before each git push

Additional compilation options¶

By default Bifrost runs in single precision IEEE::Float32, but is is possible to run any experiment in double procesion. For that define VARTYPE=vartype_dp e.g.,

make VARTYPE=vartype_dp EXPERIMENT=test MACHINE=intel

Order of stagger operations¶

Similarly you can change default order of stagger operations from 6th to 4th or 2nd by setting STAGGER_OP variable to stagger_4th or stagger_2nd e.g.,

make STAGGER_OP=stagger_4th EXPERIMENT=test MACHINE=intel

Structure of `$BIFROST` folder¶

.
├── BENCHMARK                   # SRC for benchmark code
├── BOUNDARIES                  # Folder with boundary modules
├── CUDA                        # CUDA tools for post-processing and visualization
├── EXPERIMENTS                 # Experiment files
├── EXTERNAL                    # dir for external sources // e.g., `fftpack`
├── EXTRAS                      # dir for extras sources code
├── IDL                         # IDL scripts for post-processing and utils
├── INPUT                       # Input files for `photo_tr_*` experiments
├── JULIA                       # Julia package for basic I/O and post-processing
├── OPSYS
│   └── MACHINES                # files with machine specific setup
├── PYTHON                      # Python scripts for visualization and post-processing
│   ├── BQ_t4_Look              # GUI tool for visualization
│   ├── Bifrost_to_art          # Script to translate Bifrost raw format to ART
│   ├── Bifrost_to_vtk          # Script to translate Bifrost raw to VTK
│   ├── Stagger
│   ├── TopoMaFiCuCo
│   ├── field_extrapolation     # script for field extrapolation
│   └── visualization_prepping  # script for quick interpolation to regular grids
├── SUPPORT                     # utis scripts
│   ├── auto_tests              # auto test foundation
│   ├── crc_hash.               # FORTRAN code to calculate crc hash keys
│   ├── git_hooks
│   ├── hpc_stats               # script for statistics extraction from HPC systems
│   ├── hpc_sync                # script for moving Bifrost files from HPC system
│   ├── mem_max                 # bash script to trace max mem
│   └── s2b_stagger_to_bifrost  # script to translate legacy Stagger snapshots to Bifrost format
├── TABGEN                      # Legacy Tabgen code to generate EOS table.
├── TESTS                       # Folder with Tests
├── UNITS                       # Units folder
└── docs                        # docs folder (majority in `*.md` format)

Setup for $Id string in files¶

Just copy-paste paste following lines in your shell while in the Bifrost base directory:

git config filter.gitid.smudge cat
git config filter.gitid.clean 'perl -pe "s/\\\$Id[^\\\$]*\\\$/\\\$Id\\\$/"'

Alternatively, if you do not have perl installed on your machine you can use sed:

git config filter.gitid.smudge cat
git config filter.gitid.clean 'sed "s/\\\$Id.*\\\$/\\\$Id\\\$/"'

If you are not sure which cleaning script is set up for your local repository, check $BIFROST/.git/config which should look like that:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
        ignorecase = true
        precomposeunicode = true
[remote "origin"]
        url = https://github.com/ITA-Solar/Bifrost.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master
[filter "gitid"]
        clean = sed \"s/\\\\\\$Id.*\\\\\\$/\\\\\\$Id\\\\\\$/\"
        smudge = cat

Git commits i.e., Mind your Git manners¶

The commit message is mainly for the other people, so they should be able to understand it now and six months later. Commit msg must only be one sentence and should start with a tag identifier (see end of this section).

Use the imperative form of verbs rather than past tense when referring to changes introduced by the commit in question. For example, "Remove property X", not "Removed property X" and not "I removed...".

Use following tags for commit msg (update: 29.08.2022, new tags adopted from Dispatch)

   [DEV] : code development (exc. additions and deletions)
   [ADD] : adding new feature
   [DEL] : removing files, routines
   [FIX] : fixes that occur during development, but which have essentially no impact on previous work
   [BUG] : bug with significant impact on previous work -- `grep`-ing should give restricted list
   [OPT] : optimisation
   [DBG] : debugging
   [ORG] : organisational, no changes to functionality
   [SYN] : typos and misspellings (including simple syntax error fixes)
   [DOC] : documentation only
   [REP] : repository related changes (e.g., changes in the ignore list, remove files)
   [UTL] : changes in utils (Python & IDL scripts)

Example commit msg:

[BUG] Adds missing initialisation to tg array
[FIX] Adds lowercase castig to params
[CLN] Removes unnecessary allocation for dynamic arrays.

You can find more about the Bifrost Git repository organisation here

Expanding Bifrost¶

Read develper notes for more details