> For the complete documentation index, see [llms.txt](https://docs.finngen.fi/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.finngen.fi/working-in-the-sandbox/which-tools-are-available/anaconda-python-module-with-ready-set-of-scientific-packages.md).

# Anaconda Python module with ready set of scientific packages

The Python Lmod modules provide a convenient environment for working with different Python versions (3.9.18, 3.10.13, 3.11.7, 3.12.0), each equipped with pre-installed packages. These modules come pre-configured with both conda and pip environments, allowing users to seamlessly install additional packages using pip.

**Loading Python Lmod Modules**

To utilize the Python Lmod modules, load the desired version (for instance 3.10.3) with the following command:

```bash
module load fg-python/3.10.13
```

**Package Management**

* **Listing Installed Packages:** To view the packages installed in the current Python environment, use:

  ```
  pip list
  ```
* **Checking Package Availability:** Confirm if a specific package is available in the environment:

  ```bash
  pip show <package name>
  ```
* **Interactive Python Session:** Open an interactive Python session effortlessly:

  ```bash
  python
  ```
* **Installing Additional Packages:** Install a new package using pip:

  ```bash
  python -m pip install <package name>
  ```

**Jupyter Integration**

After loading the Python Lmod module, you can launch **JupyterLab** for an interactive working environment

```
jupyter lab
```

By default, JupyterLab will start in your current working directory (typically your home directory), which is where your notebooks should be saved.

#### Accessing data from `/finngen/red`

The FinnGen RED directory is **read-only** and intended for data access only. You do not need to copy data from `/finngen/red` into your home directory to use it in Jupyter.

To make `/finngen/red` available inside Jupyter, bind-mount it using Singularity:

```
export SINGULARITY_COMMAND_OPTS="-B /finngen/red"
```

You may bind-mount multiple directories by adding more `-B` options if needed.

After this, start JupyterLab **from a writable directory**, such as your home directory:

```
jupyter lab
```

or explicitly:

```
jupyter lab --notebook-dir="$HOME"
```

This setup allows you to:

* **Read data from `/finngen/red`** (read-only)
* **Save notebooks and outputs** in your home directory

#### Troubleshooting: Installed packages not visible in Jupyter

If you have installed Python packages after loading a Python module but they are not available in Jupyter notebooks, you may need to create an IPython kernel for that environment.

To create the kernel, run the following command:

```bash
python -m ipykernel install [--user] [--name <machine-readable-name>] [--display-name <"User Friendly Name">]
```

For example:

```bash
python -m ipykernel install --user --name python310
```

This will create a kernel named `python310` ,which can be selected inside JupyterLab and will have access to the packages installed in the loaded module. For more information, please visit the [IPython documentation on kernel installation](https://ipython.readthedocs.io/en/stable/install/kernel_install.html).

[Click here to visit the site with the full Jupyter official documentation.](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html)

**BigQuery Integration**

The following BigQuery packages are pre-installed:

* **Google Cloud BigQuery**
* **Pandas\_GBQ**

  Example command to check BigQuery data in the terminal:

  ```bash
  python -c 'from google.cloud import bigquery; import pandas_gbq; client = bigquery.Client(); query = """ SELECT FINNGENID FROM finngen-production-library.sandbox_tools_r10.finngen_r10_minimum_v1 LIMIT 10 """; df = pandas_gbq.read_gbq(query, project_id = client.project); print(df) '
  ```

  The above command retrieves 10 FINNGENIDs from the DF10 release minimum table in BigQuery.

**Visualization Packages**

The Python Lmod modules include the following visualization packages:

* **Plotly**
* **Matplotlib**
* **Seaborn**
* **UpSetPlot**

More visualization packages can be added based on your requirements.

**Working in Virtual Environments**

To activate virtual environments, shell into the container using alias commands that you can see when you do:

```bash
module spider <module name>
```

To create a virtual environment inside the `fg-python/3.10.13` Lmod module for instance, follow these steps:

1. Load the Python module:

   ```bash
   module load fg-python/3.10.13
   ```
2. Shell into the environment:

   ```bash
   fg-python-shell
   ```
3. Create a virtual environment named `my-venv`:

   ```bash
   python3 -m venv --system-site-packages my-venv
   ```
4. Activate the virtual environment:

   ```bash
   source my-venv/bin/activate
   ```
5. Install packages within the virtual environment:

   ```bash
   python -m pip install <package name>
   ```