# Anaconda Python module with ready set of scientific packages

The Python Lmod modules provide a convenient environment for working with different Python versions (3.9.18, 3.10.13, 3.11.7, 3.12.0), each equipped with pre-installed packages. These modules come pre-configured with both conda and pip environments, allowing users to seamlessly install additional packages using pip.

**Loading Python Lmod Modules**

To utilize the Python Lmod modules, load the desired version (for instance 3.10.3) with the following command:

```bash
module load fg-python/3.10.13
```

**Package Management**

* **Listing Installed Packages:** To view the packages installed in the current Python environment, use:

  ```
  pip list
  ```
* **Checking Package Availability:** Confirm if a specific package is available in the environment:

  ```bash
  pip show <package name>
  ```
* **Interactive Python Session:** Open an interactive Python session effortlessly:

  ```bash
  python
  ```
* **Installing Additional Packages:** Install a new package using pip:

  ```bash
  python -m pip install <package name>
  ```

**Jupyter Integration**

After loading the Python Lmod module, you can launch **JupyterLab** for an interactive working environment

```
jupyter lab
```

By default, JupyterLab will start in your current working directory (typically your home directory), which is where your notebooks should be saved.

#### Accessing data from `/finngen/red`

The FinnGen RED directory is **read-only** and intended for data access only. You do not need to copy data from `/finngen/red` into your home directory to use it in Jupyter.

To make `/finngen/red` available inside Jupyter, bind-mount it using Singularity:

```
export SINGULARITY_COMMAND_OPTS="-B /finngen/red"
```

You may bind-mount multiple directories by adding more `-B` options if needed.

After this, start JupyterLab **from a writable directory**, such as your home directory:

```
jupyter lab
```

or explicitly:

```
jupyter lab --notebook-dir="$HOME"
```

This setup allows you to:

* **Read data from `/finngen/red`** (read-only)
* **Save notebooks and outputs** in your home directory

#### Troubleshooting: Installed packages not visible in Jupyter

If you have installed Python packages after loading a Python module but they are not available in Jupyter notebooks, you may need to create an IPython kernel for that environment.

To create the kernel, run the following command:

```bash
python -m ipykernel install [--user] [--name <machine-readable-name>] [--display-name <"User Friendly Name">]
```

For example:

```bash
python -m ipykernel install --user --name python310
```

This will create a kernel named `python310` ,which can be selected inside JupyterLab and will have access to the packages installed in the loaded module. For more information, please visit the [IPython documentation on kernel installation](https://ipython.readthedocs.io/en/stable/install/kernel_install.html).

[Click here to visit the site with the full Jupyter official documentation.](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html)

**BigQuery Integration**

The following BigQuery packages are pre-installed:

* **Google Cloud BigQuery**
* **Pandas\_GBQ**

  Example command to check BigQuery data in the terminal:

  ```bash
  python -c 'from google.cloud import bigquery; import pandas_gbq; client = bigquery.Client(); query = """ SELECT FINNGENID FROM finngen-production-library.sandbox_tools_r10.finngen_r10_minimum_v1 LIMIT 10 """; df = pandas_gbq.read_gbq(query, project_id = client.project); print(df) '
  ```

  The above command retrieves 10 FINNGENIDs from the DF10 release minimum table in BigQuery.

**Visualization Packages**

The Python Lmod modules include the following visualization packages:

* **Plotly**
* **Matplotlib**
* **Seaborn**
* **UpSetPlot**

More visualization packages can be added based on your requirements.

**Working in Virtual Environments**

To activate virtual environments, shell into the container using alias commands that you can see when you do:

```bash
module spider <module name>
```

To create a virtual environment inside the `fg-python/3.10.13` Lmod module for instance, follow these steps:

1. Load the Python module:

   ```bash
   module load fg-python/3.10.13
   ```
2. Shell into the environment:

   ```bash
   fg-python-shell
   ```
3. Create a virtual environment named `my-venv`:

   ```bash
   python3 -m venv --system-site-packages my-venv
   ```
4. Activate the virtual environment:

   ```bash
   source my-venv/bin/activate
   ```
5. Install packages within the virtual environment:

   ```bash
   python -m pip install <package name>
   ```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.finngen.fi/working-in-the-sandbox/which-tools-are-available/anaconda-python-module-with-ready-set-of-scientific-packages.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.