# Table of Contents - [Cineca Documentation — CINECA HPC Documentation 1.0 documentation](#cineca-documentation-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Getting Started — CINECA HPC Documentation 1.0 documentation](#getting-started-cineca-hpc-documentation-1-0-documentation) - [General Information — CINECA HPC Documentation 1.0 documentation](#general-information-cineca-hpc-documentation-1-0-documentation) - [Introduction HPC Resources — CINECA HPC Documentation 1.0 documentation](#introduction-hpc-resources-cineca-hpc-documentation-1-0-documentation) - [Scheduler and Job Submission — CINECA HPC Documentation 1.0 documentation](#scheduler-and-job-submission-cineca-hpc-documentation-1-0-documentation) - [File Systems and Data Management — CINECA HPC Documentation 1.0 documentation](#file-systems-and-data-management-cineca-hpc-documentation-1-0-documentation) - [Environment and Customization — CINECA HPC Documentation 1.0 documentation](#environment-and-customization-cineca-hpc-documentation-1-0-documentation) - [Software — CINECA HPC Documentation 1.0 documentation](#software-cineca-hpc-documentation-1-0-documentation) - [Known Issues — CINECA HPC Documentation 1.0 documentation](#known-issues-cineca-hpc-documentation-1-0-documentation) - [Cluster Specifics — CINECA HPC Documentation 1.0 documentation](#cluster-specifics-cineca-hpc-documentation-1-0-documentation) - [FAQ — CINECA HPC Documentation 1.0 documentation](#faq-cineca-hpc-documentation-1-0-documentation) - [Services and Tools — CINECA HPC Documentation 1.0 documentation](#services-and-tools-cineca-hpc-documentation-1-0-documentation) - [Introduction to HPC Cloud — CINECA HPC Documentation 1.0 documentation](#introduction-to-hpc-cloud-cineca-hpc-documentation-1-0-documentation) - [Operative Manual — CINECA HPC Documentation 1.0 documentation](#operative-manual-cineca-hpc-documentation-1-0-documentation) - [Cloud Specifics — CINECA HPC Documentation 1.0 documentation](#cloud-specifics-cineca-hpc-documentation-1-0-documentation) - [Tutorials and Gitlab repositories — CINECA HPC Documentation 1.0 documentation](#tutorials-and-gitlab-repositories-cineca-hpc-documentation-1-0-documentation) - [EUROfusion — CINECA HPC Documentation 1.0 documentation](#eurofusion-cineca-hpc-documentation-1-0-documentation) - [Tenants Administration — CINECA HPC Documentation 1.0 documentation](#tenants-administration-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [OpenStack Overview — CINECA HPC Documentation 1.0 documentation](#openstack-overview-cineca-hpc-documentation-1-0-documentation) - [Users and Accounts — CINECA HPC Documentation 1.0 documentation](#users-and-accounts-cineca-hpc-documentation-1-0-documentation) - [Access to the Systems — CINECA HPC Documentation 1.0 documentation](#access-to-the-systems-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Cineca-hpyc and Cineca-ai modules — CINECA HPC Documentation 1.0 documentation](#cineca-hpyc-and-cineca-ai-modules-cineca-hpc-documentation-1-0-documentation) - [Matlab — CINECA HPC Documentation 1.0 documentation](#matlab-cineca-hpc-documentation-1-0-documentation) - [QuantumESPRESSO — CINECA HPC Documentation 1.0 documentation](#quantumespresso-cineca-hpc-documentation-1-0-documentation) - [Pitagora — CINECA HPC Documentation 1.0 documentation](#pitagora-cineca-hpc-documentation-1-0-documentation) - [Leonardo — CINECA HPC Documentation 1.0 documentation](#leonardo-cineca-hpc-documentation-1-0-documentation) - [Galileo100 — CINECA HPC Documentation 1.0 documentation](#galileo100-cineca-hpc-documentation-1-0-documentation) - [Miniconda — CINECA HPC Documentation 1.0 documentation](#miniconda-cineca-hpc-documentation-1-0-documentation) - [What is Cloud Computing — CINECA HPC Documentation 1.0 documentation](#what-is-cloud-computing-cineca-hpc-documentation-1-0-documentation) - [Singularity and Apptainer Containers — CINECA HPC Documentation 1.0 documentation](#singularity-and-apptainer-containers-cineca-hpc-documentation-1-0-documentation) - [CINECA HPC Cloud Model — CINECA HPC Documentation 1.0 documentation](#cineca-hpc-cloud-model-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Budget and accounting — CINECA HPC Documentation 1.0 documentation](#budget-and-accounting-cineca-hpc-documentation-1-0-documentation) - [EFGW Gateway — CINECA HPC Documentation 1.0 documentation](#efgw-gateway-cineca-hpc-documentation-1-0-documentation) - [DNS guidelines — CINECA HPC Documentation 1.0 documentation](#dns-guidelines-cineca-hpc-documentation-1-0-documentation) - [Load Balancer — CINECA HPC Documentation 1.0 documentation](#load-balancer-cineca-hpc-documentation-1-0-documentation) - [Security guidelines — CINECA HPC Documentation 1.0 documentation](#security-guidelines-cineca-hpc-documentation-1-0-documentation) - [LoadBalancer: troubleshooting — CINECA HPC Documentation 1.0 documentation](#loadbalancer-troubleshooting-cineca-hpc-documentation-1-0-documentation) - [Store sensitive data — CINECA HPC Documentation 1.0 documentation](#store-sensitive-data-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Horizon Dashboard — CINECA HPC Documentation 1.0 documentation](#horizon-dashboard-cineca-hpc-documentation-1-0-documentation) - [Command Line Interface — CINECA HPC Documentation 1.0 documentation](#command-line-interface-cineca-hpc-documentation-1-0-documentation) - [Download RCM Software — CINECA HPC Documentation 1.0 documentation](#download-rcm-software-cineca-hpc-documentation-1-0-documentation) - [Infrastructure as a Code — CINECA HPC Documentation 1.0 documentation](#infrastructure-as-a-code-cineca-hpc-documentation-1-0-documentation) - [MEGARIDE — CINECA HPC Documentation 1.0 documentation](#megaride-cineca-hpc-documentation-1-0-documentation) - [GAIA — CINECA HPC Documentation 1.0 documentation](#gaia-cineca-hpc-documentation-1-0-documentation) - [Interactive Computing — CINECA HPC Documentation 1.0 documentation](#interactive-computing-cineca-hpc-documentation-1-0-documentation) - [Shares — CINECA HPC Documentation 1.0 documentation](#shares-cineca-hpc-documentation-1-0-documentation) - [Storage — CINECA HPC Documentation 1.0 documentation](#storage-cineca-hpc-documentation-1-0-documentation) - [Database — CINECA HPC Documentation 1.0 documentation](#database-cineca-hpc-documentation-1-0-documentation) - [Network — CINECA HPC Documentation 1.0 documentation](#network-cineca-hpc-documentation-1-0-documentation) - [Compute — CINECA HPC Documentation 1.0 documentation](#compute-cineca-hpc-documentation-1-0-documentation) - [LoadBalancer operations — CINECA HPC Documentation 1.0 documentation](#loadbalancer-operations-cineca-hpc-documentation-1-0-documentation) - [ADA — CINECA HPC Documentation 1.0 documentation](#ada-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Shares operations — CINECA HPC Documentation 1.0 documentation](#shares-operations-cineca-hpc-documentation-1-0-documentation) - [Database operations — CINECA HPC Documentation 1.0 documentation](#database-operations-cineca-hpc-documentation-1-0-documentation) - [Network operations — CINECA HPC Documentation 1.0 documentation](#network-operations-cineca-hpc-documentation-1-0-documentation) - [Storage operations — CINECA HPC Documentation 1.0 documentation](#storage-operations-cineca-hpc-documentation-1-0-documentation) - [Compute operations — CINECA HPC Documentation 1.0 documentation](#compute-operations-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [LoadBalancer: create — CINECA HPC Documentation 1.0 documentation](#loadbalancer-create-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Instance: create — CINECA HPC Documentation 1.0 documentation](#instance-create-cineca-hpc-documentation-1-0-documentation) - [Network: create — CINECA HPC Documentation 1.0 documentation](#network-create-cineca-hpc-documentation-1-0-documentation) - [Volume: create and attach — CINECA HPC Documentation 1.0 documentation](#volume-create-and-attach-cineca-hpc-documentation-1-0-documentation) - [Instance: delete — CINECA HPC Documentation 1.0 documentation](#instance-delete-cineca-hpc-documentation-1-0-documentation) - [Instance: snapshot download — CINECA HPC Documentation 1.0 documentation](#instance-snapshot-download-cineca-hpc-documentation-1-0-documentation) - [Key Pair: create — CINECA HPC Documentation 1.0 documentation](#key-pair-create-cineca-hpc-documentation-1-0-documentation) - [Instance: resize — CINECA HPC Documentation 1.0 documentation](#instance-resize-cineca-hpc-documentation-1-0-documentation) - [Instance: rescue — CINECA HPC Documentation 1.0 documentation](#instance-rescue-cineca-hpc-documentation-1-0-documentation) - [Image: upload — CINECA HPC Documentation 1.0 documentation](#image-upload-cineca-hpc-documentation-1-0-documentation) - [Instance: root storage increase — CINECA HPC Documentation 1.0 documentation](#instance-root-storage-increase-cineca-hpc-documentation-1-0-documentation) - [Volume: format and mount — CINECA HPC Documentation 1.0 documentation](#volume-format-and-mount-cineca-hpc-documentation-1-0-documentation) - [Instance: manage and monitor — CINECA HPC Documentation 1.0 documentation](#instance-manage-and-monitor-cineca-hpc-documentation-1-0-documentation) - [Security groups: create — CINECA HPC Documentation 1.0 documentation](#security-groups-create-cineca-hpc-documentation-1-0-documentation) - [Database: access — CINECA HPC Documentation 1.0 documentation](#database-access-cineca-hpc-documentation-1-0-documentation) - [Create and use a GENERIC_TYPE share — CINECA HPC Documentation 1.0 documentation](#create-and-use-a-generic-type-share-cineca-hpc-documentation-1-0-documentation) - [Floating IP: allocate and associate — CINECA HPC Documentation 1.0 documentation](#floating-ip-allocate-and-associate-cineca-hpc-documentation-1-0-documentation) - [Create and use a CEPHFS_TYPE share — CINECA HPC Documentation 1.0 documentation](#create-and-use-a-cephfs-type-share-cineca-hpc-documentation-1-0-documentation) - [Database: create — CINECA HPC Documentation 1.0 documentation](#database-create-cineca-hpc-documentation-1-0-documentation) - [Instance: snapshot create — CINECA HPC Documentation 1.0 documentation](#instance-snapshot-create-cineca-hpc-documentation-1-0-documentation) - [Volume: resize — CINECA HPC Documentation 1.0 documentation](#volume-resize-cineca-hpc-documentation-1-0-documentation) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) - [Unknown](#unknown) --- # Cineca Documentation — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html#) * Cineca Documentation * [View page source](https://docs.hpc.cineca.it/_sources/index.rst.txt) * * * ![_images/hdr.png](https://docs.hpc.cineca.it/_images/hdr.png) ![_images/spacer1.png](https://docs.hpc.cineca.it/_images/spacer1.png) ![_images/get_str.png](https://docs.hpc.cineca.it/_images/get_str.png) [Getting Started](https://docs.hpc.cineca.it/general/getting_started.html#get-str-card) * * * ![](https://docs.hpc.cineca.it/_images/gi.png) [General Information](https://docs.hpc.cineca.it/general/general_info.html#general-info-card) ![](https://docs.hpc.cineca.it/_images/hpc_s.png) [Introduction HPC Resources](https://docs.hpc.cineca.it/hpc/hpc_intro.html#hpc-card) ![](https://docs.hpc.cineca.it/_images/cloud_systems_icon.png) [Introduction to HPC Cloud](https://docs.hpc.cineca.it/cloud/general/general_info.html#cloud-card) ![](https://docs.hpc.cineca.it/_images/su.png) [EUROfusion](https://docs.hpc.cineca.it/specific_users/specific_users.html#spec-users-card) ![](https://docs.hpc.cineca.it/_images/st.png) [Services and Tools](https://docs.hpc.cineca.it/services/services_and_tools.html#serv-tools-card) ![](https://docs.hpc.cineca.it/_images/fqs.png) [FAQ](https://docs.hpc.cineca.it/faq.html#faq-card) * * * Cineca Documentation[](https://docs.hpc.cineca.it/index.html#cineca-documentation "Link to this heading") =========================================================================================================== Here you may find information regarding the usage of the Supercomputing Facilities hosted at CINECA. This site’s purpose is to provide actual and accurate information for all users of our facilities independently of the way they gain access. --- # Unknown .. User Guide documentation master file, created by sphinx-quickstart on Tue Nov 12 14:23:14 2024. You can adapt this file completely to your liking, but it should at least contain the root \`toctree\` directive. .. figure:: img/hdr.png :align: center :class: no-scaled-link .. figure:: img/spacer.png :align: center :class: no-scaled-link :height: 20px .. card:: :link: get\_str\_card :link-type: ref .. figure:: img/get\_str.png :align: center :class: no-scaled-link :height: 75px ------- .. grid:: 3 .. grid-item-card:: :img-background: img/gi.png :link: general\_info\_card :link-type: ref .. grid-item-card:: :img-background: img/hpc\_s.png :link: hpc\_card :link-type: ref .. grid-item-card:: :img-background: img/cloud\_systems\_icon.png :link: cloud\_card :link-type: ref .. grid:: 3 .. grid-item-card:: :img-background: img/su.png :link: spec\_users\_card :link-type: ref .. grid-item-card:: :img-background: img/st.png :link: serv\_tools\_card :link-type: ref .. grid-item-card:: :img-background: img/fqs.png :link: faq\_card :link-type: ref ------ Cineca Documentation ==================== Here you may find information regarding the usage of the Supercomputing Facilities hosted at CINECA. This site's purpose is to provide actual and accurate information for all users of our facilities independently of the way they gain access. .. toctree:: :maxdepth: 2 :caption: Start Here :hidden: general/getting\_started .. toctree:: :maxdepth: 2 :caption: General Information :hidden: general/general\_info general/users\_account general/access .. toctree:: :maxdepth: 2 :caption: HPC Clusters :hidden: hpc/hpc\_intro hpc/hpc\_data\_storage hpc/hpc\_scheduler hpc/hpc\_enviroment hpc/hpc\_clusters hpc/hpc\_software .. toctree:: :maxdepth: 2 :caption: HPC Cloud :hidden: cloud/general/general\_info cloud/os\_overview/index\_openstack\_overview cloud/operative/index\_operative\_manual cloud/systems/index\_system\_specifics cloud/tenant\_adm/index\_tenants\_administration cloud/tutorials/index\_tutorials\_and\_repos cloud/known\_issues .. toctree:: :maxdepth: 2 :caption: Specific Users :hidden: specific\_users/specific\_users .. toctree:: :maxdepth: 2 :caption: Services and Tools :hidden: services/services\_and\_tools .. toctree:: :maxdepth: 2 :caption: FAQ and Known Issues :hidden: faq --- # Getting Started — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Getting Started * [View page source](https://docs.hpc.cineca.it/_sources/general/getting_started.rst.txt) * * * Getting Started[](https://docs.hpc.cineca.it/general/getting_started.html#getting-started "Link to this heading") =================================================================================================================== **Welcome to CINECA HPC !!!** Here you will find in few simple steps the instructions to get your first access to CINECA HPC resources. **Let’s start !!!** 1\. Create your personal **User account** on [UserDB](https://userdb.hpc.cineca.it/) portal. * Visit the [How to become a User](https://docs.hpc.cineca.it/general/users_account.html#how-to-become-a-user) section for detailed information. * Please, consider that, once completed, the registration alone does not grant the access to the HPC resources. 2\. Get associated to a valid **Project Account**. * Visit the [Project Accounts](https://docs.hpc.cineca.it/general/users_account.html#project-accounts) section to find all the ways you can be granted a **Project Account** on our HPC resources. * If you are the _Principal Investigator (PI)_ of a project, please write to [superc@cineca.it](mailto:superc%40cineca.it) to be associated. * If you are a collaborator, please ask to the PI of the Project Account to be associated. * Visit the [PI and Collaborators](https://docs.hpc.cineca.it/general/users_account.html#pi-and-collaborators) section for more information. 3\. Submit a request to get access to CINECA HPC resources * by clicking on Submit button in HPC Access page on UserDB ([Submit a request to have a User Account](https://docs.hpc.cineca.it/general/users_account.html#submit-a-request-to-have-a-user-account) ) * Once enabled, we will provide you with a HPC username and a link to configure the 2FA (password + OTP token) 4\. Configure your 2FA * click on the link arrived via email and configure your HPC password and OTP token as described here [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems) 5\. Select the **Infrastracture** > * According to resources assigned to your **Project Account**, choose the HPC, or Cloud infrastructure. > HPC Infrastructure > 6\. Configure the **smallstep client** > > The smallstep client is needed to get the temporary 2FA certificate to access the cluster ([How to configure smallstep client](https://docs.hpc.cineca.it/general/access.html#how-to-configure-smallstep-client) > ) > > 7\. **Connect to the Cluster** > > Open a new shell/terminal and use the following commands to connect: > > $ step ssh login \--provisioner cineca-hpc > $ ssh @login..cineca.it > > Copy to clipboard > > Visit the page [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems) > to find instructions about other possible ways to connect > > Important > > You can login only to clusters where you have active budgets on it. > > 8\. **Managing Data Files** > > Get familiar with the FS areas and where they are located. On our HPC clusters there are several storage areas. The basic ones are: $HOME, $WORK and $SCRATCH, but sometimes there may be more. In section [File Systems and Data Management](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#file-systems-and-data-management) > you can find a description of all of them with their properties and limitations. > > Explore the **Enviroment** > > Check available software and/or compilers and pick-up the most convenient for your purpose. In the [Environment and Customization](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#environment-and-customization) > section is described how to check, load and get info of the available software, enviroment and compilers. > > 9\. Check your **budget status** > > Verify the status of your Project account budgets and their name with saldo command. A descrption of the saldo command and additional flags that may be needen in some cases are described in [Budget and Accounting](https://docs.hpc.cineca.it/hpc/hpc_intro.html#budget-and-accounting) > . > > 10\. **Submit your jobs** > > Execute your simulation on compute nodes of our HPC cluster by submitting a job script to our SLURM scheduler. In [Scheduler and Job Submission](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scheduler-and-job-submission) > you can find how to properly create a job script and to submit it. Cloud Infrastructure > 6\. Learn about **OpenStack** > > Follow the link for more information: [What is OpenStack?](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#what-is-openstack) > > 7\. Access the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) > > You can find the link to the dashboard and other useful information on the page specific to the cloud you have been assigned to: [ADA](https://docs.hpc.cineca.it/cloud/systems/ada.html#ada) > or [GAIA](https://docs.hpc.cineca.it/cloud/systems/gaia.html#gaia) > > Important > > You can only login on clouds where you have active budgets on. > > 8\. Setup your **project** > > Create your network stack and your first virtual machine instance: > > > * [Network: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/network_create.html#network-create) > > > > * [Security groups: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/secgroups_create.html#security-groups-create) > > > > * [Instance: create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#instance-create) > > > > Note > > Be sure to check our [Tenants Administration](https://docs.hpc.cineca.it/cloud/tenant_adm/index_tenants_administration.html#tenants-administration) > tips --- # General Information — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * General Information * [View page source](https://docs.hpc.cineca.it/_sources/general/general_info.rst.txt) * * * General Information[](https://docs.hpc.cineca.it/general/general_info.html#general-information "Link to this heading") ======================================================================================================================== This is the main section of Cineca HPC documentation. Here you will find all the information to common Cineca’s facilities. Access to the Systems [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-card) Users and Accounts [Users and Accounts](https://docs.hpc.cineca.it/general/users_account.html#users-card) HPC Service Desk[](https://docs.hpc.cineca.it/general/general_info.html#hpc-service-desk "Link to this heading") ------------------------------------------------------------------------------------------------------------------ Services provided by the User Support ca be found at [Service Desk](https://www.hpc.cineca.it/user-support/) website. Cite **CINECA** in scientific publications[](https://docs.hpc.cineca.it/general/general_info.html#cite-cineca-in-scientific-publications "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------ Users must acknowledge **CINECA** and the awarded resources in every published paper. Please use appropriate phrases depending on the origin of the awarded resource: Citation List > * **Awards granted by ISCRA @ LEONARDO:** We acknowledge ISCRA for awarding this project access to the LEONARDO supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CINECA (Italy) . > > * **Awards granted by EuroHPC @ LEONARDO:** We acknowledge the EuroHPC Joint Undertaking for awarding this project access to the EuroHPC supercomputer LEONARDO, hosted by CINECA (Italy) and the LEONARDO consortium through an EuroHPC \[Extreme/Regular/Benchmark/Development/…\] Access call. > > * **Awards granted by LEONARDO consortium countries @ LEONARDO:** We acknowledge \[Grant organization, consortium country\] for awarding this project access to the LEONARDO supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CINECA (Italy) and the LEONARDO consortium. > > * **Awards granted by ICSC @ LEONARDO:** We acknowledge the ICSC for awarding this project access to the EuroHPC supercomputer LEONARDO, hosted by CINECA (Italy). > > * **Awards granted by ISCRA @ G100 and ADA Cloud:** We acknowledge the CINECA award under the ISCRA initiative, for the availability of high performance computing resources and support. > > > To cite Leonardo architecture and the technologies adopted for its GPU-accelerated partition, please cite article: > > CINECA Supercomputing Centre, SuperComputing Applications and Innovation Department. (2024). “LEONARDO: A Pan-European Pre-Exascale Supercomputer for HPC and AI applications.”, Journal of large-scale research facilities, 8, A186. [DOI](https://doi.org/10.17815/jlsrf-8-186) > . --- # Introduction HPC Resources — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Introduction HPC Resources * [View page source](https://docs.hpc.cineca.it/_sources/hpc/hpc_intro.rst.txt) * * * Introduction HPC Resources[](https://docs.hpc.cineca.it/hpc/hpc_intro.html#introduction-hpc-resources "Link to this heading") =============================================================================================================================== This section provides a broader context for HPC Resources and the essential characteristics of HPC infrastructure. It introduces to CINECA Clusters and HPC services with specific focus on managment of resources by users together with the budgeting and accounting rules in place at CINECA for HPC projects. Budget and Accounting[](https://docs.hpc.cineca.it/hpc/hpc_intro.html#budget-and-accounting "Link to this heading") --------------------------------------------------------------------------------------------------------------------- The `saldo` command allows you to quickly retrieve information about your Project Account, including the available budget, and details of the User Account. More information about the usage of the tool can be gained just executing the command without any option. User Account Balance `saldo -b ` lists the budget of all Project Accounts associated with a username. > * A single **User Account** can be associated to a multiple **Project Accounts**. > > * For clusters with independent partitions, specify the partition using: > > * `saldo -b` (default: , on Leonardo give you back the report for Booster partition) > > * `saldo -b --dcgp` (to get a report for the DCGP partition on Leonardo, the flag `--dcgp` is mandatory) > > A typical example of `saldo` usage is reported in the following: saldo \-b ----------------------------------------------------------------------------------------------------------------------------------- account start end total localCluster totConsumed totConsumed monthTotal monthConsumed (local h) Consumed(local h) (local h) % (local h) (local h) ----------------------------------------------------------------------------------------------------------------------------------- Proj\_A 20110323 20300323 50000 25000 55027726 50.0 600 600 Proj\_B 20220427 20301231 100000 10000 27086 10.0 731 731 Proj\_C 20230524 20300323 6500 0 0 0.0 0 0 Copy to clipboard **Description of columns:** * **Account:** Refers to the Project Account (approved grants). * **Start Date:** Start of the grant period. * **End Date:** End of the grant period. * **Total Hours (local):** Total CPU hours allocated to the grant on the local cluster. * **Consumed (local):** Total CPU hours used from the allocation on the local cluster. * **Total Consumed (%):** Percentage of the total hours consumed. * **Month Total:** Allocated hours for the current month. * **Month Consumed:** Hours consumed in the current month. Project Account Balance `saldo -a ` lists the the usage of the **Project Account** for each associated **User Accunt**. The report specifies the date, consumed hours for each **User Account**, and the number of jobs submitted by the user on that day. saldo \-a ------------------Resources used from 202404 to 202412\------------------ date username account localCluster num.jobs Consumed/h ------------------------------------------------------------------------ 20240907 user001 example 5553:34:39 542 20240908 user001 example 22340:07:36 2676 20240909 user001 example 1606:21:39 154 20240910 user001 example 3210:42:40 285 ------------------------------------------------------------------------ Copy to clipboard ### Billing Policy[](https://docs.hpc.cineca.it/hpc/hpc_intro.html#billing-policy "Link to this heading") The billing policy outlines the methodology employed to calculate budget consumption associated with the use of HPC resources. We strongly recommend familiarizing yourself with this policy, as understanding it is crucial to avoiding unnecessary budget losses and to effectively planning your activities. Budget consumption is measured in **effective CPU hours (CPUh)** and is calculated based on the amount of resources allocated per node and the duration of their usage. Resource allocations are exclusive by default, meaning that once assigned, the reserved resources cannot be used by other users. **Formula for Billed Hours:** > BH\=T⋅N⋅R⋅C where: * _T_ = **elapsed time** (in hours). * _N_ = **number of nodes** allocated. * _R_ = **reserved resources per node** (explained below). * _C_ = **number of CPUs per node** (depends on node architecture). The _R_ factor measures the fraction of node resources reserved by a job that are consequently unavailable to other users. It is defined as the maximum among all reserved resource types (RES) — for example, the number of CPUs, GPUs, or memory — normalized by the total capacity of each respective resource on a single node: R\=maxr∈RES{Allocated(r)Total(r)} This billing model is designed to ensure that users are charged based on the proportion of a node’s resources made unavailable to others due to their job allocation. For example, if a job reserves all of a node’s RAM — even without utilizing all its CPUs — the node becomes unusable for other jobs and is therefore billed accordingly. Similarly, if all GPUs on a node are reserved, the node is considered substantially occupied, even if some CPUs or RAM remain available. Although GPU reservations do not entirely prevent node usage by others, GPUs are high-cost resources and nodes equipped with them are dedicated to workloads that can fully leverage their capabilities. Therefore, GPU usage is considered as critical as CPU usage when determining billing, even if the node is still partially usable. Example A user requests 1 node, 4 CPUs, 4 GPUs, and 3 hours of walltime on the Booster partition of Leonardo. However, the job runs for only 2 hours. From this information, we have: * T = 2 h (elapsed time) * N = 1 node * C = 32 CPUs (number of CPUs available on a Leonardo Booster compute node — see [Hardware Details](https://docs.hpc.cineca.it/hpc/leonardo.html#hardware-details) ) and, since: Allocated(CPU)Total(CPU)\=432\=0.125 Allocated(GPU)Total(GPU)\=44\=1.0 the maximum of the resources requested per node is determined by the GPUs, therefore _R_ = 1.0, and the billed hours are then calculated as: BH\=T⋅N⋅R⋅C\=2⋅1⋅1.0⋅32\=64CPUh This means the job consumes 64 effective CPU hours from the project’s budget. * * * Note * The **serial partition** is available for limited post-production data analysis and can be used even after a Project Account has expired. Usage of this partition is excluded from STDH billing (**free of charge**). * By default, the amount of memory allocated per node is proportional to the number of CPUs requested. * When nodes are requested in **exclusive mode** (see [Scheduler and Job Submission](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scheduler-and-job-submission) section), the entire node is reserved for the job, regardless of the specific resources requested. In such cases, the allocated resources may exceed the explicitly requested ones. * The **resources per node** are listed in the **Hardware Details** section for each cluster. Refer to the [Cluster Specifics](https://docs.hpc.cineca.it/hpc/hpc_clusters.html#cluster-specifics) section for the complete list of Cineca’s HPC systems. ### Budget Linearization[](https://docs.hpc.cineca.it/hpc/hpc_intro.html#budget-linearization "Link to this heading") A linearization policy governs the priority of scheduled jobs across Cineca clusters. To each Project Account is assigned a monthly quota (MQ) calculated as: MQ\=TB/NM TB = total assigned budget NM = total number of months Beginning on the first day of each month, any User Accounts belonging a Project Account may utilize their quota at full priority. As the budget is consumed, submitted jobs progressively lose priority until the monthly quota is exhausted. Subsequently, these jobs are still considered for execution but with reduced priority compared to accounts with remaining quota. This policy aligns with practices at other prominent HPC centers globally, aiming to enhance response times by aligning CPU hour usage with budget sizes. Note It’s recommended to adhere to a linearized usage of your budget, as non-linear consumption may impact the welfare of all users concurrently utilizing our HPC systems. A simple working scheme of budget linearization is showed in the figure below. [![../_images/bud_lin.png](https://docs.hpc.cineca.it/_images/bud_lin.png)](https://docs.hpc.cineca.it/_images/bud_lin.png) [![../_images/spacer2.png](https://docs.hpc.cineca.it/_images/spacer2.png)](https://docs.hpc.cineca.it/_images/spacer2.png) --- # Scheduler and Job Submission — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Scheduler and Job Submission * [View page source](https://docs.hpc.cineca.it/_sources/hpc/hpc_scheduler.rst.txt) * * * Scheduler and Job Submission[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scheduler-and-job-submission "Link to this heading") ======================================================================================================================================= **CINECA** HPC clusters are accessed via a dedicated set of login nodes. These nodes are intended for simple tasks such as customizing the user environment by installing applications, transferring data, and performing basic pre- and post-processing of simulation data. Access to the compute nodes is managed by the workload manager. To ensures fair access to resources for all users, production jobs must be submitted using a scheduler. **CINECA** uses Slurm (Simple Linux Utility for Resource Management) manager and batch system. Slurm is an open-source, highly scalable job scheduling system with three key functions: * Allocating access to resources (compute nodes) to users for a specified duration, allowing them to perform their work. * Providing a framework for starting, executing, and monitoring work (usually parallel jobs) on the set of allocated nodes. * Managing resource contention by handling the queue of pending jobs. * There are two main modes of using compute nodes: **Batch Mode:** This mode is intended for production runs. Users must prepare a shell script with all the operations to be executed once the requested resources are available. The job will then run on the compute nodes. Store all your data, programs, and scripts in the $WORK or $SCRATCH filesystems, as these are best for compute node access. You must have valid active projects to run batch jobs, and be aware of any specific policies regarding project budgets on our systems. **Interactive Mode:** Jobs submitted in this mode are similar to batch mode in that the user must specify the resources to allocate. The job is then managed like any other submitted job. The key difference from batch mode is that once the job is running, the user can interactively execute applications within the limits of the allocated resources. All allocated resources are available for the entire requested walltime (and consequently billed) during the submission process. Important * **Interactive** mode under SLURM has a different meaning compared to the common understanding of interactive execution of an application under a Linux shell or prompt. * **Interactive** execution of applications is allowed on compute nodes only via SLURM (see the next sections). * On login nodes, it is permitted to perform tasks such as data movement, archiving, code development, compilations, basic debugging, and very short test runs, provided these tasks do not exceed 10 minutes of CPU time and are free of charge under the current billing policy. * A comprehensive documentation of SLURM and some examples on how to submit your job is described in a separate section under this chapter, as well as on the original [SchedMD site](https://slurm.schedmd.com/documentation.html) . Basic Usage of Slurm[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#basic-usage-of-slurm "Link to this heading") ----------------------------------------------------------------------------------------------------------------------- With SLURM, you can specify the tasks you want to execute, and the system will manage running these tasks and returning the results to you. If the resources are not available, SLURM will hold your jobs and run them when resources become available. Typically, you create a **batch job**, which is a file (a shell script in UNIX) containing the set of commands you want to run. This file also includes `directives` that specify the job’s characteristics and resource requirements, such as the number of processors and CPU time needed. Once you create your job script, you can reuse it or modify it for subsequent runs. **Basic Workflow** * Create a job script with Slurm `directives`. * Submit the job using `sbatch`. * Monitor the job using commands like `squeue` and `scontrol`. * Cancel a job if needed with `scancel`. Here is a simple SLURM job script example to run a user’s application, setting a maximum wall time of one hour and requesting **1** node with **32** cores: #!/bin/bash #SBATCH --nodes=1 # 1 node #SBATCH --ntasks-per-node=32 # 32 tasks per node #SBATCH --time=1:00:00 # time limit: 1 hour #SBATCH --error=myJob.err # standard error file #SBATCH --output=myJob.out # standard output file #SBATCH --account= # project account #SBATCH --partition= # partition name #SBATCH --qos= # quality of service ./my\_application Copy to clipboard As shown in the example, a job requests resources through SLURM syntax. Resources can be allocated by including `directives` in the job script, or within the **interactive mode** via `sbatch` or `salloc` command. in a Once resources are allocated, the job can be executed. In the table below, a list of the main SLURM `directives` is reported. **Main Slurm Directives** | **Directive** | **Description** | **Example** | | --- | --- | --- | | `--job-name` | Stes the job name | `#SBATCH --job-name=my_job` | | `--output` | Specifies the output file | `#SBATCH --output=output.log` | | `--error` | Specifies the error file | `#SBATCH --error=error.log` | | `--time` | Sets the max execution time | `#SBATCH --time=01:00:00` | | `--partition` | Selects the partition | `#SBATCH --partition=compute` | | `--ntasks` | Nubmber of tasks | `#SBATCH --ntasks=1` | | `--cpus-per-task` | CPUs per task | `#SBATCH --cpus-per-task=4` | | `--mem` | Memory per node | `#SBATCH --mem=8GB` | | `--gres` | Specifies generic resources (e.g. GPUs) | `#SBATCH --gres=gpu:1` | | `--qos` | Quality of service (refer to specific clusters) | `#SBATCH --qos=` | | `--account` | Name of the project | `--account=` | How to prepare a script to submit Jobs[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#how-to-prepare-a-script-to-submit-jobs "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Serial Job This SLURM batch script is intended for running a serial (single-core) application on a Cineca’s HPC cluster. It requests one node and allocates a single CPU core to execute a task that does not require parallel processing. This setup is ideal for lightweight computations, preprocessing steps, or applications that are not parallelized. #!/bin/bash #SBATCH --job-name=serial\_job             # Descriptive name for the job #SBATCH --time=00:30:00                   # Maximum wall time (hh:mm:ss) #SBATCH --nodes=1                         # Request one node #SBATCH --ntasks=1                        # One task (process) total #SBATCH --cpus-per-task=1                 # One CPU core per task #SBATCH --partition=     # Partition (queue) to submit to #SBATCH --qos=                 # Quality of Service #SBATCH --mem=2G                          # Memory per node (adjust as needed) #SBATCH --output=serialJob.out           # File to write standard output #SBATCH --account=    # Project account number Copy to clipboard OpenMP Job This SLURM batch script is designed to run a pure OpenMP application on Cienca’s HPC clusters. It requests a single node and allocates all available physical CPU cores to a single task, making it ideal for shared-memory parallel programs. The script sets up the environment, loads the necessary modules, and configures OpenMP-specific variables to ensure optimal performance. It is tailored for systems without hyperthreading and can be easily adapted by modifying the number of CPUs per task and other resource parameters. #!/bin/bash #SBATCH --job-name=openmp\_job           # Job name #SBATCH --time=01:00:00                 # Walltime (hh:mm:ss) #SBATCH --nodes=1                       # Number of nodes #SBATCH --ntasks-per-node=1            # One MPI task per node #SBATCH --cpus-per-task=48             # Number of physical CPU cores per task (adjust to 32 for MARCONI100) #SBATCH --partition=   # Partition to submit to #SBATCH --qos=              # Quality of Service #SBATCH --mem=           # Memory per node (e.g., 128G) #SBATCH --output=myJob.out             # Standard output file #SBATCH --error=myJob.err              # Standard error file #SBATCH --account=    # Project account number \# Load required modules module load intel                      \# Load Intel compiler and libraries \# Set environment variables for OpenMP export SRUN\_CPUS\_PER\_TASK\=$SLURM\_CPUS\_PER\_TASK export OMP\_NUM\_THREADS\=$SLURM\_CPUS\_PER\_TASK  \# Set number of OpenMP threads \# Run the application using srun srun ./myprogram < myinput \> myoutput Copy to clipboard MPI Job For a typical MPI job you can take one of the following scripts as a template, and modify it depending on your needs. In this example we ask for 8 tasks, 2 SKL nodes and 1 hour of wallclock time, and runs an MPI application (myprogram) compiled with the intel compiler and the mpi library. The input data are in file “myinput”, the output file is “myoutput”, the working directory is where the job was submitted from. Through `–cpus-per-task=1` istruction each task will bind 1 physical cpu (core). This is a default option. #!/bin/bash #SBATCH --time=01:00:00 #SBATCH --nodes=2 #SBATCH --ntasks-per-node=4 #SBATCH --ntasks-per-socket=2 #SBATCH --cpus-per-task=1 #SBATCH --mem= #SBATCH --partition= #SBATCH --qos= #SBATCH --job-name=jobMPI #SBATCH --err=myJob.err #SBATCH --out=myJob.out #SBATCH --account= module load intel intelmpi srun myprogram < myinput \> myoutput Copy to clipboard GPU Job This SLURM batch script is designed to run a pure OpenMP application on Cienca’s HPC clusters. It requests a single node and allocates all available physical CPU cores to a single task, making it ideal for shared-memory parallel programs. The script sets up the environment, loads the necessary modules, and configures OpenMP-specific variables to ensure optimal performance. It is tailored for systems without hyperthreading and can be easily adapted by modifying the number of CPUs per task and other resource parameters. #!/bin/bash #SBATCH --job-name=multi\_gpu\_job # Descriptive job name #SBATCH --time=04:00:00 # Maximum wall time (hh:mm:ss) #SBATCH --nodes=4 # Number of nodes to use #SBATCH --ntasks-per-node=4 # Number of MPI tasks per node (e.g., 1 per GPU) #SBATCH --cpus-per-task=10 # Number of CPU cores per task (adjust as needed) #SBATCH --gres=gpu:4 # Number of GPUs per node (adjust to match hardware) #SBATCH --partition= # GPU-enabled partition #SBATCH --qos= # Quality of Service #SBATCH --output=multiGPUJob.out # File for standard output #SBATCH --error=multiGPUJob.err # File for standard error #SBATCH --account= # Project account number \# Load necessary modules (adjust to your environment) module load cuda/12.2 \# Load CUDA toolkit module load openmpi \# Load MPI implementation module load your\_app\_dependencies \# Load any other required modules \# Optional: Set environment variables for performance tuning export OMP\_NUM\_THREADS\=$SLURM\_CPUS\_PER\_TASK \# Set OpenMP threads per task export NCCL\_DEBUG\=INFO \# Enable NCCL debugging (for multi-GPU communication) \# Launch the distributed GPU application \# Replace with your actual command (e.g., mpirun or srun) srun \--mpi\=pmix ./my\_distributed\_gpu\_app \--config config.yaml Copy to clipboard Interactive Job Submission with SLURM[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#interactive-job-submission-with-slurm "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------- SLURM allows users to run jobs interactively using two main methods: `salloc` and `srun`. These methods are useful for debugging, testing, or running short tasks that require real-time interaction. ### Using `salloc`[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#using-salloc "Link to this heading") The `salloc` command is used to allocate resources (nodes, cores, GPUs, etc.) for an interactive session. Once the allocation is granted, you can run commands on the allocated compute nodes using `srun`. **Key Characteristics:** * The job is queued and scheduled like a batch job. * Once started, the terminal session is connected to the allocated resources. * Input/output/error streams are tied to your terminal. * You can exit the session using `exit` or `CTRL-D`. **Important Note:** Even though you’re in an interactive session, your shell prompt may still appear as if you’re on the login node. Any command not prefixed with `srun` will run on the login node, not the compute node. **Example:** salloc \-N 1 \--ntasks-per-node\=8 squeue \-u $USER \# Check if the allocation is ready hostname \# Runs on the login node srun hostname \# Runs on the allocated compute node exit \# Ends the interactive session Copy to clipboard **Tip:** You can also specify a command directly with `salloc`: salloc \-N 1 \--ntasks\=8 ./myscript.sh Copy to clipboard This will run the script on the allocated resources and return output to your terminal. ### Using `srun --pty`[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#using-srun-pty "Link to this heading") The `srun` command can also be used to start an interactive shell directly on the allocated compute node. **Syntax:** srun \-N 1 \--ntasks-per-node\=8 \--pty /bin/bash Copy to clipboard **Behavior:** * SLURM allocates the requested resources and launches a shell. * Any additional `srun` commands inside this shell may hang if no resources are left. * To allow multiple `srun` commands within the session, use the `--overlap` flag. **Recommendation:** While `srun --pty` is convenient, it is generally recommended to use `salloc` for interactive jobs, especially when you plan to run multiple commands or scripts within the session. **Summary** | **Method** | **Description** | | --- | --- | | `salloc` | Allocates resources and opens an interactive session. Use `srun` inside to run commands on compute nodes. | | `srun --pty` | Directly opens a shell on compute nodes. Use `--overlap` for multiple `srun` calls. | Monitoring Jobs[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#monitoring-jobs "Link to this heading") ------------------------------------------------------------------------------------------------------------- ### squeue Command Reference[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#squeue-command-reference "Link to this heading") The `squeue` command is used to display the status of jobs in a SLURM-managed cluster. It shows jobs that are pending, running, or recently completed. **Common Options** | **Option** | **Description** | | --- | --- | | `-u ` | Show jobs for a specific user. Example: `squeue -u alice` | | `-j ` | Show information for a specific job ID. Example: `squeue -j 123456` | | `-p ` | Filter jobs by partition (queue). Example: `squeue -p gpu` | | `-t ` | Filter jobs by state (e.g., `R` for running, `PD` for pending). | | `-o ` | Customize the output format. | | `--sort ` | Sort the output by specified fields. Example: `--sort=-t` to sort by time left. | | `--start` | Estimate job start times (useful for pending jobs). | | `--help` | Display help information for `squeue`. | **Example: Custom Output Format** To display a custom set of job details: squeue \-o "%.18i %.9P %.8j %.8u %.2t %.10M %.6D %R" Copy to clipboard This format shows: * Job ID * Partition * Job name * Username * State * Time used * Number of nodes * Reason (why pending or where running) `squeue` is a powerful tool for monitoring job status and diagnosing scheduling issues. Combine it with other SLURM commands like `sinfo` and `scontrol` for full cluster visibility. ### sinfo[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#sinfo "Link to this heading") The `sinfo` command provides information about the state of SLURM nodes and partitions. **Common Options:** | **Option** | **Description** | | --- | --- | | `-s` | Display a summary of node states. | | `-N` | Show information by node rather than by partition. | | `-p ` | Show information for a specific partition. | | `-o ` | Customize the output format. | **Example:** sinfo \-o "%P %D %t %C" Copy to clipboard This shows partition name, number of nodes, state, and CPU allocation. ### scontrol[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scontrol "Link to this heading") The `scontrol` command is used for querying and modifying SLURM configuration and job details. **Common Uses:** | **Command** | **Description** | | --- | --- | | `scontrol show job ` | Display detailed information about a specific job. | | `scontrol show node ` | Show detailed info about a specific node. | | `scontrol hold ` | Place a hold on a job to prevent it from starting. | | `scontrol release ` | Release a held job. | **Example:** scontrol show job 123456 Copy to clipboard This displays detailed job configuration, resource usage, and node assignment. ### scancel[](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scancel "Link to this heading") The `scancel` command is used to **cancel jobs** that are pending, running, or held in the SLURM job queue. It is useful for terminating jobs that are no longer needed or were submitted in error. **Common Options** | **Option** | **Description** | | --- | --- | | `scancel ` | Cancel a specific job by its job ID. | | `-u ` | Cancel all jobs belonging to a specific user. | | `-n ` | Cancel jobs by job name. | | `-p ` | Cancel jobs in a specific partition. | | `-t ` | Cancel jobs in a specific state (e.g.,\`\`PD\`\`,\`\`R\`\`). | | `--help` | Display help information for `scancel`. | **Examples** Cancel a specific job by ID: scancel 123456 Copy to clipboard Cancel all jobs for the current user: scancel \-u $USER Copy to clipboard Cancel all pending jobs in the GPU partition: scancel \-p gpu \-t PD Copy to clipboard Note * You must have permission to cancel the job (typically your own jobs). * Use with caution, especially when canceling multiple jobs at once. --- # File Systems and Data Management — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * File Systems and Data Management * [View page source](https://docs.hpc.cineca.it/_sources/hpc/hpc_data_storage.rst.txt) * * * File Systems and Data Management[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#file-systems-and-data-management "Link to this heading") ================================================================================================================================================== All HPC systems share the same logical disk structure and file system definition. In Cineca, all the filesystems are based on Lustre. The available storage areas can have multiple definitions/purposes: * **temporary**: data are accessible for a defined time window, after that data will be canceled. * **permanent**: data are accessible for additional six months after the _end_ of the project. Storage areas can be also: * **user specific**: each user has exclusive data area. * **shared**: area accessible by all _collaborators_ belonging the same project. * **open**: area accessible by all users of an HPC system. Note The available data areas are defined, on all HPC systems, through predefined `environment variables`. You can access on these areas simply using the name reported in the following table. Users hare strongly encouraged to use predefined `environment variables` instead of the full path (e.g: in scripts and codes data). | | | | | | | --- | --- | --- | --- | --- |Overview of Available Data Areas[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#id2 "Link to this table") | **Name** | **Area Attributes** | **Quota** | **Backup** | **Note** | | --- | --- | --- | --- | --- | | $HOME | permanent, user specific | 50 GB | daily | | | $WORK | permanent, shared | 1 TB | no | Large data to be shared with project’s collaborators. | | $FAST | permanent, shared | 1 TB | no | Only on [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo)
.

Faster I/O compared with outer areas. | | $SCRATCH | temporary, user specific | \-/20 TB | no | files older than 40 days

are deleted | | $TMPDIR | temporary, user specific | (-) | no | directory removed

at job completion | | $PUBLIC | permanent, open, user specific | 50 GB | no | Only on [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo)
. | | $DRES | permanent, shared | defined

by project | no | | Warning **Ethical Use of the SCRATCH Area** Users are encouraged to respect the intended use of the various areas. Users are reminded that the SCRATCH area is not subject to restrictions (quota) to facilitate the production of data, even large amounts. However, the SCRATCH area should not be used as a temporary storage area. Users are warned against using **“touch”** commands or similar methods to extend the retention of files beyond the 40-day limit. The use of such **“improper”** procedures will be monitored, and users will be subject to various degrees of _restrictions up to a ban_! Areas Details[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#areas-details "Link to this heading") ------------------------------------------------------------------------------------------------------------ $HOME **$HOME: permanent, user specific** $HOME is a local area where you are placed after the login procedure. It is where system, and user applications store their dot-files and dot-directories (`.nwchemrc`, `.ssh`, …) and where users keep initialization files specific for the systems (`.cshrc`, `.profile`, …). There is a $HOME area for each username on the machine. This area is conceived to store programs and small personal data. It has a quota of 50 GB. Files are never deleted from this area. Moreover, they are guaranteed by daily backups: if you delete or accidentally overwrite a file, you can ask our Help Desk to restore it. A maximum of 3 versions of each file is stored as a backup. The last version of the deleted file is kept for two months, then definitely removed from the backup archive. File retention is related to the life of the username; data are preserved until the username remains active. $WORK **$WORK: permanent,shared** $WORK is a scratch area for collaborative work within a given project. File retention is related to the life of the project. Files in $WORK will be conserved up to 6 months after the project end, and then they will be cancelled. Please note that there is no back-up in this area. This area is conceived for hosting large working data files since it is characterized by the high bandwidth of a parallel file system. It behaves very well when I/O is performed accessing large blocks of data, while it is not well suited for frequent and small I/O operations. This is the main area for maintaining scratch files resulting from batch processing. There is one $WORK area for each active project on the machine. The default quota is 1 TB per project, but extensions can be considered by the Help Desk if motivated. The owner of the main directory is the PI (Principal Investigator) of the project. All collaborators are allowed to read/write in there. Collaborators are advised to create a personal directory in $WORK for storing their personal files. By default, the personal directory will be protected (only the owner can read/write), but protection can be easily modified, for example by allowing write permission to project collaborators through `chmod` command. This second approach does not affect global files security. The `chprj` change project command makes it easier to manage the different $WORK areas for different projects. **Summary** * created when a project is opened. * each project has its own area. * all _collaborators_ can write in the area. * each user has as many $WORK areas as active projects. * by default files are private. * users can change file permission to make them visible, readable and writable to project’s collaborators. $FAST **$FAST: permanent, shared** ([Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo) only) $FAST is a scratch area for collaborative work within a given project. File retention is related to the life of the project. Files in $FAST will be conserved up to 6 months after the project end, and then they will be cancelled. Please note that there is **no back-up** in this area. This area is conceived for hosting working data files whenever the I/O operations constitute the bottleneck for your applications. It behaves well both when I/O is performed accessing large blocks of data, and for frequent and small I/O operations. Due to the limited size of the area, the main space for maintaining the data resulting from batch processing is the corresponding $WORK area. There is one $FAST area for each active project on the machine. The fixed quota is 1 TB per project, and due to the total dimension of the storage, extensions cannot be considered. The owner of the main directory is the PI (Principal Investigator) of the project. All collaborators are allowed to read/write in there. Collaborators are advised to create a personal directory in $FAST for storing their personal files. By default, the personal directory will be protected (only the owner can read/write), but protection can be easily modified, for example by allowing write permission to project collaborators through chmod command. This second approach does not affect global files security. $SCRATCH **$SCRATCH: temporary, user specific** This is a local temporary storage conceived for temporary files from batch applications. There are important differences with respect to $WORK area. It is user specific (not project specific). By default, file access is closed to everyone, in case you need less restrictive protections, you can set them with chmod command. On this area, a periodic cleaning procedure could be applied, with a normal retention time of 40 days: files are daily cancelled by an automatic procedure if not accessed for more than 40 days. In each user’s home directory ($HOME) a file lists all deleted files for a given day. CLEAN\_.log \= date when files were cancelled Copy to clipboard **Summary** * created when a user has granted access. * each user has it own area (exclusively). * files older than 40-days are cancelled. * no quota * by default files are public (read only). * user can change file permission to make files private. Warning Users are encouraged to respect the intended use of the various areas. Users are reminded that the SCRATCH area is not subject to restrictions (quota) to facilitate the production of data, even large amounts. However, the SCRATCH area should not be used as a temporary storage area. Users are warned against using “touch” commands or similar methods to extend the retention of files beyond the 40-day limit. The use of such “improper” procedures will be monitored, and users will be subject to various degrees of restrictions up to a ban. $TMPDIR **$TMPDIR: temporary, user specific** Each compute node is equipped with a local area whose dimension differs depending on the cluster. When a job starts, a **temporary area** is defined on the storage _local to each compute node_. * On **login nodes**: `TMPDIR=/scratch_local` * On **Galileo100**: `TMPDIR=/scratch_local/slurm_job.$SLURM_JOB_ID` * On **Leonardo**: `TMPDIR=/tmp` (visible with the command `df -h /tmp`). Special behavior can be found in the specific section [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo) . If more jobs share one node, each one will have a `private/tmp` in the job’s user space. The _TMPFS_ are removed at the end of each job (data will be deleted). Whatever the mechanism, the _TMPDIR_ can be used **exclusively** by the job’s owner. During a job, user can get access to the area with _local_ variable `$TMPDIR`. In a sbatch script, for example, user can move the input data of simulations to the `$TMPDIR` before the beginning of job run and also write on `$TMPDIR` job output. This would further improve the I/O speed of a code. Please note that the area is located on local disks, so it can be accessed only by the processes running on the specific node. For multinode jobs, if you need all the processes to access some data, please use the shared filesystems `$HOME`, `$WORK` and `$SCRATCH`. $PUBLIC **$PUBLIC: permanent, open, user specific** (LEONARDO ONLY) $PUBLIC is a shared area. Each username on the machine owns a $PUBLIC area with a quota of 50 GB. This area is accessible by every other user of the cluster. File retention is related to the life of the username; data are preserved until the username remains active. Please note that there is no back-up in this area. $DRES **$DRES: permanent, shared (among platforms and projects)** This is a repository area for collaborative work among different projects and across platforms. You need to explicitly ask for this kind of resource: it does not come as part of a project contact the user support. File retention is related to the life of DRES itself. Files in DRES will be conserved up to 6 months after DRES completion, then they will be cancelled. Several types of DRES are available, at present: * **FS**: normal filesystem access oh high throughput disks, shared among all systems (mounted only on login nodes). * **ARCH**: magnetic tape archiving with a disk-like interface via LTSFS. * **REPO**: smart repository on iRODS. **Summary** * created on request. * non linked to a specific project. * all collaborators can write in. * compute nodes **cannot** access to data in $DRES. * by default files are public (read only). * Quota based on needs. * No backup. Backup Policy and Data Availability[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#backup-policy-and-data-availability "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------- Daily backups guarantee the $HOME filesystem. In particular, the daily backup procedure preserves a maximum of three different copies of the same file. Older versions of files are kept for 1 month. The last version of deleted files is kept for 2 months, then definitely removed from the backup archive. Different agreements about Backup policies are possible. For more information contact the HPC support ([superc@cineca.it](mailto:superc%40cineca.it) ). Data, both backed up and non-backed up, are available for the entire duration of the project. After a project expires, users will still have full access to the data for an additional six months. Beyond this six-month period, data availability is not guaranteed. Important Users have **responsibility** to backup their important data !!! A scheme of data availability is reported in the figure below. [![../_images/file_time.png](https://docs.hpc.cineca.it/_images/file_time.png)](https://docs.hpc.cineca.it/_images/file_time.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) Lustre Best Practice[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#lustre-best-practice "Link to this heading") -------------------------------------------------------------------------------------------------------------------------- **Overview** Lustre is a parallel distributed filesystem ideal in handling large files accessed by many compute nodes. However, it struggles with small files and certain access patterns common in desktop and enterprise environments. Following best practices can minimize expensive operations and improve performance. **Key Recommendations** > * _Minimize Metadata Operations_ > > Avoid frequent access to file attributes (e.g., size, type, permissions) and commands like `ls -l`. Use simpler alternatives ( `ls` or `lfs` commands). > > * _Avoid Metadata-Intensive Commands_ > > Refrain from using commands like `ls -R` , `find` , `du` , and `df` . Instead, use Lustre-specific tools like `lfs find` . > > * _Limit Wildcard Usage_ > > Expanding wildcards (e.g., \* or ?) is resource-intensive, especially when matching many files in a large directory. For instance, commands like `rm *.tmp` can significantly degrade performance on Lustre. Instead, precompile a list of target files (e.g., `lfs find . -name "*.tmp" > files_to_delete.txt`) and process them iteratively. This method avoids the overhead of expanding wildcards directly on the filesystem. For large-scale operations, ensure scripts are designed to handle smaller batches of files to reduce the impact on metadata servers. > > * _Organize Files_ > > Avoid storing large numbers of files in a single directory. This creates contention as Lustre locks the parent directory when files are accessed, leading to performance bottlenecks. Use subdirectories to distribute files. A common approach is to create directories based on the square root of the total number of files. For example, 90,000 files could be split into 300 directories with 300 files each. Logical data grouping (e.g., by date or project) can further streamline access and maintenance. > > * _Avoid Small Files_ > > Accessing small files is inefficient. Where possible, combine them into larger files (e.g., using `tar`) or use formats like HDF5 or NetCDF. If the total size of the small files is manageable (e.g., a few GB), copy them to a local directory (/tmp) on the compute nodes at the start of a job and clean up afterward. Alternatively, create read-only disk images (e.g., ISO) that can be mounted via loopback. Tools like Singularity can facilitate this approach for containers. > > * _Minimize Repeated Operations_ > > Perform all I/O in a single session instead of frequent, small operations. For example, avoid operations such as appending small amounts of data repeatedly. Instead, open the file once, perform all operations in a single session, and close the file. For append-heavy workloads, consider buffering data in memory and writing it in larger chunks. > > * _Prevent File Access Contention_ > > Avoid multiple processes accessing the same file region or appending to the same file simultaneously. Use a single “master” process for such operations. Use strategies like file replication, splitting files, or delegating access to a single master process. Ensure processes access distinct file regions whenever possible. > > * _File Locking and Backups_ > > Use file locking (flock) only when necessary, as it can impact performance. Lustre generally manages non-overlapping writes and concurrent append operations effectively. Regularly back up data to a secure location, as Lustre does not provide built-in backup capabilities. > **File Striping** File striping is a method employed in Lustre to enhance data access and storage performance by distributing the contents of a single file across multiple storage devices, or **OSTs** (Object Storage Targets). Rather than storing a file as a single block on one device, striping breaks the file into smaller pieces, (or “_stripes_”) with each chunk written to a different device according to a set pattern. This approach helps increase throughput, improve parallelism, and reduce bottlenecks. The striping of a file can be defined by different parameters, the most important are: * the _Stripe Count_, which indicates the number of OSTs across which a file is distributed. A stripe count of 1 means the file is stored on a single OST, while higher values spread the file across multiple OSTs in a round-robin fashion. * the _Stripe Size_, which refers to the amount of data (typically in bytes) written consecutively to a single OST before moving to the next OST. Common defaults are around 1 MB, but it can range from 512 KB up to several GB. Choosing the right stripe size balances between overhead and parallelism. In the following scheme, different striping examples are reported: _File C_ has a larger _stripe size_ than _File A_, allowing more data per stripe. _File A_ is striped across three OSTs (_stripe count_ = 3), while _Files B_ and _C_ are stored on a single OST (_stripe count_ = 1). No space is reserved on the OST for unwritten data. ![../_images/striping_example.png](https://docs.hpc.cineca.it/_images/striping_example.png) The **Parallel File Layouts** (**PFL**) defines how this striping is applied to files (or directories). It allows to specify how files should be split across storage resources, setting the for example the _stripe count_ and _the stripe size_. The layout can vary based on different factors like **file size**, **filesystem**, and **hardware configuration**. For example, the PFL can be configured to limit the number of stripes for small files, while setting a higher stripe count for large files. In the following scheme, one **PFL** example is reported in details: ![../_images/pfl.png](https://docs.hpc.cineca.it/_images/pfl.png) The **PFL** consists of three _Components_, (defined by **range**: 0-2MB, 2MB-256MB and 256MB-EOF), mapping a 2055MB file. The first two components use a 1MB _stripe size_, while the third uses a 4MB _stripe size_. The stripe count increases with each component and accordingly with the file size, (Component 1: _stripe count_ =1, Component 2: _stripe count_ =4 and Component 3: _stripe count_ = 32). The first component of our 2055MB file, contains two 1MB blocks in a single 2MB object. The second component spans 254MB across four OST objects, each holding 64MB, with a 1MB hole in the first two objects. The third component distributes 1800MB across 32 OST objects, each holding 64MB, except obj 3,0 and obj 3,1, which contain extra chunks. Additional data would only expand component 3. Here are the default Parallel File Layouts (PFL) for all CINECA HPC systems utilizing Lustre. Note Since Lustre cannot know the final size of a file _a priori_, it starts creating it incrementally following the **PFL**. A file of many GB will have the first MB with striping rules similar to those of small files, then the rest like large files. Here are the default Parallel File Layouts (PFL) for all CINECA HPC systems utilizing Lustre. Leonardo | | | | | --- | --- | --- |Parallel File Layouts (PFL) for Leonardo.[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#id3 "Link to this table") | **Filesystem** | **PFL ranges** | **PFL Parameters** | | --- | --- | --- | | $HOME | File size = 64 kB - 10 GB

File size = 10 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB | | $WORK | File size = 64 kB - 10 GB

File size = 10 GB - 100 GB

File size = 100 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 2, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB | | $FAST | File size = 64 kB - 10 GB

File size = 10 GB - 100 GB

File size = 100 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 2, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB | | $SCRATCH | File size = 64 kB - 10 GB

File size = 10 GB - 100 GB

File size = 100 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 2, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB | | $PUBLIC | File size = 64 kB - 10 GB

File size = 10 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB | G100 | | | | | --- | --- | --- |Parallel File Layouts (PFL) for G100.[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#id4 "Link to this table") | **Filesystem** | **PFL ranges** | **PFL Parameters** | | --- | --- | --- | | $HOME | File size = 64 kB - EOF | Stripe count = 1, Stripe size = 1 MB | | $WORK | File size = 64 kB - 1 GB

File size = 1 GB - 4 GB

File size = 4 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB

Stripe count = -1, Stripe size = 1 MB | | $SCRATCH | File size = 64 kB - 1 GB

File size = 1 GB - 4 GB

File size = 4 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB

Stripe count = -1, Stripe size = 1 MB | Pitagora | | | | | --- | --- | --- |Parallel File Layouts (PFL) for Pitagora.[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#id5 "Link to this table") | **Filesystem** | **PFL ranges** | **PFL Parameters** | | --- | --- | --- | | $HOME | File size = 64 kB - 10 GB

File size = 10 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB | | $WORK | File size = 64 kB - 10 GB

File size = 10 GB - 100 GB

File size = 100 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB

Stripe count = -1, Stripe size = 1 MB | | $SCRATCH | File size = 64 kB - 10 GB

File size = 10 GB - 100 GB

File size = 100 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB

Stripe count = -1, Stripe size = 1 MB | | $PUBLIC | File size = 64 kB - 10 GB

File size = 10 GB - EOF | Stripe count = 1, Stripe size = 1 MB

Stripe count = 4, Stripe size = 1 MB | `Stripe count = -1` _means that Lustre shall use every OST available for striping the file, rather than a fixed number of OSTs._ **lfs setstripe command** `lfs setstripe` command is intended to create new files or directories with a specific PFL configuration. lfs setstripe \[\--size|\-s stripe\_size\] \[\--stripe-count|\-c stripe\_count\] \[\--component-end|\-E \] filename|dirname Copy to clipboard The most useful flags are: * `-E `: This flag sets the extent of file sizes that the following striping options should apply to. For example, -E 1G applies to files that are 1GB or smaller, -E 100G applies to files between 1GB and 100GB, and so on. * `-c `: Set the stripe count, which specifies how many Object Storage Targets (OSTs) the file will be striped across. For example, -c 1 means the file will be stored on 1 OST, -c 2 means the file will be stored on 2 OSTs, and so on. * `-S `: Set the stripe size, which specifies how large each stripe will be. For example, -S 1M means each stripe will be 1MB. **Examples** > * Create a file with striping on a single OST: This will ensure that myfile is stored entirely on a single OST, which is useful for small files. > > > `lfs setstripe -c 1 myfile` > > * Create a file with striping on 2 OSTs: This distributes the data of bigfile across two OSTs. It’s suitable for medium-sized files (between 10GB and 100GB). > > > `lfs setstripe -c 2 bigfile` > > * Create a directory with default striping: This sets the default striping for all files in /mydir to 2 OSTs. New files created in this directory will inherit this configuration. > > > `lfs setstripe -c 2 /mydir` > > * Set both the stripe count and stripe size: This command not only sets the number of OSTs (4) but also configures the stripe size to 4MB, which is great for optimizing performance on large files. A good stripe size for sequential I/O using high-speed networks is between 1 MB and 4 MB. > > > `lfs setstripe -c 4 -S 4M hugefile` > > * View the current striping configuration of a file: To check the current striping configuration of a file, use: > > > `lfs getstripe largefile.txt` > Data Occupancy Monitoring Tools[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#data-occupancy-monitoring-tools "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------ The occupancy status of all areas accessible to a user, along with the storage quota limits, can be monitored using a simple command available on all HPC cluster. There are two commands named `cindata` , `cinQuota`. For both commands the flag `-h` can be used to show the help. Both tools are available in the module cintools, which is automatically loaded in your environment. In the following, an example of `cindata` and `cinQuota` outputs is shown. cindata $ cindata USER AREADESCR AREAID FRESH USED QTA USED% aUSED aQTA aUSED% myuser00 /gpfs/work/ galileo\_work-Acc-name 9hou 114G \-- \--% 14T 30T 48.8% myuser00 /gpfs/scratch/ galileo\_scr 9hou 149G \-- \--% 341T 420T 81.2% myuser00 /galileo/home galileo\_hpc-home 9hou 5.7G 50G 11.4% 16T \-- \--% myuser00 /gss/gss\_work/DRES\_myAcc work\_OFFLINE-DRES\_myAcc-FS 9hou 2.9G \-- \--% 11T 15T 73.3% myuser00 /gss/gss\_work/DRES\_myAcc work\_ONLINE-DRES\_myAcc-FS 9hou 1.2T \-- \--% 2.8T 4T 70.0% Copy to clipboard Interpreting the storage status can be complex. Here’s a breakdown: > * **OFFLINE** area: this represents `DRES` data that has been stored on tape after three months of storage. > > * **ONLINE** area: this represents `DRES` data that is still in the filesystem or `ARCH` area. > The total storage quota assigned to your DRES is indicated by the aQTA parameter in the OFFLINE line. When the `DRES` is empty, the **ONLINE** value will be the same as **OFFLINE**. As files begin to be moved to tape, the **ONLINE** value will decrease, while the _aUSED_ parameter in **OFFLINE** will increase accordingly. This indicates that you have less space available for storing new data since some of the used space has been moved to tape. Similarly, deleting offline data will decrease the _aUSED_ parameter in OFFLINE and increase the _aQTA_ parameter in **ONLINE** by the same amount. Remember this formula: TOTALDRESSTORAGE\=aQTA−OFF\=aQTA−ON+aUSED−OFF cinQuota The additional tool for monitoring the disk occupancy is named `cinQuota` . A typical output of the command will contain the following information: $ cinQuota ----------------------------------------------------------------------------------------------------------------------------------- Filesystem used quota grace files ------------------------------------------------------------------------------------------------------------------------------------ /g100/home/userexternal/myuser00 22.66G 50G \- 194295 /g100\_scratch/userexternal/myuser00 1.955T 0k \- 41139 /g100\_work/ 366.3G 1T \- 548665 ----------------------------------------------------------------------------------------------------------------------------------- Copy to clipboard Manage File Permissions[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#manage-file-permissions "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------- As explained above, `$WORK` and `$DRES` are environmental variables automatically set in the user environment. > * $WORK variable points to a directory (fileset) specific for one of the user projects: `/gpfs/work/`. > > * $DRES variable points to space where all of the dres are defined: `/gss/gss_work/`. > > > * in order to use a specific DRES type, the path is `$DRES/`. > > > The owner of the root directory is the “Principal Investigator” (PI) of the project or the “owner” of the DRES, the group corresponds to the name of the project or the name of the DRES. Default permissions are: > own: rwx > > group: rwx > > other: \- > > Copy to clipboard in this way, all project’s collaborators sharing the same project _group_ can **read/write** into the **project/dres fileset**, whereas others users can not. Users are advise to create a personal _subdirectory_ under `$WORK` and `$DRES`. By default, files into the _subdirectory_ are private, but the owner can easily share the files with other collaborators by opening the _subdirectory_: > chmod 777 mydir > > chmod 755 mydir > > Copy to clipboard since the `$WORK/$DRES` fileset is closed to non-collaborators, the data sharing is active only among project’s collaborators. **Pointing $WORK to a different project: the chprj command** The user can modify the project pointed to by the variable `$WORK` using the `change project` command. To list all your accounts (both active or completed) and the default project: > chprj \-l > > Copy to clipboard To set `$WORK` to point to a different project: > chprj \-d > > Copy to clipboard More details are in the help page of the command: > chprj \-h > chprj \--help > > Copy to clipboard Data Transfer[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#data-transfer "Link to this heading") ------------------------------------------------------------------------------------------------------------ Users can use login nodes to transfer small files, but we **strongly** suggest to use the dedicated services: CINECA provides a data transfer service based on two main tools: **data movers** and **GridFTP**. **Data movers** are dedicated, containerized nodes without interactive access, supporting only a limited set of commands (scp, rsync, sftp, wget, curl, rclone, aws s3 and s3). User authentication is done via SSH certificates with 2-Factor Authentication or host-based authentication from within CINECA clusters. **GridFTP** is also available on these nodes but can only be used through the [globus-url-copy](https://gridcf.org/gct-docs/6.2/gridftp/user/index.html) client, which must be run from the user’s local machine. Further details on how to use these tools are provided in the two tabs below. Data Mover Datamovers are dedicated nodes on each HPC cluster that are designed for transferring data FROM/TO a cluster. **Hostnames and IPs** * **Galileo100**: alias: data.g100.cineca.it hostnames and IPs: * dmover1.g100.cineca.it - 130.186.16.212 * dmover2.g100.cineca.it - 130.186.16.213 * **Leonardo**: alias: data.leonardo.cineca.it hostnames and IPs: * dmover1.leonardo.cineca.it - 131.175.44.50 * dmover2.leonardo.cineca.it - 131.175.44.51 * dmover3.leonardo.cineca.it - 131.175.44.52 * dmover4.leonardo.cineca.it - 131.175.44.53 **Main features** This transfer service is containerized, and there are many differences between these nodes and the login nodes. First of all, on datamovers, there is no CPU time limit, that allows long data transfers. Unlike, on login nodes, there is a 10-minute of CPU time limit that usually interrupts the transfer of a large amount of data. By construction, the shell is not available, so it is not possible to open interactive sessions. In other words you cannot connect directly to the datamover via SSH. The only available commands are scp, rsync, sftp, wget, curl, rclone, s3 and aws s3. However, the authentication is still based on SSH protocol. There are only 2 possible authentication methods: * _publickey_: it only accepts valid SSH certificates, obtained via 2-Factor Authentication. No private/public keys generated in other ways are accepted. * _hostbased_: if you are already logged into a CINECA HPC cluster and try to use a datamover from a login node, the SSH daemon on the datamover recognizes you are already authenticated on a CINECA HPC cluster and that is enough. Warning The host-based authentication is not enabled inside a job batch. If you want to use a datamover inside a job batch you have to copy a valid 2FA SSH certificate inside your ~/.ssh directory on the cluster where you are submitting the job batch. Important When you are authenticated on a datamover, the environment variables $HOME, $WORK and $CINECA\_SCRATCH (as well as ~ or \* ) are not defined. This property has 2 side effects: 1. if you want to transfer files FROM/TO your cluster personal areas, you have to specify the absolute path of them. 2. You cannot make use of the SSH configuration files stored in your remote ~/.ssh/ directory (such as $HOME/.ssh/config). **Listing Directory via sftp** If you need to list files on a cluster where login nodes are offline, you can rely on datamover service via the sftp command: sftp @data..cineca.it:/path/to/be/listed/ Connected to data..cineca.it Changing to: /path/to/be/listed/ sftp> Copy to clipboard One entered the sftp session, the familiar pwd, cd /path/to/, ls commads are available to explore the remote filesystem, together with the sftp command lpwd, lcd /path/to/, lls. You can also transfer data from the sftp session, see the appropriate section below. **Available transfer tools** > * **rsync** > > > There are 2 possible ways to use rsync via datamovers: > > > > 1. You need to upload or download data FROM/TO your local machine TO/FROM a CINECA HPC cluster > > > > > > rsync \-PravzHS /absolute/path/from/file @data..cineca.it:/absolute/path/to/ > > rsync \-PravzHS @data..cineca.it:/absolute/path/from/file /absolute/path/to/ > > > > Copy to clipboard > > > > 2. You need to transfer files between 2 CINECA HPC clusters > > > > > > ssh \-xt @data..cineca.it rsync \-PravzHS /absolute/path/from/file @data..cineca.it:/absolute/path/to/ > > ssh \-xt @data..cineca.it rsync \-PravzHS @data..cineca.it:/absolute/path/from/file /absolute/path/to/ > > > > Copy to clipboard > > * **scp** > > > There are 3 possible ways to use scp via datamovers: > > > > 1. You need to upload or download data FROM/TO your local machine TO/FROM a CINECA HPC cluster > > > > > > scp /absolute/path/from/file @data..cineca.it:/absolute/path/to/ > > scp @data..cineca.it:/absolute/path/from/file /absolute/path/to/ > > > > Copy to clipboard > > > > 2. You need to transfer files between 2 CINECA HPC clusters > > > > > > ssh \-xt @data..cineca.it scp /absolute/path/from/file @data..cineca.it:/absolute/path/to/ > > ssh \-xt @data..cineca.it scp @data..cineca.it:/absolute/path/from/file /absolute/path/to/ > > > > Copy to clipboard > > > > 3. You need to transfer files between 2 CINECA HPC clusters using your local machine as a bridge. We strongly suggest not using this option because it has very low transfer performance, each file you move from one cluster to another will pass through your local machine > > > > > > scp \-3 @data..cineca.it:/absolute/path/from/file data..cineca.it:/absolute/path/from/file > > > > Copy to clipboard > > * **sftp** > > > There are 2 possible ways to use sftp via datamovers: > > > > 1. You need to upload or download data FROM/TO your local machine TO/FROM a CINECA HPC cluster > > > > > > sftp @data..cineca.it:/absolute/remote/path/to/ > > sftp> put relative/local/path/to/file > > Uploading /absolute/local/path/to/file to /absolute/remote/path/to/file > > file 100% 414 365.7KB/s 00:00 > > sftp> get relative/remote/path/to/file > > Fetching /absolute/remote/path/to/file to file > > file 100% 1455KB 19.0MB/s 00:00 > > sftp> > > > > Copy to clipboard > > > > 2. You need to transfer files between 2 CINECA HPC clusters > > > > > > ssh \-xt @data..cineca.it sftp @data..cineca.it:/absolute/path/to/ > > > > Copy to clipboard > > > > It is also possible to use the flag -b and execute sftp in batch mode. > > * **wget** > > > Sometimes, the 10-minute CPU time limit or the 4-hour wall time limit on the serial queue is not enough to download a large dataset for ML. In this case, you can use wget from the datamover. Here you can find a simple example > > > > ssh \-xt @data..cineca.it wget http://ftp.gnu.org/gnu/wget/wget2-2.0.0.tar.gz \-P /absolute/path/to/ > > > > Copy to clipboard > > > > Please note that is mandatory to use the flag -P with the absolute path of the destination folder, because of the fake /home directory. > > * **curl** > > > Sometimes, the 10-minute CPU time limit or the 4-hour wall time limit on the serial queue is not enough to download a large dataset for ML. In this case, you can use curl from the datamover. Here you can find a simple example > > > > ssh \-xt @data..cineca.it curl https://curl.se/download/curl-8.2.1.tar.gz \--output /absolute/path/to/curl-8.2.1.tar.gz > > > > Copy to clipboard > > > > Please note that is mandatory to use the flag –output with the absolute path of the destination file, because of the fake /home directory. > > * **rclone** > > > Rclone is a powerful tool that supports different transfer protocols, and a lot of data \[providers\]([https://rclone.org/#providers](https://rclone.org/#providers) > > ). At the moment it is available on Leonardo and Galileo100 datamovers. It needs a configuration file. If you are able, you car write the configuration file using your favourite editor (VIM) or you can rely on the rclone config command: > > > > ssh \-xt @data.leonardo.cineca.it rclone \--config /leonardo/home/userexternal//.rclone.conf config > > > > Copy to clipboard > > > > When your configuration is ready you can use rclone to manage data between Leonardo filesystem and the remote host you have configures. For example: > > > > ssh \-xt @data.leonardo.cineca.it rclone \--config /leonardo/home/userexternal//.rclone.conf copy /absolute/path/to/{file|directory} my\_remote: > > ssh \-xt @data.leonardo.cineca.it rclone \--config /leonardo/home/userexternal//.rclone.conf move my\_remote:{file|directory} /absolute/path/to/{file|directory} > > ssh \-xt @data.leonardo.cineca.it rclone \--config /leonardo/home/userexternal//.rclone.conf sync /absolute/path/to/directory my\_remote:remote/directory > > > > Copy to clipboard > > > > Please note that is mandatory to use the flag –config with the absolute path of the config file, because of the fake /home directory. > > * **aws s3** > > > AWS is the official command line tool from Amazon to manage s3 buckets. This command is available on Leonardo and Galileo100 datamovers and you can use only the s3 service, no other service are allowed at the moment. We discourage you to use ~/.aws/credentials and ~/.aws/config for two reasons: > > > > * for security reason, it is not a good idea writing secrets on a shared filesystem > > > > * there is a fake home on the datamover /home and the users cannot write inside any configuration file. > > > > > > We strongly suggest to define the environment variables AWS\_ACCESS\_KEY\_ID and AWS\_SECRET\_ACCESS\_KEY on your local computer and use the ssh option “SendEnv” to export them on the Leonardo datamovers. > > > > AWS\_ACCESS\_KEY\_ID\="" AWS\_SECRET\_ACCESS\_KEY\="" ssh \-xt \-o "sendEnv=AWS\_\*" @data.leonardo.cineca.it aws s3 ls s3:// > > AWS\_ACCESS\_KEY\_ID\="" AWS\_SECRET\_ACCESS\_KEY\="" ssh \-xt \-o "sendEnv=AWS\_\*" @data.leonardo.cineca.it aws s3 sync /absolute/path/to/ s3:// > > AWS\_ACCESS\_KEY\_ID\="" AWS\_SECRET\_ACCESS\_KEY\="" ssh \-xt \-o "sendEnv=AWS\_\*" @data.leonardo.cineca.it aws s3 cp s3:// /absolute/path/to/ > > > > Copy to clipboard > > * **s3** > > > On Leonardo and Galileo100 datamovers it is available also the s3 command from the libs3 system package. Here you can find the git repo, [https://github.com/bji/libs3](https://github.com/bji/libs3) > > . Since it is not possible to define environment variable on the datamover, it is mandatory to set the environment variable S3\_ACCESS\_KEY\_ID and S3\_SECRET\_ACCESS\_KEY and send these environment to the datamovers, using the ssh option “SendEnv=S3\_\*”. Our suggestion is to define this option in the local ssh\_config file. We strongly discourage to define these variable inside a file on the Leonardo filesystem, for security reason. > > > > Usage examples: > > > > ssh \-xt @data.leonardo.cineca.it s3 help > > S3\_ACCESS\_KEY\_ID\="" S3\_SECRET\_ACCESS\_KEY\="" ssh \-xt \-o "SendEnv=S3\_\*" @data.leonardo.cineca.it s3 test s3:// > > S3\_ACCESS\_KEY\_ID\="" S3\_SECRET\_ACCESS\_KEY\="" ssh \-xt \-o "SendEnv=S3\_\*" @data.leonardo.cineca.it s3 put s3:// filename\=/absolute/path/to/{file|directory} > > S3\_ACCESS\_KEY\_ID\="" S3\_SECRET\_ACCESS\_KEY\="" ssh \-xt \-o "SendEnv=S3\_\*" @data.leonardo.cineca.it s3 get s3:// filename\=/absolute/path/to/{file|directory} > > > > Copy to clipboard > GridFTP **Introduction** In this section, we shall provide an easy way to transfer data **to** and **from** any CINECA clusters using GridFTP protocol via [globus-url-copy](https://gridcf.org/gct-docs/6.2/gridftp/user/index.html) client. GridFTP is a highly efficient and robust protocol designed for transferring large volumes of data, significantly enhancing the standard FTP service by providing faster and more reliable transfers. It is widely used in large-scale scientific projects and supercomputing centers due to its ability to handle very large files securely and efficiently. Key features of GridFTP include: > * **Multiple simultaneous TCP streams**: Maximizes bandwidth utilization by allowing parallel downloads from multiple sources or striped/interleaved transfers. > > * **Partial file transfers**: Enables downloading specific portions of large files, useful for scientific data processing. > > * **Fault tolerance and automatic restart**: Resumes interrupted transfers from the last successful byte to improve reliability over unstable networks. > > * **Security integration**: Supports Grid Security Infrastructure (GSI), Kerberos, and SSH-based authentication, encryption, and data integrity. > > * **TCP buffer/window size negotiation**: Optimizes transfer speed and reliability based on file size and network conditions. > > * **Cluster-to-cluster transfers**: Uses multiple nodes at source and destination to increase transfer performance. > > * **Data channel reuse**: Avoids repeated connection setups when transferring multiple files between the same endpoints. > > * **Third-party control**: Allows secure initiation of transfers between remote sites without the client being directly involved in the data path. > The command-line utility used to perform GridFTP transfers is called `globus-url-copy`. Since 2018, the client software `globus-url-copy` can be installed via packages from the Grid Community Forum (GridCF), a global community supporting core grid software. GridCF maintains the Grid Community Toolkit (GCT), an open-source fork of the original Globus Toolkit developed by the Globus Alliance. Although GCT is derived from the Globus Toolkit, it is a distinct project, and GridCF operates independently from the Globus Alliance. Example usage: > * User Local PC <==> CINECA HPC Cluster > > * CINECA HPC Cluster A <==> CINECA HPC Cluster B > > * CINECA HPC Cluster <==> Other site HPC Cluster > **How to install standard client on your local workstation** The following instructions applies to both Debian/Ubuntu users and Windows users running WSL1 or WSL2. > sudo apt install globus-gass-copy-progs > > Copy to clipboard Otherwise, if you are a RedHat/Fedora user, execute the following command to install the client: > sudo dnf install globus-gass-copy-progs > > Copy to clipboard For detailed installation guidance, users are directed to the official Grid Community Toolkit documentation at [https://gridcf.org/gct-docs/](https://gridcf.org/gct-docs/) . **Authentication to the service** The authentication process is delegated to SSH, which manages secure user authentication through mechanisms such as public key cryptography and, in CINECA’s case, enhanced with two-factor authentication. For this reason, you have to generate the ssh certificate on your workstation using the step client, via: > step ssh login '' \--provisioner cineca-hpc > > Copy to clipboard For more info please refer to the page [How to configure smallstep client](https://docs.hpc.cineca.it/general/access.html#how-to-configure-smallstep-client) and [How to activate the 2FA and the OTP generator](https://docs.hpc.cineca.it/general/access.html#how-to-activate-the-2fa-and-the-otp-generator) . > Note > > For data transfers between a CINECA HPC cluster and an external HPC site, please ensure that the appropriate external access method is verified and properly configured. **Use the standard client** From the workstation with the ssh certificate, you may transfer data from CINECA HPC Cluster A to CINECA HPC Cluster B by using the standard client `globus-url-copy`. > globus-url-copy \-vb \-cd sshftp://@gftp..cineca.it:22/absolute/path/from/directory/ \\ > sshftp://@gftp..cineca.it:22/absolute/path/to/ > > Copy to clipboard In addition, you may list files in a specific cluster by the command: > globus-url-copy \-list sshftp://@gftp..cineca.it:22/absolute/path/from/directory/ > > Copy to clipboard > > Warning > > **Do not switch off the workstation during data transfer!** > > Client process is hosted on your workstation: switching it off, will kill data transfer process. You may also transfer data FROM/TO local machine TO/FROM a CINECA HPC cluster, via: > globus-url-copy \-vb \-cd /absolute/path/from/directory/ sshftp://@gftp..cineca.it:22/absolute/path/to/ > > globus-url-copy \-vb \-cd sshftp://@gftp..cineca.it:22/absolute/path/from/directory/ /absolute/path/to/ > > Copy to clipboard where can be: **g100** or **leonardo**. For more info about globus-url-copy command please refer to the official guide, or simply use the command line help: > globus-url-copy \-help > man globus-url-copy > > Copy to clipboard **GridFTP TCP Port Range configuration** Please note that GridFTP servers on our clusters are configured to use the port range 20000 - 25000 for the incoming and outgoing connections. Endianness[](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#endianness "Link to this heading") ------------------------------------------------------------------------------------------------------ Endianness is the attribute of a system that indicates whether integers are represented from left to right or right to left. At present, all clusters in Cineca are _“little-endian”_. --- # Environment and Customization — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Environment and Customization * [View page source](https://docs.hpc.cineca.it/_sources/hpc/hpc_enviroment.rst.txt) * * * Environment and Customization[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#environment-and-customization "Link to this heading") ========================================================================================================================================== The Software Catalog[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#the-software-catalog "Link to this heading") ------------------------------------------------------------------------------------------------------------------------ CINECA offers a variety of third-party applications and community codes that are installed on its HPC systems. Most of the third-party software is installed using software modules mechanism (see The module command section). Information on the available packages and their detailed descriptions are organized in a catalog, divided by discipline ([link](https://www.hpc.cineca.it/systems/software/) ). The catalog is also accessible directly on HPC clusters by using the commands `module` and `modmap` descrived in next sections. The module command[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#the-module-command "Link to this heading") -------------------------------------------------------------------------------------------------------------------- All softwares installed on the CINECA clusters are available as modules. As default, a set of basic modules are preloaded in the enviroment at login. To manage modules in the production enviroment, the user can execute the command module  with a variety of options. A short description of the most useful module command usage is reported in the following table. | **Command** | **Action** | | --- | --- | | module avail | show the available modules on the machine | | module load | load the module in the current shell session,

preparing the enviroment for the application. | | module load autoload | load the module and all dependencies in the current session | | module help | show specific information and basic help on the application | | module list | show the module currently loaded in the shell session | | module purge | unload all the loaded modules | | module unload | unload a specific module | | module av -a | show also the hidden modules available on the machine. These are modules usable but not guaranteed | | module load / | to load an hidden module you must specify its version | The modmap command[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#the-modmap-command "Link to this heading") -------------------------------------------------------------------------------------------------------------------- For an easy reading, the modules are collected in different profiles. Only the **base** profile is automatically loaded at login. `modmap` is a very useful command to look for a specific module in all the profiles at once. It shows at standard output all the modules with the searched name showing in wgicg profile they can be found. For example, suppose you are looking for the lammps software: $ modmap \-m lammps Profile: archive applications lammps 20220623\--openmpi--4.1.4--gcc--11.3.0-cuda-11.8 Profile: astro Profile: base Profile: bioinf Profile: chem-phys applications lammps 29aug2024 2aug2023 2aug2023--intel-oneapi-compilers--2023.2.1 Profile: deeplrn Profile: eng Profile: geo-inquire Profile: lifesc Profile: meteo Profile: quantum Profile: spoke7 Profile: statistics Copy to clipboard The output of modmap is showing that several lammps versions are present in the **chem-phys** profile and an old one in the **archive** profile. To load the module is now easy: $ module load profile/chem-phys $ module load lammps/29aug2024 Copy to clipboard Compilers[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#compilers "Link to this heading") -------------------------------------------------------------------------------------------------- You can check the complete list of available compilers on a specific cluster with the command: $ modmap \-c compilers Copy to clipboard For **GPU compilation** the available compilers are: * For **NVIDIA GPUs** cuda-aware * GNU Compilers Collection (GCC) * NVIDIA nvhpc (ex PGI) * NVIDIA cuda For **CPU compilation** the available compilers are: * For **INTEL CPUs** * Intel oneAPI compilers (x and classic compilers) * GNU Compilers Collection (GCC) * For **AMD CPUs** * AOCC compilers * GNU Compilers Collection (GCC) ### GCC[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#gcc "Link to this heading") **Serial** The GNU compilers are always available. A GCC version is available on the system (gcc –version ) without the need to load any module. In the module environment you can find more recent version though: $ modmap \-m gcc Copy to clipboard To use a specific version: $ module load gcc/ Copy to clipboard The name of the GNU compilers are: * **gfortran**: fully compliant with the Fortran 95 Standard and includes legacy F77 support * **gcc**: C compiler * **g++**: C++ compiler The gcc module loading set a specific environment variable for each compiler: * **CC**: gcc * **CXX**: g++ * **FC**: gfortran * **F90**: gfortran * **F77**: gfortran The documentation can be obtained with the “man” command after loading the gcc module: $ module load gcc/ Copy to clipboard On the **accelerated clusters** the available gcc modules support the offloading to the device. For NVIDIA GPUs the target is nvptx. On the **cluster provided with accelerated and non-accelerated partitions** that share the same modules environment the available offloading gcc modules can be used on both. As a result there is one only installation of a specific gcc version that supports the offload-device and you can use also on CPUs partition. **MPI wrappers** The **GCC OpenMPI** implementation is always available on accelerated and non accelerated clusters. The version installed for NVIDIA GPUs is configured to support CUDA, but you can use it also for partitions non accelerated of a cluster. In this case, however, it is **highly recommended** to compile with the MPI implementation specific for their architecture (e.g intel-oneapi-mpi module for INTEL CPUs). You can check the list of available OpenMPI modules on a specific cluster with the command: $ modmap \-m openmpi Copy to clipboard To use a specific one: $ module load openmpi/ Copy to clipboard After loading a specific GCC openmpi module select the MPI compiler wrapper for Fortran, C or C++ codes. * **mpicc**: gcc compiler mpi wrappers * **mpic++** **mpiCC** **mpicxx**: g++ compiler mpi wrappers * **mpif77** **mpif90** **mpifort**: gfortran compiler mpi wrappers e.g. Compiling C code: $ module load openmpi/ $ mpicc \-o myexec myprog.c Copy to clipboard ### NVIDIA nvhpc[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#nvidia-nvhpc "Link to this heading") (ex PORTLAND PGI + NVIDIA CUDA) **Serial** The NVHPC compilers are always available on the NVIDIA GPUs clusters. In the module environment you can find more recent version though: $ modmap \-m nvhpc Copy to clipboard To use a specific version: $ module load nvhpc/ Copy to clipboard The name of the NVHPC compilers are: * **nvc**: Compile C source files (C11 compiler. It supports GPU programming with OpenACC, and supports multicore CPU programming with OpenACC and OpenMP) * **nvc++**: Compile C++ source files (C++17 compiler. It supports GPU programming with C++17 parallel algorithms (pSTL) and OpenACC, and supports multicore CPU programming with OpenACC and OpenMP) * **nvfortran**: Compile FORTRAN source files (supports ISO Fortran 2003 and many features of ISO Fortran 2008. It supports GPU programming with CUDA Fortran and OpenACC, and supports multicore CPU programming with OpenACC and OpenMP) * **nvcc**: CUDA C and CUDA C++ compiler driver for NVIDIA GPUs As of August 5, 2020, the “PGI Compilers and Tools” technology is a part of the NVIDIA HPC SDK product, available as a free download from NVIDIA. For legacy reasons, the NVIDIA nvhpc suite also offers the PGI C, C++, and Fortran compilers with their original names, as follows. * **pgcc**: Compile C source files. * **pgc++**: Compile C++ source files. * **pgf77**: Compile FORTRAN77 source files. * **pgf90**: Compile FORTRAN90 source files. * **pgf95**: Compile FORTRAN95 source files. * **pgfortran**: Compile PGI Fortran The documentation can be obtained with the “man” command after loading the gcc module: $ module load nvhpc/ $ man nvc Copy to clipboard To enable CUDA C++ or CUDA Fortran, and link with the CUDA runtime libraries, use the -cuda option (-Mcuda is deprecated). Use the -gpu option to tailor the compilation of target accelerator regions. The OpenACC parallelization is enabled by the -acc flag. GPU targeting and code generation can be controlled by adding the -⁠gpu flag to the compiler command line. The OpenMP parallelization is enabled by the -mp compiler option. The GPU offload via OpenMP is enabled by the -mp=gpu option. **MPI wrappers** The **NVHPC MPI** implementation is always available on the clusters provided with NVIDIA gpus. The OpenMPI nvhpc version, if installed, is available as **openmpi/** module. The version built-in from NVIDIA is available within nvhpc installation as **hpcx-mpi/** module. You can check the list of available NVHPC OpenMPI/hpcx-mpi modules on a specific cluster with the command: $ modmap \-m openmpi OR hpcx-mpi Copy to clipboard To use a specific one: $ module load openmpi/ OR hpcx-mpi/ Copy to clipboard After loading a specific nvhpc openmpi module select the MPI compiler wrapper for Fortran, C or C++ codes. * **mpicc**: nvc compiler mpi wrappers * **mpic++** **mpiCC** **mpicxx**: nvc++ compiler mpi wrappers * **mpif77** **mpif90** **mpifort**: nvfortran compiler mpi wrappers e.g. Compiling C code: $ module load openmpi/ OR hpcx-mpi/ $ mpicc \-o myexec myprog.c (uses the nvc compiler) Copy to clipboard ### Intel oneAPI[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#intel-oneapi "Link to this heading") **Serial** The Intel compilers are the best choice on the Intel CPUs clusters. In the module environment you can find more recent version though: $ modmap \-m intel-oneapi-compilers Copy to clipboard To use a specific version: $ module load intel-oneapi-compilers/ Copy to clipboard Starting from 2021 version up to 2023 intel-oneapi-compilers module makes available two types of compilers, classic and oneAPI. Intel **classic** compilers: * **icc**: Compile C source files * **icpc**: Compile C++ source files * **ifort**: Compile FORTRAN source files LLVM-based Intel **oneAPI** compilers: * **icx**: Compile C source files * **icpx**: Compile C++ source files * **ifx**: Compile FORTRAN source files * **dpcpp**: Compile C++ source files with SYCL extensions Starting from 2024 version intel-oneapi-compilers module makes available only oneAPI compilers set and ifort classic compiler which is no longer available from 2025 version. In order to use the Intel classic compilers load: $ module load intel-oneapi-compilers-classic Copy to clipboard e.g. Compiling Fortran code with oneAPI: $ module load intel-oneapi-compilers/ $ ifx \-o myexec myprog.f90 Copy to clipboard **MPI wrappers** The Intel MPI implementation is the best choice on the Intel CPUs clusters. In the module environment you can find more recent version though: $ modmap \-m intel-oneapi-mpi Copy to clipboard To use a specific module: $ module load intel-oneapi-mpi/ Copy to clipboard This module makes available classic and oneAPI compilers wrappers. After loading a specific intel-oneapi-mpi module select the MPI compiler wrapper, classic or oneaAPI, for Fortran, C or C++ code. Intel **OneAPI** compilers wrappers: * **mpiicx** (C code) * **mpiicpx** (C++ code) * **mpiifx** (Fortran code) Intel **classic** compilers wrappers: * **mpiifort** (Fortran code) * **mpiicc** (C code) * **mpiicpc** (C++ code) Intel **GNU** compilers wrappers: * **mpifc**, **mpif77**, **mpif90** (Fortran MPI wrapper) * **mpicc** (C MPI wrapper) * **mpicxx**: (C++ MPI wrapper) e.g. Compiling C code: $ module load intel-oneapi-mpi/ $ mpiicx \-o myexec myprog.c Copy to clipboard ### AMD AOCC[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#amd-aocc "Link to this heading") **Serial** The AOCC compilers are available on the AMD CPUs clusters. In the module environment you can find more recent version though: $ modmap \-m aocc Copy to clipboard To use a specific version: $ module load aocc/ Copy to clipboard The AOCC compilers allow the development for x86 applications written in C, C++, and Fortran. AMD **AOCC** compilers: * **clang**: Compile C source files * **clang++**: Compile C++ source files * **flang**: Compile FORTRAN source files e.g. Compiling Fortran code with AOCC: $ module load aocc/ $ flang \[command line flags\] \-o myexec myprog.f90 Copy to clipboard AOCC compiler offers target-dependent and target-independent optimizations, with a particular focus on AMD “Zen” processors. You can read more about these in the command line option AMD section [https://docs.amd.com/r/en-US/57222-AOCC-user-guide/Command-line-Options](https://docs.amd.com/r/en-US/57222-AOCC-user-guide/Command-line-Options) **MPI wrappers** The **AOCC OpenMPI** implementation is available on AMD clusters. You can check the list of available OpenMPI modules on a specific cluster with the command: $ modmap \-m openmpi Copy to clipboard To use a specific one: $ module load openmpi/ Copy to clipboard After loading a specific AOCC openmpi module select the MPI compiler wrapper for Fortran, C or C++ codes. * **mpicc**: gcc compiler mpi wrappers * **mpic++** **mpiCC** **mpicxx**: g++ compiler mpi wrappers * **mpif77** **mpif90** **mpifort**: gfortran compiler mpi wrappers e.g. Compiling C code: $ module load openmpi/ $ mpicc \-o myexec myprog.c Copy to clipboard ### Basic MPI execution[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#basic-mpi-execution "Link to this heading") To test if your parallel executable works, you can execute it with mpirun on the login node and with a single process: module load mpirun ./myexec Copy to clipboard To run it in the parallel way you have to allocate the compute nodes via interactive job or sbatch job and execute it with mpirun or srun launcher . **Example:** 2 GPU compute nodes allocation and 2 tasks execution **via interactive job (salloc):** module load salloc \-N 2 \--ntasks-per-node\=1 \--cpus-per-task\=1 \--gres\=gpu:1 \-A \--time\= \--partition\= \--qos\= srun \-n 2 ./myexec Copy to clipboard **via interactive job (srun):** module load srun \-N 2 \--ntasks-per-node\=1 \--cpus-per-task\=1 \--gres\=gpu:1 \-A \--time\= \--partition\= \--qos\= \--pty /bin/bash mpirun \-n 2 ./myexec Copy to clipboard **via sbatch job:** sbatch my\_batch\_script.sh cat my\_batch\_script.sh #!/bin/sh #SBATCH --job-name osu #SBATCH -N2 --ntasks-per-node=1 #SBATCH --cpus-per-task=1 #SBATCH --gres=gpu:1 #SBATCH --time= #SBATCH --account= #SBATCH --partition= #SBATCH --qos= module load mpirun ./myexec or srun ./myexec Copy to clipboard ### Totalview[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#totalview "Link to this heading") This document introduces the user how to launch totalview through an [Access via Remote Visualization (RCM)](https://docs.hpc.cineca.it/general/access.html#access-via-remote-visualization-rcm) session. With respect to other GUIs that can be run on RCM, Totalview is a little peculiar and must be run directly on the nodes that execute the parallel code. In the following, we will detail how to establish a Totalview debugging session through RCM with a SLURM job. Once you have established a connection through RCM with one of our systems, GALILEO100 or Leonardo, please follow the instructions below. 1) Setup the .tvdrc file - only the first time > The first time you estabilish a Totalview session, a folder named .totalview will be created in your $HOME (it is not visible with the standard “ls” command, you have to add the flag -a for the hidden directories and files). Inside it, create a text file named .tvdrc, that should contain the following lines documented also in the [official Slurm manual](https://slurm.schedmd.com/faq.html#totalview) > : > > dset \-set\_as\_default TV::bulk\_launch\_enabled true > dset \-set\_as\_default TV::bulk\_launch\_string {srun \--mem-per-cpu\=0 \-N%N \-n%N \-w\`awk \-F. 'BEGIN {ORS=","} {if (NR==%N) ORS=""; print $1}' %t1\` \-l \--input\=none %B/tvdsvr%K \-callback\_host %H \-callback\_ports %L \-set\_pws %P \-verbosity %V \-working\_directory %D %F} > dset \-set\_as\_default TV::bulk\_launch\_tmpfile1\_host\_lines {%R} > > Copy to clipboard 2) Prepare the job (job.sh script) and submit it Example `job.sh` for GALILEO100: > #!/bin/bash > > #SBATCH -t 30:00 > #SBATCH -N 1 > #SBATCH -o totaljob.out > #SBATCH -e totaljob.err > #SBATCH -A > #SBATCH -p g100\_usr\_prod > > module load totalview > module load > > tvconnect srun ./your\_executable > > Copy to clipboard > > Submit the job via: > > $ sbatch job.sh > > Copy to clipboard 3) Open a Totalview terminal > In the RCM shell, load the module of Totalview and launch “totalview” to open the GUI. When the job starts, you will be asked by a prompt to connect to it and you will see that the tool is trying to debug the “srun” command. 4) Launch the simulation > Press the green “Go” button to launch the simulation. Eventually, a prompt will ask you if you want to stop the parallel job: if you choose “Yes”, you will finally see the main code of the executable you want to debug and you can start working on it. Installing packages with python environment[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#installing-packages-with-python-environment "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- In Cineca clusters you can find the available versions for python and py-mpi4py with the command `modmap -m python` and `modmap -m py-mpi4py`, respectively. In case you need to install packages through a python virtual environment you can do: $ module load python/ \# In case you need py-mpi4py $ module load py-mpi4py/ $ python \-m venv my\_env\_test $ source my\_env\_test/bin/activate $ pip install Copy to clipboard Note * my\_env\_test: choose an arbitrary name for your personal virtual env. * It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. * Once you source your virtual environment you will see on your shell (before the login node name), something like this: `(my_env_test) [otrocon1@login02 UserGuideTests]$` . * Once you finish to work on your env, you can deactivate it with the command `deactivate`. * In case you need specific python or artificial intelligence packages optimized for Cineca’s clusters you can refer to the section: **Cineca-ai** and **Cineca-hpyc modules**. **Cineca-ai** [The cineca-ai module](https://docs.hpc.cineca.it/hpc/hpc_cineca-ai-hpyc.html#cineca-ai-card) **Cineca-hpyc** [The cineca-hpyc module](https://docs.hpc.cineca.it/hpc/hpc_cineca-ai-hpyc.html#cineca-hpyc-card) SPACK[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#spack "Link to this heading") ------------------------------------------------------------------------------------------ To assist users in customizing their production environment by installing fresh software, we offer a powerful tool named Spack. Spack is a multi-platform package manager that facilitates the easy installation of multiple versions and configurations of software. Below, you will find a step-by-step guide to install software using Spack. For a comprehensive and detailed guide, please refer to the [official Spack documentation](https://spack.readthedocs.io/) . ### Quick usage[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#quick-usage "Link to this heading") $ ml spack $ spack spec \-Il \# to check current specs $ spack install \# to actually install $ ml \# load the created module Copy to clipboard For a fine-grained control, you can select the Spack version (see [Loading the preconfigured Spack available on the cluster](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#loading-the-spack-module-available-on-the-cluster) ), and you can add specs (see [Variants and dependencies](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#variants-and-dependencies) ) to the `spec` and the `install` commands (see [Spec and install commands](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#spec-command) ). It may happen that the module created by Spack will miss some dependencies, you can create the missing modulefiles via `spack module tcl refresh` (see [Module command and Spack managing](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#module-command-and-spack-managing) ). Additional useful steps are: * check beforehand if the package exists in Spack and what is its _Spack name_ (see [Listing the software that can be installed via Spack](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#listing-recipe) ) * check if the package or its dependencies are already installed ([Find already installed packages](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#listing-installed) ) ### Installing a new package[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#installing-a-new-package "Link to this heading") Loading the preconfigured Spack available on the cluster > We provide a module to load a pre-configured Spack instance: > > $ modmap \-m spack > $ module load spack/ > > Copy to clipboard > > The directory `/spack-` is automatically created into a default space, containing some sub-directories created and used by Spack during the package installation. On GALILEO100, the default area is `$WORK/$USER`, while on LEONARDO is `$PUBLIC`. You will find, for example on LEONARDO: > > * software installation root: `$PUBLIC/spack-/install` > > * modulefiles location: `$PUBLIC/spack-/modules` > > * user scope: `$PUBLIC/spack-/user_cache` > > * sources cache: `$PUBLIC/spack-/cache` > > > For GALILEO100 users, please be aware that `$WORK` space will be removed after six months since project expiration. If you want to define different paths for installations, modules, user scope directories, and cache, please refer to Spack manual (a simple workaround is to redefine `WORK` to a different path, e.g. `export WORK=/your/different/path`, before loading Spack module). Listing the software that can be installed via Spack > You can check if the software package you want to install is known to Spack via the command `spack list`, which will print out the list of all the packages you can install via Spack. You can also specify the name of the package (or only part of its name): > > $ spack list > $ spack list > > Copy to clipboard > > or > > $ spack list | grep > > Copy to clipboard Find already installed packages > You will find a suite of compilers, libraries, tools and applications already installed by Cineca staff via Spack. It is strongly recommended you use them to install additional software. > > Find the already installed packages > > $ spack find > > Copy to clipboard > > Check if a specific package is already installed or what packages have been already installed to provide a specific [virtual package](https://spack.readthedocs.io/en/latest/basic_usage.html#sec-virtual-dependencies) > (e.g mpi) > > $ spack find > $ spack find > > Copy to clipboard > > List the packages already installed and see e.g. the used variants (-v), dependencies (-d), the installation path (-p) and the hash (-l). The meaning of the hash is discussed in the next paragraph. > > $ spack find \-ldvp > > Copy to clipboard > > You can also list the packages already installed with a specific variant > > $ spack find \-l + > e.g. $ spack find \-l +cuda > > Copy to clipboard > > or which depends on a specific package (e.g openmpi) or a generic virtual package (e.g. mpi) > > $ spack find \-l ^ > e.g. $ spack find \-l ^openmpi > e.g. $ spack find \-l ^mpi > > Copy to clipboard > > or installed with a specific compiler > > $ spack find % > > Copy to clipboard Add a new compiler to Spack compilers > The list of all the compilers already installed and ready to be used can be seen with > > $ spack compilers > > Copy to clipboard > > To add a compiler to the ones known to Spack: > > $ module load > $ spack compiler add > $ module unload > > Copy to clipboard Variants and dependencies If the package of your interest is listed by `spack list`, you can inspect its build _variants_ via $ spack info Copy to clipboard You can activate (`+`) or deactivate (`-`) variants via $ spack spec \-Il +variant\_1 \-variant\_2 variant\_3\=value $ spack install +variant\_1 \-variant\_2 variant\_3\=value Copy to clipboard and also for a dependency $ spack spec \-Il ^" +variant\_1 -variant\_2 variant\_3=value" $ spack install ^" +variant\_1 -variant\_2 variant\_3=value" Copy to clipboard #### Spec and install commands[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#spec-and-install-commands "Link to this heading") In order to install a package with the Spack module, you have to select for it a version (`@`), a compiler (`%`), the dependencies (`^`) and the building variants (`+`/`-`). The combination of all these parameters is the _spec_ with which the package will be installed. If you don’t select any combination during the installation, a default _spec_ is selected. Before installing a package, it is strongly recommended to check the default _spec_ with which the package would be installed: $ spack spec \-Il Copy to clipboard The suggested options to the `spec` command used in the example above are: `-I` (install), which shows the installation status of the package and its dependencies with a symbol preceding the hash of the _spec_ (`-` not installed, `+/^` installed/installed from another user); `-l` (long) which shows the unique identifier (“hash”) of the package installation (e.g. aouyzha). Important On Cineca clusters it’s recommended to execute always `spec` command before installing a package to make sure its dependencies are satisfied with Cineca installations (`^`) where available. The Cineca installations are optimised and tested for the architecture of the specific cluster. This is especially true for e.g. openmpi. Note Even when a Cineca installation is available to satisfy a dependency, the default _spec_ for that dependency may differ, thus a symbol `-` may be shown. If possible, force the _spec_ to match the one corresponding to the Cineca one (so the symbol will become `^`). A simple way to force this is to force the dependency via its hash: $ spack spec \-Il ^/hash $ spack install ^/hash e.g. $ spack spec \-Il ^/aouyzha e.g. $ spack install ^/aouyzha Copy to clipboard Once you select the _spec_, a `spack install` is all you need: $ \# default spec $ spack install $ $ \# custom spec $ spack install @ +/~/ \= %@ ^ Copy to clipboard #### Module command and Spack managing[](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#module-command-and-spack-managing "Link to this heading") You can load the installed software by loading the correspondent modulefile Spack automatically created. To force its creation, you can run: $ spack module tcl refresh \--upstream-modules Copy to clipboard Then you can find and load the new modulefile by adding the “modules” folder to the search path via `module use` (this is done implicitly also when loading Spack), e.g. on Leonardo: $ module use $PUBLIC/spack-/modules $ module av $ module load Copy to clipboard Please refer to section [Loading the preconfigured Spack available on the cluster](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#loading-the-spack-module-available-on-the-cluster) to know the correct path to the modulefiles folder. --- # Software — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Software * [View page source](https://docs.hpc.cineca.it/_sources/hpc/hpc_software.rst.txt) * * * Software[](https://docs.hpc.cineca.it/hpc/hpc_software.html#software "Link to this heading") ============================================================================================== On CINECA clusters, several softwares are already available through the [The module command](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#the-module-command) . Please check our [Software Catalog](https://www.hpc.cineca.it/systems/software/) page that indicates which softwares and version are available on each specific cluster. It is also possible to install software by yourself using the available compilers or using the Spack package manager. Many softwares are free of use, but some are covered by a license. CINECA already provides licenses for several softwares available via module, but in some cases the access to these softwares is not automatic and additional actions are requested to the user. For other softwares for which CINECA does not provide licenses, it is possible to configure a connection with your own license server following the procedure described below. Software available with CINECA license[](https://docs.hpc.cineca.it/hpc/hpc_software.html#software-available-with-cineca-license "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------- Here we provide the list of softwares directly available using CINECA license. No additional actions are requested. User just needs to load the corresponding module. > * Amber24 > > * AMS > > * IDL > > * Molcas > > * Molpro > > * Q-Chem > > * Totalview > Software available using your own license[](https://docs.hpc.cineca.it/hpc/hpc_software.html#software-available-using-your-own-license "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- Here we describe the additional actions requested to the user in order to make use of the following licensed softwares: ### Gaussian[](https://docs.hpc.cineca.it/hpc/hpc_software.html#gaussian "Link to this heading") CINECA provides its own license. If you would like to use Gaussian, you need to write to [superc@cineca.it](mailto:superc%40cineca.it) asking to be enabled to load the corresponding module. ### VASP[](https://docs.hpc.cineca.it/hpc/hpc_software.html#vasp "Link to this heading") CINECA does not provide license for VASP. User needs to make use of his/her own license. Please write to [superc@cineca.it](mailto:superc%40cineca.it) stating that you possess a VASP license indicating also for which software version. If you are a collaborator of a research group with a license, please provide the license responsible name and the email you are registered on the VASP portal. After a check with VASP developer we will enable you to load the corresponding module. ### MATLAB[](https://docs.hpc.cineca.it/hpc/hpc_software.html#matlab "Link to this heading") Thanks to an agreement with MathWorks, **CINECA provides several MATLAB licenses** through its internal license server that can be used on CINECA clusters. Usage of the CINECA MATLAB licenses is allowed **exclusively for Open Science** (non-commercial) activities. In case you are interested in using those licenses and you declare us that your activity is devoted to Open Science, please write to [superc@cineca.it](mailto:superc%40cineca.it) to be enabled to use CINECA licenses. ### Crystal[](https://docs.hpc.cineca.it/hpc/hpc_software.html#crystal "Link to this heading") CINECA does not provide a license for Crystal. User needs to make use of his/her own license. Please write to [superc@cineca.it](mailto:superc%40cineca.it) declaring the you or your responsible have a crystal license specifying the type (Basic or Basic+MPP). In the email you have to add in CC the responsible and [info@crystalsolutions.eu](mailto:info%40crystalsolutions.eu) After a check with Crystal developers we will enable you to load the corresponding module. How to connect your license server[](https://docs.hpc.cineca.it/hpc/hpc_software.html#how-to-connect-your-license-server "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------- In case you are entitled of a software FlexLM license and you would like to use it on CINECA clusters, we need to connect our compute nodes with your license server. Please write to CINECA’s staff at [superc@cineca.it](mailto:superc%40cineca.it) and provide us the following information: > * the port and host (IP or alias) of the license server where the license is installed. > > * the license holder needs to sign a document ([`template`](https://docs.hpc.cineca.it/_downloads/5f06a7444bf288186b0992dabb10a69e/License_request.odt) > ) in which the holder declares to have a valid license and relieves CINECA of future responsibilities for the usage of that license on CINECA’s cluster. > > * We will provide the IPs of CINECA’S cluster so that the license server administrators can open their firewall to them. > When the aforementioned steps have been completed, your usernames and account(s) will be authorized to use your license running your jobs on CINECA infrastructure. In case you would like to use an academic license, you will also have to indicate us a representative (not necessarily the license holder but with his/her approval) to be contacted by CINECA HPC User Support to allow future requests to use the same license. Advanced Software Specific details[](https://docs.hpc.cineca.it/hpc/hpc_software.html#advanced-software-specific-details "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------- ![ico1](https://docs.hpc.cineca.it/_images/matlab.png) **Matlab** [Matlab](https://docs.hpc.cineca.it/hpc/software/matlab.html#matlab-card) ![ico2](https://docs.hpc.cineca.it/_images/qe_logo.png) **QuantumESPRESSO** [QuantumESPRESSO](https://docs.hpc.cineca.it/hpc/software/qe.html#quantum-espresso-card) --- # Known Issues — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Known Issues * [View page source](https://docs.hpc.cineca.it/_sources/cloud/known_issues.rst.txt) * * * Known Issues[](https://docs.hpc.cineca.it/cloud/known_issues.html#known-issues "Link to this heading") ======================================================================================================== This section collects currently known issues affecting CINECA HPC Cloud systems. The list below is intended as a quick reference for users who may experience problems on the system. We strongly encourage all users to report any issues they encounter - whether listed here or not - to the user support team. PCI interfaces restrictions on a single VM **Status:** Open | **Last Update:** 2025-11-05 | **Systems:** All **Description** There is a restriction imposed by libvirt which allows a maximum of 28 virtual PCI interfaces used for attaching block devices: 2 of these virtual PCIs are used for server needs (mainly boot device) which leaves 26 virtual PCI interfaces available for block device attaching. For this reason, it is possible to attach maximum 26 volumes to an instance created with the default Ubuntu images provided by CINECA. **SOLUTION:** To avoid this issue the solution is to upload a custom image, editing its metadata from Horizon Dashboard as follows: `hw_scsi_model = virtio-scsi` and `hw_disk_bus = scsi`. Docker MTU issues on virtual machines **Status:** Open | **Last Update:** 2025-11-05 | **Systems:** All **Description** Docker containers built on virtual machines are unable to communicate outside of the vm. **SOLUTION:** To use Docker in your virtual machine please set the MTU value at 1400 in the file /etc/docker/daemon.json. > { > > "mtu" : 1400 > > } > > Copy to clipboard Creating shares from snapshots **Status:** Open | **Last Update:** 2025-11-04 | **Systems:** ADA **Description** In order to enable the creation of shares from snapshots, the corresponding openstack share-type needs to include the attribute: `create_share_from_snapshot_support=True`. On ADA, the value of the attribute was set to `False` for both share types (generic, and cephfs) until 03/11/2025. This implies that shares created on ADA **before 4th of November 2025** can generate snapshots, but those are unusable since it is not possible to create new shares from them. Share networks creation: differences between dashboard and CLI **Status:** Open | **Last Update:** 2025-12-15 | **Systems:** ADA **Description** > When creating a share network, the following two parameters are set with different values, depending whether the network has been created via CLI or Horizon dashboard. | **Parameter** | **Horizon Dashboard** | **CLI** | | --- | --- | --- | | security\_service\_update\_support | false | true | | network\_allocation\_update\_support | false | true | This is significant when trying to deploy manila-csi-plugin in the openstack cloud controller manager for kubernetes. In particular to make the manila CSI provider work, the network shall be created via Horizon Dashboard. References: * [https://specs.openstack.org/openstack/manila-specs/specs/wallaby/security-service-updates-in-use-share-network.html](https://specs.openstack.org/openstack/manila-specs/specs/wallaby/security-service-updates-in-use-share-network.html) * [https://docs.openstack.org/manila/latest/user/share-network-operations.html](https://docs.openstack.org/manila/latest/user/share-network-operations.html) --- # Cluster Specifics — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Cluster Specifics * [View page source](https://docs.hpc.cineca.it/_sources/hpc/hpc_clusters.rst.txt) * * * Cluster Specifics[](https://docs.hpc.cineca.it/hpc/hpc_clusters.html#cluster-specifics "Link to this heading") ================================================================================================================ In this section, we highlight the specific features of the HPC systems at CINECA, focusing on deviations from the general behavior described earlier. ![ico1](https://docs.hpc.cineca.it/_images/hpc_icon1.png) **Leonardo** [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo-card) ![ico1](https://docs.hpc.cineca.it/_images/hpc_icon1.png) **Galileo100** [Galileo100](https://docs.hpc.cineca.it/hpc/galileo.html#galileo-card) ![ico1](https://docs.hpc.cineca.it/_images/hpc_icon1.png) **Pitagora** [Pitagora](https://docs.hpc.cineca.it/hpc/pitagora.html#pitagora-card) --- # FAQ — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * FAQ * [View page source](https://docs.hpc.cineca.it/_sources/faq.rst.txt) * * * FAQ[](https://docs.hpc.cineca.it/faq.html#faq "Link to this heading") ======================================================================= This is a page collecting answers to requests arrived to the HPC Helpdesk. Please check the page before sending a specific request. General[](https://docs.hpc.cineca.it/faq.html#general "Link to this heading") ------------------------------------------------------------------------------- > [I still didn’t receive the username and the link for the 2FA configuration?](https://docs.hpc.cineca.it/faq.html#i-still-didnt-receive-the-username-and-the-link-for-the-2fa-configuration) > > You have to do the complete registration on the UserDB page and to be associated with a project (PI has to add you). Once you have inserted all the necessary information and you are associated with a project a new access button will appear, just click on it and you will receive in two mails the username and the link for the 2FA configuration. > > [I have finished my budget but my project is still active, how can I do?](https://docs.hpc.cineca.it/faq.html#i-have-finished-my-budget-but-my-project-is-still-active-how-can-i-do) > > Non-expired projects with exhausted budgets may be allowed to keep using the computational resources at the cost of minimal priority. Ask [superc@cineca.it](mailto:superc%40cineca.it) > to motivate your request and, in case of a positive evaluation, you will be enabled to use the qos\_lowprio QOS. > > [Information about my project on CINECA clusters (end data, total end monthly amount of hours, the usage?)](https://docs.hpc.cineca.it/faq.html#information-about-my-project-on-cineca-clusters-end-data-total-end-monthly-amount-of-hours-the-usage) > > You can list all the Accounts attached to your username on the current cluster, together with the “budget” and the consumed resources, with the command: > > `> saldo -b` > > Find more information in [Data Occupancy Monitoring Tools](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#data-occupancy-monitoring-tools) > section. Access and Login[](https://docs.hpc.cineca.it/faq.html#access-and-login "Link to this heading") ------------------------------------------------------------------------------------------------- > [My new password isn’t accepted, with error “Could not execute the password modify extended operation for DN”](https://docs.hpc.cineca.it/faq.html#my-new-password-isn-t-accepted-with-error-could-not-execute-the-password-modify-extended-operation-for-dn) > > > The error message can be difficult to interpret, but it means that the new password you have chosen does not respect our password policies. Please check the [Users and Accounts](https://docs.hpc.cineca.it/general/users_account.html#users-and-accounts) > > and choose your new password accordingly. > > [I receive the error message “WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!” when trying to login](https://docs.hpc.cineca.it/faq.html#i-receive-the-error-message-warning-remote-host-identification-has-changed-when-trying-to-login) > > > The problem may happen because we have reinstalled the login node changing the fingerprint. We should have informed you through an HPC-news. If this is the case you can remove the old fingerprint from your known\_hosts file with the command > > > > `ssh-keygen -f "~/.ssh/known_hosts" -R "login..cineca.it"` > > [I keep receiving the error message “WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!” even if I modify known\_host file](https://docs.hpc.cineca.it/faq.html#i-keep-receiving-the-error-message-warning-remote-host-identification-has-changed-even-if-i-modify-known-host-file) > > > Please, follow the procedure described below to solve the problem. > > > > Linux/MacOS > > > > \# LEONARDO > > mkdir \-p ~/.ssh; ssh-keygen \-f ~/.ssh/known\_hosts \-R login.leonardo.cineca.it; for address in login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it; do ssh-keygen \-f ~/.ssh/known\_hosts \-R $address; done; for keyal in rsa ecdsa ed25519 dsa; do for address in login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it; do ssh-keyscan \-t ${keyal} ${address} | sed "s/${address}/login\*.leonardo.cineca.it/g" \>> ~/.ssh/known\_hosts; done; done > > > > Copy to clipboard > > > > \# G100 > > mkdir \-p ~/.ssh; ssh-keygen \-f ~/.ssh/known\_hosts \-R login.g100.cineca.it; for address in login01-ext.g100.cineca.it login02-ext.g100.cineca.it login03-ext.g100.cineca.it; do ssh-keygen \-f ~/.ssh/known\_hosts \-R $address; done; for keyal in rsa ecdsa ed25519 dsa; do for address in login01-ext.g100.cineca.it login02-ext.g100.cineca.it login03-ext.g100.cineca.it; do ssh-keyscan \-t ${keyal} ${address} | sed "s/${address}/login\*.g100.cineca.it/g" \>> ~/.ssh/known\_hosts; done; done > > > > Copy to clipboard > > > > \# PITAGORA > > mkdir \-p ~/.ssh; ssh-keygen \-f ~/.ssh/known\_hosts \-R login.pitagora.cineca.it; for address in login01-ext.pitagora.cineca.it login02-ext.pitagora.cineca.it login03-ext.pitagora.cineca.it login04-ext.pitagora.cineca.it login05-ext.pitagora.cineca.it login06-ext.pitagora.cineca.it; do ssh-keygen \-f ~/.ssh/known\_hosts \-R $address; done; for keyal in rsa ecdsa ed25519 dsa; do for address in login01-ext.pitagora.cineca.it login02-ext.pitagora.cineca.it login03-ext.pitagora.cineca.it login04-ext.pitagora.cineca.it login05-ext.pitagora.cineca.it login06-ext.pitagora.cineca.it; do ssh-keyscan \-t ${keyal} ${address} | sed "s/${address}/login\*.pitagora.cineca.it/g" \>> ~/.ssh/known\_hosts; done; done > > > > Copy to clipboard > > > > Windows > > > > \# LEONARDO > > mkdir \-Force $HOME\\.ssh\\known\_hosts; ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R "login.leonardo.cineca.it"; foreach ($address in "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it") { ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R $address }; foreach ($keyal in "rsa", "ecdsa", "ed25519", "dsa") { foreach ($address in "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it") { ssh-keyscan \-t $keyal $address | ForEach\-Object { $\_ \-replace "$address", "login\*.leonardo.cineca.it" } | Out-File \-Encoding UTF8 \-Append "$HOME\\.ssh\\known\_hosts" } } > > > > Copy to clipboard > > > > \# G100 > > mkdir \-Force $HOME\\.ssh\\known\_hosts; ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R "login.g100.cineca.it"; foreach ($address in "login01-ext.g100.cineca.it", "login02-ext.g100.cineca.it", "login03-ext.g100.cineca.it") { ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R $address }; foreach ($keyal in "rsa", "ecdsa", "ed25519", "dsa") { foreach ($address in "login01-ext.g100.cineca.it", "login02-ext.g100.cineca.it", "login03-ext.g100.cineca.it") { ssh-keyscan \-t $keyal $address | ForEach\-Object { $\_ \-replace "$address", "login\*.g100.cineca.it" } | Out-File \-Encoding UTF8 \-Append "$HOME\\.ssh\\known\_hosts" } } > > > > Copy to clipboard > > > > \# PITAGORA > > mkdir \-Force $HOME\\.ssh\\known\_hosts; ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R "login.pitagora.cineca.it"; foreach ($address in "login01-ext.pitagora.cineca.it", "login02-ext.pitagora.cineca.it", "login03-ext.pitagora.cineca.it", "login04-ext.pitagora.cineca.it", "login05-ext.pitagora.cineca.it", "login06-ext.pitagora.cineca.it") { ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R $address }; foreach ($keyal in "rsa", "ecdsa", "ed25519", "dsa") { foreach ($address in "login01-ext.pitagora.cineca.it", "login02-ext.pitagora.cineca.it", "login03-ext.pitagora.cineca.it", "login04-ext.pitagora.cineca.it", "login05-ext.pitagora.cineca.it", "login06-ext.pitagora.cineca.it") { ssh-keyscan \-t $keyal $address | ForEach\-Object { $\_ \-replace "$address", "login\*.pitagora.cineca.it" } | Out-File \-Encoding UTF8 \-Append "$HOME\\.ssh\\known\_hosts" } } > > > > Copy to clipboard > > [Windows WSL issue DNS resolution failing](https://docs.hpc.cineca.it/faq.html#windows-wsl-issue-dns-resolution-failing) > > > If the DNS resolution fails with Temporary failure in name resolution or resolution timing out, an automatic change in `/etc/resolv.conf` occured. You can change it manually by replacing the name server value with 8.8.8.8 . This file is automatically generated by WSL: to stop the automatic generation of this file, add the following entry to /etc/wsl.conf: \[network\] generateResolvConf = false. > > > > Then, add in your `.bashrc` the following commands for the automatic creation of the name server value in the `resolv.conf` file: > > > > if \[ ! \-f /etc/resolv.conf \]; then > > echo "nameserver 8.8.8.8" \> /etc/resolv.conf > > fi > > > > Copy to clipboard > > [The message “perl: warning: Setting locale failed” appear when I login. How do I solve?](https://docs.hpc.cineca.it/faq.html#the-message-perl-warning-setting-locale-failed-appear-when-i-login-how-do-i-solve) > > > This warning is typical of Mac users (but can happen with other OS too). It is actually innocuous and can be safely ignored, but if you want to get rid of it you can add these lines to the `.bashrc` of your workstation, or in general you can execute them before trying to login to our systems: > > > > export LANGUAGE\=en\_US.UTF-8 > > export LANG\=en\_US.UTF-8 > > export LC\_ALL\=en\_US.UTF-8 > > > > Copy to clipboard > > > > if you try to login afterwards, the warnings should have disappeared. > > [Can I login with ssh inside a compute node?](https://docs.hpc.cineca.it/faq.html#can-i-login-with-ssh-inside-a-compute-node) > > > In general is forbidden to login via ssh in a compute node, as the network is local-only restricted, It is always possible to access to compute node by submitting an interactive job with `srun` or `salloc` (check the [Interactive Job Submission with SLURM](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#interactive-job-submission-with-slurm) > > section), but apart from that, `ssh` is **only** possible if there is a _running_ job from the user that want to login into compute node. > > > > Specifically, in Leonardo, due to recent system updates, the connection via `ssh` can be denied even for that specific case described before. To avoid such limitation is now mandatory to have a pair of SSH keys generated inside Leonardo cluster. > > > > The user can create the keys with the command `ssh-keygen` and copy the output , the public key, in their configuration file `~/.ssh/autorized_keys`. After the generation of SSH Keys, the connection via `ssh` in a compute node will be possible (with the condition of a job running from the user that whant to connect to compute node). > > [Users using smartcards and GPG agent with SSH support enabled](https://docs.hpc.cineca.it/faq.html#users-using-smartcards-and-gpg-agent-with-ssh-support-enabled) > > > Step CLI doesn’t play well with non-default ssh-agent. The solution is to either: > > > > > * Configure the shell to start the default `ssh-agent` on launch (or find the existing one); it can be done on Unix systems or on Windows WSL e.g., by adding to `$HOME/.bashrc` file: > > > > > > if \[ \-f ~/.bash\_agent \]; then > > > > > > . ~/.bash\_agent > > > > > > fi > > > > > > steptest\=$(step ssh list \--raw ''| step ssh inspect | grep "Valid") > > > > > > if \[ \-z "$steptest" \] > > > > > > then > > > > > > eval $(ssh-agent) > > > > > > echo "export SSH\_AUTH\_SOCK=$SSH\_AUTH\_SOCK" \> ~/.bash\_agent > > > > > > echo "export SSH\_AGENT\_PID=$SSH\_AGENT\_PID" \>> ~/.bash\_agent > > > > > > step ssh login '' \--provisioner cineca-hpc > > > > > > fi > > > > > > Copy to clipboard > > > > > > * Avoid using ssh-agent, downloading your certificate launching the following command in any path of your local PC (we suggest using the ~/.ssh folder): > > > > > > step ssh certificate 'user-email' \--provisioner cineca-hpc my\_key > > > > > > Copy to clipboard > > > > > > You can change **my\_key** with the name you prefer. > > > > > > A passphrase to encrypt the private key is request as input in the shell command line: > > > > > > Please enter the passphrase to encrypt the private key: > > > > > > Copy to clipboard > > > > > > > > > Both options are explained in greater detail in [How to configure smallstep client](https://docs.hpc.cineca.it/general/access.html#how-to-configure-smallstep-client) > > > section. 2FA[](https://docs.hpc.cineca.it/faq.html#fa "Link to this heading") ---------------------------------------------------------------------- > [ERROR: The term ‘step’ is not recognized as the name of a cmdlet (Powershell)](https://docs.hpc.cineca.it/faq.html#error-the-term-step-is-not-recognized-as-the-name-of-a-cmdlet-powershell) > > > If running the command step to verify the installation of smallstep you incour in the following error: > > > > PS C:\\Users\\user \> step > > step : The term 'step' is not recognized as the name of a cmdlet, > > function, script file, or operable program. Check > > the spelling of the name, or if a path was included, verify that the path > > is correct and try again. > > At line:1 char:1 > > + step > > + \~~~~ > > + CategoryInfo : ObjectNotFound: (step:String) \[\], > > ParentContainsErrorRecordException > > + FullyQualifiedErrorId : CommandNotFoundException > > > > Copy to clipboard > > > > check if you find the executable step.exe inside the folder: `C:\Users\user\scoop\shims` > > > > The installation command should have placed it there. If you don’t find it, run on your Powershell the command: `scoop install smallstep/step` > > > > **ERROR associated to X11 execution** > > > > 1. install Xming: [https://sourceforge.net/projects/xming/](https://sourceforge.net/projects/xming/) > > , it will open a window in the background that you won’t be able to see but you can see that it’s there looking between the icons in the Windows’ applications bar (bottom right) > > > > 2. follow the steps reported at [https://x410.dev/cookbook/built-in-ssh-x11-forwarding-in-powershell-or-windows-command-prompt/](https://x410.dev/cookbook/built-in-ssh-x11-forwarding-in-powershell-or-windows-command-prompt/) > > for PowerShell > > > > 3. then you can run the command ssh to login on the cluster > > > > [undefined method ‘cellar’ when installing step on MacOS](https://docs.hpc.cineca.it/faq.html#undefined-method-cellar-when-installing-step-on-macos) > > > You may encounter an error that looks like this: > > > > Error: step: undefined method `cellar for #` > > > > In this case, the problem is in your homebrew. It may refer to the directories of different processes, e.g. Intel, while you need to make it refer to AMD. You can reinstall homebrew: > > > > brew tap homebrew/core and then set the proper paths with simple shell commands: > > > > (echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') \>> /Users//.zprofile > > > > eval "$(/opt/homebrew/bin/brew shellenv)" > > > > Copy to clipboard > > > > If this is not the solution of your error, the command _brew doctor_ should give you an hint about how to proceed in your specific case. Scheduler and Job Execution[](https://docs.hpc.cineca.it/faq.html#scheduler-and-job-execution "Link to this heading") ----------------------------------------------------------------------------------------------------------------------- > [My job has been waiting for a long time](https://docs.hpc.cineca.it/faq.html#my-job-has-been-waiting-for-a-long-time) > > > The priorities in the queue are composed of several factors and the value may change due to the presence of other jobs, of the resources required, and your priority. You can see the reason for your job in the queue with the squeue command. If the state is PD, the job is pending. Some reasons for the pending state that could bee displayed: > > > > > * Priority= The job is waiting for free resources. > > > > > > * Dependency= It is depending to the end of another job. > > > > > > * QOSMaxJobsPerUserLimit = The maximum number of jobs a user can have running at a given time. > > > > > > > You can also consult the estimated starting run time with the SLURM command `scontrol`: `scontrol show job #JOBID` > > > > or you can see the priority of your job with the sprio SLURM command: `sprio -j #JOBID` > > [Can I modify SLURM settings of a waiting job?](https://docs.hpc.cineca.it/faq.html#can-i-modify-slurm-settings-of-a-waiting-job) > > > Some Slurm settings of a pending job can be modified using the command scontrol update. For example, setting the new job name and time limit of the pending job: > > > > `scontrol update JobId=2543 Name=newtest TimeLimit=00:10:00` > > [How can I place and release a job from hold state?](https://docs.hpc.cineca.it/faq.html#how-can-i-place-and-release-a-job-from-hold-state) > > > In order to place a job on hold type: `scontrol hold JOB_ID`. > > > > To release the job from the hold state, issue: `scontrol release JOB_ID`. > > [Error invalid account when submitting a job: Invalid account or account/partition combination specified](https://docs.hpc.cineca.it/faq.html#error-invalid-account-when-submitting-a-job-invalid-account-or-account-partition-combination-specified) > > > The error Invalid account might depend on the lack of resources associated to your project or there is an error with the account name in your batch script. Just use the `saldo` command. If the account is correct and valid, are you lunching the job on the right partition? To see which partition is right for your case and account, please consult the [Scheduler and Job Submission](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scheduler-and-job-submission) > > section. > > [I get the following message: “srun: Warning: can’t honor –ntasks-per-node set to xx which doesn’t match the requested tasks yy with the number of requested nodes yy. Ignoring –ntasks-per-node.” What does it mean?](https://docs.hpc.cineca.it/faq.html#i-get-the-following-message-srun-warning-can-t-honor-ntasks-per-node-set-to-xx-which-doesn-t-match-the-requested-tasks-yy-with-the-number-of-requested-nodes-yy-ignoring-ntasks-per-node-what-does-it-mean) > > > This is a message that can appear when using mpirun and Intelmpi parallel environment. It is a known problem that can be safely ignored,since mpirun does not read the proper Slurm variables and thinks that the environment is not set properly, thus generating the warning: in reality, the instance of srun behind it will respect the setting you requested with your Slurm directives. While there are workarounds for this, the best solution (apart from ignoring the message) is to use srun instead of mpirun: with this command, the Slurm environment is read properly and the warning does not appear. HPC Cloud[](https://docs.hpc.cineca.it/faq.html#hpc-cloud "Link to this heading") ----------------------------------------------------------------------------------- > [I can’t SSH to my virtual machine](https://docs.hpc.cineca.it/faq.html#i-can-t-ssh-to-my-virtual-machine) > > The most common reasons for not being able to login to your VM are related to: > > * **SSH command**: be sure to use the correct username, address and access key > > * **Floating IP**: If no FIP is associated to your VM, it is not possible to reach it. (see [Allocate a floating IP](https://docs.hpc.cineca.it/cloud/operative/network_ops/fip_association.html#allocate-a-floating-ip) > ) > > * **Security Rules**: In the “Overview” tab, under the section “Security Group” check that the rule for port 22 is defined, and what ranges of IP is allowed to access. (see [Security groups: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/secgroups_create.html#security-groups-create) > ) > > * **Network issues**: Check that Network, Subnet and Router are set up properly. > > * **Machine Boot**: Check if the VM booted correctly. On the Horizon Dashboard, enter the VM page and check the “Console” tab. If an error message appears related to “kernel panic” or “no bootable device”, then the problem lies on either the specific image or the bootable device used. If no error appears, then check the full Boot Log in the tab “Log”. Depending on the error found, could be necessary to perform the rescue of the VM (see [Instance: rescue](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_rescue.html#instance-rescue) > ) > > > [I can’t create an OpenStack resource](https://docs.hpc.cineca.it/faq.html#i-can-t-create-an-openstack-resource) > > > The main reason a user is blocked when creating new resources (virtual machines, volumes, etc.) is that they have reached their project quota. If you need to increase your project quota please contact our user support. > > [If I resize my virtual machine, will I lose my data?](https://docs.hpc.cineca.it/faq.html#if-i-resize-my-virtual-machine-will-i-lose-my-data) > > > No, you won’t lose your data, but you will have to re-partition your disk to use the space you added with the resize operation. > > [Can I create a virtual machine using a Windows image?](https://docs.hpc.cineca.it/faq.html#can-i-create-a-virtual-machine-using-a-windows-image) > > > No, on CINECA HPC Cloud systems users are not allowed to upload and/or use Windows images, even if they own a personal license. --- # Services and Tools — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Services and Tools * [View page source](https://docs.hpc.cineca.it/_sources/services/services_and_tools.rst.txt) * * * Services and Tools[](https://docs.hpc.cineca.it/services/services_and_tools.html#services-and-tools "Link to this heading") ============================================================================================================================= High-Performance Computing (HPC) services typically provide advanced computational resources. These tools and services are designed to support research, simulations, data processing, and other high-performance applications. **Interactive Computing** [Interactive Computing](https://docs.hpc.cineca.it/services/interactive_computing.html#interactive-computing-card) **Singularity and Apptainer Containers** [Singularity and Apptainer Containers](https://docs.hpc.cineca.it/services/singularity.html#hpc-containers-card) **Miniconda** [Miniconda](https://docs.hpc.cineca.it/services/miniconda.html#miniconda-card) --- # Introduction to HPC Cloud — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Introduction to HPC Cloud * [View page source](https://docs.hpc.cineca.it/_sources/cloud/general/general_info.rst.txt) * * * Introduction to HPC Cloud[](https://docs.hpc.cineca.it/cloud/general/general_info.html#introduction-to-hpc-cloud "Link to this heading") ========================================================================================================================================== Cloud computing is a crucial paradigm in today’s digital era. It encompasses the delivery of diverse services and resources, including computing power, storage, databases, networking, software, and more, via the internet. The **CINECA HPC Cloud infrastructure** integrates and completes the HPC ecosystem, providing a tightly-integrated infrastructure that covers both high performance and high flexible computing. The flexibility of the cloud adapts better to the diversity of user workloads, while still providing high-end computing power. **What is cloud computing** [What is Cloud Computing](https://docs.hpc.cineca.it/cloud/general/what_is_cloud.html#what-is-cloud-card) **CINECA HPC Cloud model** [CINECA HPC Cloud Model](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#cineca-cloud-model-card) **Budget and accounting** [Budget and accounting](https://docs.hpc.cineca.it/cloud/general/budget_accounting.html#budget-accounting-card) **Section Contents** ![](https://docs.hpc.cineca.it/_images/os_overview_icon.png) [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#os-overview-card) ![](https://docs.hpc.cineca.it/_images/operative_manual_icon.png) [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html#operative-manual-card) ![](https://docs.hpc.cineca.it/_images/cloud_specifics_icon3.png) [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics-card) ![](https://docs.hpc.cineca.it/_images/tenants_admin_icon.png) [Tenants Administration](https://docs.hpc.cineca.it/cloud/tenant_adm/index_tenants_administration.html#tenants-administration-card) ![](https://docs.hpc.cineca.it/_images/tutorials_repos_icon.png) [Tutorials and Gitlab repositories](https://docs.hpc.cineca.it/cloud/tutorials/index_tutorials_and_repos.html#tutorials-and-repos-card) ![](https://docs.hpc.cineca.it/_images/faqs.png) [HPC Cloud](https://docs.hpc.cineca.it/faq.html#cloud-faq-card) * * * ![../../_images/known_issues.png](https://docs.hpc.cineca.it/_images/known_issues.png) [Known Issues](https://docs.hpc.cineca.it/cloud/known_issues.html#known-issues-card) --- # Operative Manual — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Operative Manual * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/index_operative_manual.rst.txt) * * * Operative Manual[](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html#operative-manual "Link to this heading") ==================================================================================================================================== This section collects all the operative guidelines on how to perform the most common operations. The operations are grouped whenever possible according to the respective Components. ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Compute operations** [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html#compute-ops-card) ![oscinder](https://docs.hpc.cineca.it/_images/cinder_logo.png) **Storage operations** [Storage operations](https://docs.hpc.cineca.it/cloud/operative/storage_ops/index_storage_ops.html#storage-ops-card) ![osneutron](https://docs.hpc.cineca.it/_images/neutron_logo.png) **Network operations** [Network operations](https://docs.hpc.cineca.it/cloud/operative/network_ops/index_network_ops.html#network-ops-card) ![osmanila](https://docs.hpc.cineca.it/_images/manila_logo.png) **Shares operations** [Shares operations](https://docs.hpc.cineca.it/cloud/operative/shares_ops/index_shares_ops.html#shares-ops-card) ![ostrove](https://docs.hpc.cineca.it/_images/trove_logo.png) **Database operations** [Database operations](https://docs.hpc.cineca.it/cloud/operative/db_ops/index_db_ops.html#database-ops-card) ![osoctavia](https://docs.hpc.cineca.it/_images/octavia_logo.png) **Load Balancer operations** [LoadBalancer operations](https://docs.hpc.cineca.it/cloud/operative/lb_ops/index_lb_ops.html#load-balancer-ops-card) * * * ![../../_images/known_issues.png](https://docs.hpc.cineca.it/_images/known_issues.png) [Known Issues](https://docs.hpc.cineca.it/cloud/known_issues.html#known-issues-card) --- # Cloud Specifics — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Cloud Specifics * [View page source](https://docs.hpc.cineca.it/_sources/cloud/systems/index_system_specifics.rst.txt) * * * Cloud Specifics[](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics "Link to this heading") ================================================================================================================================ In the corresponding sub-sections are described the details specific to each HPC Cloud machine in production at CINECA. For the general information on the HPC Cloud infrastructure and service, please refer to [Introduction to HPC Cloud](https://docs.hpc.cineca.it/cloud/general/general_info.html#introduction-to-hpc-cloud) . ![ico1](https://docs.hpc.cineca.it/_images/hpc_icon.png) **ADA** [ADA](https://docs.hpc.cineca.it/cloud/systems/ada.html#ada-card) ![ico1](https://docs.hpc.cineca.it/_images/hpc_icon.png) **GAIA** [GAIA](https://docs.hpc.cineca.it/cloud/systems/gaia.html#gaia-card) ![ico1](https://docs.hpc.cineca.it/_images/hpc_icon.png) **MEGARIDE** [MEGARIDE](https://docs.hpc.cineca.it/cloud/systems/megaride.html#megaride-card) --- # Tutorials and Gitlab repositories — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Tutorials and Gitlab repositories * [View page source](https://docs.hpc.cineca.it/_sources/cloud/tutorials/index_tutorials_and_repos.rst.txt) * * * Tutorials and Gitlab repositories[](https://docs.hpc.cineca.it/cloud/tutorials/index_tutorials_and_repos.html#tutorials-and-gitlab-repositories "Link to this heading") ========================================================================================================================================================================= Tutorials for OpenStack dashboard[](https://docs.hpc.cineca.it/cloud/tutorials/index_tutorials_and_repos.html#tutorials-for-openstack-dashboard "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- This section provides you with a list of guides for basic operations. ![tutorial](https://docs.hpc.cineca.it/_images/tutorial_icon.png) **Creation of a VM (step-by-step)** [`Download the pdf`](https://docs.hpc.cineca.it/_downloads/48e828372fc20da9f32cf0200f1c67d3/ADA_getting_started.pdf) ![tutorial](https://docs.hpc.cineca.it/_images/tutorial_icon.png) **Deletion of resources (step-by-step)** [`Download the pdf`](https://docs.hpc.cineca.it/_downloads/a95b6ec1cde5eebebfea6f561b3516b0/ADA_getting_started_cancel_resources.pdf) Terraform/OpenTofu/Ansible repositories[](https://docs.hpc.cineca.it/cloud/tutorials/index_tutorials_and_repos.html#terraform-opentofu-ansible-repositories "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- This section provides you with links to CINECA Gitlab repositories that contain tutorials or modules as examples for the use of [Infrastructure as a Code](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/infrastructure_as_code.html#infrastructure-as-a-code) tools. Warning The GitLab repositories are accessible only if you have an HPC user account * [Infrastructure as Code (IaC) modules](https://gitlab.hpc.cineca.it/adacloud/tf-modules) * [Infrastructure as Code (IaC) tutorial](https://gitlab.hpc.cineca.it/adacloud/tf-tutorial) * [Customized ansible-scripts for OpenStack](https://gitlab.hpc.cineca.it/adacloud/openstack-ansible-repo) * [Slurm mini-hpc cluster on ADA Cloud](https://gitlab.hpc.cineca.it/adacloud/ada-slurm-mini-hpc) * [Graphics on HPC Cloud](https://gitlab.hpc.cineca.it/adacloud/graphics-on-hpc-cloud) * [RKE2 kubernetes deploy guide](https://gitlab.hpc.cineca.it/adacloud/k8s-guide) --- # EUROfusion — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * EUROfusion * [View page source](https://docs.hpc.cineca.it/_sources/specific_users/specific_users.rst.txt) * * * EUROfusion[](https://docs.hpc.cineca.it/specific_users/specific_users.html#eurofusion "Link to this heading") =============================================================================================================== ![../_images/spacer1.png](https://docs.hpc.cineca.it/_images/spacer1.png) The ![ico1](https://docs.hpc.cineca.it/_images/EUROfusion.png) community has access to the following CINECA HPC systems: > * Leonardo **(until July 31st 2025)** > > > * Booster partition > > > > * DCGP partition > > > > * Pitagora > > * EUROFusion Gateway (EFGW) > Important The general environment defined on our clusters is the same for all the users, so EUROfusion users are invited to refer to the general documentation. Essential links below. For general information regarding the access to the HPC clusters: * [Getting Started](https://docs.hpc.cineca.it/general/getting_started.html#getting-started) * [Users and Accounts](https://docs.hpc.cineca.it/general/users_account.html#users-and-accounts) * [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems) For general information regarding the environment on the HPC clusters: * [Introduction HPC Resources](https://docs.hpc.cineca.it/hpc/hpc_intro.html#introduction-hpc-resources) * [File Systems and Data Management](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#file-systems-and-data-management) * [Scheduler and Job Submission](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scheduler-and-job-submission) * [Environment and Customization](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#environment-and-customization) For specific information regarding the HPC clusters used by the EUROfusion community: * [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo) * [Pitagora](https://docs.hpc.cineca.it/hpc/pitagora.html#pitagora) * [EFGW Gateway](https://docs.hpc.cineca.it/specific_users/gateway.html#efgw-gateway) Dedicated tutorials[](https://docs.hpc.cineca.it/specific_users/specific_users.html#dedicated-tutorials "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- A presentation of each supercomputer, and of the access method via two-factor authentication (2FA), have been dedicated to the EUROfusion community. You can find the slides, including a report of the final Q&A session, and the recording at the following links (you should log in through the button Access as a guest). ![tutorial](https://docs.hpc.cineca.it/_images/tutorial_icon1.png) Pitagora: _Introduction to Pitagora for Eurofusion_ June 23th, 2025 [Pitagora webinar page](https://learn.cineca.it/course/view.php?id=2121) with slides and recording. [`slides`](https://docs.hpc.cineca.it/_downloads/84493ea9f614a4a9b4d303f0fb5f34cd/Pitagora_EF.pdf) ![tutorial](https://docs.hpc.cineca.it/_images/tutorial_icon1.png) 2FA: _Introduction to two-factor authentication (2FA) on CINECA HPC clusters_ June 7th, 2023 [`2FA slides`](https://docs.hpc.cineca.it/_downloads/c6608c051cbc5638235edbb527019060/2FA_EF.pdf) SLURM Partitions[](https://docs.hpc.cineca.it/specific_users/specific_users.html#slurm-partitions "Link to this heading") --------------------------------------------------------------------------------------------------------------------------- On [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo) and [Pitagora](https://docs.hpc.cineca.it/hpc/pitagora.html#pitagora) _Job Managing and SLURM Partitions_ sections, you can find the description of SLURM partitions and QOS to submit your jobs. Notice that EUROfusion users have dedicated partitions and QOS: you are allowed to use the **“\_fua\_”** partitions and the related QOS, besides the **“\_all\_serial”** partition which is shared among all users. ### Low-priority jobs[](https://docs.hpc.cineca.it/specific_users/specific_users.html#low-priority-jobs "Link to this heading") 1. **If all the budget assigned to your Project Account has been consumed**, you can keep running on Leonardo boost\_fua\_prod and dcgp\_fua\_prod partitions at low priority by requesting in your submission script the **qos\_fualowprio** QOS: #SBATCH --account= #SBATCH --qos=qos\_fualowprio Copy to clipboard The QOS is _automatically_ added to your Project Account upon budget exhaustion. 2. You can also request to run low priority jobs, **without having consumed all the budget of yout active Project Account**, by association to the **FUPB1\_LOWPRIO** account on Booster and **FUPA1\_LOWPRIO\_0** account on DCGP (write a mail to [superc@cineca.it](mailto:superc%40cineca.it) ). You always need to specify also the **qos\_fualowprio** QOS in your submission script. #SBATCH --account= #SBATCH --qos=qos\_fualowprio Copy to clipboard --- # Tenants Administration — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Tenants Administration * [View page source](https://docs.hpc.cineca.it/_sources/cloud/tenant_adm/index_tenants_administration.rst.txt) * * * Tenants Administration[](https://docs.hpc.cineca.it/cloud/tenant_adm/index_tenants_administration.html#tenants-administration "Link to this heading") ======================================================================================================================================================= This section provides additional information related to the management of cloud projects. In particular, it explains how you can request to associate a DNS name to your instance, and how to safely manage sensitive data. A dedicated section provides guidelines on how to best manage security for your instances to reduce vulnerabilities and risks of attacks. ![guidelines](https://docs.hpc.cineca.it/_images/guidelines_icon.png) **DNS guidelines** [DNS guidelines](https://docs.hpc.cineca.it/cloud/tenant_adm/dns_guidelines.html#dns-guidelines-card) ![guidelines](https://docs.hpc.cineca.it/_images/guidelines_icon.png) **Security guidelines** [Security guidelines](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#security-guidelines-card) ![guidelines](https://docs.hpc.cineca.it/_images/guidelines_icon.png) **Store sensitive data** [Store sensitive data](https://docs.hpc.cineca.it/cloud/tenant_adm/store_sens_data.html#store-sensitive-data-card) --- # Unknown PKM�Z^�2 ''mimetypeapplication/vnd.oasis.opendocument.textPKM�ZConfigurations2/accelerator/PKM�ZConfigurations2/images/Bitmaps/PKM�ZConfigurations2/toolpanel/PKM�ZConfigurations2/floater/PKM�ZConfigurations2/statusbar/PKM�ZConfigurations2/toolbar/PKM�ZConfigurations2/progressbar/PKM�ZConfigurations2/popupmenu/PKM�ZConfigurations2/menubar/PKM�Z styles.xml�\[�s۸�\_���� ")َ�F����暤3M�|� �8hJ��;�"�/��}wu�x&3�����%���t\*��q�=�: V�&,�4?���Wp���oY���S��%�y&H�N%�b�������ڇa�4�f�f<��3T4 �I��E�z�}dF2�˥� ��P36݆��,��>:�Y����zy�c�8�\`�;W���Y�T�I�1�����PhsQjñD�\]h����B+j��6M��Pa/��=�C�\*�e���⥸�g2!) \_�(�\_��C����v��0�}K$�Rf��y {DB؊���OI��������"È��i�7� c�S�8�����rz�m��=!3F%�\`�@�"޽5z��+�\[�=�q��HE��������a�ğ83�zS+<�E'������2)�=�Xga�\]��Ϩ�h�ϥ�8\[���&�c��?�{Ha9�Wn��}��fz��+s�O��3�L�����\_}y?���lڅs�h����Y�� ���g��Vќê�I��7�8�� m����!�0��!�a{us ���zd�3LHKy��eI�2�o8������۲���,���t��6)'��QB.�Q����oLU5��^�+��1�t�#�Ĝ�Ʉ5���f��A"�� ���=ok��XK�d�C�S� ���N�V���;Br��\`e+���)R5G�)�v�J���r��H=q�)�7ՙ�,pR���y�Z��jdk-h0Uk���P��F���K ?���+�3t���CG��cҼ���HXM%?�\_\[�")w�Sm�Qc @�!��P w@���\`�p�o��Xu���\`��"�4\]#�\[l����R <�ߒ>|��핑%�� ��b�gl\_��Q����4Eh\]t� ��������J$C\*�� ����l�v ��i"k3��\[���4�4��y���2/^o���\`�Մ�yy)�&r%r��S�N=.�7#PU�b7e ��h%M"4��\`��w=���(�� ����A~���ّq��\*9��^+�2既g�@8g͠���r$ u�����@�m\_$�)�i0\[\]4(�!PI�\`�,!�1�U�Qލ���'�=�Fլ5$�g��3U�����y������(�ԏ���� H� ���1����\_i� �I�2��\]ʨ��By�p�s'G�l�+�S��W,�֌�ȓa7��Zļ�y��@�����)��3�� 0\]���N���/��#K�O�W��v�f�7�fi���Y��>PA���x��� ��y�Xi���\]e��$�J������J+���D}�=�zs�fi��SO�p^�C��?.��G,�c̿P���ra꿄�\\���Pw՞1��Ir��%�����t�bμ��}&ѧ�psoE��2?� 3cR�e��,ڳtl�:��Rbj��3��\[�����3�$��\] �g ��1qy���˟4X2�~��������vv� |��������}���k��|����KY��K��A�Ue���Z�����!�֟�O��84vK{d$��o.!�ȭ}�|�A9T72������A��6�G��Хֺ״QE�Ѩ�G����\_};�� ,�h�d-��)��-��k Fp:ip����6��b� ��%�Nݣ��L�U�^&�Z��-��FV����:��P���{D,��8mҺ��$���\]4u����Jᓘ��9i�����\\���U�25�b�փ#�5(dz9^Ix��1�ۻ��� �yuH�!�W��r�ꐾC�\_�w�ͫC�y�ꐾCn\_�w���!��Y��>�we �0�ἶ���lw.cL��S����\\����V���c���(��z�M5�����\]�!�霂xZA7��H�����#�����b�\]���򎝤�A��4L�?�QO����n�(wϬ�\]{�w�x�q �mLؠ�w��\[z���Ȟ���|������>�#G���P)�w�;����H/�Q�\_��\]�S�6�7�{XLD�zƷpmfk������!#�n��WC�\[m�H�2&�\]r٩��f̸�ct��8W�ip7���"k\[Og績�᭩���v�uu��U���'�F���$|��8x �3f(�����Q��&�������l �"8�0��GA�3.9����F��˾��\*b�AnZtݯVm(D�o akp|u3E=B��#Z�on:|���7�1��\[K����OA�a ��@� � m�\]����D��P�b�)3O^P�S�X�v��Z\]K3��;�\*P�V�)ŷ���38��q�׎�� ����)�l7�ꆢy2��爴 q�����Rj9k � T�#�\]� �'���~J���n��cO��m���)�?;9��.���B��)� �\*���c %�!��vP �>|o�;�+Y�s������Ri 5�����PK2\`��Q �?PKM�Z manifest.rdf\]�An�0E���r�x�������j��<���Wu�.���?�w��C}b��4�Ɯ�B��-�޲�\_�u�.���>�Ա>H�����s�=���f1�h���s m\['�kyR�����J��1��bL�;�o��A��ψeK(�%�\*�V ���T�$���ИV�3ď�4��V�����{�~��t�/PK$#��PKM�Z content.xml�ZYo�~���\`�A D��%�;V1ʹ�s�d�I�.OZ:��P�JR^�� RKd;����7FD~��,�CJo\\� �@\*\*���F�CQ����\_�3�c���"�i�H�y \\;���F��q5 ���1�j��\\w>����B&��'��9��%:�Q�C�<��&nD4qf��T��N�����\]�\[Cݢ�w�{uyN!%�JƒT�\_��B�j ���8�V͋�90�k� %��璏QT�8IA�t8�\*��&zd�V����JlS6%z�%�g��S�suY� J��$Pc���u�\[7Ăk'&!8�Lo ��fT<;���N@��\[�Fj�TД���!�P?��F�VT��I�1�"%|�QN�xF$����d�@�Ԝ\*u(�@Ҹ�m�"�S��{�N"�v�ր�q��H���wff�4�O��Zhay�.�����j�:�\`ֹ�&b�Nr-R�i�X=� ��+����X���$�$�V��\`�B��K��2�W�kA'�"�)( ��;�ф�1�R�{�4������|>,��\_\`j>���{Ϟ��ȄrG�l��0ō����\\�yǽ�iXI�fr�s�e�j�1;��n� 2\_l��(81����jc.dJ�C���'�E+g�;�(�R��B�l7W ����}�6j�������7��F�t�8n\]K��Q�4�\[y�'O��F��N����Q弬����b�n�E�5��T��e��%gӼ�� F%��n@���?j�nhXV�r%d�'f�����o��c՗ ��\_/�"�H�>܂l�v���������nM6�m���Y:"���\`l�=\\t���X�tqg,}�����iZ ����}����z~�I. �n~VH4��W#g�o���8��p���;���z��\\wB֠�r���S( �x���ߔ�+�p1�N�>ڔx�>� �0�)(��p������G��ݻ��U~e�W>f��Z+�w;�Zap�O~�P��'����TH�gq�����vi��� �?���qp��/��~~�t�q�����H@��� �Q����@��\[&�5\_��Dރt6��U��'��S���G��H �.K�?+�1�S�=;�����R�〨��ya�m�hؑ'����W�L� H���|�.D���IP"��=/\[\_.WB�I�\`ތ��7\`�HQr�#��cX-ZG�rD ��>�S=E��u��k�Z���44i�%�"?���S�P�I���mh�&��D�ʉ��?��(L�J�-�-K�JU"4Y�g�IG����Jr����'2�̨�&�<3��"7(\\r�GH�\\� �pj�1��MYT�OT��τ7L���V��)"|i��LpE&�Q�,Rd�L�xݛ��<�U�M�L$(c���$!��uE�"� ���wݿ�Io��룏�S S���\_y���6Q��\[��a9�q�� o��N�|4�F��cox:�x'>z'�7t �4���v���3�djm �Q�%g�������?�u)�8�0���w���M8ѹ���P+B>���zюG3�p��1���\\�D����Y�?+���jR�5 Ȏ�=�?�Q���Q��gN�$�v���\_�h{��\\�TO�߮PKT�S�.PKM�Zmeta.xml�S�n�0��+ 6W�-+!+@ �h��@oMnd&)PT��}!���7����΂��u�z�jkv(�(Z��ViS����W�Eŧ�>=i \\Y��\`<���չ�L�K�T�C'�NH��Q���+ �tMJ���UC�MC�u�p+Z�r#jh���6\`f ���a��Vr�j:W#% T04�$�b2kϕ6/�6��,#�����E8l1�c$d�u��7�t�b��8��u��EK�kk� F�4�t��7�f<���.W�W”�(������y�� f�>�yBy�����,��,��(��)��++h�l����kU��^����}O~\]�,��U�'�uţ>:��AXm�4bw��t�ßmzH�Յ��8� ғdK�t�R�MFoG���^x�z-W��X��3~�(A\]��h���5�\\(�s�t�9-���:��i6��$���B��~�5�?im#$��,a�9y��ȭ���PK�tS��PKM�Z settings.xml�\[\[s�:~�\_1��V&@�����!���jHxْ�4��.I0���drdX�n^�tK��|�����"a��AH���P�\\,|aL���0ܓ��\_7���� ��\*�(K�� JQ>�� ��\*B>���B&�I�' �+\]a |��w\_e�+ F��ua�Tzuz:��?��>����\*�ʩ�ts+"�ܨe�Vfn.�秫�/w�w�.l�K�����LsV���a���o뽬^N��D�����Vv\]@īg ��v}��R�;H�ia�Z�p\]\`ȧ����E���\[)�Kn�D�\]�'wDc5�)�|���<��;����e�.˕#M��p�A�(�ڌ�)�- !"� 7Jdp��&� �Khc ��O��?IHzBy ��k����Wby�ɛ��R�T\[�s���{��R9��}�ryqQ:V��!��#�Ft���0�Ax�Kv��d��b��ȅ��\`�ndž��P�rR�,1S5dY·�z-�h��H�">}X\`���K"�b���Gڦ)}\`)�\]AK��zDQ� ��� �,T�Ą�S--(\[ �)�2hQ.r5G�dAU 9}� >'��v��攣����L�v���ϥ����! w\\|\]l�}��\_�o��X� ����������@�B���e��+d U=Mf|���Fɭ s�o;�\]���kPpz�4��c�� �稌�H��ꉑG��N<�vPOR���b���0�������\`��|�\*'MٲG� ���D<�(�4ԙ�Quw'/7Y�ۆ�M# ��\`'KB����yuo:^��2�' ^k�>�T��L�'\`B@|�kn������QA��\`X�ڹ�6�\`#S ���"��Ί� 3Y�ޑ�2h���u#=�j�F:B>^���J�H�l)7��1�,�τ)}:s�I��0f)��A d�'@� ��y��)��\_ \*I�����H�ُ�әB� �tFn����R���X\_Ҋ�����&bJ�Nj�'!�5LR ?4U�\[��ȑ�p�j�\_���1� Y��f�i�t��j���:�jcH)$��0� \[�o�$!��d��A%� �J��j$��0����q�;l H�IeD\\�)|�bí5.3!t�a��,�'�� �3U�;�H�� PAdh���'��ı�=P���DO��1����\\�P��y�, �.�z|\\��=4��Ng'$�L���t�gRщ)R�Q&(zg{9�̓4���c��R���R:r����Jg�I%�zY���5���p&>��J�� �A Q$$j�$�M�v8�%N����7��p�>y�\`�%!'�B7(!�x��\\�׀��-й�%1XH��$ �!a��v�r+��-G( �Ѥ���;��B^m9\[��\\���������u�������<6x)m/����n�C�j�!\_� +��c2���pQ��9��-\*���Y�L�-��\[젪�TetO� �.��l F���9�1w\]�'��P=�%RY�B�3 W{uy���a���$�x��6��m\* O��X�F�ã��{R�(�ム s%�W&����;n�^�#��Y��c��ڄg���$�)�3��~� ���?����l�^p�-�E��+����H�r^���e�f>���i)n��f�T'��pIFß���\_�4�~�-.;A}�P�(a,���u�8m���^w���\_c�q��B��\[|��E�W�dtQ?4+��j�,�n%?ܧ�����>�.��^��+�Fe����C��c�2���u�� nq�����縟�LJ{NF�q#-Eg��rz�#۵9o���Z�y��1�Ǎi�����sv�4~���A����Aݻ �z���Yw?�?�b6(�/� �v�Ʀ�۠X�����s���>||(�z^}8u�5 ��qc�ˆ)\_<����V}p1�������x�sg�ZN������J�?(��y�v8�\]����2C"�N��a�N��� �.gˁ�ǩ�m A���4i���\_�?�PU��Yޚ6\`�-\\\_(�����4q�j2�?�s�#�/���H0�Ƀ���E���7��X���v�ԐK�R����Gm�Yj�@v�&$c����{7��n�� ���-;��d� �2U���d�;�|�\_ʋ^=�\`2=-�.�Nff��, �tx�b\]4�T��(c����Yc4M!��Hw�\\�MԬJ$hb�at����9\*���W�nҞ�gi����5��9��<�4T�p���1� ��5��%�����Dڠ�;�1���~����wx��ny|�P�@�3�J��F�;Dž��寗GZ��M�����i����|씦M���O�Kw?U�/{�?}���}���PK ��P m:PKM�Z��N��M�MThumbnails/thumbnail.png�PNG  IHDRj���pPLTE     %%!\*'6&%%!\*"5&(%&%+6)1;6+$2.1:1+866-C$/@\*7G+>Q4:F1>Q;BJ8GW>P\_;Mb=RhA,G8+E:5R=+P>1C?@KA9UA/VG9\`K9iS>FFGFKTJQZUKEQOQYQKXWYGVfH\[rV\\dQ\]sMby\\biXfvbLBgWGd\\Wr\\Jp\_Qb\_aia\[xbLwgYhghgltkrztlgqnqzrksrsru{ty}{urzvx|xt{z|Wk�\]s�ao�jv�g}�u|�r�k��{��y��v��}���lW�r\\�wi�|v�}i�r�~���z��m��y��x��|�����������������������������������������������������������������������������������������������������������������������������������������������š����ŧ�ҵ�ľ����ͮ�Ӻ�ɹ�պ�������­�Ų�Ҹ����ŷ�ļ�ҽ�ҿ��¹�¬�ȹ�κ�о����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������sP� pHYs  �� IDATx��� \\���ߝ�g����Z몝�l\]Oߎ��u�j�3/��\*���%��$!7��ܤp�M�h{� l�1�4&�€Ѓ� /}�c���|�� B�<�������Z�2����ת��~�+���~���&D��$75AM��&���5AM��&���5AM��&���5AM��&���5AM��&���5AM��&���5���&���\_Q�=��O�3��^O\[�=� BCX��z�vOMy\`��i �\_U�Ku��uhCRߙ骜I5\[U����ae���h�aY�\`�SW �3\[65�P$�h��t�8���t���f�����P���t<�ub�k~T���\]�&͡�̩M�oe��m�� Q�Y�-4�ː6 �պ-%qN!�����t}�toi:A�ْ�� ���ks#��1(g^��=֏�����GWӝ���9{�+1�"�Q����:��(9�+�L� , �?\[,��d���Fo��y �Y���f� �,@H�F?o�CS-�D~��1�d�E��-iZ�f�y~X0�2�\]�1;��� �뱮bMv�=����\\V��U�>�Y2&P�Q�#� 4�3�f��y,�� �m�a�b+�Ҭ�Ⱥd!eu7�.�Λ �V�2��g��F��Y��ǟ�>�?��� '���2GZ��ބ��M���ZB�kve�i� Э��h���GP�\_<�����v�<^��,z�'�݇B힀ǃF�g�?�mHc<��O��߆f�UN������׍I{�hД\]էD��>�I��-I��f4�� Ō\\WyD}�<�TLD�CJ�Wy�\_�����Y��e���"3����vy�&W���ۧ\*3y�^���s����M>CPi9�\`��l Bl��Wo/˕4K�� ����b�\*Ո���=F�ƒ4�d�d�Y���Kw$��m����� �V� �����m���s����g�Z�\]͸�� �#���t #�t{{��SB� �^W׾DP/�0�i՟j�c��,Sc�� |��?~�.n��t(u�rk �#����T�\[�C��>�1�<MQ�ň�6��l^E��\`��,��7,~Ԍ��2� ��2����g+��HQvV����T,�cM3k|Ⱦ�4\*�G���^=�s�-�C�5���s�8ͅ�5���g�?L���q+cv3�4i�0t1U��c%\*�&^?�2�%J9�1x�m<�ҳ�a������ӈ\\|vK���� Z��"/ͿEԵVyyA4>bK���� �k�� �{)�X������K\*��EfaDԈv�~X�1Y��\_��i�T��ɜI<��=h�Jb�����O�q��eDG�h�֯G�����H��@\`��:�m�s���K���s���篵^}i���XS��"Z� �GP!� j"5AMP!� j"5AMP!� j"5AM��&� j"7�P�ɜ���UA8^O���wm��)���\\�s�8�� {:��#�� ��N��&���vy!�D��Y#���amǧC\_\*M^0�����gGm)tA��٨��I&�9"���3S/�\\��r��0=�7#��L.�|�B� \`������Ulz?�=�hlB��b|�#�V8�Z��N;�����8\[��N~�gh�� ��M��ɨ�n����Ӄ/�� O5����g�1�ҽ�45Ν ;+�rK��,��4Փ�}�|� ش����"��Q�7u�j'\*�\_f�Ȩ5���:�F�����=��# m�2i��G�\]NS1�v\`��� |J�FW�t\*�KL�8�����i�q�;��j\]�E-=��(;$�lɈ��E)q �7���)�U}L�܄������z����\\����>��a�d\_����1b��\*��/;�c�}h��s��Ew琧�)����O�Υ�A��p�9�B��H�6��+��zܒ�.?��K�l�e�nr� ��C���D8�:X6���a�O��Gع/�a�c�t���<���ؙ��!t��F!g^8v��\\�a�O���VBphne�O �Dz��Q\_?���w���qN{~� v��>�5��(�|�O��ݥV:�����/��7�-b�y��7�@Go���M��媈,j�V���<\\h���ulR�N��y�Z ^B;Jӏ>P��ϭ�f��4�� ����Ad�I��ap���UJ\*����g��� �C����&+t�hh��}X �j�IA���f3tM��ܫ�k�I��n�\[R�X7�S��M\]lC�ψ-t��B+8���&��d\_\\�D&�isYK{�{\\�IX�3�h��9�n��(�֧0�MC�� <�;��p��}�y�=�Ή��\]�����=����s<����۞��-s��^fZ Ԋ�Ug�ƃd�Cv�&/�S�ߕ���@Bo�\[��3��>�l���\`�ô��l���Lvh3i� tr�!\[�X\\#2�ol#:��8r2�C�r�g���鐐 嘤�u{뙜^%#�2,3½�w �\[���T���#��8�>�^�\_������΢��5���\*���\[,/m,��PjP!S�O)�� |d�.%��?���--��ʫ��8��է��9�Đ��x:e�����,�������+�&���wíƨ����<���U?{��LYMJ2� �v�RR�EG�Q���3��3��=(���jy�C>���N�MFT�z�iQqa�9x �uH�ʏ��s���"�yQ�\`~�P��q��R��|;���A S��.��։�K{Z�~�~IZ�L�{ �E�Z�Y���lm�$l�\_ G�iVC 98��Bt��� �z��Il>�\*|%y\\�҅�h��{�����r���Y����}�P���L�8����atR�G����:��Fv�=���q(�o��o�L?��那ߘx�g���Q���\]�DoD��f�>�B�k�a\]ᘔ�,\]C�Kȯ�U�C�A\*��R�n��4�(��3� Z��l�) ��|��>M�n�� |�\`Ŕ� }E��a���&�a���0G�W�N�)7�f��a+����%�h�d�1�V�t���7���\*��� ���l����K�K\_��tv4�ʳQ������ ��1�Sq���X�|����5!Z;�����ݲQe5�\\����i4P�, h4U��3cF�񶼀ytSAk�LY�q�^TѩXP�b�)�� i���jQ:MRp�:<7F��?�}�zMs/�����t��\\Q�K\]ݵ���! -!� j��Z��σz�3������!�羌bG���Z�����~���ȋM�Z���v�ʞ��C �ag��\*<�^o��g ��O?�Aӗ�%.�3�v9D��j�kb�L��'\]��2j�2SUEŸ��\`��?�V��h�ӓ���,8ꩺ�)}�%�T�N0�x돍́ZhLӵ�i��9���fhԮ;V&�V9����� ������!��V�B����Z��h�k �������n�����6������9�e'Fj!�����0�& �R�C���P�M�ђ�&m- ����dO��O ��'�Q�Wq��K\`8T ��z-�Gr���ZWE$��Na?������dM�5�1��͛�%�Jա|�H����W�M\*��5�'wJ�&�nYXԃiLR�Ȩ�W����Bz\\1�H� \>���x;�C��=���/\\�73=�C\]�ao0�����r�Eox���F=}p=C�,�j��nߜ'�z�A4�ɽ��溆��\] �<��q����@\`�K�����)�ʫ�H\]��q2�Y���K7�����-��9���&���5��� �4���.m��4 �D��ݕ���/yV���GEi�d�\\8|��'�&��:�c{�kQ�S�O�p�^ �g�2�Kz-g��ߍ\\���m�r�+��=�\[?s�f\[#��hޫ�waP�t�9Ԇ��‡���I�����=qs#r�����\*�Ay��/��rVE�"�@��gV�C�\\�����'�i�� a��{Π��85ʖZnv>$H�uNm����\[�ˎ���<\*W!G4������ l��\*5ٽz\*�S'�y PGo���������s�\\/CuHh���پZ �v/��$=���.I�5i���T�)�͇�\_�\\U\]P����5���� j�b� E�S!����̶6�S��-n�i=e� �F=e.��}�>��J�|��nW?�Rr�\]� ��� ~n%�5����w����ϼ��s8��j�J�a�������ᣧx�r��'o~۴�W=���\]��U?X������^�lޱY\\S���t���נ��r@?Q�팠g�=�;���-�-}����O�}f ���stɚ���%=��~R �<��Ft�Vۢ�?�zpe�ᢥ�7<�/����l��\\���p�� ����W�P.+\[�\\�|n��$~~r�d����-R=ȬP޻�EdYu�s�MeT�-k����{i���ǻ�V�\\�ֽ��ҕE�Y����|ͽ����=;��i��Ƈ\_\]�V}�9dB�GP���}�Y�C�n �Th��3�n@���=}�Tqg��-=�^�~�=�{qp�<\\�h?z�g;���=xdGo����+������U���+ �������A��#�Z�.���穧����l�U�5�8âx�e��G/�V}Njh���\[�ï���(��V\_�����'=�3���{������~��ߎ.���{n�?ǿQdW�|�3�6n��l�\`�lj�77^f\_vw Y��ږVe��^��+ZW��M@}��\_��k�+.����g~� �m|��ꫥ������=����se�,Ւ9W�Q �\[�9N�ưdU�rE�7k~�+|�j�/7��}��x>ݶ�'��;޹�����P��=����3�q��4�������M�P�ݖ��� к���ά��Ӌ����p�}��o-�=��ˏ���a��?�SX~��ذ��;ﭒT�U��o��i.{L�J������7��:F�%~��{��vc�.��\]�\]������ʲ�4+�?��k^�ۺB'\]gY�?�F����7pﱬt\_i<~>-�Jߝ���֘���H4�|���#1�c~x.�f��A1ߘK���n+\_q���������‘�0 ��G���M/^~�΅#��/B+���w|�����\\���q��8$�^�����# \[�@\\7���)J�pY�8U/�\\�T��1�7$�ĻO#w�o���f$��(���Ѕ�}���!���"���cqp$|�a�"%so�\]M�>� z�\`����U{ }e7�kߗM{y.b��s0�O�0���馿4w�?�q#� ̉ �FK���ZJ�%7���mrh��Xү\\�k�'� 7?����v�Ul�y���iG��l���ki旹��e�#���6��-,|e�� v\\�v������{}¥\]�i��6���EѤl���.���A�4n�ˋa�sO�˗ E<:�n��T(����ٸ��"|�����s��|aP��{��\[V�q�Su��c�� �W/�s�����8ܦJwؿ��$X�����{���&���m~͞�o��A���cL������q�Ҹ�M^�&�����d�-+,��Po���͋�A�����뒛4qΧ����{����,j>�(�z��/���� ������C IDAT�ܰ�~��O�4��'���cq�Ē�mY����s˝����΄ �\_�<�҂��Ο\]����a������>\_�\]w���8�=�ҲMI˦0�\`x�I(\]�򺓋pe�.熂� ��Z����2Ko,�����+a�hS�}EO\\y�t�鑂��\]�~��S��~���o|�e������Kh�9.��m��4�\_d|����GT?��N��w ��o��TK�-�וw��M�g\[�Z����o{�.�mզY�\`\_�>znx���\*0� ���/�-+Z6�/V\`k�/6g�}��M o� �a�@�������g~q�����������n~���-�-��������ĦPd\_� ۄ7$<����z���/�y����d���G/�+\]��|�;���٧N�����������N�~箕�+�8#��<�.u�!noxt��ݙp�7o���L�r(H3Z��Jz�k���+0>T����cw��W�sF?�Ԣ";W�/�ZW�laZ��I� ����-��R$N�%�uStnɊ�Z��k�jt6�X2t\[�������{�R�mu��D6���Q�!�y��A͖^K��H�kU�.�� ґ� \]��-pQ�sZR��\\�N)F��BZ��� �ɑPL��G����e��5T�a(�wKs�q7�T�ڠ��� ��LJiq�H����Z�/�hO�<��a���Ѭ\[�ʤ���.։��^����K^ZU�+\[���%cSk�)|����>ݖ�ͫ�R2Z��Gn���#/�h}�(S��s7�Q u>����o��x���H�/�͵;?�=#�)��֠:�) �?��ۉ���@��O5z��|����rMu�z�q8.z��G�0d�HP��A!�����=���\_� Ih��/���!�?\_t2��^g\]U���UY��y'Y�z?w�q��v���:��-gO��|���n�eN��$�骪���i掟r��(��9Ϙ�l{˚��+�\]��w �W�u�oZK;�lP��$���\\nx!l/�u�v�C��kV��m1��=^���\_��4l�lټ�^z�b��1��K�(���+���3ϥ?�Z�f%��T@P�<����� o݃��K��\]�m���<�ڜ��$\_�MO��\]ڈ~�����%�O>�� �#�|� o�.��)������U��26ԟ!�֨�W�~z���یOm��|"x�ې �(~M��\`J{��f��ϲ����1ߛ��.��.�^�o�ͫiU�V$Y%Z��m����~\\�z�3�g �#�� �� #QW�ev��8����6��;!'�g��{����N�mF���|�O��ņH����Wi|\_o��&2!N�5���&BP�5��ou84����p�0>����9�\])-׳��W}�^wt3� \\5�p�y?�\_i8:��}���橚+�9�ގs}����ŲG���T� U"�/j��ABNy���.$�B fe�ڏ6����:��5\]yO��u�?����s�EU�z,=���q co^Zx����\\�F��-��s~��+-��4���(k�b���4\_�G�p��6����ՇW~�t� � �z�J5�-���\_��\*��÷���K��^�� ����G�- k��Մt�� ��uy�A���q�5㬛�&��D�Ocv��/�b�>9�e=L�\_crQ2G;��b�b�#��W����8����!�?8C�i:�Bz���M<����� ��24��������hxR=�(�Ƿ��zD�!�}익��FG�t E��Rpy��8%y��\_��b��������e}L��(X��#z�1�:�\\hTÌg-P���|(���t�s�r�z�)T�%��6|�K s�D?��d?����9w����t{�{Q~��c�X^�������L�)̮\\\]z�(;K{J�S�O�T�l�5wW�d�>��\*�\[�^�/f�=%Y�?�\]N�6Jb�\[R�"h��\*{f�������w���\[q��o����y��Bs�H�3e�!�$�B��M��,���{q�� Q��K�&�����\]�C6���BQ>�3i\\�T�3g����H�=\_s�8\\����)���ɥg���^��MDZ+�@!z�;HQ\]�wg5�wl?| ��0�\*�\\BV\[r�����hw�(s�b�Ko� s�|��^�����0�"�>�ȥ:�t@-½�=��;ǡ6��$(%#S��\[� ��O��Bd|nٿj(�"=^&�G<�&MX9���ƣ��B����l()�?���#�PAŝ�N�U�Ӡq������dG�U7�c�h����a�Ke/Di�<�qFR��� ��rH͔R��=��z.�.s�Q����9�b�����ʸ�\_����g�S����<�y(�s<$�����"��qD�i��>�W\[io�O Ӛ�o�7Q�-�����A۬ck4����a\\� z��� l�%{��\\%��<�c.U���p��p�T�A��m�w�\*����E�Z)�ɹiYoQ��j �bO��88{(�N�����+4��0���(F5< ���b�v���| -4 ��b�'f! VJ�+��\]~W�×済 ; ���E�� P6Zj.\_i��?����/g ���q�d��>M�!�w>!2����,����G�AY�c��4S� \\�� ЫáP�/��<��Q�=o�G��q���/\`s !2�!BP�D�>�Qpa��quq���+$1绺~mW,+U��S��#��i��^�,\`ؘ�|5ե�����+j��=M�����X��0�V� ���N3QN�Ŗ\[0����=ӌ���鼳~l�𳜚�.7!t8ߵ�D�L�L�߆/�ܩ $�c���r�\_ \]Vy�ٱ� �^����%�j6ʙ�� �Μ���W�q��y͟���<~����a�5�U��p)d�h��L�f$90���—0�K�L3��fx8)��)�t��r����B���^>��!j,7+�B���ph�6\*�-A �?\[i��rr��\*���ц"\\���ŗ���M�n�ҝ?�;�2��C�\]b���6rf��᧱ymw�%%�i�q2���\\$60��#tjU^�xD�jb�X� �!�>����R�8�l�S�)� )>���P^�5Y�.Kzх�$6�?���q���Wz@c��ʲ�4m\*��ɽ��(�ke�8\\o��jb��4�$bh�R�LEyF�SǙEN&�R{��TJ�\]��o����¬����w2�bMe�����G��Vm��\]Ċ��f~W�C�2L�dƫI K���n�@n�Ʒ�l\]~4ISi)���쿨���f���B k��\*:��P�\]���B���Wģ��b|�fD�i��t(�˯��V$G$�o��S���ai�tjP�\[)��)(��T����!}�06�y5�sY5���=R�J�t��h\\\]2�l��MJ�7����#9b���e����<©�&��t�4KF��y�꫟Ό�9Q��0+<;�ʈ��\[F���\*p'���c}�'$4�����a�(|� 6��{�́ T,�nO�f��ݹE��� �b��y��zPQ�����0z�$VZX�Q��cs/��L�,3\\��z��$e��q���de{,�PٶNec��C�V��3G\*��w=���\\؁B��jt)p;���e�zt��t�ft+�,N��:�ԏ��n�M�2Y3�/˚$��^����tK�\[���p1�A0�b��}���^�F՝tk��ng-ӫ3���(��U�s�d ������L)!��D�Q ��W�m46���ι��z�wt��t�����OeFˍ�J/���} �%�}F;�U��O��G���T��~\\ �Ӊ�tO�ȧ}'?.22❽2��J������|�fMyAo��J�N���9Z�|\_�V%�I���������Z?:�T{�\[DѴ��esm�C^ߧG�C}�r�����\*C^qC����I��)SDz&wvP�F�&��BVV\[��m�O���SȘs~�' ��\]ӈ3Q�Q��Z�������W.;ϥOC%�̌~�Ͳ���uG�f%v-��C��C)\`t���/���Q�Mh���n����C:�Y�3�s i�m���+\`��ui���4�j�E�������ޣ�����s�HϞ�C�P���m�ޫk�9A��:F�l3�;�ƒ'�(�{�����.�� ��:��Ў2����N~R9����5E��#�\_@��o���������A"�}֚z<�r1��z6���L�$c�f�phǜՌ�����8���b��. ��f���ko#�a�!�����V=�g{m��ͼ@��J�b��\`kM��-��� �z��MКء�9\\��=AWZYf��f�/pc ����^Б"n��k݈��ѵj�j���j�c?�(�zz�C� Ƌ��r +��tb�<֌?����e��}���\[s�������/�Y�?X�h>���U+��=����wc�GЅ��>��K���^���nͪu���a��%?,�l��bI��R?�o۴f�r�t��>ϮUKj�����ޮ�%�9����6 ��?�z,�{�z4Y#V +��jS2#���u�Q�X��Y&�!/g��l���ΐ �ew�0"��2X~R�'~�E>�ΞH@ �}�������ѥ��y#&3�z=��&ל��ؓ��xGT!gLEM�Ԩ��x45��e��B������凗���&.ٛ��w�.齛{��� ���K���%���2�C���o\\��G���Η� mo�1J�O�zj摼%o��ӝe+틻�\_�\`�����dY$����H�}n����p�R��\[�vf�?�����L! !ⅱ��ʕfp�ڵ?�Z� ���~7ϳ��� P���gj�)/g$�<���'S�6�뱋TtM����a���թQ�j�vU��UOc5g����D�;�R�L����;���b��I���V�.�./\]V���׃��"-�vߌ���}�f��}������V݅�8 ���&�;^��=�7�\[+~g�v'�9Lm�|����\[�)�lN��Y���=�wV�|�ǫ�Z"�|��!�}?X�\*\[�蚷rH ǭ&!�!5�\`"U�Mp̖vva4H:�۠�746���궣�����wl(�5�fG}#�/i����u�C��!O��uy\`4����gl���3~7���9;Xen'�uw|����;#N4z��9��~�w^�7� ey�M���\`<�Ƨ�;��pI�r^��?t���︮'��0b�B��f~D->���6&��N���ik���;��P���=vʦ�h� �(\_�\[��o�X�����Z1�@�(�1������O������qi�u�v3����� C�蕹�d���"㪣 b$sҪ j"7uh>h\*����=%օ��}��:V��\`���'��y��3��|�Y� 5�8�,�?W:��wF���kO\_�^�����+���3��ɲm���¹+SW����8Mb����\*�U� ����gj��7���(�־�L�#��(;��ݕ�TYn9~��V#d��ὤ~<̋>�����>��� 4�3,��B쵞4wM��"W|��L�p�w�S?��(W;����L/ ���A�KV�˜ܔB�g�5f���P:\`�}R����!��NF�q1Ճ����vt1�r>=\\r����CX�m:@���1}���Υq���6���m�='l�s�1(!��h����i=ճ{���o5����T?tk���E�h�Mo ܾ�ME;�%2h�Sr�^+�lE�5-� ���SҪhr@ѳ�4�Ow�^=ny� ?Ă�q\*�nƩ���j�N����27m~������8���ټᑮ\]�������x��f�aɈU\`˷������'d������ũ�X�������(���9��� =S�7�����W� �:$o���M"JQ�/���I��1\] ����\]�u���'o\*@O��Y�\`�jB��\[L�Ҋ�A�h�\*k9zx�?;c�i\\�ى�!g�6��G!g����\`�!uy��'�9��4�@)�8sv��{���e��9�h\`�)�\\\*� ��=#%��{�3�b >��N��xn�Z�\\Ж��G�����P4g߻�;�y�q �<I�����@2�G�����1����ޱ�����$e�;�RJ�k�GsKz�zJ�ҷg�b?�b�t�qJ�s��� � {�i�����$�S?Ҩ����#��c빣��\[7x��w����Hb�yK���拿�\\�?E�n��!g\[���9l��9s ������!~kBjJ�t��\[���s�b+p˒"�ڞ͖��qU�Y��7Z��$�?)n݁�2��f�o�Sg��<�������Jc���e�=1 c�rq�.�Q��#�\\\]��Igm��Z���>S���Ll��έ�e���"cIa���@��R%\*�fN%���;�"\\BѾyP��a�Avq+Mss���f���s\\X/�2r4�.F�Ej��v��=�Mvr�� �e>� 9S�TZ���B�F!M���gӝ��}�G�)���۴�ؽ�-�r��/\*�Sڲ�a�ܷ�D�\\Ț�k\`�Р��0��@�9 ���Ǝ�x�,g��"�����|�L�T��UO�a�٫�J2\\������3L�V�+b�³�8�"�K%��cUa���� r-\`��'K2��+3�8�lG\\\]�1g��H,X�"V��σ�.b� x��B������JLh�b�|�ll�h,� ��ga1� ���DC�}zu�����u��n}F�� Ɯ�i�w�^�7�N$ ?�{A�g�\]��BP������}��m��n�^�1Ng�}�%3�FP\_���L|¬���v���xwn�/��C�6O�'��l���\]T����v�ya�/����s�!�� ڎ�lBi��Q�q3P9 G�0�;�FF����m��ί#j\*����r�Ѧͣ !�B���T4C�J��8�.EeQF;� � ����\*�,����'��:WOS�(���Tmn ��lt( i��@�~�E �oW���L!�Ĭ0��\`<��C\]��\\�V �M6���4AAU��Y��ʭ�����E ��{4\`�|7�D�l��\*�'u�����q� �a�6K������E%�I�\\�Nd� 'E�U�XjR����ՊfF�������Ba�2j�f;z��!:�/�\_�����㓛��E��� mRI>N�e�1�qx\[Е��J�ݩ21t�3m��R\*�I��r0X�(as�St ����g hE����{K��&����\[�nq��\[ M�;�\\Z��X��0�?�s����>�+���C�6X��Ӫ��\*T�W�1t�?�,r���)�w��\*SHیj����j5@�1X5U�1t��um���u5�{uU��Bt�߬��V��7b�V���!��ԫ#�=�w=jH� ��!� j"5AMP!� j"5AMP!� j"5AMP!� j"5AMP!� j" �ZW��gb�3ը��A�N�\*R�\\�&�Z�gF�� ��ir!���G�<�z�w=3��T\_���Z�B �c���e��3�m�/L��i���b�c248�ػb�-�r~��6/̎CYI����nF������Pg�хMU�n@=��q�BU0��h�܃�r�Iz�G>Ž���#韂z\`�SS��a����(��Xd�+�l=�?��2)�9�l �� �?��8��&����ii�L��c\\�\\Ԣ�n�T��}���<��A�e��&���5���&��\\K�������\_4�!A��b\_\_���(�&�v�C�z!J�l>�g̃�s�v��Gӳ�uL^���U5iyfk�,A���\*�C�&C��R�/�ki�B���Ϻ�~��x��Mt�V�0�:�y�F?$X���H��������W�� "#�Fw�q�zA�����z \\IDATI��\`�� 5Iv��0�:,д�2� M�V�NՒBP/����͐�p�����FpN�"���{,A}����{���dCP!� j"5A��H� �\]��� �\*���\*&�O�Ds7���Z8�y���Jf��M��}��}d��r \[�+�^�{W<"u��W��^Z/�p#dj/���421ހڸ�C����I!�����ʋK�B�\\��N5{!�e���'t�q�w3�C�KV��@�Ǡ>�F��R��.S�gkZeT��p��$3�IXv��V�锕~���F'�#��ߥ)�w�E3�7q&Q�,5�4L\* ����� ��x��hR�K���ٌ1(>{��ʆ�\*w�"Sjό�PB�Zss�yZ(L�R�&Ͷ���^^ �|�� ��(�R(2!h�g�����4Di�j�\[�vF\]���V �����(�I̡���&Q?�Y��b,R\*�Cr�64�B�\[򪒽R6�� ��>���Br��6�1�ދ���e����b�(�E}\\y�.>��T��\]�u��#�=�w�(\]R��C�q�,1)��-fW��F�(��Z����,�M��tڱ��:�A:P.�mn��3rinj WQ6� E�b��\\�T1��$� \\��\\=��Wu-z�+��?���\`}%�����%�B7/2=NP\_S��� �s���D!C��AMP!� �v \\G� � (\\4H b�b�\\CF� '3I�\\l��1�Rm �q��v��q��V�����^0࡬|Tl�:����\]I�L�����%;��3����QiA�~K��#�n��Ne�suƁ|�D�z��C^���W Ձ(C�;���KsJ�%{�Rv�}>1�.��D X��(g�4�\*c�1<�8�tF�W/�DcQ݃��>�n�C�=2�!���5���&���5���&BP�5���&BP�5���&BP�5���&BP�5���&BP�5���&BP�Dj���&BP�Dj���&�ߌzn��\[p�6m�d�� f� e>�\_�� �l�?����E\]Z0��D�6o�ӇF:P���:�j��#��E���ק�����P�@�a�u�P�"@�m�aO8��:Д'2�񝗝&��!�U��H��B�q���)\*��~d�P�\\=τ�Z��\*SI�ޛH��K��h<����G�Q�L�JbTҴB�cx ����m� �k�T�v�ܪ�8Ǖ��^��� ���T��F��̖\*{�5�2JIN�SŒ���u�=R� M�^�t9KUI�\*E���"ȵD�ȴg�"�S�M:>�� �xL���LHa�!U�L�r^{�?E�\*�Gk�kξ�\_�~��#�q��Mf�s\`w��J؀v9rj���5d�Pv�u�JASg����vk��hH;;X5<�^����~���k7׿c�jf�g� ��!� Y+�4���9mNk� ����zj���W�L\_��F��( \_��+K�GcK����+�(Fc\_Ca�� j��A���:J��V����\]o�k�az����\*�F'�N5@Ԩ΁lW��Z�B������\\f{ڰ�!K�o��5��\*�{��܄����tV>�8Z B\`����pf���#��t�:ш3<��@~'�v������'�?���\]�j��։��z��� ���%�uZ��2�ddt� ���=�B)�|����|9��t�f �rE&�J�-:0���T���'�%�?��"{:b�����Mh��t�Ү��Ќ�4�����W��'0\]��ZhW)�54��x '���F��E��d��?��( JM��\_\_��!����{��$CK"Y��Mp�Uc�F�2q�逢�\_Bm�FE��ԋɟm��(MRSZ��D�+�i<� Zu�N\]&Of�u��$�H\\�?сD����\]\_�sW c�3��2�P�����D.(X�jY'������O������x.�j|ȶo���N#������nhjioko1�)U�?�� ��$��\]�)��u6aM:�@�g���k�����\\��'6�4�����o��&BP�Dj���F\_ҒJ� �� ��#w�mL�4��g���rPU�\\�mI�O@1���������o�&�e�3A=���T��U��ب����Ϣ {�{�)Tn�������j�~CP�#��ꡜm%?�M1"�@�c4�����d�+~O��ےt\`�(@Gף�XTx<��A�qg�2��-U�V"I��P�������H��9�|ov #$�������c�QJ��t�n�9$4\*| �AZ�8cQϰ �ϩ �\_��%���V�R�i�����w��\[�K�N/�M5${RjiL��B��Y$i2(�����|E���R����b����u����7��v���7��io��z9h��7�� G;��3��D\]�4ѫ j��AMP!� j��AMP!� j��AMP!� j��AMP!� j��AMP!� j�� � j"5AM��&� j"5AM��&� j"5AM��&� j"5AM��&� j"5AM��&� j"5AM��&���5AM��&���5AM��&���5AM��&���5AM��&���5AM��&���5���&���5���&���5���&���5���&���5���&���5���&� ���&BP����:V���n�IEND�B\`�PKM�ZMETA-INF/manifest.xml��Kj�0��9�ѶXJ�M��(��T{� H##��s�b'.%Cv���h��+N�ī܊� -RW���W�!��f� ���z>�w��c%r$L¤�xH�z�64����^OJ�ia�\]\\�.�0sc�g� �Z��"CǕq:X� �aR�7�-�E%��̀�Ε��c%�\]\_��ES�J��w�L�ԉZ9u ��%��B����b��Mo�A�I�Ɍ�Y։'>;H#��G��U��N��>�'��e�F��Ɵ��<���{~Ӈc�?d�%��Q���Ao:P�^՛�����\_PKѠA�+%PKM�Z^�2 ''mimetypePKM�ZMConfigurations2/accelerator/PKM�Z�Configurations2/images/Bitmaps/PKM�Z�Configurations2/toolpanel/PKM�Z�Configurations2/floater/PKM�Z2Configurations2/statusbar/PKM�ZjConfigurations2/toolbar/PKM�Z�Configurations2/progressbar/PKM�Z�Configurations2/popupmenu/PKM�ZConfigurations2/menubar/PKM�Z2\`��Q �? Hstyles.xmlPKM�Z$#�� � manifest.rdfPKM�ZT�S�. � content.xmlPKM�Z�tS��Tmeta.xmlPKM�Z ��P m: csettings.xmlPKM�Z��N��M�M�!Thumbnails/thumbnail.pngPKM�ZѠA�+%�oMETA-INF/manifest.xmlPKe\_q --- # OpenStack Overview — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * OpenStack Overview * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/index_openstack_overview.rst.txt) * * * OpenStack Overview[](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#openstack-overview "Link to this heading") ============================================================================================================================================ What is OpenStack?[](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#what-is-openstack "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- In CINECA, we use OpenStack to virtualize and manage cloud computing resources. [OpenStack](https://www.openstack.org/) is an open-source cloud operating system that controls and manages the compute resources of a cloud data center. ![../../_images/openstack_software-overview-diagram-new.png](https://docs.hpc.cineca.it/_images/openstack_software-overview-diagram-new.png) Management Tools[](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#management-tools "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------- To manage the resources assigned to a project, the users can interact with OpenStack using in-house tools like Horizon Web Dashboard and the Command Line Interface (CLI) or using Infrastructure as a Code tools. ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Horizon Dashboard** [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#dashboard-card) ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Command Line** [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-card) ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Infrastructure as code** [Infrastructure as a Code](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/infrastructure_as_code.html#infrastructure-as-code-card) Components[](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#components "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------- OpenStack is a flexible tool that has a modular structure: it is split up in different components that can be installed and activated independently to address every user’s need. ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Compute** [Compute](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#compute-card) ![oscinder](https://docs.hpc.cineca.it/_images/cinder_logo.png) **Storage** [Storage](https://docs.hpc.cineca.it/cloud/os_overview/os_components/storage.html#storage-card) ![osneutron](https://docs.hpc.cineca.it/_images/neutron_logo.png) **Network** [Network](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#network-card) ![osmanila](https://docs.hpc.cineca.it/_images/manila_logo.png) **Shares** [Shares](https://docs.hpc.cineca.it/cloud/os_overview/os_components/shares.html#shares-card) ![ostrove](https://docs.hpc.cineca.it/_images/trove_logo.png) **Database** [Database](https://docs.hpc.cineca.it/cloud/os_overview/os_components/database.html#database-card) ![osoctavia](https://docs.hpc.cineca.it/_images/octavia_logo.png) **Load Balancer** [Load Balancer](https://docs.hpc.cineca.it/cloud/os_overview/os_components/load_balancers.html#load-balancers-card) --- # Users and Accounts — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Users and Accounts * [View page source](https://docs.hpc.cineca.it/_sources/general/users_account.rst.txt) * * * Users and Accounts[](https://docs.hpc.cineca.it/general/users_account.html#users-and-accounts "Link to this heading") ======================================================================================================================= Usage of CINECA HPC resources is allowed only to users with an **User Account** (or HPC username) and provided with a **Project Account**. UserDB[](https://docs.hpc.cineca.it/general/users_account.html#userdb "Link to this heading") ----------------------------------------------------------------------------------------------- The **UserDB** is the portal containing all User Accounts and Project Accounts and where CINECA users can manage their profile and monitor their computational resources. ### How to become a User[](https://docs.hpc.cineca.it/general/users_account.html#how-to-become-a-user "Link to this heading") To obtain a **User Account**, you need to: > 1. register on the **UserDB** Portal; > > 2. get associated to a valid **Project Account**; > > 3. request to be validated and enabled to access CINECA clusters. > #### Register on UserDB[](https://docs.hpc.cineca.it/general/users_account.html#register-on-userdb "Link to this heading") You can reach the portal at [https://userdb.hpc.cineca.it](https://userdb.hpc.cineca.it/) . Click on Create new user and fill in the form. ![../_images/userdb_login.png](https://docs.hpc.cineca.it/_images/userdb_login.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) Important In case your Name or Surname contain special (non ASCII) characters, please use the corresponding ASCII one Once the registration is completed and you have set the password of your **UserDB credentials**, please go to my User, click on Edit and complete your profile: > * upload a valid Identity document (Passport, ID, Italian driving license) in **Documents for HPC** page and sign the CINECA Access policies; > > * insert your affiliation in **Institution** page. > You can use the **UserDB credentials** to login to UserDB. Alternatively, you can also use your [HPC credentials](https://docs.hpc.cineca.it/general/users_account.html#hpc-credentials) with 2FA by clicking on OpenID button. Warning Each user can have **only one profile** on UserDB. If the profile already exists or the email is already used you need to recover your previous profile by clicking on Request new password and inserting the email that you used to register the first time. Write to [superc@cineca.it](mailto:superc%40cineca.it) for any issues. #### Get associated to a valid **Project Account**[](https://docs.hpc.cineca.it/general/users_account.html#get-associated-to-a-valid-project-account "Link to this heading") ![../_images/registration.png](https://docs.hpc.cineca.it/_images/registration.png) Each user can be either a _Principal Investigator (PI)_, or a _Collaborator_ of a **Project Account**. > * **Principal Investigators**: apply for a project or acquire HPC resources through the processes described below. > > * **Collaborators**: request the PI of a project to be associated to it. See [PI and Collaborators](https://docs.hpc.cineca.it/general/users_account.html#pi-and-collaborators) > section. > Currently, there are several ways to become **Principal Investigator**: > 1. **ISCRA Projects**: dedicated to all scientific researchers affiliated to an Italian research organization. > > 2. **EuroHPC Projects**: dedicated to European researchers. > > 3. **Agreements**: For Italian research institutions, contact [superc@cineca.it](mailto:superc%40cineca.it) > . > > 4. **General Users and Industrial Applications**: Send a request to [superc@cineca.it](mailto:superc%40cineca.it) > . > You can find on [HPC Access](https://www.hpc.cineca.it/hpc-access/) page of our HPC portal a detailed description of each case and useful links. Once your project has been approved, we will create the corresponding **Project Account** on UserDB portal and associate you as **PI**. Important Write to [superc@cineca.it](mailto:superc%40cineca.it) in case the project is approved but you still do not see it in the portal. #### Submit a request to have a **User Account**[](https://docs.hpc.cineca.it/general/users_account.html#submit-a-request-to-have-a-user-account "Link to this heading") Once the steps above are all completed you can request us to create a **User Account** by going to the page HPC Access and click Submit. ![../_images/submituserdb.png](https://docs.hpc.cineca.it/_images/submituserdb.png) Warning If the Submit button does not appear, it means that one or more of the above steps are still not completed. In the **HPC Access** page, the portal will indicate you which step is still **not OK**. We will receive your request and after a check of the inserted data we will grant you access to our HPC resources. Within 24 hours you will receive via email the information needed to set your **HPC credentials**: > * your **User account** name (or HPC username) > > * a link to setup your **2FA** access (the link will be **valid only for 12h**) > You can visit the dedicated page [How to activate the 2FA and the OTP generator](https://docs.hpc.cineca.it/general/access.html#how-to-activate-the-2fa-and-the-otp-generator) for the steps to configure the 2FA and use it to access our HPC resources. ### HPC credentials[](https://docs.hpc.cineca.it/general/users_account.html#hpc-credentials "Link to this heading") **HPC credentials** are different from the **UserDB credentials**. The former are needed to access CINECA HPC and Cloud infrastructures, while the latter are only to be used inside the UserDB portal. Caution Both **HPC credentials** and **UserDB credentials** are strictly personal and for any reasons cannot be shared with others. ### PI and Collaborators[](https://docs.hpc.cineca.it/general/users_account.html#pi-and-collaborators "Link to this heading") Each **Project Account** has a _PI (Principal Investigator)_. The PI is usually who have applied for obtaining the budget and is responsible of everything that happens using that account. Each **Project Account** may have one or more _Collaborators_. The PI can add autonomously collaborators to its account. ![../_images/userdb_collaborator.png](https://docs.hpc.cineca.it/_images/userdb_collaborator.png) The Collaborator needs to be registered on UserDB. To add the Collaborator, the PI needs to go to the page of the Project Account visible in the My projects page and click on Edit. In the Collaborators section, insert the **UserDB username** in a blank line and click on the correct name of the collaborator among the ones proposed by the appearing scroll-down menu. Important For any issues in finding the collaborator, please write to [superc@cineca.it](mailto:superc%40cineca.it) Warning If the PI of the project account is registered to UserDB but **is not yet approved to access the cluster (HPC STATUS: Not Configured)**, the project is not loaded on the infrastructure and collaborators **may not be able to access** the cluster. Once the PI is approved, the project account will be activated. Project Accounts[](https://docs.hpc.cineca.it/general/users_account.html#project-accounts "Link to this heading") ------------------------------------------------------------------------------------------------------------------- CINECA users can have access to HPC resources in several ways listed in [Get associated to a valid Project Account](https://docs.hpc.cineca.it/general/users_account.html#get-associated-to-a-valid-project-account) and described in detail in our [HPC portal](https://www.hpc.cineca.it/hpc-access/) . Each approved grant is uniquely identified by a **Project Account** name. Each Project Account is provided with at least one budget, usually in standard core hours (STDH) for HPC clusters or VCPUs for Cloud resources, on a specific CINECA cluster. A Project Account may have multiple budgets on different CINECA systems. A User Account may be associated as _Principal Investigator (PI)_ or as _Collaborator_ to one or multiple Project Accounts. ### Budgets[](https://docs.hpc.cineca.it/general/users_account.html#budgets "Link to this heading") In the context of HPC (High-Performance Computing) resources, **budget** refers to the allocation of some computational resources granted to a Project Account. The resources differs if the Project is defined on HPC or Cloud systems. HPC The utilization of our High-Performance Computing (HPC) resources operates within a “pay-per-use” framework. Usually short interactive serial jobs run on login nodes for testing incur no charges. Parallel and Production simulations instead must be executed on compute nodes by submitting jobs to a scheduler. Compute nodes of a HPC system are usually divided into different partitions. Some partitions may be free of charge. Please check on [Cluster Specifics](https://docs.hpc.cineca.it/hpc/hpc_clusters.html#cluster-specifics) section for the available partitions and their cost on the cluster where you have budget. On UserDB all the budgets are expressed in **standard core-hours**. On the cluster they are translated into **local core-hours**. The convertion factor depends on the different cost of a single core-hour among the available clusters. Each approved **Project Account** may have multiple budgets on several clusters. In this case the budget name may differ among the clusters. Please check with the command `saldo` the correct name of your budget on the specific cluster. Further information about how to check the availabe `saldo` of a project are reported in the [Budget and Accounting](https://docs.hpc.cineca.it/hpc/hpc_intro.html#budget-and-accounting) section. On CINECA clusters a budget linearization policy is enforced. A detailed description is found in the specific [Budget Linearization](https://docs.hpc.cineca.it/hpc/hpc_intro.html#budget-linearization) section. CLOUD Cloud budget is assinged in Virtual CPUs (VCPUs) and Storage Your accounts are in the menu on the top left in Horizon Dashboard. ### Projects validity and expiration[](https://docs.hpc.cineca.it/general/users_account.html#projects-validity-and-expiration "Link to this heading") Project accounts allocated in UserDB have a fixed duration. Once reached the end date, the account enters a so called **expired** status. In the **expired** period, the budget cannot be used to submit jobs, but the data in the project specific areas are preserved. The expiration duration depends on the kind of project. Usually it is **6 months** for HPC accounts and **1 month** for Cloud accounts, but in special cases it may differ. At the end of the expiration delay all the data in the project specific areas are deleted. Please check the [Backup Policy and Data Availability](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#backup-policy-and-data-availability) for further details. Manage your UserDB credentials[](https://docs.hpc.cineca.it/general/users_account.html#manage-your-userdb-credentials "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------- In the UserDB portal, the users can manage the following information: * Changing the UserDB password * Request a new UserDB password * Update the affiliation Note It is **not possible** to update the email address directly on UserDB. The change has to be made on SSO portal and the UserDB will automatically update the email (see [Manage your HPC credentials](https://docs.hpc.cineca.it/general/users_account.html#manage-your-hpc-credentials) ). How to change the UserDB password > Login to UserDB portal, then click on Edit. Insert the Current Password and then below the New Password. If the quality is Good, click on Save at the bottom of the page. > > ![../_images/UserDBpassword.png](https://docs.hpc.cineca.it/_images/UserDBpassword.png) > > Please follow the [Policy for UserDB password definition](https://docs.hpc.cineca.it/general/users_account.html#policy-for-userdb-password-definition) > . How to recover the UserDB password > If you do not remember the password you can reset it by clicking on Request new password. You will be asked to insert the email that you used to register on UserDB. The portal will send you an email with a link to set a new password. > > Note > > If you insert a different email address, you will **not receive** any link. How to update the affiliation > Any time you chnge your affiliation if you still make use of CINECA resources, we kindly ask you to update your affiliation. > > ![../_images/Institution.png](https://docs.hpc.cineca.it/_images/Institution.png) > > Login to UserDB portal, click on Edit and then on Institution and update it. ### Policy for UserDB password definition[](https://docs.hpc.cineca.it/general/users_account.html#policy-for-userdb-password-definition "Link to this heading") If you change the password on UserDB portal here you can find the password policies: > * Password must not match last 3 passwords. > > * Password must not contain the username. > > * Password must be at least 10 characters in length. > > * Password must contain at least 2 digits, 4 letters, 2 lowercase and 2 uppercase characters. > > * The following special characters are allowed **(!”#$%&’()\*+,-./:;<=>?@\[\]^\_\`{|}~)** > Manage your HPC credentials[](https://docs.hpc.cineca.it/general/users_account.html#manage-your-hpc-credentials "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------- Thanks to the new Identity Provider, [sso.hpc.cineca.it](https://sso.hpc.cineca.it/) , users can now manage their HPC authentication credentials independently. This includes: * Changing the HPC password * Updating the email address associated to their User Account * Recovering the OTP generator and re-generating recovery codes For any issues with these procedures or if you have any questions, please contact [superc@cineca.it](mailto:superc%40cineca.it) . How to change the HPC _password_ > * If you have already configured the 2FA but do not remember the password > > * by clicking on Forgot password while authenticating on [https://sso.hpc.cineca.it](https://sso.hpc.cineca.it/) > > > ![../_images/pwd_1.png](https://docs.hpc.cineca.it/_images/pwd_1.png) > > * specify your _HPC Username_ or the _email_ address used to register on [UserDB](https://docs.hpc.cineca.it/general/users_account.html#userdb) > portal. You will receive on that email a link to reset the password. > > > ![../_images/pwd_2.png](https://docs.hpc.cineca.it/_images/pwd_2.png) > > * click on the link and insert a _new password_ according to the [Policy for HPC password definition](https://docs.hpc.cineca.it/general/users_account.html#policy-for-hpc-password-definition) > and click on submit > > > ![../_images/pwd_3.png](https://docs.hpc.cineca.it/_images/pwd_3.png) > > * If you **remember the password**, but just want to update it on SSO portal > > > * login on the [SSO](https://sso.hpc.cineca.it/) > > portal. > > > > * choose **Account security**. > > > > > > ![../_images/key_2.png](https://docs.hpc.cineca.it/_images/key_2.png) > > > > * select Update in **my password** section. Choose a _new password_ according to the [Policy for HPC password definition](https://docs.hpc.cineca.it/general/users_account.html#policy-for-hpc-password-definition) > > > > > > ![../_images/key_3.png](https://docs.hpc.cineca.it/_images/key_3.png) > > > Warning > > The `passwd` command has been disabled. If you still need to configure your 2FA but you don’t remember the password, or your password is expired, the above solutions **will not work**. Please write to [superc@cineca.it](mailto:superc%40cineca.it) > in order to solve the issue. How to change the _email_ address > > * login on the [SSO](https://sso.hpc.cineca.it/) > > portal. > > > > * select **Personal Info**. > > > > > > ![../_images/key_2.png](https://docs.hpc.cineca.it/_images/key_2.png) > > > > * edit the field `email` and `save`. > > > > > > ![../_images/key_1.png](https://docs.hpc.cineca.it/_images/key_1.png) > > > > Within 24 hours the email will be updated also on UserDB portal. > > Warning > > Users cannot change their email address directly on the UserDB portal. If you encounter issues while attempting to update your email address, please contact [superc@cineca.it](mailto:superc%40cineca.it) > for assistance. Recover OTP One-Time Code Generator If you lost access to your OTP generator, you can configure a new one using the recovery authentication codes saved during the initial OTP setup. After inserting username and password on SSO login procedure, the page will request the One-Time Password. Click on Try another way. ![../_images/otp_recover_1.png](https://docs.hpc.cineca.it/_images/otp_recover_1.png) * choose Recovery Authentication Code ![../_images/otp_recover_2.png](https://docs.hpc.cineca.it/_images/otp_recover_2.png) * Insert the requested **Recovery Code** (obtained during the first configuration of the OTP). ![../_images/otp_recover_3.png](https://docs.hpc.cineca.it/_images/otp_recover_3.png) Note The system requires a specific recovery code of the list of available codes. It is identified by the “#” on top of the Recovery code field. In the above example it is asking the first (#1) recovery code of the list. Important **Renew the recovery codes**: It is also possible to generate new recovery codes by clicking on Set up Recovery authentication codes in the section **Recovery authentication codes** of **Account Security** page. ### Policy for HPC password definition[](https://docs.hpc.cineca.it/general/users_account.html#policy-for-hpc-password-definition "Link to this heading") If you change the password on the portal [sso.hpc.cineca.it](https://sso.hpc.cineca.it/) , it will be automatically changed on all the clusters (the propagation can take up to one hour). Here we report the HPC password policies: > * The new password has to be at least 10 characters long and contains at least 1 capital letter, 1 number, and 1 special character **(!”#$%&’()\*+,-./:;<=>?@\[\]^\_\`{|}~)** > > * The password has a validity of 3 months. You will receive a reminder 10 days before the expiration when you login. > > * The new password has to be different from the previous 5 ones. > > * Any password change will be notified to the user by email. > --- # Access to the Systems — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * Access to the Systems * [View page source](https://docs.hpc.cineca.it/_sources/general/access.rst.txt) * * * Access to the Systems[](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems "Link to this heading") ====================================================================================================================== Accessing any section of the Cineca HPC systems requires activating two-factor authentication (**2FA**) for each user account. This enhanced security measure verifies a user’s identity by requiring a second, independent factor in addition to the account password. Even if the correct account password is used, **2FA** ensures that unauthorized access is prevented, significantly improving the system’s overall security. This access modality operates seamlessly for users, who continue to utilize standard protocols such as the SSH client. Before connecting to the cluster, users must request an SSH certificate from our Identity Provider (IP) via the _smallstep_ client. Upon making the request, a web page will automatically open in the browser, prompting users to authenticate with our IP by entering a one-time password (**OTP**). Following successful authentication, the server will issue a time-limited certificate valid for **12 hours**. This certificate allows users to connect to CINECA systems via _SSH client_. Important For the **First-time Access** and the activation of **2FA**, a user must complete the following steps: * Registration on [UserDB](https://docs.hpc.cineca.it/general/users_account.html#userdb) . * Correct configuration of 2FA and OTP ([How to activate the 2FA and the OTP generator](https://docs.hpc.cineca.it/general/access.html#how-to-activate-the-2fa-and-the-otp-generator) ). Important * If you are a **Cloud** user only, _smallstep_ installation is not required. * For **HPC Resources** you must use _smallstep_ client to allow **ssh** protocol ([How to configure smallstep client](https://docs.hpc.cineca.it/general/access.html#how-to-configure-smallstep-client) ). How to activate the **2FA** and the **OTP** generator[](https://docs.hpc.cineca.it/general/access.html#how-to-activate-the-2fa-and-the-otp-generator "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Follow the instructions below to activate the 2FA authenticator method and configure the OTP generator. 2FA activation and OTP configuration, step-by-step guide > * Step 1 - Activate the **2FA** > > * point the website [https://sso.hpc.cineca.it](https://sso.hpc.cineca.it/) > > * Sign in using your CINECA cluster username and password (received by email). > > * **At first login**, you’ll be prompted to verify your email, change your password, and configure your OTP (One-Time Password). > > > > > ![../_images/2fa_1.png](https://docs.hpc.cineca.it/_images/2fa_1.png) > > Important > > New users will automatically receive the link. Once it is set up, you’ll use the OTP alongside your password for logging into the cluster. > > If it’s your first time logging in and you haven’t received a valid access link, contact [superc@cineca.it](mailto:superc%40cineca.it) > . > > > if you already have access to the cluster via username and password but you haven’t yet done your first login on the new sso portal and you haven’t configured the 2FA, you need to write at [superc@cineca.it](mailto:superc%40cineca.it) > > to receive a valid link for the access. > > * Step 2 > > * After your first login, you will receive an email from CINECA with the subject “CINECA HPC Single Sign-On: Verify Your Email.” This email will be sent to the address you registered with on the UserDB portal. > > * The email contains a verification link labeled `Click on the link to proceed`. Once you click it, you will be prompted to define a new _password_. Please ensure that your chosen password complies with the [Policy for HPC password definition](https://docs.hpc.cineca.it/general/users_account.html#policy-for-hpc-password-definition) > . > > > > ![../_images/2fa_2.png](https://docs.hpc.cineca.it/_images/2fa_2.png) > > * Step 3 > > After the definition of a new valid password (that will replace the password used to login to CINECA clusters), you will be asked to configure the 2FA following few simple steps. > > > ![../_images/2fa_3.png](https://docs.hpc.cineca.it/_images/2fa_3.png) > > * Step 4 - OTP Applications > > > * Authentication codes can be generated using either _FreeOTP_ or _Google Authenticator_, or other compatible apps. If you don’t already have one of these apps, download it onto your smartphone. > > > > * Once the app is installed, use it to scan the QR code displayed on the setup page. The OTP will be automatically configured on your authenticator. > > > > * As a final step, enter the **6-digit** code that appears in the app to verify the correct configuration. If you have multiple OTPs in the app, the correct one will be labeled **“CINECA HPC: ”**. > > > > * After verifying the correct configuration, the following page will display your **Recovery codes**. Save these codes by downloading, printing, or copying them into a text file. > > > > > ![../_images/2fa_4.png](https://docs.hpc.cineca.it/_images/2fa_4.png) > > Warning > > > **Recovery codes** are requested to the user in case of problems with your management of the OTP codes (for example, issues with the app or smartphone lost), so saving them somewhere is very important. > > * Step 5 > > > * At this point, your 2FA is active and your OTP configuration is done. In case you are a user of HPC resources (not Cloud), you can follow the guide below to setup the smallstep client for the authentication certificate and to connect to the HPC clusters via ssh protocol. > > > Important If you need to regenerate the OTP, please refer to the procedure outlined in the [Manage your HPC credentials](https://docs.hpc.cineca.it/general/users_account.html#manage-your-hpc-credentials) . Users that want to get access _only_ to **Cloud** resources can skip the next steps. How to configure _smallstep_ client[](https://docs.hpc.cineca.it/general/access.html#how-to-configure-smallstep-client "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------ Once 2FA is enabled as authentication method for CINECA clusters, you will need to install and configure a 2FA-compatible program on your PC, to download the temporary certificate locally. At CINECA, we recommend using the **Smallstep** client. Note **smallstep** client is not required to access to cloud resources. To obtain and install the Smallstep executable, visit the Smallstep [website](https://smallstep.com/docs/step-cli/installation/index.html) and follow the installation instructions for your operating system. Alternatively, you can download the executable directly from the [GitHub repository](https://github.com/smallstep/cli/releases/tag/v0.26.1) . After installation, you must configure Smallstep according to your operating system’s requirements. If you encounter any issues or need further assistance, please contact [superc@cineca.it](mailto:superc%40cineca.it) . Linux/MacOS Systems > Warning > > Users with **Ubuntu Linux** operating systems (also for other Linux distributions) should not run the command sudo apt install step because this will install a different software that will give errors when following the rest of the instructions. > > * **Step 1** > > > > Configure _smallstep_ on your system by running the following command line instructions in your shell: > > > > step ca bootstrap \--ca-url\=https://sshproxy.hpc.cineca.it \--fingerprint 2ae1543202304d3f434bdc1a2c92eff2cd2b02110206ef06317e70c1c1735ecd > > > > Copy to clipboard > > the command output should be the following: > > > The root certificate has been saved in /.step/certs/root\_ca.crt. > > > > The authority configuration has been saved in /.step/config/defaults.json. > > > > Copy to clipboard > > > > Note > > > > If you have a previous version of smallstep installed and configured on your system, the client will ask if you want to overwrite the existing configuration. To save a copy of a previous version of smallstep installed and configured on your system, make a copy of the directory _.step_. > > * **Step 2** > > > > To use the certificate created in **Step 1**, the user should activate the **ssh-agent running** with the following command: > > > > eval $(ssh-agent) > > > > Copy to clipboard > > > > Note > > > > If the agent is already active, this step is not required. > > * **Step 3** > > To use the certificate for the authentication, use the command: > > step ssh login '' \--provisioner cineca-hpc > > Copy to clipboard > > The command will prompt an output as in this figure: > > ![../_images/ca_linux.png](https://docs.hpc.cineca.it/_images/ca_linux.png) > > * **Step 4** > > Once the certificate is created, a webpage will automatically open in your default browser. You will need to **sign in** using your cluster credentials (username and password). Afterwards, you will be prompted to enter a temporary code generated by your OTP application to complete the process. > > ![../_images/otp.png](https://docs.hpc.cineca.it/_images/otp.png) > > Once authenticated, you will see a **Success message** on your browser, meaning that the certificate has been generated and it is available on your PC. > > ![../_images/success_ca.png](https://docs.hpc.cineca.it/_images/success_ca.png) > > Note > > the certificate is valid for **12 hours** !!! If you reboot your PC, the certificate is lost and you need to download a new one (repeating step 3 step ssh login … ) ! > Windows users have multiple options: Windows Powershell * **Step 1** * Open the `Powershell`. Windows O.S will show the standard prompt ![../_images/powershell_1.png](https://docs.hpc.cineca.it/_images/powershell_1.png) * **Step 2** - Download and install step * From the `Powershell` prompt type the command: winget install Smallstep.step Copy to clipboard and type “Yes” or “Y” if prompted. After the installation, close and reopen Powershell so that the alias for step will become available. * **Step 3** - Configure smallstep client * Initialize smallstep client with the command: step ca bootstrap \--ca-url\=https://sshproxy.hpc.cineca.it \--fingerprint 2ae1543202304d3f434bdc1a2c92eff2cd2b02110206ef06317e70c1c1735ecd Copy to clipboard A successful configuration will show the message: The authority configuration has been saved in C:\\Users\\m.rossi\\.step\\config\\default.json. PS C:\\Users\\m.rossi> Get-Service \-Name ssh-agent Copy to clipboard * **Step 4** - Activation of the `ssh-agent` * On the O.S. Windows 11, the `ssh-agent` may not be active by default. You can verify the activation status with the command: Get-Service \-Name ssh-agent Copy to clipboard The output on `Powershell` should be: Status Name Display Name ------ \---- \------------- Running ssh-agent OpenSSH Authentication Agent Copy to clipboard * If the service is not in `running status`, it can be activated with: Start-Service \-Name ssh-agent Copy to clipboard Note If the service will not start after the comamnd `Start-Service -Name ssh-agent`, the user can force the activation by opening a `Powershell` with administration privileges (from the control panel of Windows) and use the following commands: Set-Service \-Name ssh-agent \-StartupType Auto Start-Service ssh-agent Copy to clipboard * **Step 5** - Check the certificate * Run the following command to get the timed certificate: step ssh login \--provisioner cineca-hpc Copy to clipboard * Please, refer to the [How to manage authentication certificates](https://docs.hpc.cineca.it/general/access.html#how-to-manage-authentication-certificates) section for furter information. Windows Subsystem Linux (**WSL**) To proceed, open a shell and install `Step` by carefully following the installation instructions for the Linux environment. **WSL** does not support separate tabs, and any new window will not recognize a previously issued Step certificate by default. To avoid generating a new timed certificate for each session, we recommend adding an automatic certificate verification process in your WSL .bashrc file. This process can utilize variables initialized by the `eval $(ssh-agent)` command and redirect them to an appropriate text file. For example: if \[ \-f ~/.bash\_agent \]; then . ~/.bash\_agent fi steptest\=$(step ssh list \--raw ''| step ssh inspect | grep "Valid") if \[ \-z "$steptest" \] then eval $(ssh-agent) echo "export SSH\_AUTH\_SOCK=$SSH\_AUTH\_SOCK" \> ~/.bash\_agent echo "export SSH\_AGENT\_PID=$SSH\_AGENT\_PID" \>> ~/.bash\_agent step ssh login '' \--provisioner cineca-hpc fi Copy to clipboard Clients SSH/SFTP under Windows There are many SSH or SFTP Clients available for Windows, that are of common usage but are not automatically configured for working with the new 2FA system. It is possible to login with them by exploiting the OpenSSH agent forwarding that can be set by taking advantage of another tool installable on Powershell, that is [WinSSH-Pageant](https://github.com/ndbeals/winssh-pageant) . * **Step 1** - as a prerequisite, complete the configuration for `Powershell` as explained in the dedicated paragraph above. * **Step 2** Download WinSSH-Pegeant > winget install winssh-pageant > > Copy to clipboard > > after downloading, you should find a new executable in the following path: > > `C:\Users\$Env:UserName\AppData\Local\Programs\WinSSH-Pageant\winssh-pageant.exe` > > The variable `$Env:UserName` will be specific for your personal workstation. > > * Create an alias to simplify the execution of `winssh-pageant.exe` > > > New-Alias winssh-pageant C:\\Users\\$Env:UserName\\AppData\\Local\\Programs\\WinSSH-Pageant\\winssh-pageant.exe > > Copy to clipboard > > Important > > Keep in mind though that Powershell keeps an alias alive only until the shell is closed. An easy permanent solution would be to copy the program winssh-pageant.exe to another folder, for example `C:\Users\$Env:UserName\scoop\shims` that has been already included permanently in the `PATH` variable by the previous installation of step and is therefore recognized by `Powershell` without the need of expliciting the full path. * **Step 3** > * Launch the WinSSH-pageant with the command: `winssh-pageant --sshpipe` > > * Check if `winssh-pageant` is active and works properly > > > Get-Process | Select-String pageant > > > > Copy to clipboard > > > > You should expect an output like: > > > > System.Diagnostic.Process (winssh-pageant) > > > > Copy to clipboard > * **Step 4** - Create a new certificate > step ssh login \--provisioner cineca-hpc > > Copy to clipboard At this point you can **connect** via SSH/SFTP client by using your preferred client. In the following, we report the proper settings for the most popular clients (Putty, WinSCP, Filezilla, MobaXterm). **Putty:** In the login window, check the category “Connection –> SSH –> Auth” and be sure that the boxes “Attempt authentication using Pageant” and “Allow agent forwarding” are ticked. ![../_images/putty_1.png](https://docs.hpc.cineca.it/_images/putty_1.png) **WinSCP:** In the login window, from the Advanced settings go to “SSH–> Authentication” and tick the boxes “Attempt authentication using Pageant” and “Allow agent forwarding”. Be sure that the file protocol is set to “SCP”. ![../_images/wscp_1.png](https://docs.hpc.cineca.it/_images/wscp_1.png) ![../_images/wscp_2.png](https://docs.hpc.cineca.it/_images/wscp_2.png) **Note**: It is possible that if you try to edit an already saved site, the ssh-agent won’t be recognized. If this is the case, we recommend to create a new site from scratch and configure it accordingly. The new site can then be saved and will keep working. **Note**: In certain cases, we noted that the procedure may not work at first try, and you will get an error at login even if everything is in order. In most cases, a simple reboot of your workstation solves the problem and the issue will not occur again. **FileZilla:** In your site configuration, be sure that the Protocol is set to “SFTP - SSH File Transfer Protocol” and the Logon type is set to “Normal”. ![../_images/filezilla.png](https://docs.hpc.cineca.it/_images/filezilla.png) **MobaXTerm:** In the upper menu bar with the general options, make sure that in “Settings” → “Configuration” → “SSH” the box “Use external Pageant” is ticked (it should be by default). ![../_images/moba_1.png](https://docs.hpc.cineca.it/_images/moba_1.png) After that, opening a simple ssh session should be enough. ![../_images/moba_2.png](https://docs.hpc.cineca.it/_images/moba_2.png) Other SSH/SFTP clients don’t seem to be working with this method and are currently not supported by CINECA (for example BitviseSSH), or haven’t been tested yet. We will keep updating the Userguide when other clients will be proven compatible. ### How to manage authentication _certificates_[](https://docs.hpc.cineca.it/general/access.html#how-to-manage-authentication-certificates "Link to this heading") Managing authentication certificates in HPC environments involves ensuring secure access to resources. Users typically generate certificates via an Identity Provider or a local client, like _Smallstep_, for secure session authentication. Key operations include: > * Certificate generation: Performed during initial setup or when required by the system. > > * Re-generating certificates: Necessary if the current certificate expires or becomes invalid. Users typically log in to the portal or use a client tool to request a new certificate. > > * Certificate renewal: Ensures continued access without disruptions, often automated if supported by the system. > For issues or detailed procedures, users should consult the documentation or contact: [superc@cineca.it](mailto:superc%40cineca.it) . How to re-generate the certificate > $ step ssh login '' \--provisioner cineca-hpc > > Copy to clipboard How to check the presence of a valid certificate > It is possible to check for the presence of a valid certificate either via ssh-agent or via step with one of the following commands: > > $ ssh-add \-L > ecdsa-sha2-nistp256-cert-v01@openssh.com AAAAKGVjZHNhLXNoYTItbmlzdHAyNTYtY2VydC12MDFAb3BlbnNzaC5jb20AAAAgYjJfSnpeTTNrMHB4Lm9yX3YjZWNxXyRxcHM9blRzU1gAAAAIbmlzdHAyNTYAAABBBAJRZ11/PIo0VJknlFMDa5BIaJp/w0OWd95ueZbWlQ4uG92aSZ+K8aKgkyDiOGla3x7l+saVT/pIR+x3zBgvwgkLrbmYufPPVAAAAA > EAAAAUbS5tb3Jnb3R0aUBjaW5lY2EuaXQAAAAMAAAACG1tb3Jnb3R0AAAAAGILhpwAAAAAYgv3HAAAAAAAAACCAAAAFXBlcm1pdC1YMTEtZm9yd2FyZGluZwAAAAAAAAAXcGVybWl0LWFnZW50LWZvcndhcmRpbmcAAAAAAAAAFnBlcm1pdC1wb3J0LWZvcndhcmRpbmcAAAAAAAAACnBlcm1pdC1wdHkAAAAAAAAADnBlcm1pdC11c2VyLXJjAAAAAAAAAAAAAABoA > AAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAE3K7f5piMLWXDm9c6kd+VAJmBClKXkQ9i/8E1UA9DcBFofX+r9JyBOULZSDkGtr84oqpNX0fa5DMCar3AQp1YAAABkAAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAABJAAAAIDg33ohPQ6BgzV1ATGsSVSbRwrbYa8LprV2EEHk4mMgWAAAAIQCkd8QKYS+zbeyD1nXeuRAXVWJXJeoxMScgDVx2 > qqu2Mg\== > > $ step ssh list > 256 SHA256:x+QEW8xmDBtRjVRtAukc7v7zKEHef/9joyFP9n/gZtk (ECDSA-CERT) > > Copy to clipboard How to examine the validity of the current certificate > $ step ssh list \--raw '' | step ssh inspect > > -: > Type: ecdsa-sha2-nistp256-cert-v01@openssh.com user certificate > Public key: ECDSA-CERT SHA256:TdhIpD5KFZD37roGYcDstS7180TruOnNgNJeS8eJJPk > Signing CA: ECDSA SHA256:e0ZF6AnnUzi0g7Db9nOaXxkEjRq9D6Ka4tV04XqiIgM > Key ID: "" > Serial: 841532770994081620 > Valid: from 2025\-05-12T11:55:24 to 2025\-05-12T19:55:24 > Principals: > > Critical Options: (none) > Extensions: > permit-X11-forwarding > permit-port-forwarding > permit-pty > > Copy to clipboard How to create a certificate in file format If it is necessary to avoid using ssh-agent, you can download your certificate launching the following command in any path of your local PC (we suggest using the ~/.ssh folder): step ssh certificate 'user-email' \--provisioner cineca-hpc my\_key Copy to clipboard You can change **my\_key** with the name you prefer. A passphrase to encrypt the private key is request as input in the shell command line: Please enter the passphrase to encrypt the private key: Copy to clipboard Note **Three keys** will be generated in the path where you executed the above command. To use the keys to access the cluster you can place the three files in the `~/.ssh folder` (default path), or you can specify `-i ` within the ssh command, and enter the passphrase you selected in the previous step: $ ssh \-i /path/my\_key @login..cineca.it Enter passphrase for key \`\`my\_key\`\` Copy to clipboard Access via Secure Shell (**SSH**)[](https://docs.hpc.cineca.it/general/access.html#access-via-secure-shell-ssh "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------- SSH is commonly employed for remote access to a machine, allowing users to execute commands (remote console), run programs, and transfer files securely. On Linux and Mac systems, the SSH client is typically pre-installed. However, on Windows systems, users need to download and install an SSH client. Some popular SSH clients for Windows include [Powershell](https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/powershell) , [openSSH](https://docs.microsoft.com/en-gb/windows-server/administration/openssh/openssh_install_firstuse) , [Putty](https://www.putty.org/) or [Tectia](https://www.ssh.com/products/tectia-ssh) . Connection adopting 2FA procedure does not require to provide password. The access is done via one of the following commands: $ ssh @login.marconi.cineca.it $ ssh @login.g100.cineca.it $ ssh @login.leonardo.cineca.it $ ssh @login.pitagora.cineca.it Copy to clipboard You can use the option `-X` to enable **X11** display forwarding. Important * After **12 hours**, a new certificate must be generated using the smallstep client ([How to manage authentication certificates](https://docs.hpc.cineca.it/general/access.html#how-to-manage-authentication-certificates) ). * You will login to our systems with one of the two shells: **bash** or **tcsh**. Contact the HPC support ([superc@cineca.it](mailto:superc%40cineca.it) ) if you want to change your default login shell. * Login is prevented on systems in which you don’t have a budget account. * We have identified a potential issue for local PC with openssh 8.6 (check with the command `ssh -V`). The solution can be found here in our [FAQ](https://docs.hpc.cineca.it/faq.html#faq) page. Troubleshooting - WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! The DNS listed above to access our clusters are aliases pointing at different login nodes. When accessing multiple times on the same cluster, if you end-up on a different login node than the previous session you may get the following error: **“WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!”** the reason is a possible mismatch between the DNS and the node’s fingerprint saved in your known\_hosts file. To solve this issue, we sugget the following steps. Linux/MacOS and Windows WSL Open your terminal, get the 2FA certificate, and run the following command: ssh-keygen \-f ~/.ssh/known\_hosts \-R ; for keyal in ssh-rsa ecdsa-sha2-nistp256; do for address in ; do ssh-keyscan \-t ${keyal} ${address} | sed "s/\\b${address}//g" \>> ~/.ssh/known\_hosts; done; done Copy to clipboard where `` is the domain name system of the cluster, and `` is a wildcard pattern derived from the cluster DNS used to generalize host entries in the known\_hosts file. An example for Leonardo cluster: ssh-keygen \-f ~/.ssh/known\_hosts \-R login.leonardo.cineca.it; for keyal in ssh-rsa ecdsa-sha2-nistp256; do for address in login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it; do ssh-keyscan \-t ${keyal} ${address} | sed "s/\\b${address}/login\*.leonardo.cineca.it/g" \>> ~/.ssh/known\_hosts; done; done Copy to clipboard Note You can find the previously configured one-line command for each cluster in our [FAQ](https://docs.hpc.cineca.it/faq.html#faq) . To generalize you can use the following: \# Define the main cluster DNS name cluster\_dns\=login.leonardo.cineca.it \# Define an array of explicit DNS names for each login node clustes\_explicit\_dns\=( login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it ) \# Generate a generic DNS pattern by replacing the first '.' with '\*.' \# (e.g., "login.leonardo.cineca.it" → "login\*.leonardo.cineca.it") cluster\_generic\_dns\="${cluster\_dns/./\*.}" \# Remove any existing SSH key entries for the main cluster DNS from known\_hosts if \[ \-f ~/.ssh/known\_hosts \] then ssh-keygen \-f ~/.ssh/known\_hosts \-R ${cluster\_dns} fi \# Loop over the key algorithms to scan (e.g., ssh-rsa, ecdsa-sha2-nistp256) for keyal in ssh-rsa ecdsa-sha2-nistp256 do \# Loop over all explicit login node DNS entries for address in ${clustes\_explicit\_dns\[@\]} do \# Retrieve the SSH public key for the node using ssh-keyscan \# Replace the explicit node address with the generic pattern in the output \# Append the result to the known\_hosts file ssh-keyscan \-t ${keyal} ${address} | sed "s/\\b${address}/${cluster\_generic\_dns}/g" \>> ~/.ssh/known\_hosts done done Copy to clipboard Windows Powershell and other SSH clients Open your **Powershell** terminal, get the 2FA certificate, and run the following command: ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R ""; foreach ($keyal in "ssh-rsa", "ecdsa-sha2-nistp256") { foreach ($address in ) { ssh-keyscan \-t $keyal $address | ForEach\-Object { $\_ \-replace "\\b$address\\b", "" } \>> "$HOME\\.ssh\\known\_hosts" } } Copy to clipboard where `` is the domain name system of the cluster, and `` is the list of the explicit DNS of the login nodes. An example for Leonardo cluster: ssh-keygen \-f "$HOME\\.ssh\\known\_hosts" \-R "login.leonardo.cineca.it"; foreach ($keyal in "ssh-rsa", "ecdsa-sha2-nistp256") { foreach ($address in "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it") { ssh-keyscan \-t $keyal $address | ForEach\-Object { $\_ \-replace "$address", "login\*.leonardo.cineca.it" } \>> "$HOME\\.ssh\\known\_hosts" } } Copy to clipboard Note You can find the previously configured one-line command for each cluster in our [FAQ](https://docs.hpc.cineca.it/faq.html#faq) . To generalize you can use the following: \# Define the main cluster DNS name $cluster\_dns \= "login.leonardo.cineca.it" \# Define an array of explicit DNS names for each login node $clusters\_explicit\_dns \= @( "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it" ) \# Generate a generic DNS pattern by replacing the first '.' with '\*.' \# (e.g., "login.leonardo.cineca.it" → "login\*.leonardo.cineca.it") $cluster\_generic\_dns \= $cluster\_dns \-replace '\\.', '\*.', 1 \# Path to known\_hosts $known\_hosts\_path \= "$HOME\\.ssh\\known\_hosts" \# Remove any existing SSH key entries for the main cluster DNS from known\_hosts if (Test-Path $known\_hosts\_path) { ssh-keygen \-f $known\_hosts\_path \-R $cluster\_dns | Out-Null } \# Loop over the key algorithms to scan $keyal\_list \= @("ssh-rsa", "ecdsa-sha2-nistp256") foreach ($keyal in $keyal\_list) { foreach ($address in $clusters\_explicit\_dns) { \# Retrieve the SSH public key for the node $keyscan\_output \= ssh-keyscan \-t $keyal $address 2\>$null \# Replace the explicit node address with the generic pattern $updated\_output \= $keyscan\_output \-replace \[regex\]::Escape($address), $cluster\_generic\_dns \# Append the result to the known\_hosts file Add-Content \-Path $known\_hosts\_path \-Value $updated\_output } } Copy to clipboard Access via Remote Visualization (**RCM**)[](https://docs.hpc.cineca.it/general/access.html#access-via-remote-visualization-rcm "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------- Remote visualization has become popular as an HPC service since it allows to: * **visualize** the data produced on HPC infrastructure (scientific visualization) * **analize** and **inspect** data directy on the HPC infrastructure. * **debug** and **optmization** of codes via GUI tools directly on the HPC infrastructure. All the aforementioned use cases can take advantage of use applications on the **server side**. The requirements to access to this service are the same for the access to HPC infrastructure. The remote visualization service at Cineca is provided through the Remote Connection Manager (**RCM**) application. Using this tool you can graphically inspect your data without moving them to your local work station. Important It can be used by any user with valid credentials to access CINECA clusters. * [Download RCM Software](https://docs.hpc.cineca.it/rcm/rcm.html) * [Basic Usage of RCM](https://docs.hpc.cineca.it/rcm/rcm.html#basic-usage-of-rcm) * [How to create a new Display](https://docs.hpc.cineca.it/rcm/rcm.html#how-to-create-a-new-display) * [How to share a Display](https://docs.hpc.cineca.it/rcm/rcm.html#how-to-share-a-display) * [How to kill a Display](https://docs.hpc.cineca.it/rcm/rcm.html#how-to-kill-a-display) * [Running a GUI-based Software](https://docs.hpc.cineca.it/rcm/rcm.html#running-a-gui-based-software) Continuous Integration (**CI/CD**) at CINECA[](https://docs.hpc.cineca.it/general/access.html#continuous-integration-ci-cd-at-cineca "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------- Continuous Integration, continuous delivery and deployment, also known as **CI/CD**, is the practice of a constant monitoring of the code development through the activation of an automatic pipeline. Every time a developer applies a change to the code, the automatic pipeline validates it by building the code and running some simple tests (typically unit tests). CINECA has activated a new service where to run your Continuous Integration (CI/CD) pipelines on CINECA clusters, based on CINECA GitLab service. On [Gitlab website](https://docs.gitlab.com/ee/ci/) there is a detailed documentation about CI/CD. This service is active on Galileo100, and is at the moment in an experimental phase. How to use it You can activate our CI/CD service in projects defined into our [GitLab instance](https://gitlab.hpc.cineca.it/) . If you are already a CINECA HPC user, you can access the CINECA GitLab using the same credentials. If you are interested and you are still not an HPC User you can find [here](https://docs.hpc.cineca.it/general/users_account.html#how-to-become-a-user) the instructions on how to get access. Once logged to CINECA GitLab, you can activate the CI/CD service by enabling shared runners that pick up and execute your CI/CD pipeline on our cluster. They can be enabled as in the following: > 1. From your project’s web page, select “_Settings_” and then on “_CI/CD_”, from the menu on the left; > > 2. Once in the web page, expand the section “_Runners_”; > > 3. Activate the switch under “_Enable shared runners for this project_” on the right, in the “_Shared Runners_” right column. The _shared runners_ are listed in that section along with blue labels specifying the _tags_ associated to them. > Now shared runners are available to your CI/CD pipeline. The CI/CD pipeline has to be specified inside the `.gitlab-ci.yml` file through tags (see [Gitlab documentation](https://docs.gitlab.com/ee/ci/) for how to create and manage pipelines). **IMPORTANT**: There are **two different kind of runners**. You have to identify **which runner** you would like to run your pipeline by **specifying one or more tags** summarized in the table at the bottom of the page. **IMPORTANT**: If you do not select any tag, the pipeline **will never start**. We set a **time limit** for the execution of each single job of a given pipeline that cannot last for more than **10 minutes**. Runners description All shared runners are based on [docker images](https://docs.gitlab.com/runner/install/docker.html) , so in your CI/CD pipeline you can optionally choose in which **container image** your pipeline job will run. You will find 4 distinct shared runners, consisting of: * **2 CPU-only** runners, with access to up to 24 CPUs each. Jobs are executed in concurrent execution. (specific tags: **x86\_64, cpu, docker**) * **2 CPU+GPU** runners, limited to run 1 CI job each at the time. Each runner has access to a dedicated GPU. **No concurrent execution** is allowed on these runners. (specific tags: **x86\_64, docker, nvidia-sm70, nvidia-volta, nvidia-cuda**) All shared runners run on a dedicated node of Galileo100 with **Intel x86\_64 architectures** (2 x CPU Intel CascadeLake 8260 processors with 24 cores each, 2.4 GHz, 384GB RAM). GPU runners make use of **Nvidia** V100 GPUs. Summary Below we summarize the runners and the tags needed to select the correct one. | **Runners** | **Tags** | **Notes** | | --- | --- | --- | | 2 CPU-only | x86\_64,cpu,docker | Up to 24 cpus each. Concurrent execution | | 2 CPU+GPU | x86\_64, docker, nvidia-sm70, nvidia-volta, nvidia-cuda | Each runner has a dedicated GPU. No concurrent execution. | **IMPORTANT**: **If you do not select any tag**, the pipeline **will never start**. Access via X.509 certificate[](https://docs.hpc.cineca.it/general/access.html#access-via-x-509-certificate "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------ An X.509 certificate is issued by a trusted Certificate Authority (CA), which verifies the user’s identity and ensures the certificate is both valid and associated with a real individual. It is used as an authentication mechanism, serving as an alternative to traditional username/password credentials, and helps avoid the need to replicate user accounts across systems. When connecting to a service, the user’s certificate is mapped to a local account, under which all commands and operations are executed. Important **Access to CINECA clusters no longer requires or supports X.509 certificates for authentication.** However, X.509 certificates may still be necessary for accessing certain external services or resources, such as data repositories, collaboration platforms, or grid computing infrastructures. The following describes the procedure for obtaining an X.509 certificate and generating a proxy certificate for temporary use. **How to get your X.509 certificate** * Users in need of an X.509 certificate can visit the [HARICA website](https://cm.harica.gr/) . Academic users can select _Academic Login_ and authenticate using their institutional credentials. * In the dashboard’s left-hand menu, click on _IGTF Certificate_, then select _GÉANT Personal Authentication_. * Review and accept the terms and conditions, then click _Submit Request_. ![../_images/X509_1.png](https://docs.hpc.cineca.it/_images/X509_1.png) * Under _Ready Certificates_, click _Enroll your Certificate_. Choose your preferred algorithm (RSA or ECDF), then click _Enroll Certificate_ again. After enrollment, click _Download_. ![../_images/X509_2.png](https://docs.hpc.cineca.it/_images/X509_2.png) * A `.p12` file (containing your personal certificate and private key) will be downloaded. **How to Use Your X.509 Certificate (Browser and Command Line)** Once you have downloaded your `.p12` certificate file, you can either: * Import it into your browser for web-based authentication. * Convert it into PEM format (`cert.pem` and `key.pem`) to use it with command-line tools such as `grid-proxy-init`, `voms-proxy-init`, or `globus-url-copy`. Using the .p12 Certificate in Your Browser > _Firefox_: > > > * Go to Settings → Privacy & Security → Certificates → View Certificates. > > > > * Open the Your Certificates tab. > > > > * Click Import, then select your .p12 file. > > > > * Enter the password used to protect the certificate. > > > > > > Your certificate is now ready for web authentication. > > _Chrome / Edge / Safari_: > > > These browsers use the system certificate store. > > > > * On Linux (GNOME): Use the Passwords and Keys application (Seahorse) → Import your .p12 file. > > > > * On macOS: Double-click the .p12 file to open Keychain Access → import it into the login or system keychain. > > > > * On Windows: Double-click the .p12 file → follow the Certificate Import Wizard, choose “Current User”, and confirm the installation. > > Converting .p12 to cert.pem and key.pem for Command-Line Use > To use your certificate with command-line tools, you’ll need to copy the `cert.p12` file in your `$HOME`, create the directory `$HOME/.globus` and finally extract the certificate and private key from the `.p12` file using OpenSSL: > > **Extract the private key:** > > > $ mkdir $HOME/.globus > > $ openssl pkcs12 \-nocerts \-in cert.p12 \-out $HOME/.globus/userkey.pem > > > > Copy to clipboard > > **Extract the User certificate:** > > > $ openssl pkcs12 \-clcerts \-nokeys \-in cert.p12 \-out $HOME/.globus/usercert.pem > > > > Copy to clipboard > > **Protect your keys:** > > > $ chmod 600 $HOME/.globus/userkey.pem $HOME/.globus/usercert.pem > > > > Copy to clipboard --- # Unknown ADA cloud user documentation Getting started (2): How to cancel cloud resources HPC Cloud support group Last update: 30 May 2025 STEP BY STEP user GUIDE Getting started (2) workflow: deleting resources Access your cloud resources Disassociate floating IP Delete instance Delete your VM Delete your network Clear gateway & delete interface Delete network Delete router Delete security group Delete keypair Visit the HPC Cloud User guide for more information STEP BY STEP user GUIDE Getting started (2) workflow: deleting resources Access your cloud resources Disassociate floating IP Delete instance Delete your VM Delete your network Clear gateway & delete interface Delete network Delete router Delete security group Delete keypair Visit the HPC Cloud User guide for more information ADA Cloud dashboard 2 -Access your cloud resources •Go to https://adacloud.hpc.cineca.it •Select "CINECA HPC" as Authentication method •Insert your HPC-CINECA credentials to log in •NOTE: the 2nd factor needs to be activated (see sectionManaging password, 2FA and OTP) Access your cloud resources Delete your network Delete your VM STEP BY STEP user GUIDE Getting started (2) workflow: deleting resources Access your cloud resources Disassociate floating IP Delete instance Delete your VM Delete your network Clear gateway & delete interface Delete network Delete router Delete security group Delete keypair Visit the HPC Cloud User guide for more information 1.1 - Disassociate thefloating IP from your VM 1–Delete your VM Click Click on «Disassociate» for the IP Go to the section «Network/FloatingIPs» on the left-side menu Access your cloud resources Delete your network Delete your VM 1.1 - Disassociate thefloating IP from your VM 1–Delete your VM Click If desired, you can also release the floating IP. Important: once you release it, there is no guarantee the same IP can be allocated again. Access your cloud resources Delete your network Delete your VM 1.2 - Delete instance 1–Delete your VM Go to section «Compute/Instances» on the left-side menu Access your cloud resources Delete your network Delete your VM Display thedrop-down menu of the instance youwant to delete Click the action «Delete Instance» STEP BY STEP user GUIDE Getting started (2) workflow: deleting resources Access your cloud resources Disassociate floating IP Delete instance Delete your VM Delete your network Clear gateway & delete interface Delete network Delete router Delete security group Delete keypair Visit the HPC Cloud User guide for more information 2.1 - Clear Gateway and deleteinterface 2-Delete your network Go to «Network/Routers» on the left-side menu Then,click on your router name and select the «Interfaces» tab Access your cloud resources Delete your network Delete your VM Click «Delete Interface» button Clickthe «Clear Gateway» button corresponding to your router 2.2 - Deletenetwork 2-Delete your network Click the action «Delete Network» Access your cloud resources Delete your network Delete your VM Go to «Network/Networks» on the left-side menu Display the drop-down menu for your network 2.3 - Deleterouter 2-Delete your network Click the action "Delete Router" Access your cloud resources Delete your network Delete your VM Go to "Network/Routers" on the left-side menu Display the drop-down menu for your router 2.4 - Deletesecurity groups 2-Delete your network Click the action "Delete Security Group" Access your cloud resources Delete your network Delete your VM Go to «Network/Security Groups» on the left-side menu Display the drop- down menu for your security group Click on «Delete Key Pair» 2.5 - Delete your keypair 2-Delete your network Access your cloud resources Delete your network Delete your VM Go to «Compute/Key Pairs» on the left-sidemenu CINECA For any issue or question, please contact the HPC User support at superc@cineca.it --- # Unknown .. \_get\_str\_card: Getting Started =============== \*\*Welcome to CINECA HPC !!!\*\* Here you will find in few simple steps the instructions to get your first access to CINECA HPC resources. \*\*Let's start !!!\*\* .. card:: 1. Create your personal \*\*User account\*\* on \`UserDB \`\_ portal. \* Visit the :ref:\`general/users\_account:How to become a User\` section for detailed information. \* Please, consider that, once completed, the registration alone does not grant the access to the HPC resources. .. card:: 2. Get associated to a valid \*\*Project Account\*\*. \* Visit the :ref:\`general/users\_account:Project Accounts\` section to find all the ways you can be granted a \*\*Project Account\*\* on our HPC resources. \* If you are the \*Principal Investigator (PI)\* of a project, please write to superc@cineca.it to be associated. \* If you are a collaborator, please ask to the PI of the Project Account to be associated. \* Visit the :ref:\`general/users\_account:PI and Collaborators\` section for more information. .. card:: 3. Submit a request to get access to CINECA HPC resources \* by clicking on Submit button in HPC Access page on UserDB (:ref:\`general/users\_account:Submit a request to have a \*\*User Account\*\*\`) \* Once enabled, we will provide you with a HPC username and a link to configure the 2FA (password + OTP token) .. card:: 4. Configure your 2FA \* click on the link arrived via email and configure your HPC password and OTP token as described here :ref:\`general/access:Access to the Systems\` .. card:: 5. Select the \*\*Infrastracture\*\* \* According to resources assigned to your \*\*Project Account\*\*, choose the :bdg-primary:\`HPC\`, or :bdg-secondary:\`Cloud\` infrastructure. .. dropdown:: HPC Infrastructure :animate: fade-in-slide-down :color: primary .. card:: 6. Configure the \*\*smallstep client\*\* The smallstep client is needed to get the temporary 2FA certificate to access the cluster (:ref:\`general/access:How to configure \*smallstep\* client\`) .. card:: 7. \*\*Connect to the Cluster\*\* Open a new shell/terminal and use the following commands to connect: .. code-block:: bash $ step ssh login --provisioner cineca-hpc $ ssh @login..cineca.it Visit the page :ref:\`general/access:Access to the Systems\` to find instructions about other possible ways to connect .. important:: You can login only to clusters where you have active budgets on it. .. card:: 8. \*\*Managing Data Files\*\* Get familiar with the FS areas and where they are located. On our HPC clusters there are several storage areas. The basic ones are: $HOME, $WORK and $SCRATCH, but sometimes there may be more. In section :ref:\`hpc/hpc\_data\_storage:File Systems and Data Management\` you can find a description of all of them with their properties and limitations. .. card:: Explore the \*\*Enviroment\*\* Check available software and/or compilers and pick-up the most convenient for your purpose. In the :ref:\`hpc/hpc\_enviroment:Environment and Customization\` section is described how to check, load and get info of the available software, enviroment and compilers. .. card:: 9. Check your \*\*budget status\*\* Verify the status of your Project account budgets and their name with saldo command. A descrption of the saldo command and additional flags that may be needen in some cases are described in :ref:\`hpc/hpc\_intro:Budget and Accounting\`. .. card:: 10. \*\*Submit your jobs\*\* Execute your simulation on compute nodes of our HPC cluster by submitting a job script to our SLURM scheduler. In :ref:\`hpc/hpc\_scheduler:Scheduler and Job Submission\` you can find how to properly create a job script and to submit it. .. dropdown:: Cloud Infrastructure :animate: fade-in-slide-down :color: secondary .. card:: 6. Learn about \*\*OpenStack\*\* Follow the link for more information: :ref:\`cloud/os\_overview/index\_openstack\_overview:what is openstack?\` .. card:: 7. Access the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` You can find the link to the dashboard and other useful information on the page specific to the cloud you have been assigned to: :ref:\`cloud/systems/ada:ada\` or :ref:\`cloud/systems/gaia:gaia\` .. important:: You can only login on clouds where you have active budgets on. .. card:: 8. Setup your \*\*project\*\* Create your network stack and your first virtual machine instance: - :ref:\`cloud/operative/network\_ops/network\_create:network: create\` - :ref:\`cloud/operative/network\_ops/secgroups\_create:security groups: create\` - :ref:\`cloud/operative/compute\_ops/instance\_create:instance: create\` .. note:: Be sure to check our :ref:\`cloud/tenant\_adm/index\_tenants\_administration:tenants administration\` tips --- # Unknown Introductionto two-factor authentication (2fa) on CINECA HPC clusters June 7th, 2023 Francesco Talpo –f.talpo@cineca.it CINECA -High Performance Computing Department Summary How to activate 2FA for HPC access and configure the mobile authenticator How to connect to the HPC clusters with SSH certificates using Smallstep FAQs and common problems How to recover HPC password and OTP generator Three possible scenarios: 1–You have a valid CINECA HPC username and password 2 – You have a valid HPC username, but your password is expired, or you forgot it 3 – You are registering as a new HPC user Registeringto the Identity Provider Authenticate on ournew Identity Providerat: https://sso.hpc.cineca.it using username and password you use to connect to CINECA clusters 1 –Registeringto the Identity Provider with a validCINECA HPC password At the first login you will be forced to: ❑verify your email ❑change the password ❑configure your One-Time Password(OTP) code An e-mail containing a link will be sent to the e-mail address indicated into the UserDB site: Subject"CINECA HPC Single Sign On: verify your email" 1 –Registeringto the Identity Provider with a validCINECA HPC password Write to superc@cineca.it, and we will send you the link (with a validity of 12 hours) to verify your e-mail address and register to the Identity Provider 2 –Registeringto the Identity Provider WITHOUT a validCINECA HPC password 3 –Registeringto the Identity Provider asa new CINECA HPC user You can register following the procedure reported on the User Guide, and you will receive,in two separate e-mails, the username and the link for e-mail verification and for registering to the Identity Provider Following the link received in the e-mail you will be forced to change the password: The new defined password will replace the password used to login to CINECA cluster How to activate2FAand configurethe OTP How to activate2FAand configurethe OTP Please refer to the policy for password definition on our User Guide; specifically: •The new password must be 10 characters long and contains at least 1 capital letter, 1 number, and 1 special character (!"#$%&'()\*+,-./:;<=>?@\[\\\]^\_\`{|}~) •The password has a validity of 3 months. You will receive a reminder 10 days before the expiration when you login. •The new password must be different from the previous 5 ones. •Any password change will be notified to the user by email. WARNING: in this case the process will fail silently without any error ! ! Next step after the definition of the new password is the activation of the 2FA via OTP following these simple steps: How to activate2FAand configurethe OTP First step: install on your mobile an App to generate authentication codes: - FreeOTP -Google Authenticator -other Second step: once the app is installed, you can use your authenticator toscan the QRcode shown in the page. The OTP will be automatically configured on your authenticator. If you have problems in configuring the 2FAon your smartphone, contact us at: superc@cineca.it Third step: you will be asked to insert the 6 digits code that appears on the App toverify the correct configuration. If you have multiple OTP defined in the App, the correct one has the name "CINECA HPC: ". How to activate2FAand configurethe OTP Oncetheconfiguration is complete the subsequent page will show you theRecovery codes. Pleasesave these codes somewhereby downloading, printing or copying them in a text file These codes are requested to the user in case of problems in the OTP configuration (issue with the app or smartphone lost) so they are very important (ALL of them). They are one-shot codes, and more can be generated. Now 2FA and OTP are enabled and configured. How to activate2FAand configurethe OTP Summary How to activate 2FA for HPC access and configure the mobile authenticator How to connect to the HPC clusters with SSH certificates using Smallstep FAQs and common problems How to recover HPC password and OTP generator How to configureSSH access to the HPC Clusters HPC clusters can be accessed through SSH with a temporary certificate obtained via the smallstepsoftware. You can setup the smallstep client in several ways: Either follow the instructions on the Smallstep website Or download an executable from the official GitHub repository !! WARNING !! Some Linux distributions (Ubuntu ...) may have a completely different "step" package available in the distribution's official repositories; DO NOT INSTALL IT UNLESS SURE THAT IT'S THE SAME SOFTWARE, OTHERWISE IT MAY LEAD TO ERRORS How to configureSSH access to the HPC Clusters –Linux/MacOS Configure the smallstep client for SSH access with the following commands: Get the CA certificate: $ step ca bootstrap --ca-url=https://sshproxy.hpc.cineca.it --fingerprint 2ae1543202304d3f434bdc1a2c92eff2cd2b02110206ef06317e70c1c1735ecd Activate the ssh-agent: $ eval $(ssh-agent) Get the temporary certificate: $ step ssh login '' --provisioner cineca-hpc IS THE USERDB MAIL ADDRESS How to configureSSH access to the HPC Clusters –Linux/MacOS You will be redirected to a web page asking for your HPC credentials (username, password) and OTP: At this point, the temporary certificate will be passed to the ssh-agent, and you will be able to connect to the cluster via SSH. How to configureSSH access to the HPC Clusters –SSH keys Alternatively, the command: $ step ssh certificate 'user-email' --provisioner cineca-hpc my\_key Followed by the same login procedure on the Identity Provider page Will download to your system a pair of public/private ssh keyswith a limited validity of 12 hours How to configureSSH access to the HPC Clusters –SSH keys If you passed "my\_key" as the last argument to the previous command, you will find the files "my\_key", "my\_key.pub" and "my\_key-cert.pub" in your current directory. Only "my\_key" and "my\_key-cert.pub" are needed to access the cluster, which can be done: With the command: ssh -i my\_key @login..cineca.it passing the correct identity directly to the ssh command Or with the commands: ssh-add my\_key ssh @login..cineca.it which will add the key to the ssh agent before connecting to the cluster How to configureSSH access to the HPC Clusters –Windows For Windows some of the commands are slightly different: Get the CA certificate: > step ca bootstrap --ca-url=https://sshproxy.hpc.cineca.it --fingerprint 2ae1543202304d3f434bdc1a2c92eff2cd2b02110206ef06317e70c1c1735ecd Activate the ssh-agent: > Get-Service -Name ssh-agent > Start-Service -Name ssh-agent Get the temporary certificate: > step ssh login '' --provisioner cineca-hpc How to configureSSH access to the HPC Clusters –Windows If, when activating the ssh agent, these commands don't work: > Get-Service -Name ssh-agent > Start-Service -Name ssh-agent the following commands need to be executed in a Powershell instance with admin rights: > Set-Service -Name ssh-agent -StartupType Auto > Start-Service ssh-agent How to configureSSH access to the HPC Clusters –Windows Alternatively on Windows it is possible to install WSL2 (https://learn.microsoft.com/en- us/windows/wsl/install) and configure the subsystem following theinstructions for Linux In this case, to share the certificate between WSL tabs, the following lines can be added to .bashrc: if \[ -f ~/.bash\_agent \]; then . ~/.bash\_agent fi steptest=$(step ssh list --raw ''| step ssh inspect | grep "Valid") if \[ -z "$steptest" \] then eval $(ssh-agent) echo "export SSH\_AUTH\_SOCK=$SSH\_AUTH\_SOCK" > ~/.bash\_agent echo "export SSH\_AGENT\_PID=$SSH\_AGENT\_PID" >> ~/.bash\_agent step ssh login '' --provisioner cineca-hpc fi How to configureSSH access to the HPC Clusters –Usefulcommands You can check for the presence of a valid certificate, in both Linux/MacOS and Windows systems, with the commands: > ssh-add –L > step ssh list And, to display the validity of the certificate, you may run the command: > step ssh list --raw '' | step ssh inspect If you want to "clean" the ssh-agent memory from any memorized key and certificate you can do so with the command: > ssh-add -D Summary How to activate 2FA for HPC access and configure the mobile authenticator How to connect to the HPC clusters with SSH certificates using Smallstep FAQs and common problems How to recover HPC password and OTP generator How to reset the HPC password – ONLY IF ALREADY REGISTERED ON "sso.hpc.cineca.it" The new Identity Provider system allows users to recover their HPC password: you can do so by clicking Forgot Password? on the Identity Provider login webpage You will then receive an e-mail with a temporary link for password reset How to reset the HPC password – ONLY IF ALREADY REGISTERED ON "sso.hpc.cineca.it" If you just need to change your password, you can do it by clicking on the My Password- Update button in the Account Security section of your Indentity Provider page at https://sso.hpc.cineca.it NOTE:the passwd command has been disabled on the clusters How to recoverthe OTP generator If you have any issues with these procedures, you can contact us at: superc@cineca.it If you lose your OTP generator, you can reset it by clicking on Forgot Password? in the Identity Provider loginpage, then following the link received via e-mail and then clicking on Try Another Way when prompted for the OTP code How to recoverthe OTP generator You will then be asked to insert a specific code from the Recovery codes that you were provided with during the registration with the Identity Provider (https://sso.hpc.cineca.it) Summary How to activate 2FA for HPC access and configure the mobile authenticator How to connect to the HPC clusters with SSH certificates using Smallstep FAQs and common problems How to recover HPC password and OTP generator FAQsand common problems Q: I forgot my password, or it is expired, but "Forgot your password?" takes me to this page -> A: you can see the "Forgot password?" link even if you're not registered to the Identity Provider, but it will be broken like this. Write to superc@cineca.it to get your registration link. FAQsand common problems Q: Trying to install smallstep through scoop on Windows, but the commands scoop bucket add smallstep https://github.com/smallstep/scoop-bucket.git scoop install smallstep/step don't seem to install "step.exe" in my PATH? A: make sure that you're launching the two commands separately. Look for the "step.exe" file and if you find it, make sure that its directory is in the system's PATH. FAQsand common problems Q: How can I connect to CINECA HPC clusters from a machine on which Network access has been restricted and a web browser is not available? A: You can refer to theHow to configureSSH access to the HPC Clusters –SSH keys section of the presentation; You can generate a pair of public/private keys on a computer with an available browser and transfer themto the machine from which you intend to connect to CINECA's clusters FAQsand common problems Q: I have too many ssh keys memorized in the ssh agent and the ssh connection does not work? A: having more than 5 ssh keys in a single ssh agent may lead to connection problems. Common workaround is to pass the identity to the ssh command through the –i flag. It is not possible to do so while using the agent-embedded certificate; you will need to download the public/private keys as described in theHow to configureSSH access to the HPC Clusters –SSH keys section of the presentation. --- # Unknown ADA cloud user documentation Getting started (1): How to create an instance HPC Cloud support group Last update: 30 May 2025 STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources Configure your network Configure your network STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources Configure your network Configure your network Set-up network and router Set-up network and router Set-up keypairs Set-up keypairs Set-up security rules Set-up security rules STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources Configure your network Configure your network Set-up network and router Set-up network and router Set-up keypairs Set-up keypairs Set-up security rules Set-up security rules Create your VM Create your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources Launch an instance Launch an instance Associate IP Associate IP Configure your network Configure your network Set-up network and router Set-up network and router Set-up keypairs Set-up keypairs Set-up security rules Set-up security rules Create your VM Create your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources Launch an instance Launch an instance Associate IP Associate IP Connect to your VM Connect to your VM Configure your network Configure your network Set-up network and router Set-up network and router Set-up keypairs Set-up keypairs Set-up security rules Set-up security rules Create your VM Create your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources Launch an instance Launch an instance Associate IP Associate IP Connect to your VM Connect to your VM Configure your network Configure your network Set-up network and router Set-up network and router Set-up keypairs Set-up keypairs Set-up security rules Set-up security rules Create your VM Create your VM Secure your VM Secure your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Get a CINECA HPC user and a cloud project Access your cloud resources Access your cloud resources Launch an instance Launch an instance Associate IP Associate IP Connect to your VM Connect to your VM Configure your network Configure your network Set-up network and router Set-up network and router Set-up keypairs Set-up keypairs Set-up security rules Set-up security rules Visit the ADA Cloud User guide for more information Create your VM Create your VM Secure your VM Secure your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Access your cloud resources Launch an instance Associate IP Connect to your VM Configure your network Set-up network and router Set-up keypairs Set-up security rules Create your VM Secure your VM How to get a CINECA HPC account and cloud resources 1 -Account and project For more info: Become a user in User Guide Get a HPC user •Account = "personal" username for HPC systems in CINECA •Register to CINECA UserDB portal •Ask to be associated with a valid project, as "Collaborator" or as "Principal Investigator" •Important:The access is possible only through two-factors (2FA) authentication Get cloud resources •ISCRA Projects:Researchers affiliated with an Italian University or an Italian Research Agency •EuroHPC Projects: European researchers •Italian research Institutions, General users andIndustrial applications: contactthe HPC User support Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Access your cloud resources Launch an instance Associate IP Connect to your VM Configure your network Set-up network and router Set-up keypairs Set-up security rules Create your VM Secure your VM Visit the ADA Cloud User guide for more information ADA Cloud dashboard 2 -Access your cloud resources •Go to https://adacloud.hpc.cineca.it •Select "CINECA HPC" as Authentication method •Insert your HPC-CINECA credentials to log in •NOTE: the 2nd factor needs to be activated (see sectionManaging password, 2FA and OTP) Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Access your cloud resources Launch an instance Associate IP Connect to your VM Configure your network Set-up network and router Set-up keypairs Set-up security rules Create your VM Secure your VM Visit the HPC Cloud User guide for more information 3.1 – Create Network and subnet for the project 3 -Configure your network Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM Go to the «Networks» tab under «Network» in the left side menu Click 3.1 – Create Network and subnet for the project 3 -Configure your network Insert a name for your network Click Follow the wizard steps Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM Insert network IP address: 192.168.0.0/24 3.1 – Create Network and subnet for the project 3 -Configure your network Insert name of your subnet Click Insert gateway IP: 192.168.0.254 Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.1 – Create Network and subnet for the project 3 -Configure your network Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.2 – Create Router for the Project 3 -Configure your network Go to «Routers» tab under «Network» in the left side menu Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.2 – Create Router for the Project 3 -Configure your network Click Insert a name for your router Select «externalNetwork» from the menu Follow the wizard steps Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.2 – Create Router for the Project 3 -Configure your network From the list of Routers, click on your Router name Click Go to the «Interfaces» tab Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.2 – Create Router for the Project 3 -Configure your network Select the network created in the previous step Click In the wizard Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.3 - Set-up keypairs 3 -Configure your network Go to the «Key Pairs» tab under «Compute» in the left side menu Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.3 - Set-up keypairs 3 -Configure your network •The publickey stays on the OpenStack dashboard •The privatekey is AUTOMATICALLY downloaded locally •IMPORTANTNOTES: •The download of the private key will be done ONLY when the keypair is created. If you lose the private key, you will have to create a new keypair. •If you are a Linux user, modify the permission of the private key (downloaded file) to read-write for only the user (chmod600 ) Click Insert a name for the new Key Pair Select «SSH Key» In the wizard Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.4 – Set-up security rules 3 -Configure your network •A security rule defines which traffic is allowed to instances assigned to the security group. •A security group is a group of security rules that can be assigned to an instance. Go to the «Security groups» tab under «Network» in the left side menu Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.4 – Set-up security rules 3 -Configure your network Insert a name for the security group Click In the wizard Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.4 – Set-up security rules 3 -Configure your network By default, only security rules to get out of your VM are created Security rules to access your VM needs to be added For the security group just created, select «Manage Rules» on the right side Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 3.4 – Set-up security rules 3 -Configure your network Select «SSH» from the list Click By default, access is enabled for all IPs In the wizard Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Access your cloud resources Launch an instance Associate IP Connect to your VM Configure your network Set-up network and router Set-up keypairs Set-up security rules Create your VM Secure your VM Visit the HPC Cloud User guide for more information 4.1 - Launch an instance 4 -Create your VM Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM Go to the «Instances» tab under «Compute» in the left side menu Click «Launch instance» 4.1 - Launch an instance 4 -Create your VM Insert a name for your VM Click Follow the wizard steps Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.1 - Launch an instance 4 -Create your VM Select an operative system of your VM Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.1 - Launch an instance 4 -Create your VM Select the flavourof your VM Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.1 - Launch an instance 4 -Create your VM Select the network created in the previous «configure your network» step and click «Next» Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.1 - Launch an instance 4 -Create your VM Select the Key Pair created in the previous «configure your network» step Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.1 - Launch an instance 4 -Create your VM Select the Security Group created in the previous «configure your network» step and click «Launch Instance» Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.2 - Associate a floating IP to your VM 4 -Create your VM Go to the «Floating IPs» tab under «Network» in the left side menu Click Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.2 - Associate a floating IP to your VM 4 -Create your VM Click In the wizard Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM 4.2 - Associate a floating IP to your VM 4 -Create your VM Click In the wizard Click By default, the IP, you have just allocated, will be selected From the menu, select your VM Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM STEP BY STEP user GUIDE Getting started workflow Get a CINECA HPC user and a cloud project Access your cloud resources Launch an instance Associate IP Connect to your VM Configure your network Set-up network and router Set-up keypairs Set-up security rules Create your VM Secure your VM Visit the HPC Cloud User guide for more information 5.1 – Log in to your VM 5 -Connect to your VM Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM •Your VM is now ready to be used •Login using the default user (of the OS you have chosen for your VM) and your private key (see step 3.3) •Suppose you have used the default ubuntu cloud image, you can login as: $ ssh -imy\_keypair.pemubuntu@ 5.2 – Secure your VM 5 -Connect to your VM At the first log in, remember to: •Update the OS and relative packages Follow the basic security guidelines: •activate automatic updates •only install software from reputable sources •disable unneeded services •use encrypted and secure communication protocols to avoid man in the middle attacks •keep logs of your applications •monitor accounts created on your system and do not enable password login,useSSHkeysinstead More information at: Security guidelines Get an HPC CINECA user and a project Get an HPC CINECA user and a project Access your cloud resources Access your cloud resources Create your VM Create your VM Configure your network Configure your network Connect to your VM Connect to your VM CINECA For any issue or question please contact the HPC User support at superc@cineca.it --- # Unknown .. \_general\_info\_card: General Information =================== This is the main section of Cineca HPC documentation. Here you will find all the information to common Cineca's facilities. .. grid:: 2 .. grid-item-card:: Access to the Systems :link: access\_card :link-type: ref .. grid-item-card:: Users and Accounts :link: users\_card :link-type: ref HPC Service Desk ---------------- Services provided by the User Support ca be found at \`Service Desk \`\_ website. Cite \*\*CINECA\*\* in scientific publications ------------------------------------------ Users must acknowledge \*\*CINECA\*\* and the awarded resources in every published paper. Please use appropriate phrases depending on the origin of the awarded resource: .. dropdown:: Citation List :animate: fade-in-slide-down :color: info \* \*\*Awards granted by ISCRA @ LEONARDO:\*\* We acknowledge ISCRA for awarding this project access to the LEONARDO supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CINECA (Italy) . \* \*\*Awards granted by EuroHPC @ LEONARDO:\*\* We acknowledge the EuroHPC Joint Undertaking for awarding this project access to the EuroHPC supercomputer LEONARDO, hosted by CINECA (Italy) and the LEONARDO consortium through an EuroHPC \[Extreme/Regular/Benchmark/Development/…\] Access call. \* \*\*Awards granted by LEONARDO consortium countries @ LEONARDO:\*\* We acknowledge \[Grant organization, consortium country\] for awarding this project access to the LEONARDO supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CINECA (Italy) and the LEONARDO consortium. \* \*\*Awards granted by ICSC @ LEONARDO:\*\* We acknowledge the ICSC for awarding this project access to the EuroHPC supercomputer LEONARDO, hosted by CINECA (Italy). \* \*\*Awards granted by ISCRA @ G100 and ADA Cloud:\*\* We acknowledge the CINECA award under the ISCRA initiative, for the availability of high performance computing resources and support. To cite Leonardo architecture and the technologies adopted for its GPU-accelerated partition, please cite article: CINECA Supercomputing Centre, SuperComputing Applications and Innovation Department. (2024). “LEONARDO: A Pan-European Pre-Exascale Supercomputer for HPC and AI applications.”, Journal of large-scale research facilities, 8, A186. \`DOI \`\_. --- # Unknown Introduction to Pitagora June 23th, 2025 Jonathan Frassineti j.frassineti@cineca.it CINECA - High Performance Computing Department Outline ➢ Pitagora infrastructure ➢ Access HPC resources and filesystems ➢ Software environment ➢ Programming environment ➢ Production environment ➢ Final remarks Pitagora login nodes Connect with: login{01-06}-ext.pitagora.cineca.it - CPU (01, 03, 05): same as compute nodes (see later) - GPU (02, 04, 06): same as compute nodes, only GPU differs at the moment (same GPU, less memory, air cooled) Data Centric and General Purpose (CPU-only) partition Lenovo SD665V3 DWC CPU compute node ➢ 1008 nodes ➢ Processors (dual-socket): 2x AMD Turin Dense 128c - Zen5 microarch (128 cores/node per CPU), 2.4 GHz ➢ RAM: 768 (24 x 32) GB (744 GB available) DDR5 6400 MHz ➢ Internal network: Nvidia ConnectX-7 NDR 200Gbit/s network interface ➢ No storage ➢ Performance: R max = 17 Pflops, R peak ~ 20 Pflops BOOSTER (CPU+GPU) partition Lenovo SD650-N V3 GPU compute node ➢ 168 nodes ➢ Processors (dual-socket): 2x Intel Emerald Rapids 6548Y 32c (32 cores/node per CPU), 2.4 Ghz ➢ GPU: 4x NVIDIA H100 SXM 100GB HBM2e ➢ RAM: 512 (16 x 32) GB (494 GB available) DDR5 5200 MHz ➢ Internal network: Nvidia ConnectX-7 NDR200 200Gbit/s network interface ➢ Storage: 894 GB ➢ Performance: R max ~ 28 Pflops, R peak ~ 37 Pflops Storage (DDN) - 1x ES200NVX2E appliance for metadata handling housing: - 12 SSD NVMe 3,84TB - 3x ES400NVX2E-S appliances for data storage with 5x SS9024 expansion JBOD each appliance housing: - 24 SSD NVMe 3,84TB (ES400NVX2E controller) - 440 Hard Disk 12TB SAS Enterprise Edition (JBOD SS9024) Raw total HDD Capacity (WORK, SCRATCH): 15.7 PB RAW flash Capacity (HOME, PUBLIC): 0.27 PB (3x 24 SSD NVMe 3,84TB (ES400NVX2E controller)) Sequential Read/Write HDD: ~150/190 GB/s Outline ➢ Pitagora infrastructure ➢ Access HPC resources and filesystems ➢ Software environment ➢ Programming environment ➢ Production environment ➢ Final remarks Become a new HPC user ● Register on the UserDB Portal: https://userdb.hpc.cineca.it/ ● Get associated to an active account → Principal Investigator (PI): we create the account and set you as PI on the UserDB → Collaborator: ask your PI to associate you to the account on the UserDB ● Request the “HPC Access” on UserDB → You will receive two mails: one with your HPC username, and one to set an HPC password and configure the 2FA https://docs.hpc.cineca.it/general/getting\_started.html Any access to the cluster ● Request the ssh certificate to our Identity Provider via the smallstep client from your local shell. → A web page will open on the browser and you will be asked to insert a One-Time Password (OTP) from the app → Valid for 12 hours ● Access to the cluster via ssh: $ ssh @login.pitagora.cineca.it Access to Pitagora First time ● Activate the 2FA: authenticate on our Identity Provider at https://sso.hpc.cineca.it using your HPC username and password. → You will need an app to generate authentication codes (e.g. Google Authenticator) ● Install and configure the smallstep client (depending on your OS) The access to CINECA HPC systems requires a two-factor authentication (2FA). https://docs.hpc.cineca.it/general/access.html Access to Pitagora $ ssh @login.pitagora.cineca.it Motto of the day ➔ Short system description ➔ “In evidence” messages ➔ “Important” messages (e.g. scheduled maintenances) $PUBLIC ● 50 GB per user ● user specific (permissions 755) ● Permanent ● No backup $HOME ● 50 GB per user ● user specific ● permanent ● Backup (in future) Filesystems $SCRATCH ● no quota ● user specific ● temporary (data removed after 40 days) ● No backup $PUBLIC ● 50 GB per user ● user specific (permissions 755) ● Permanent ● No backup $HOME ● 50 GB per user ● user specific ● Permanent ● Backup (in future) Filesystems $WORK ● quota per account (default 1 TB) ● account specific ● Permanent ● No backup $SCRATCH ● no quota ● user specific ● temporary (data removed after 40 days) ● No backup $PUBLIC ● 50 GB per user ● user specific (permissions 755) ● Permanent ● No backup $HOME ● 50 GB per user ● user specific ● Permanent ● Backup (in future) Filesystems $WORK ● quota per account (default 1 TB) ● account specific ● Permanent ● No backup $SCRATCH ● no quota ● user specific ● temporary (data removed after 40 days) ● No backup All the filesystems are based on Lustre $TMPDIR ● local on nodes ● job specific ● fast I/O DRES ● To be defined ● No backup Outline ➢ Pitagora infrastructure ➢ Access HPC resources and filesystems ➢ Software environment ➢ Programming environment ➢ Production environment ➢ Final remarks Module environment Any available software is offered on the clusters in a module environment. The modules are organized in functional categories and collected in different profiles. Installed software Module Category Profile Compilers Libraries Tools Applications Data Base is the default profile: automatically loaded after login, containing basic modules for programming activities Programming (base): compilation, profiling, debugging... Domain (chem-phys, lifesc, ...): production activities Module environment $ module avail Almost all the modules on Pitagora have been installed with Spack, and they report the Spack package name. Module environment $ module load profile/chem-phys $ module avail $ module show / $ module help / Print information about the module, such as dependencies, paths Print the help of the software, its brief description and examples of the use Loaded profiles are added to the environment Module environment $ modmap -m $ module load $ module load / $ module list Detect all profiles, categories and modules available (e.g. different releases) → TO BE INSTALLED SOON List all the profiles and modules loaded so far all the dependencies are automatically loaded; we recommend to specify the module version! You will find modules compiled to support GPUs and modules suitable only for CPUs. You can check the compiler in the full name of the module, where the version is specified (e.g. gromacs/2024.2--intel-oneapi-mpi--2021.12.1--oneapi--2024.1.0). Remind that modules compiled with nvhpc, cuda should be used only on the GPU partition, while modules compiled with gcc, aocc are suitable for running on the CPU partition. I m p o r t a n t ! Install new software In case you don't find a software, you can choose to install it by yourself. ● Install without sudo permissions ● Install with pip in virtual env ➢ $ module load python/3.11.7 $ python3 -m venv $ source /bin/activate ● Install with Spack Write to superc@cineca.it if you need guidance on the installation or if you want to request a new module. Install with Spack “Spack” environment provided by the package manager Spack and available as modules. $ module avail spack \[jfrassi1@login01 ~\]$ module avail spack ---------------------------------------------- /pitagora/prod/opt/modulefiles/base/tools ----------------------------------------------- spack/0.22.2\_6.1 Load the module: $ module load spack/0.22.2\_6.1 ➢ setup-env.sh file is sourced ➢ $SPACK\_ROOT is initialized ➢ spack command is added to your PATH, and some nice command line integration tools as well ➢ Folder /spack- is created into your $PUBLIC area (on Pitagora and Leonardo, and $WORK on the other clusters) and it contains some subfolders created and used by spack during the phase of the packages installation: ● sources cache: /cache ● software installation root: /install ● modulefiles location: /modules ● user scope: /user\_cache https://docs.hpc.cineca.it/hpc/hpc\_enviroment.html Install with Spack Install with Spack $ spack list $ spack info $ spack spec -Il e.g. $ spack spec -Il scorep $ spack install $ spack load Check if the package is available for installation with Spack Show available versions, building variants and dependencies Show version, compiler, dependencies, building variants with which the package will be installed (-Il for installation status and hash) → options can be specified Install the package → options as spec command Load the package installed to use it (you can also create a module) Some fundamental Spack commands Outline ➢ Pitagora infrastructure ➢ Access HPC resources and filesystems ➢ Software environment ➢ Programming environment ➢ Production environment ➢ Final remarks Programming environment Check with commands module av, module show, module help, and man MPI libraries ➢ OpenMPI (GNU compiler) → CUDA-aware ➢ hpcx-mpi (NVHPC compiler) → CUDA-aware Compilers and MPI libraries are available as modules in profile/base. Use the ones suitable for the architecture: on Pitagora DCGP, AMD aocc compilers and libraries are recommended. Compilers ➢ GCC (GNU compilers: gcc, g++, gfortran) ➢ gcc was installed to offload the NVIDIA GPUs through the nvptx target. However, if the device is not present, as in the CPU partition, it runs on the CPU. ➢ NVHPC (ex hpc-sdk, ex PGI + CUDA → NVIDIA compilers: nvc, nvc++, nvcc, nvfortran) ➢ CUDA ➢ INTEL ONEAPI (Oneapi compilers: icx, icpx, ifx, ifort) → no Nvidia GPU support ➢ From the 2024 version, intel-oneapi-compilers contains only the oneAPI compilers (the x ones) and the classic Fortran (ifort), which will no longer be available from 2025 version. ➢ INTEL ONEAPI CLASSIC (Intel compilers: icc, icpc, ifort) → no Nvidia GPU support ➢ Load intel-oneapi-compilers-classic to use the above compilers ➢ AMD AOCC → no Nvidia GPU support ➢ OpenMPI (AOCC compiler) → no CUDA-aware ➢ Intel Oneapi MPI (Intel compilers) → no CUDA-aware Outline ➢ Pitagora infrastructure ➢ Access HPC resources and filesystems ➢ Software environment ➢ Programming environment ➢ Production environment ➢ Final remarks Login and compute nodes CINECA HPC clusters are shared among many users, so a responsible use is crucial! Login nodes ➢ Interactive runs on login nodes are strongly discouraged and should be limited to short test runs → 10 minutes cpu-time limit ➢ Avoid running large and parallel applications on login nodes ➢ GPUs only on certain login nodes (login02, login04 and login06) Compute nodes ➢ Long production jobs should be submitted on compute nodes using the scheduler → SLURM ➢ Jobs can be submitted in two main ways: via batch mode and via interactive mode ➢ Nodes shared, but the allocated resources (cores, RAM, $TMPDIR) are assigned in an exclusive way Resources per node Each node → max resources you can request per node ➢ 64 cores (GPU), 256 cores (CPU) ntasks-per-node \* cpus-per-tasks ≤ 64 (BOOSTER), 256 (DCGP) ➢ 495 GB (GPU), 744 GB (CPU) of RAM (memory) ➢ 894 GB of temporary local memory on $TMPDIR (GPU) (gres=tmpfs) The accounting will consider ● the requested number of CPUs ● the requested memory on RAM ● the requested memory on $TMPDIR and calculates the number of equivalent cores → it takes the maximum among ● number of cpus ● memory / memory-per-core ( = requested memory / memory-per-node \* cores-per-node ) ● tmpfs / tmpfs-per-core ( = requested tmpfs / tmpfs-per-node \* cores-per-node ) Accounting is currently not active, so it is not necessary to specify the budget account in the slurm scripts! Eurofusion resources Production partition → dcgp\_fua\_prod/boost\_fua\_prod 1) Normal QOS: normal - max 16 nodes (BOOSTER) – max 64 nodes (DCGP) - max walltime: 24 h 2) Big production QOS: dcgp\_qos\_fuaprod/boost\_qos\_fuaprod 640 (DCGP) – 96 (BOOSTER) compute nodes dinamically allocated - min 65 full, max 128 nodes (DCGP) – min 17 full, max 32 nodes (BOOSTER) - max walltime: 24 h Debug partition → dcgp\_fua\_dbg/boost\_fua\_dbg Normal QOS: normal - max 2 nodes - max walltime: 30 min 3) Long production QOS: dcgp\_qos\_fualprod/boost\_qos\_fualprod - max 3 (BOOSTER), 3 (DCGP) nodes - max walltime: 4 days Submit jobs with SLURM #!/bin/bash #SBATCH --nodes=1 # nodes #SBATCH --ntasks-per-node=4 # tasks per node #SBATCH --cpus-per-task=8 # cores per task #SBATCH –mem=MB # memory on RAM #SBATCH –gres=tmpfs:GB # memory on $TMPDIR #SBATCH --time=1:00:00 # time limit (d-hh:mm:ss) #SBATCH --account= # account #SBATCH --partition= # partition name #SBATCH --qos= # quality of service module load srun my\_application #SBATCH directives (also contracted syntax, e.g. -N for --nodes) Loading modules and setting variables Launch executable (for parallel applications, use srun or mpirun) Batch mode ● Write a batch script like the example ● Launch the batch script $ sbatch \[options\] start.sh ● The job is queued and scheduled shell Submit jobs with SLURM Interactive mode ● Ask for the needed resources with the same SLURM directives with srun or salloc ● The job is queued and scheduled but, when executed, the standard input, output, and error streams are connected to the terminal session from which srun or salloc were launched ● Run your application from that prompt ● Exit from the terminal session: $ exit Non MPI programs $ srun -N 1 --ntasks-per-node=8 --cpus-per-task=4 -t 01:00:00 -p -A --pty /bin/bash The session starts on the compute node: \[username@r328c03s09 ~\] $ Also MPI programs $ salloc -N 1 --ntasks-per-node=8 --cpus-per-task=4 -t 01:00:00 - p -A A new session starts on the login node: \[username@login02 ~\]$ Submit jobs with SLURM Only on Pitagora “diskful” nodes (GPU), it’s possible to increase the space of the $TMPDIR area. Remind that the area is local to nodes, and job specific (i.e. “temporary”): created at the begging of a job and deleted at its end, and accessible only by the user who launched the job. Specify the space on $TMPDIR=/tmp (default=10GB): #SBATCH --gres=tmpfs:200GB on the local disks on GPU compute nodes (max 894 GB). On the diskless CPU compute nodes, the $TMPDIR=/tmp area is hosted on the RAM, with a fixed size of 10 GB (no increase is allowed, and the gres=tmpfs resource is disabled). Remind that for the GPU jobs the requested amount of gres=tmpfs resource contributes to the consumed budget, changing the number of accounted equivalent core hours. Submit jobs with SLURM #SBATCH --account= or -A Specifies the account with a budget of core-hours available to run jobs. Remind also that, on Pitagora, you can check the status of your accounts with “saldo” (TO BE INSTALLED SOON) Monitor your jobs with SLURM $ squeue -u or $ squeue --me Shows the list of all your scheduled jobs, along with their status (pending, running, closing, ...). Also, shows you the jobID required for other SLURM commands. $ scontrol show job Provides a long list of informations for the job requested. In particular, if your job isn’t running yet, you'll be notified about the reason it has not started yet and, if it is scheduled with top priority, you will get an estimated start time. $ scancel Removes the job (queued or running) from the scheduled job list by killing it. $ sinfo (e.g. $ sinfo -o "%10D %a %20F %P") Provides information about SLURM nodes and partitions. $ sacct (e.g. $ sacct -Bj ) Displays accounting data for all jobs and job steps in the SLURM job accounting log or SLURM database. Outline ➢ Pitagora infrastructure ➢ Access HPC resources and filesystems ➢ Software environment ➢ Programming environment ➢ Production environment ➢ Final remarks Final remarks ★ Login nodes should only be used for installation (connection to external network!), compilation, and small tests. GPUs only on certain login nodes! ★ Consider to use Pitagora GPU/BOOSTER for applications on GPUs and Pitagora CPU/DCGP for applications only on CPUs. ★ Recommended compilers are gcc and Nvidia compilers (nvhpc, cuda) for Pitagora Booster, and gcc and AMD (aocc) for Pitagora DCGP. ★ Rely on the already available software stack, tested and optimized for the cluster architecture, and on Spack for autonomously installing additional software. https://docs.hpc.cineca.it/index.html Write to superc@cineca.it in case of need! Additional info SW architecture  Pitagora CPU (AMD) = zen5  Pitagora GPU (Intel) = sapphire rapids  Software stack = zen4. Zen 4 includes instruction set extensions that are compatible with both AMD and Intel architectures, allowing us to maintain a unified GCC software stack across both AMD and Intel partitions. Zen 4 supports AVX-512, but compared to Intel’s Sapphire Rapids architecture, it lacks AMX (Advanced Matrix Extensions) instructions. Additionally, compared to Zen 5, it misses a minor extension that is not particularly critical. Additional info (2) INTEL-ONEAPI-MPI on AMD  job that fail with the error (es. Ascot): = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = RANK 1920 PID 2666886 RUNNING AT r332c06s03 = KILLED BY SIGNAL: 9 (Killed) srun: error: r333c01s06: task 15: Broken pipe \[mpiexec@r332c04s12\] wait\_proxies\_to\_terminate (../../../../../src/pm/i\_hydra/mpiexec/intel/i\_mpiexec.c:554): downstream from host r332c04s12 exited with status 141 The issue appears to be related to the default Mellanox provider. For now, it can be resolved by explicitly selecting the verbs provider (using export I\_MPI\_FABRICS=shm:ofi and export FI\_PROVIDER=verbs). After some investigation, we found that the code compiled with Intel oneAPI MPI using the Mellanox provider runs without issues on the Intel CPU side. So we attribute the problem to Intel MPI on AMD, where it can be resolved by selecting the fabrics provider, which slightly reduces performance. We still don't know exactly by how much.  job that hang (es. starwall): still investigating.  jobs that concern PETSc, which produces incorrect output at runtime. In this case too, it appears to be a problem with the Mellanox provider, which, however, is not resolved even when using the verbs provider. It actually works only with the tcp provider, which uses the service network and is therefore even more inefficient. Additional info (2) INTEL-ONEAPI-MPI on AMD To summarize: Intel MPI on AMD might (because there are no issues with OSU benchmarks and presumably other applications) show problems with the mlx provider. The workaround (which doesn't apply in all cases) is to use the verbs provider, with some performance penalty (NOTE: the fabrics is always ofi, both for mlx and verbs), or even tcp, which we do not recommend since it uses the service network and is significantly less efficient. Additional info (3) SLURM  SLURM version is not definitive, possible errors may occur  If you encounter this error: sbatch: error: Batch job submission failed: Unexpected message received you can try cleaning your module environment with "module purge" and then resubmit. Load all the modules you need inside the jobscript itself 1. Q: how can we install software/libraries so that they are available to all the members of one project? A: You can install them in your $PUBLIC (with open access) or in your WORK (shared with collaborators) with Spack. You can also create your own modules. See more information regarding installing with SPACK on CINECA clusters at https://docs.hpc.cineca.it/hpc/hpc\_enviroment.html#spack Q&A 2. Q: if I am already user and with active projects, when can i start running on Pitagora? A: Pitagora is open to projects selected for EUROfusion cycle 9. Projects related to cycle 8 (started in March 2024), will run on Leonardo until July. The projects of cycle 9 are already defined on the cluster. To access the cluster you must be associated with one of these. Q&A 3. Q: If no memory or tempfs specified, what's happening with that resources? A: The default for memory is the memory-per-core. The default for tmpfs is 10 GB. Q&A 4. Q: Who do I have to contact to ask for operation and allocation resources? A: In case of doubt: richard.kamendje@euro-fusion.org (operation), Duarte.Borba@euro-fusion.org (allocation, see call) Q&A --- # Unknown .. \_hpc\_card: Introduction HPC Resources ========================== This section provides a broader context for HPC Resources and the essential characteristics of HPC infrastructure. It introduces to CINECA Clusters and HPC services with specific focus on managment of resources by users together with the budgeting and accounting rules in place at CINECA for HPC projects. Budget and Accounting --------------------- The \`\`saldo\`\` command allows you to quickly retrieve information about your Project Account, including the available budget, and details of the User Account. More information about the usage of the tool can be gained just executing the command without any option. .. tab-set:: .. tab-item:: User Account Balance \`\`saldo -b \`\` lists the budget of all Project Accounts associated with a username. \* A single \*\*User Account\*\* can be associated to a multiple \*\*Project Accounts\*\*. \* For clusters with independent partitions, specify the partition using: \* \`\`saldo -b\`\` (default: , on Leonardo give you back the report for Booster partition) \* \`\`saldo -b --dcgp\`\` (to get a report for the DCGP partition on Leonardo, the flag \`\`--dcgp\`\` is mandatory) A typical example of \`\`saldo\`\` usage is reported in the following: .. code-block:: bash saldo -b ----------------------------------------------------------------------------------------------------------------------------------- account start end total localCluster totConsumed totConsumed monthTotal monthConsumed (local h) Consumed(local h) (local h) % (local h) (local h) ----------------------------------------------------------------------------------------------------------------------------------- Proj\_A 20110323 20300323 50000 25000 55027726 50.0 600 600 Proj\_B 20220427 20301231 100000 10000 27086 10.0 731 731 Proj\_C 20230524 20300323 6500 0 0 0.0 0 0 \*\*Description of columns:\*\* \* \*\*Account:\*\* Refers to the Project Account (approved grants). \* \*\*Start Date:\*\* Start of the grant period. \* \*\*End Date:\*\* End of the grant period. \* \*\*Total Hours (local):\*\* Total CPU hours allocated to the grant on the local cluster. \* \*\*Consumed (local):\*\* Total CPU hours used from the allocation on the local cluster. \* \*\*Total Consumed (%):\*\* Percentage of the total hours consumed. \* \*\*Month Total:\*\* Allocated hours for the current month. \* \*\*Month Consumed:\*\* Hours consumed in the current month. .. tab-item:: Project Account Balance \`\`saldo -a \`\` lists the the usage of the \*\*Project Account\*\* for each associated \*\*User Accunt\*\*. The report specifies the date, consumed hours for each \*\*User Account\*\*, and the number of jobs submitted by the user on that day. .. code-block:: bash saldo -a ------------------Resources used from 202404 to 202412------------------ date username account localCluster num.jobs Consumed/h ------------------------------------------------------------------------ 20240907 user001 example 5553:34:39 542 20240908 user001 example 22340:07:36 2676 20240909 user001 example 1606:21:39 154 20240910 user001 example 3210:42:40 285 ------------------------------------------------------------------------ Billing Policy ^^^^^^^^^^^^^^ The billing policy outlines the methodology employed to calculate budget consumption associated with the use of HPC resources. We strongly recommend familiarizing yourself with this policy, as understanding it is crucial to avoiding unnecessary budget losses and to effectively planning your activities. Budget consumption is measured in \*\*effective CPU hours (CPUh)\*\* and is calculated based on the amount of resources allocated per node and the duration of their usage. Resource allocations are exclusive by default, meaning that once assigned, the reserved resources cannot be used by other users. \*\*Formula for Billed Hours:\*\* .. math:: B\_{H} = T \\cdot N \\cdot R \\cdot C where: \* \*T\* = \*\*elapsed time\*\* (in hours). \* \*N\* = \*\*number of nodes\*\* allocated. \* \*R\* = \*\*reserved resources per node\*\* (explained below). \* \*C\* = \*\*number of CPUs per node\*\* (depends on node architecture). The \*R\* factor measures the fraction of node resources reserved by a job that are consequently unavailable to other users. It is defined as the maximum among all reserved resource types (RES) — for example, the number of CPUs, GPUs, or memory — normalized by the total capacity of each respective resource on a single node: .. math:: R = \\max\_{r \\in RES}\\left\\{ \\frac{\\text{Allocated}(r)}{\\text{Total}(r)} \\right\\} This billing model is designed to ensure that users are charged based on the proportion of a node's resources made unavailable to others due to their job allocation. For example, if a job reserves all of a node's RAM — even without utilizing all its CPUs — the node becomes unusable for other jobs and is therefore billed accordingly. Similarly, if all GPUs on a node are reserved, the node is considered substantially occupied, even if some CPUs or RAM remain available. Although GPU reservations do not entirely prevent node usage by others, GPUs are high-cost resources and nodes equipped with them are dedicated to workloads that can fully leverage their capabilities. Therefore, GPU usage is considered as critical as CPU usage when determining billing, even if the node is still partially usable. .. dropdown:: Example :animate: fade-in-slide-down :chevron: down-up A user requests 1 node, 4 CPUs, 4 GPUs, and 3 hours of walltime on the Booster partition of Leonardo. However, the job runs for only 2 hours. From this information, we have: \* T = 2 h (elapsed time) \* N = 1 node \* C = 32 CPUs (number of CPUs available on a Leonardo Booster compute node — see :ref:\`hpc/leonardo:Hardware Details\`) and, since: .. math:: \\frac{\\text{Allocated}(\\text{CPU})}{\\text{Total}(\\text{CPU})} = \\frac{4}{32} = 0.125 .. math:: \\frac{\\text{Allocated}(\\text{GPU})}{\\text{Total}(\\text{GPU})} = \\frac{4}{4} = 1.0 the maximum of the resources requested per node is determined by the GPUs, therefore \*R\* = 1.0, and the billed hours are then calculated as: .. math:: B\_{H} = T \\cdot N \\cdot R \\cdot C = 2 \\cdot 1 \\cdot 1.0 \\cdot 32 = 64 \\text{CPUh} This means the job consumes 64 effective CPU hours from the project's budget. ---- .. note:: \* The \*\*serial partition\*\* is available for limited post-production data analysis and can be used even after a Project Account has expired. Usage of this partition is excluded from STDH billing (\*\*free of charge\*\*). \* By default, the amount of memory allocated per node is proportional to the number of CPUs requested. \* When nodes are requested in \*\*exclusive mode\*\* (see :ref:\`hpc/hpc\_scheduler:Scheduler and Job Submission\` section), the entire node is reserved for the job, regardless of the specific resources requested. In such cases, the allocated resources may exceed the explicitly requested ones. \* The \*\*resources per node\*\* are listed in the \*\*Hardware Details\*\* section for each cluster. Refer to the :ref:\`hpc/hpc\_clusters:Cluster Specifics\` section for the complete list of Cineca's HPC systems. Budget Linearization ^^^^^^^^^^^^^^^^^^^^ A linearization policy governs the priority of scheduled jobs across Cineca clusters. To each Project Account is assigned a monthly quota (MQ) calculated as: .. math:: MQ = TB/NM TB = total assigned budget NM = total number of months Beginning on the first day of each month, any User Accounts belonging a Project Account may utilize their quota at full priority. As the budget is consumed, submitted jobs progressively lose priority until the monthly quota is exhausted. Subsequently, these jobs are still considered for execution but with reduced priority compared to accounts with remaining quota. This policy aligns with practices at other prominent HPC centers globally, aiming to enhance response times by aligning CPU hour usage with budget sizes. .. note:: It's recommended to adhere to a linearized usage of your budget, as non-linear consumption may impact the welfare of all users concurrently utilizing our HPC systems. A simple working scheme of budget linearization is showed in the figure below. .. image:: img/bud\_lin.png :align: center :height: 180 px .. image:: img/spacer.png :align: center :height: 20 px --- # Unknown Scheduler and Job Submission ============================ \*\*CINECA\*\* HPC clusters are accessed via a dedicated set of login nodes. These nodes are intended for simple tasks such as customizing the user environment by installing applications, transferring data, and performing basic pre- and post-processing of simulation data. Access to the compute nodes is managed by the workload manager. To ensures fair access to resources for all users, production jobs must be submitted using a scheduler. \*\*CINECA\*\* uses Slurm (Simple Linux Utility for Resource Management) manager and batch system. Slurm is an open-source, highly scalable job scheduling system with three key functions: \* Allocating access to resources (compute nodes) to users for a specified duration, allowing them to perform their work. \* Providing a framework for starting, executing, and monitoring work (usually parallel jobs) on the set of allocated nodes. \* Managing resource contention by handling the queue of pending jobs. \* There are two main modes of using compute nodes: \*\*Batch Mode:\*\* This mode is intended for production runs. Users must prepare a shell script with all the operations to be executed once the requested resources are available. The job will then run on the compute nodes. Store all your data, programs, and scripts in the \`$WORK\` or \`$SCRATCH\` filesystems, as these are best for compute node access. You must have valid active projects to run batch jobs, and be aware of any specific policies regarding project budgets on our systems. \*\*Interactive Mode:\*\* Jobs submitted in this mode are similar to batch mode in that the user must specify the resources to allocate. The job is then managed like any other submitted job. The key difference from batch mode is that once the job is running, the user can interactively execute applications within the limits of the allocated resources. All allocated resources are available for the entire requested walltime (and consequently billed) during the submission process. .. important:: \* \*\*Interactive\*\* mode under SLURM has a different meaning compared to the common understanding of interactive execution of an application under a Linux shell or prompt. \* \*\*Interactive\*\* execution of applications is allowed on compute nodes only via SLURM (see the next sections). \* On login nodes, it is permitted to perform tasks such as data movement, archiving, code development, compilations, basic debugging, and very short test runs, provided these tasks do not exceed 10 minutes of CPU time and are free of charge under the current billing policy. \* A comprehensive documentation of SLURM and some examples on how to submit your job is described in a separate section under this chapter, as well as on the original \`SchedMD site \`\_. Basic Usage of Slurm -------------------- With SLURM, you can specify the tasks you want to execute, and the system will manage running these tasks and returning the results to you. If the resources are not available, SLURM will hold your jobs and run them when resources become available. Typically, you create a \*\*batch job\*\*, which is a file (a shell script in UNIX) containing the set of commands you want to run. This file also includes \`\`directives\`\` that specify the job's characteristics and resource requirements, such as the number of processors and CPU time needed. Once you create your job script, you can reuse it or modify it for subsequent runs. \*\*Basic Workflow\*\* \* Create a job script with Slurm \`\`directives\`\`. \* Submit the job using \`\`sbatch\`\`. \* Monitor the job using commands like \`\`squeue\`\` and \`\`scontrol\`\`. \* Cancel a job if needed with \`\`scancel\`\`. Here is a simple SLURM job script example to run a user's application, setting a maximum wall time of one hour and requesting \*\*1\*\* node with \*\*32\*\* cores: .. code-block:: bash #!/bin/bash #SBATCH --nodes=1 # 1 node #SBATCH --ntasks-per-node=32 # 32 tasks per node #SBATCH --time=1:00:00 # time limit: 1 hour #SBATCH --error=myJob.err # standard error file #SBATCH --output=myJob.out # standard output file #SBATCH --account= # project account #SBATCH --partition= # partition name #SBATCH --qos= # quality of service ./my\_application As shown in the example, a job requests resources through SLURM syntax. Resources can be allocated by including \`\`directives\`\` in the job script, or within the \*\*interactive mode\*\* via \`\`sbatch\`\` or \`\`salloc\`\` command. in a Once resources are allocated, the job can be executed. In the table below, a list of the main SLURM \`\`directives\`\` is reported. \*\*Main Slurm Directives\*\* .. list-table:: :widths: 50 50 70 :header-rows: 1 \* - \*\*Directive\*\* - \*\*Description\*\* - \*\*Example\*\* \* - \`\`--job-name\`\` - Stes the job name - \`\`#SBATCH --job-name=my\_job\`\` \* - \`\`--output\`\` - Specifies the output file - \`\`#SBATCH --output=output.log\`\` \* - \`\`--error\`\` - Specifies the error file - \`\`#SBATCH --error=error.log\`\` \* - \`\`--time\`\` - Sets the max execution time - \`\`#SBATCH --time=01:00:00\`\` \* - \`\`--partition\`\` - Selects the partition - \`\`#SBATCH --partition=compute\`\` \* - \`\`--ntasks\`\` - Nubmber of tasks - \`\`#SBATCH --ntasks=1\`\` \* - \`\`--cpus-per-task\`\` - CPUs per task - \`\`#SBATCH --cpus-per-task=4\`\` \* - \`\`--mem\`\` - Memory per node - \`\`#SBATCH --mem=8GB\`\` \* - \`\`--gres\`\` - Specifies generic resources (e.g. GPUs) - \`\`#SBATCH --gres=gpu:1\`\` \* - \`\`--qos\`\` - Quality of service (refer to specific clusters) - \`\`#SBATCH --qos=\`\` \* - \`\`--account\`\` - Name of the project - \`\`--account=\`\` How to prepare a script to submit Jobs -------------------------------------- .. tab-set:: .. tab-item:: Serial Job This SLURM batch script is intended for running a serial (single-core) application on a Cineca's HPC cluster. It requests one node and allocates a single CPU core to execute a task that does not require parallel processing. This setup is ideal for lightweight computations, preprocessing steps, or applications that are not parallelized. .. code-block:: bash #!/bin/bash #SBATCH --job-name=serial\_job             # Descriptive name for the job #SBATCH --time=00:30:00                   # Maximum wall time (hh:mm:ss) #SBATCH --nodes=1                         # Request one node #SBATCH --ntasks=1                        # One task (process) total #SBATCH --cpus-per-task=1                 # One CPU core per task #SBATCH --partition=     # Partition (queue) to submit to #SBATCH --qos=                 # Quality of Service #SBATCH --mem=2G                          # Memory per node (adjust as needed) #SBATCH --output=serialJob.out           # File to write standard output #SBATCH --account=    # Project account number .. tab-item:: OpenMP Job This SLURM batch script is designed to run a pure OpenMP application on Cienca's HPC clusters. It requests a single node and allocates all available physical CPU cores to a single task, making it ideal for shared-memory parallel programs. The script sets up the environment, loads the necessary modules, and configures OpenMP-specific variables to ensure optimal performance. It is tailored for systems without hyperthreading and can be easily adapted by modifying the number of CPUs per task and other resource parameters. .. code-block:: bash #!/bin/bash #SBATCH --job-name=openmp\_job           # Job name #SBATCH --time=01:00:00                 # Walltime (hh:mm:ss) #SBATCH --nodes=1                       # Number of nodes #SBATCH --ntasks-per-node=1            # One MPI task per node #SBATCH --cpus-per-task=48             # Number of physical CPU cores per task (adjust to 32 for MARCONI100) #SBATCH --partition=   # Partition to submit to #SBATCH --qos=              # Quality of Service #SBATCH --mem=           # Memory per node (e.g., 128G) #SBATCH --output=myJob.out             # Standard output file #SBATCH --error=myJob.err              # Standard error file #SBATCH --account=    # Project account number # Load required modules module load intel                      # Load Intel compiler and libraries # Set environment variables for OpenMP export SRUN\_CPUS\_PER\_TASK=$SLURM\_CPUS\_PER\_TASK export OMP\_NUM\_THREADS=$SLURM\_CPUS\_PER\_TASK  # Set number of OpenMP threads # Run the application using srun srun ./myprogram < myinput > myoutput .. tab-item:: MPI Job For a typical MPI job you can take one of the following scripts as a template, and modify it depending on your needs. In this example we ask for 8 tasks, 2 SKL nodes and 1 hour of wallclock time, and runs an MPI application (myprogram) compiled with the intel compiler and the mpi library. The input data are in file "myinput", the output file is "myoutput", the working directory is where the job was submitted from. Through \`\`–cpus-per-task=1\`\` istruction each task will bind 1 physical cpu (core). This is a default option. .. code-block:: bash #!/bin/bash #SBATCH --time=01:00:00 #SBATCH --nodes=2 #SBATCH --ntasks-per-node=4 #SBATCH --ntasks-per-socket=2 #SBATCH --cpus-per-task=1 #SBATCH --mem= #SBATCH --partition= #SBATCH --qos= #SBATCH --job-name=jobMPI #SBATCH --err=myJob.err #SBATCH --out=myJob.out #SBATCH --account= module load intel intelmpi srun myprogram < myinput > myoutput .. tab-item:: GPU Job This SLURM batch script is designed to run a pure OpenMP application on Cienca's HPC clusters. It requests a single node and allocates all available physical CPU cores to a single task, making it ideal for shared-memory parallel programs. The script sets up the environment, loads the necessary modules, and configures OpenMP-specific variables to ensure optimal performance. It is tailored for systems without hyperthreading and can be easily adapted by modifying the number of CPUs per task and other resource parameters. .. code-block:: bash #!/bin/bash #SBATCH --job-name=multi\_gpu\_job # Descriptive job name #SBATCH --time=04:00:00 # Maximum wall time (hh:mm:ss) #SBATCH --nodes=4 # Number of nodes to use #SBATCH --ntasks-per-node=4 # Number of MPI tasks per node (e.g., 1 per GPU) #SBATCH --cpus-per-task=10 # Number of CPU cores per task (adjust as needed) #SBATCH --gres=gpu:4 # Number of GPUs per node (adjust to match hardware) #SBATCH --partition= # GPU-enabled partition #SBATCH --qos= # Quality of Service #SBATCH --output=multiGPUJob.out # File for standard output #SBATCH --error=multiGPUJob.err # File for standard error #SBATCH --account= # Project account number # Load necessary modules (adjust to your environment) module load cuda/12.2 # Load CUDA toolkit module load openmpi # Load MPI implementation module load your\_app\_dependencies # Load any other required modules # Optional: Set environment variables for performance tuning export OMP\_NUM\_THREADS=$SLURM\_CPUS\_PER\_TASK # Set OpenMP threads per task export NCCL\_DEBUG=INFO # Enable NCCL debugging (for multi-GPU communication) # Launch the distributed GPU application # Replace with your actual command (e.g., mpirun or srun) srun --mpi=pmix ./my\_distributed\_gpu\_app --config config.yaml Interactive Job Submission with SLURM ------------------------------------- SLURM allows users to run jobs interactively using two main methods: \`\`salloc\`\` and \`\`srun\`\`. These methods are useful for debugging, testing, or running short tasks that require real-time interaction. Using \`\`salloc\`\` ^^^^^^^^^^^^^^^^ The \`\`salloc\`\` command is used to allocate resources (nodes, cores, GPUs, etc.) for an interactive session. Once the allocation is granted, you can run commands on the allocated compute nodes using \`\`srun\`\`. \*\*Key Characteristics:\*\* - The job is queued and scheduled like a batch job. - Once started, the terminal session is connected to the allocated resources. - Input/output/error streams are tied to your terminal. - You can exit the session using \`\`exit\`\` or \`\`CTRL-D\`\`. \*\*Important Note:\*\* Even though you're in an interactive session, your shell prompt may still appear as if you're on the login node. Any command not prefixed with \`\`srun\`\` will run on the login node, not the compute node. \*\*Example:\*\* .. code-block:: bash salloc -N 1 --ntasks-per-node=8 squeue -u $USER # Check if the allocation is ready hostname # Runs on the login node srun hostname # Runs on the allocated compute node exit # Ends the interactive session \*\*Tip:\*\* You can also specify a command directly with \`\`salloc\`\`: .. code-block:: bash salloc -N 1 --ntasks=8 ./myscript.sh This will run the script on the allocated resources and return output to your terminal. Using \`\`srun --pty\`\` ^^^^^^^^^^^^^^^^^^^^ The \`\`srun\`\` command can also be used to start an interactive shell directly on the allocated compute node. \*\*Syntax:\*\* .. code-block:: bash srun -N 1 --ntasks-per-node=8 --pty /bin/bash \*\*Behavior:\*\* - SLURM allocates the requested resources and launches a shell. - Any additional \`\`srun\`\` commands inside this shell may hang if no resources are left. - To allow multiple \`\`srun\`\` commands within the session, use the \`\`--overlap\`\` flag. \*\*Recommendation:\*\* While \`\`srun --pty\`\` is convenient, it is generally recommended to use \`\`salloc\`\` for interactive jobs, especially when you plan to run multiple commands or scripts within the session. \*\*Summary\*\* +----------------+-------------------------------------------------------------+ | \*\*Method\*\* | \*\*Description\*\* | +================+=============================================================+ | \`\`salloc\`\` | Allocates resources and opens an interactive session. | | | Use \`\`srun\`\` inside to run commands on compute nodes. | +----------------+-------------------------------------------------------------+ | \`\`srun --pty\`\` | Directly opens a shell on compute nodes. | | | Use \`\`--overlap\`\` for multiple \`\`srun\`\` calls. | +----------------+-------------------------------------------------------------+ Monitoring Jobs --------------- squeue Command Reference ^^^^^^^^^^^^^^^^^^^^^^^^ The \`\`squeue\`\` command is used to display the status of jobs in a SLURM-managed cluster. It shows jobs that are pending, running, or recently completed. \*\*Common Options\*\* +--------------------+-------------------------------------------------------------+ | \*\*Option\*\* | \*\*Description\*\* | +====================+=============================================================+ | \`\`-u \`\` | Show jobs for a specific user. | | | Example: \`\`squeue -u alice\`\` | +--------------------+-------------------------------------------------------------+ | \`\`-j \`\` | Show information for a specific job ID. | | | Example: \`\`squeue -j 123456\`\` | +--------------------+-------------------------------------------------------------+ | \`\`-p \`\` | Filter jobs by partition (queue). | | | Example: \`\`squeue -p gpu\`\` | +--------------------+-------------------------------------------------------------+ | \`\`-t \`\` | Filter jobs by state (e.g., \`\`R\`\` for running, \`\`PD\`\` for | | | pending). | +--------------------+-------------------------------------------------------------+ | \`\`-o \`\` | Customize the output format. | +--------------------+-------------------------------------------------------------+ | \`\`--sort \`\`| Sort the output by specified fields. | | | Example: \`\`--sort=-t\`\` to sort by time left. | +--------------------+-------------------------------------------------------------+ | \`\`--start\`\` | Estimate job start times (useful for pending jobs). | +--------------------+-------------------------------------------------------------+ | \`\`--help\`\` | Display help information for \`\`squeue\`\`. | +--------------------+-------------------------------------------------------------+ \*\*Example: Custom Output Format\*\* To display a custom set of job details: .. code-block:: bash squeue -o "%.18i %.9P %.8j %.8u %.2t %.10M %.6D %R" This format shows: - Job ID - Partition - Job name - Username - State - Time used - Number of nodes - Reason (why pending or where running) \`\`squeue\`\` is a powerful tool for monitoring job status and diagnosing scheduling issues. Combine it with other SLURM commands like \`\`sinfo\`\` and \`\`scontrol\`\` for full cluster visibility. sinfo ^^^^^ The \`\`sinfo\`\` command provides information about the state of SLURM nodes and partitions. \*\*Common Options:\*\* +----------------------+-----------------------------------------------------------+ | \*\*Option\*\*           | \*\*Description\*\*                                           | +======================+===========================================================+ | \`\`-s\`\`               | Display a summary of node states.                         | +----------------------+-----------------------------------------------------------+ | \`\`-N\`\`               | Show information by node rather than by partition.        | +----------------------+-----------------------------------------------------------+ | \`\`-p \`\`   | Show information for a specific partition.                | +----------------------+-----------------------------------------------------------+ | \`\`-o \`\`      | Customize the output format.                              | +----------------------+-----------------------------------------------------------+ \*\*Example:\*\* .. code-block:: bash sinfo -o "%P %D %t %C" This shows partition name, number of nodes, state, and CPU allocation. scontrol ^^^^^^^^ The \`\`scontrol\`\` command is used for querying and modifying SLURM configuration and job details. \*\*Common Uses:\*\* +-------------------------------+----------------------------------------------------+ | \*\*Command\*\*                 | \*\*Description\*\*                                    | +===============================+====================================================+ | \`\`scontrol show job \`\` | Display detailed information about a specific job. | +-------------------------------+----------------------------------------------------+ | \`\`scontrol show node \`\` | Show detailed info about a specific node.       | +-------------------------------+----------------------------------------------------+ | \`\`scontrol hold \`\` | Place a hold on a job to prevent it from starting. | +-------------------------------+----------------------------------------------------+ | \`\`scontrol release \`\` | Release a held job.                             | +-------------------------------+----------------------------------------------------+ \*\*Example:\*\* .. code-block:: bash scontrol show job 123456 This displays detailed job configuration, resource usage, and node assignment. scancel ^^^^^^^ The \`\`scancel\`\` command is used to \*\*cancel jobs\*\* that are pending, running, or held in the SLURM job queue. It is useful for terminating jobs that are no longer needed or were submitted in error. \*\*Common Options\*\* +----------------------------+-----------------------------------------------------+ | \*\*Option\*\* | \*\*Description\*\* | +============================+=====================================================+ | \`\`scancel \`\` | Cancel a specific job by its job ID. | +----------------------------+-----------------------------------------------------+ | \`\`-u \`\` | Cancel all jobs belonging to a specific user. | +----------------------------+-----------------------------------------------------+ | \`\`-n \`\` | Cancel jobs by job name. | +----------------------------+-----------------------------------------------------+ | \`\`-p \`\` | Cancel jobs in a specific partition. | +----------------------------+-----------------------------------------------------+ | \`\`-t \`\` | Cancel jobs in a specific state (e.g.,\`\`PD\`\`,\`\`R\`\`).| +----------------------------+-----------------------------------------------------+ | \`\`--help\`\` | Display help information for \`\`scancel\`\`. | +----------------------------+-----------------------------------------------------+ \*\*Examples\*\* Cancel a specific job by ID: .. code-block:: bash scancel 123456 Cancel all jobs for the current user: .. code-block:: bash scancel -u $USER Cancel all pending jobs in the GPU partition: .. code-block:: bash scancel -p gpu -t PD .. note:: - You must have permission to cancel the job (typically your own jobs). - Use with caution, especially when canceling multiple jobs at once. --- # Unknown File Systems and Data Management ================================ All HPC systems share the same logical disk structure and file system definition. In Cineca, all the filesystems are based on Lustre. The available storage areas can have multiple definitions/purposes: \* \*\*temporary\*\*: data are accessible for a defined time window, after that data will be canceled. \* \*\*permanent\*\*: data are accessible for additional six months after the \*end\* of the project. Storage areas can be also: \* \*\*user specific\*\*: each user has exclusive data area. \* \*\*shared\*\*: area accessible by all \*collaborators\* belonging the same project. \* \*\*open\*\*: area accessible by all users of an HPC system. .. note:: The available data areas are defined, on all HPC systems, through predefined \`\`environment variables\`\`. You can access on these areas simply using the name reported in the following table. Users hare strongly encouraged to use predefined \`\`environment variables\`\` instead of the full path (e.g: in scripts and codes data). .. list-table:: Overview of Available Data Areas :widths: 30 50 20 20 80 :header-rows: 1 :align: center \* - \*\*Name\*\* - \*\*Area Attributes\*\* - \*\*Quota\*\* - \*\*Backup\*\* - \*\*Note\*\* \* - $HOME - permanent, user specific - 50 GB - daily - \* - $WORK - permanent, shared - 1 TB - no - Large data to be shared with project's collaborators. \* - $FAST - permanent, shared - 1 TB - no - Only on :ref:\`hpc/leonardo:Leonardo\`. Faster I/O compared with outer areas. \* - $SCRATCH - temporary, user specific - -/20 TB - no - files older than 40 days are deleted \* - $TMPDIR - temporary, user specific - (-) - no - directory removed at job completion \* - $PUBLIC - permanent, open, user specific - 50 GB - no - Only on :ref:\`hpc/leonardo:Leonardo\`. \* - $DRES - permanent, shared - defined by project - no - .. warning:: \*\*Ethical Use of the SCRATCH Area\*\* Users are encouraged to respect the intended use of the various areas. Users are reminded that the SCRATCH area is not subject to restrictions (quota) to facilitate the production of data, even large amounts. However, the SCRATCH area should not be used as a temporary storage area. Users are warned against using \*\*“touch”\*\* commands or similar methods to extend the retention of files beyond the 40-day limit. The use of such \*\*“improper”\*\* procedures will be monitored, and users will be subject to various degrees of \*restrictions up to a ban\*! Areas Details ------------- .. tab-set:: .. tab-item:: $HOME \*\*$HOME: permanent, user specific\*\* $HOME is a local area where you are placed after the login procedure. It is where system, and user applications store their dot-files and dot-directories (\`\`.nwchemrc\`\`, \`\`.ssh\`\`, ...) and where users keep initialization files specific for the systems (\`\`.cshrc\`\`, \`\`.profile\`\`, ...). There is a $HOME area for each username on the machine. This area is conceived to store programs and small personal data. It has a quota of 50 GB. Files are never deleted from this area. Moreover, they are guaranteed by daily backups: if you delete or accidentally overwrite a file, you can ask our Help Desk to restore it. A maximum of 3 versions of each file is stored as a backup. The last version of the deleted file is kept for two months, then definitely removed from the backup archive. File retention is related to the life of the username; data are preserved until the username remains active. .. tab-item:: $WORK \*\*$WORK: permanent,shared\*\* $WORK is a scratch area for collaborative work within a given project. File retention is related to the life of the project. Files in $WORK will be conserved up to 6 months after the project end, and then they will be cancelled. Please note that there is no back-up in this area. This area is conceived for hosting large working data files since it is characterized by the high bandwidth of a parallel file system. It behaves very well when I/O is performed accessing large blocks of data, while it is not well suited for frequent and small I/O operations. This is the main area for maintaining scratch files resulting from batch processing. There is one $WORK area for each active project on the machine. The default quota is 1 TB per project, but extensions can be considered by the Help Desk if motivated. The owner of the main directory is the PI (Principal Investigator) of the project. All collaborators are allowed to read/write in there. Collaborators are advised to create a personal directory in $WORK for storing their personal files. By default, the personal directory will be protected (only the owner can read/write), but protection can be easily modified, for example by allowing write permission to project collaborators through \`\`chmod\`\` command. This second approach does not affect global files security. The \`\`chprj\`\` change project command makes it easier to manage the different $WORK areas for different projects. \*\*Summary\*\* \* created when a project is opened. \* each project has its own area. \* all \*collaborators\* can write in the area. \* each user has as many $WORK areas as active projects. \* by default files are private. \* users can change file permission to make them visible, readable and writable to project's collaborators. .. tab-item:: $FAST \*\*$FAST: permanent, shared\*\* (:ref:\`hpc/leonardo:Leonardo\` only) $FAST is a scratch area for collaborative work within a given project. File retention is related to the life of the project. Files in $FAST will be conserved up to 6 months after the project end, and then they will be cancelled. Please note that there is \*\*no back-up\*\* in this area. This area is conceived for hosting working data files whenever the I/O operations constitute the bottleneck for your applications. It behaves well both when I/O is performed accessing large blocks of data, and for frequent and small I/O operations. Due to the limited size of the area, the main space for maintaining the data resulting from batch processing is the corresponding $WORK area. There is one $FAST area for each active project on the machine. The fixed quota is 1 TB per project, and due to the total dimension of the storage, extensions cannot be considered. The owner of the main directory is the PI (Principal Investigator) of the project. All collaborators are allowed to read/write in there. Collaborators are advised to create a personal directory in $FAST for storing their personal files. By default, the personal directory will be protected (only the owner can read/write), but protection can be easily modified, for example by allowing write permission to project collaborators through chmod command. This second approach does not affect global files security. .. tab-item:: $SCRATCH \*\*$SCRATCH: temporary, user specific\*\* This is a local temporary storage conceived for temporary files from batch applications. There are important differences with respect to $WORK area. It is user specific (not project specific). By default, file access is closed to everyone, in case you need less restrictive protections, you can set them with chmod command. On this area, a periodic cleaning procedure could be applied, with a normal retention time of 40 days: files are daily cancelled by an automatic procedure if not accessed for more than 40 days. In each user's home directory ($HOME) a file lists all deleted files for a given day. .. code-block:: bash CLEAN\_.log = date when files were cancelled \*\*Summary\*\* \* created when a user has granted access. \* each user has it own area (exclusively). \* files older than 40-days are cancelled. \* no quota \* by default files are public (read only). \* user can change file permission to make files private. .. warning:: Users are encouraged to respect the intended use of the various areas. Users are reminded that the SCRATCH area is not subject to restrictions (quota) to facilitate the production of data, even large amounts. However, the SCRATCH area should not be used as a temporary storage area. Users are warned against using “touch” commands or similar methods to extend the retention of files beyond the 40-day limit. The use of such “improper” procedures will be monitored, and users will be subject to various degrees of restrictions up to a ban. .. tab-item:: $TMPDIR \*\*$TMPDIR: temporary, user specific\*\* Each compute node is equipped with a local area whose dimension differs depending on the cluster. When a job starts, a \*\*temporary area\*\* is defined on the storage \*local to each compute node\*. \* On \*\*login nodes\*\*: \`\`TMPDIR=/scratch\_local\`\` \* On \*\*Galileo100\*\*: \`\`TMPDIR=/scratch\_local/slurm\_job.$SLURM\_JOB\_ID\`\` \* On \*\*Leonardo\*\*: \`\`TMPDIR=/tmp\`\` (visible with the command \`\`df -h /tmp\`\`). Special behavior can be found in the specific section :ref:\`hpc/leonardo:Leonardo\`. If more jobs share one node, each one will have a \`\`private/tmp\`\` in the job's user space. The \*TMPFS\* are removed at the end of each job (data will be deleted). Whatever the mechanism, the \*TMPDIR\* can be used \*\*exclusively\*\* by the job's owner. During a job, user can get access to the area with \*local\* variable \`\`$TMPDIR\`\`. In a sbatch script, for example, user can move the input data of simulations to the \`\`$TMPDIR\`\` before the beginning of job run and also write on \`\`$TMPDIR\`\` job output. This would further improve the I/O speed of a code. Please note that the area is located on local disks, so it can be accessed only by the processes running on the specific node. For multinode jobs, if you need all the processes to access some data, please use the shared filesystems \`\`$HOME\`\`, \`\`$WORK\`\` and \`\`$SCRATCH\`\`. .. tab-item:: $PUBLIC \*\*$PUBLIC: permanent, open, user specific\*\* (LEONARDO ONLY) $PUBLIC is a shared area. Each username on the machine owns a $PUBLIC area with a quota of 50 GB. This area is accessible by every other user of the cluster. File retention is related to the life of the username; data are preserved until the username remains active. Please note that there is no back-up in this area. .. tab-item:: $DRES \*\*$DRES: permanent, shared (among platforms and projects)\*\* This is a repository area for collaborative work among different projects and across platforms. You need to explicitly ask for this kind of resource: it does not come as part of a project contact the user support. File retention is related to the life of DRES itself. Files in DRES will be conserved up to 6 months after DRES completion, then they will be cancelled. Several types of DRES are available, at present: \* \*\*FS\*\*: normal filesystem access oh high throughput disks, shared among all systems (mounted only on login nodes). \* \*\*ARCH\*\*: magnetic tape archiving with a disk-like interface via LTSFS. \* \*\*REPO\*\*: smart repository on iRODS. \*\*Summary\*\* \* created on request. \* non linked to a specific project. \* all collaborators can write in. \* compute nodes \*\*cannot\*\* access to data in $DRES. \* by default files are public (read only). \* Quota based on needs. \* No backup. Backup Policy and Data Availability ----------------------------------- Daily backups guarantee the $HOME filesystem. In particular, the daily backup procedure preserves a maximum of three different copies of the same file. Older versions of files are kept for 1 month. The last version of deleted files is kept for 2 months, then definitely removed from the backup archive. Different agreements about Backup policies are possible. For more information contact the HPC support (superc@cineca.it). Data, both backed up and non-backed up, are available for the entire duration of the project. After a project expires, users will still have full access to the data for an additional six months. Beyond this six-month period, data availability is not guaranteed. .. important:: Users have \*\*responsibility\*\* to backup their important data !!! A scheme of data availability is reported in the figure below. .. figure:: img/file\_time.png :align: center :height: 200px .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link Lustre Best Practice -------------------- \*\*Overview\*\* Lustre is a parallel distributed filesystem ideal in handling large files accessed by many compute nodes. However, it struggles with small files and certain access patterns common in desktop and enterprise environments. Following best practices can minimize expensive operations and improve performance. \*\*Key Recommendations\*\* - \*Minimize Metadata Operations\* Avoid frequent access to file attributes (e.g., size, type, permissions) and commands like \`\`ls -l\`\`. Use simpler alternatives ( \`\`ls\`\` or \`\`lfs\`\` commands). - \*Avoid Metadata-Intensive Commands\* Refrain from using commands like \`\`ls -R\`\` , \`\`find\`\` , \`\`du\`\` , and \`\`df\`\` . Instead, use Lustre-specific tools like \`\`lfs find\`\` . - \*Limit Wildcard Usage\* Expanding wildcards (e.g., \* or ?) is resource-intensive, especially when matching many files in a large directory. For instance, commands like \`\`rm \*.tmp\`\` can significantly degrade performance on Lustre. Instead, precompile a list of target files (e.g., \`\`lfs find . -name "\*.tmp" > files\_to\_delete.txt\`\`) and process them iteratively. This method avoids the overhead of expanding wildcards directly on the filesystem. For large-scale operations, ensure scripts are designed to handle smaller batches of files to reduce the impact on metadata servers. - \*Organize Files\* Avoid storing large numbers of files in a single directory. This creates contention as Lustre locks the parent directory when files are accessed, leading to performance bottlenecks. Use subdirectories to distribute files. A common approach is to create directories based on the square root of the total number of files. For example, 90,000 files could be split into 300 directories with 300 files each. Logical data grouping (e.g., by date or project) can further streamline access and maintenance. - \*Avoid Small Files\* Accessing small files is inefficient. Where possible, combine them into larger files (e.g., using \`\`tar\`\`) or use formats like HDF5 or NetCDF. If the total size of the small files is manageable (e.g., a few GB), copy them to a local directory (/tmp) on the compute nodes at the start of a job and clean up afterward. Alternatively, create read-only disk images (e.g., ISO) that can be mounted via loopback. Tools like Singularity can facilitate this approach for containers. - \*Minimize Repeated Operations\* Perform all I/O in a single session instead of frequent, small operations. For example, avoid operations such as appending small amounts of data repeatedly. Instead, open the file once, perform all operations in a single session, and close the file. For append-heavy workloads, consider buffering data in memory and writing it in larger chunks. - \*Prevent File Access Contention\* Avoid multiple processes accessing the same file region or appending to the same file simultaneously. Use a single "master" process for such operations. Use strategies like file replication, splitting files, or delegating access to a single master process. Ensure processes access distinct file regions whenever possible. - \*File Locking and Backups\* Use file locking (flock) only when necessary, as it can impact performance. Lustre generally manages non-overlapping writes and concurrent append operations effectively. Regularly back up data to a secure location, as Lustre does not provide built-in backup capabilities. \*\*File Striping\*\* File striping is a method employed in Lustre to enhance data access and storage performance by distributing the contents of a single file across multiple storage devices, or \*\*OSTs\*\* (Object Storage Targets). Rather than storing a file as a single block on one device, striping breaks the file into smaller pieces, (or "\*stripes\*") with each chunk written to a different device according to a set pattern. This approach helps increase throughput, improve parallelism, and reduce bottlenecks. The striping of a file can be defined by different parameters, the most important are: - the \*Stripe Count\*, which indicates the number of OSTs across which a file is distributed. A stripe count of 1 means the file is stored on a single OST, while higher values spread the file across multiple OSTs in a round-robin fashion. - the \*Stripe Size\*, which refers to the amount of data (typically in bytes) written consecutively to a single OST before moving to the next OST. Common defaults are around 1 MB, but it can range from 512 KB up to several GB. Choosing the right stripe size balances between overhead and parallelism. In the following scheme, different striping examples are reported: \*File C\* has a larger \*stripe size\* than \*File A\*, allowing more data per stripe. \*File A\* is striped across three OSTs (\*stripe count\* = 3), while \*Files B\* and \*C\* are stored on a single OST (\*stripe count\* = 1). No space is reserved on the OST for unwritten data. .. image:: img/striping\_example.png :align: center :height: 350 px :class: no-scaled-link The \*\*Parallel File Layouts\*\* (\*\*PFL\*\*) defines how this striping is applied to files (or directories). It allows to specify how files should be split across storage resources, setting the for example the \*stripe count\* and \*the stripe size\*. The layout can vary based on different factors like \*\*file size\*\*, \*\*filesystem\*\*, and \*\*hardware configuration\*\*. For example, the PFL can be configured to limit the number of stripes for small files, while setting a higher stripe count for large files. In the following scheme, one \*\*PFL\*\* example is reported in details: .. image:: img/pfl.png :align: center :height: 350 px :class: no-scaled-link The \*\*PFL\*\* consists of three \*Components\*, (defined by \*\*range\*\*: 0-2MB, 2MB-256MB and 256MB-EOF), mapping a 2055MB file. The first two components use a 1MB \*stripe size\*, while the third uses a 4MB \*stripe size\*. The stripe count increases with each component and accordingly with the file size, (Component 1: \*stripe count\* =1, Component 2: \*stripe count\* =4 and Component 3: \*stripe count\* = 32). The first component of our 2055MB file, contains two 1MB blocks in a single 2MB object. The second component spans 254MB across four OST objects, each holding 64MB, with a 1MB hole in the first two objects. The third component distributes 1800MB across 32 OST objects, each holding 64MB, except obj 3,0 and obj 3,1, which contain extra chunks. Additional data would only expand component 3. Here are the default Parallel File Layouts (PFL) for all CINECA HPC systems utilizing Lustre. .. note:: Since Lustre cannot know the final size of a file \*a priori\*, it starts creating it incrementally following the \*\*PFL\*\*. A file of many GB will have the first MB with striping rules similar to those of small files, then the rest like large files. Here are the default Parallel File Layouts (PFL) for all CINECA HPC systems utilizing Lustre. .. tab-set:: .. tab-item:: Leonardo .. list-table:: Parallel File Layouts (PFL) for Leonardo. :widths: 25 45 55 :header-rows: 1 :align: center \* - \*\*Filesystem\*\* - \*\*PFL ranges\*\* - \*\*PFL Parameters\*\* \* - $HOME - File size = 64 kB - 10 GB File size = 10 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB \* - $WORK - File size = 64 kB - 10 GB File size = 10 GB - 100 GB File size = 100 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 2, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB \* - $FAST - File size = 64 kB - 10 GB File size = 10 GB - 100 GB File size = 100 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 2, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB \* - $SCRATCH - File size = 64 kB - 10 GB File size = 10 GB - 100 GB File size = 100 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 2, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB \* - $PUBLIC - File size = 64 kB - 10 GB File size = 10 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB .. tab-item:: G100 .. list-table:: Parallel File Layouts (PFL) for G100. :widths: 25 45 55 :header-rows: 1 :align: center \* - \*\*Filesystem\*\* - \*\*PFL ranges\*\* - \*\*PFL Parameters\*\* \* - $HOME - File size = 64 kB - EOF - Stripe count = 1, Stripe size = 1 MB \* - $WORK - File size = 64 kB - 1 GB File size = 1 GB - 4 GB File size = 4 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB Stripe count = -1, Stripe size = 1 MB \* - $SCRATCH - File size = 64 kB - 1 GB File size = 1 GB - 4 GB File size = 4 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB Stripe count = -1, Stripe size = 1 MB .. tab-item:: Pitagora .. list-table:: Parallel File Layouts (PFL) for Pitagora. :widths: 25 45 55 :header-rows: 1 :align: center \* - \*\*Filesystem\*\* - \*\*PFL ranges\*\* - \*\*PFL Parameters\*\* \* - $HOME - File size = 64 kB - 10 GB File size = 10 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB \* - $WORK - File size = 64 kB - 10 GB File size = 10 GB - 100 GB File size = 100 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB Stripe count = -1, Stripe size = 1 MB \* - $SCRATCH - File size = 64 kB - 10 GB File size = 10 GB - 100 GB File size = 100 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB Stripe count = -1, Stripe size = 1 MB \* - $PUBLIC - File size = 64 kB - 10 GB File size = 10 GB - EOF - Stripe count = 1, Stripe size = 1 MB Stripe count = 4, Stripe size = 1 MB \`\`Stripe count = -1\`\` \*means that Lustre shall use every OST available for striping the file, rather than a fixed number of OSTs.\* \*\*lfs setstripe command\*\* \`\`lfs setstripe\`\` command is intended to create new files or directories with a specific PFL configuration. .. code-block:: bash lfs setstripe \[--size|-s stripe\_size\] \[--stripe-count|-c stripe\_count\] \[--component-end|-E \] filename|dirname The most useful flags are: - \`\`-E \`\`: This flag sets the extent of file sizes that the following striping options should apply to. For example, -E 1G applies to files that are 1GB or smaller, -E 100G applies to files between 1GB and 100GB, and so on. - \`\`-c \`\`: Set the stripe count, which specifies how many Object Storage Targets (OSTs) the file will be striped across. For example, -c 1 means the file will be stored on 1 OST, -c 2 means the file will be stored on 2 OSTs, and so on. - \`\`-S \`\`: Set the stripe size, which specifies how large each stripe will be. For example, -S 1M means each stripe will be 1MB. \*\*Examples\*\* - Create a file with striping on a single OST: This will ensure that myfile is stored entirely on a single OST, which is useful for small files. \`\`lfs setstripe -c 1 myfile\`\` - Create a file with striping on 2 OSTs: This distributes the data of bigfile across two OSTs. It’s suitable for medium-sized files (between 10GB and 100GB). \`\`lfs setstripe -c 2 bigfile\`\` - Create a directory with default striping: This sets the default striping for all files in /mydir to 2 OSTs. New files created in this directory will inherit this configuration. \`\`lfs setstripe -c 2 /mydir\`\` - Set both the stripe count and stripe size: This command not only sets the number of OSTs (4) but also configures the stripe size to 4MB, which is great for optimizing performance on large files. A good stripe size for sequential I/O using high-speed networks is between 1 MB and 4 MB. \`\`lfs setstripe -c 4 -S 4M hugefile\`\` - View the current striping configuration of a file: To check the current striping configuration of a file, use: \`\`lfs getstripe largefile.txt\`\` Data Occupancy Monitoring Tools ------------------------------- The occupancy status of all areas accessible to a user, along with the storage quota limits, can be monitored using a simple command available on all HPC cluster. There are two commands named \`\`cindata\`\` , \`\`cinQuota\`\`. For both commands the flag \`\`-h\`\` can be used to show the help. Both tools are available in the module cintools, which is automatically loaded in your environment. In the following, an example of \`\`cindata\`\` and \`\`cinQuota\`\` outputs is shown. .. tab-set:: .. tab-item:: cindata .. code-block:: bash $ cindata USER AREADESCR AREAID FRESH USED QTA USED% aUSED aQTA aUSED% myuser00 /gpfs/work/ galileo\_work-Acc-name 9hou 114G -- --% 14T 30T 48.8% myuser00 /gpfs/scratch/ galileo\_scr 9hou 149G -- --% 341T 420T 81.2% myuser00 /galileo/home galileo\_hpc-home 9hou 5.7G 50G 11.4% 16T -- --% myuser00 /gss/gss\_work/DRES\_myAcc work\_OFFLINE-DRES\_myAcc-FS 9hou 2.9G -- --% 11T 15T 73.3% myuser00 /gss/gss\_work/DRES\_myAcc work\_ONLINE-DRES\_myAcc-FS 9hou 1.2T -- --% 2.8T 4T 70.0% Interpreting the storage status can be complex. Here's a breakdown: \* \*\*OFFLINE\*\* area: this represents \`\`DRES\`\` data that has been stored on tape after three months of storage. \* \*\*ONLINE\*\* area: this represents \`\`DRES\`\` data that is still in the filesystem or \`\`ARCH\`\` area. The total storage quota assigned to your DRES is indicated by the aQTA parameter in the OFFLINE line. When the \`\`DRES\`\` is empty, the \*\*ONLINE\*\* value will be the same as \*\*OFFLINE\*\*. As files begin to be moved to tape, the \*\*ONLINE\*\* value will decrease, while the \*aUSED\* parameter in \*\*OFFLINE\*\* will increase accordingly. This indicates that you have less space available for storing new data since some of the used space has been moved to tape. Similarly, deleting offline data will decrease the \*aUSED\* parameter in OFFLINE and increase the \*aQTA\* parameter in \*\*ONLINE\*\* by the same amount. Remember this formula: .. math:: TOTAL DRES STORAGE = aQTA-OFF = aQTA-ON + aUSED-OFF .. tab-item:: cinQuota The additional tool for monitoring the disk occupancy is named \`\`cinQuota\`\` . A typical output of the command will contain the following information: .. code-block:: bash $ cinQuota ----------------------------------------------------------------------------------------------------------------------------------- Filesystem used quota grace files ------------------------------------------------------------------------------------------------------------------------------------ /g100/home/userexternal/myuser00 22.66G 50G - 194295 /g100\_scratch/userexternal/myuser00 1.955T 0k - 41139 /g100\_work/ 366.3G 1T - 548665 ----------------------------------------------------------------------------------------------------------------------------------- Manage File Permissions ----------------------- As explained above, \`\`$WORK\`\` and \`\`$DRES\`\` are environmental variables automatically set in the user environment. \* $WORK variable points to a directory (fileset) specific for one of the user projects: \`\`/gpfs/work/\`\`. \* $DRES variable points to space where all of the dres are defined: \`\`/gss/gss\_work/\`\`. - in order to use a specific DRES type, the path is \`\`$DRES/\`\`. The owner of the root directory is the "Principal Investigator" (PI) of the project or the "owner" of the DRES, the group corresponds to the name of the project or the name of the DRES. Default permissions are: .. code-block:: bash own: rwx group: rwx other: - in this way, all project's collaborators sharing the same project \*group\* can \*\*read/write\*\* into the \*\*project/dres fileset\*\*, whereas others users can not. Users are advise to create a personal \*subdirectory\* under \`\`$WORK\`\` and \`\`$DRES\`\`. By default, files into the \*subdirectory\* are private, but the owner can easily share the files with other collaborators by opening the \*subdirectory\*: .. code-block:: bash chmod 777 mydir chmod 755 mydir since the \`\`$WORK/$DRES\`\` fileset is closed to non-collaborators, the data sharing is active only among project's collaborators. \*\*Pointing $WORK to a different project: the chprj command\*\* The user can modify the project pointed to by the variable \`\`$WORK\`\` using the \`\`change project\`\` command. To list all your accounts (both active or completed) and the default project: .. code-block:: bash chprj -l To set \`\`$WORK\`\` to point to a different project: .. code-block:: bash chprj -d More details are in the help page of the command: .. code-block:: bash chprj -h chprj --help Data Transfer ------------- Users can use login nodes to transfer small files, but we \*\*strongly\*\* suggest to use the dedicated services: CINECA provides a data transfer service based on two main tools: \*\*data movers\*\* and \*\*GridFTP\*\*. \*\*Data movers\*\* are dedicated, containerized nodes without interactive access, supporting only a limited set of commands (\`scp\`, \`rsync\`, \`sftp\`, \`wget\`, \`curl\`, \`rclone\`, \`aws s3\` and \`s3\`). User authentication is done via SSH certificates with 2-Factor Authentication or host-based authentication from within CINECA clusters. \*\*GridFTP\*\* is also available on these nodes but can only be used through the \`globus-url-copy \`\_ client, which must be run from the user's local machine. Further details on how to use these tools are provided in the two tabs below. .. tab-set:: .. tab-item:: Data Mover Datamovers are dedicated nodes on each HPC cluster that are designed for transferring data FROM/TO a cluster. \*\*Hostnames and IPs\*\* - \*\*Galileo100\*\*: alias: data.g100.cineca.it hostnames and IPs: - dmover1.g100.cineca.it - 130.186.16.212 - dmover2.g100.cineca.it - 130.186.16.213 - \*\*Leonardo\*\*: alias: data.leonardo.cineca.it hostnames and IPs: - dmover1.leonardo.cineca.it - 131.175.44.50 - dmover2.leonardo.cineca.it - 131.175.44.51 - dmover3.leonardo.cineca.it - 131.175.44.52 - dmover4.leonardo.cineca.it - 131.175.44.53 \*\*Main features\*\* This transfer service is containerized, and there are many differences between these nodes and the login nodes. First of all, on datamovers, there is no CPU time limit, that allows long data transfers. Unlike, on login nodes, there is a 10-minute of CPU time limit that usually interrupts the transfer of a large amount of data. By construction, the shell is not available, so it is not possible to open interactive sessions. In other words you cannot connect directly to the datamover via SSH. The only available commands are scp, rsync, sftp, wget, curl, rclone, s3 and aws s3. However, the authentication is still based on SSH protocol. There are only 2 possible authentication methods: - \*publickey\*: it only accepts valid SSH certificates, obtained via 2-Factor Authentication. No private/public keys generated in other ways are accepted. - \*hostbased\*: if you are already logged into a CINECA HPC cluster and try to use a datamover from a login node, the SSH daemon on the datamover recognizes you are already authenticated on a CINECA HPC cluster and that is enough. .. warning:: The host-based authentication is not enabled inside a job batch. If you want to use a datamover inside a job batch you have to copy a valid 2FA SSH certificate inside your ~/.ssh directory on the cluster where you are submitting the job batch. .. important:: When you are authenticated on a datamover, the environment variables $HOME, $WORK and $CINECA\_SCRATCH (as well as ~ or \* ) are not defined. This property has 2 side effects: 1. if you want to transfer files FROM/TO your cluster personal areas, you have to specify the absolute path of them. 2. You cannot make use of the SSH configuration files stored in your remote ~/.ssh/ directory (such as $HOME/.ssh/config). \*\*Listing Directory via sftp\*\* If you need to list files on a cluster where login nodes are offline, you can rely on datamover service via the sftp command: .. code-block:: bash sftp @data..cineca.it:/path/to/be/listed/ Connected to data..cineca.it Changing to: /path/to/be/listed/ sftp> One entered the sftp session, the familiar pwd, cd /path/to/, ls commads are available to explore the remote filesystem, together with the sftp command lpwd, lcd /path/to/, lls. You can also transfer data from the sftp session, see the appropriate section below. \*\*Available transfer tools\*\* - \*\*rsync\*\* There are 2 possible ways to use rsync via datamovers: 1. You need to upload or download data FROM/TO your local machine TO/FROM a CINECA HPC cluster .. code-block:: bash rsync -PravzHS /absolute/path/from/file @data..cineca.it:/absolute/path/to/ rsync -PravzHS @data..cineca.it:/absolute/path/from/file /absolute/path/to/ 2. You need to transfer files between 2 CINECA HPC clusters .. code-block:: bash ssh -xt @data..cineca.it rsync -PravzHS /absolute/path/from/file @data..cineca.it:/absolute/path/to/ ssh -xt @data..cineca.it rsync -PravzHS @data..cineca.it:/absolute/path/from/file /absolute/path/to/ - \*\*scp\*\* There are 3 possible ways to use scp via datamovers: 1. You need to upload or download data FROM/TO your local machine TO/FROM a CINECA HPC cluster .. code-block:: bash scp /absolute/path/from/file @data..cineca.it:/absolute/path/to/ scp @data..cineca.it:/absolute/path/from/file /absolute/path/to/ 2. You need to transfer files between 2 CINECA HPC clusters .. code-block:: bash ssh -xt @data..cineca.it scp /absolute/path/from/file @data..cineca.it:/absolute/path/to/ ssh -xt @data..cineca.it scp @data..cineca.it:/absolute/path/from/file /absolute/path/to/ 3. You need to transfer files between 2 CINECA HPC clusters using your local machine as a bridge. We strongly suggest not using this option because it has very low transfer performance, each file you move from one cluster to another will pass through your local machine .. code-block:: bash scp -3 @data..cineca.it:/absolute/path/from/file data..cineca.it:/absolute/path/from/file - \*\*sftp\*\* There are 2 possible ways to use sftp via datamovers: 1. You need to upload or download data FROM/TO your local machine TO/FROM a CINECA HPC cluster .. code-block:: bash sftp @data..cineca.it:/absolute/remote/path/to/ sftp> put relative/local/path/to/file Uploading /absolute/local/path/to/file to /absolute/remote/path/to/file file 100% 414 365.7KB/s 00:00 sftp> get relative/remote/path/to/file Fetching /absolute/remote/path/to/file to file file 100% 1455KB 19.0MB/s 00:00 sftp> 2. You need to transfer files between 2 CINECA HPC clusters .. code-block:: bash ssh -xt @data..cineca.it sftp @data..cineca.it:/absolute/path/to/ It is also possible to use the flag -b and execute sftp in batch mode. - \*\*wget\*\* Sometimes, the 10-minute CPU time limit or the 4-hour wall time limit on the serial queue is not enough to download a large dataset for ML. In this case, you can use wget from the datamover. Here you can find a simple example .. code-block:: bash ssh -xt @data..cineca.it wget http://ftp.gnu.org/gnu/wget/wget2-2.0.0.tar.gz -P /absolute/path/to/ Please note that is mandatory to use the flag -P with the absolute path of the destination folder, because of the fake /home directory. - \*\*curl\*\* Sometimes, the 10-minute CPU time limit or the 4-hour wall time limit on the serial queue is not enough to download a large dataset for ML. In this case, you can use curl from the datamover. Here you can find a simple example .. code-block:: bash ssh -xt @data..cineca.it curl https://curl.se/download/curl-8.2.1.tar.gz --output /absolute/path/to/curl-8.2.1.tar.gz Please note that is mandatory to use the flag --output with the absolute path of the destination file, because of the fake /home directory. - \*\*rclone\*\* Rclone is a powerful tool that supports different transfer protocols, and a lot of data \[providers\](https://rclone.org/#providers). At the moment it is available on Leonardo and Galileo100 datamovers. It needs a configuration file. If you are able, you car write the configuration file using your favourite editor (VIM) or you can rely on the rclone config command: .. code-block:: bash ssh -xt @data.leonardo.cineca.it rclone --config /leonardo/home/userexternal//.rclone.conf config When your configuration is ready you can use rclone to manage data between Leonardo filesystem and the remote host you have configures. For example: .. code-block:: bash ssh -xt @data.leonardo.cineca.it rclone --config /leonardo/home/userexternal//.rclone.conf copy /absolute/path/to/{file|directory} my\_remote: ssh -xt @data.leonardo.cineca.it rclone --config /leonardo/home/userexternal//.rclone.conf move my\_remote:{file|directory} /absolute/path/to/{file|directory} ssh -xt @data.leonardo.cineca.it rclone --config /leonardo/home/userexternal//.rclone.conf sync /absolute/path/to/directory my\_remote:remote/directory Please note that is mandatory to use the flag --config with the absolute path of the config file, because of the fake /home directory. - \*\*aws s3\*\* AWS is the official command line tool from Amazon to manage s3 buckets. This command is available on Leonardo and Galileo100 datamovers and you can use only the s3 service, no other service are allowed at the moment. We discourage you to use ~/.aws/credentials and ~/.aws/config for two reasons: - for security reason, it is not a good idea writing secrets on a shared filesystem - there is a fake home on the datamover /home and the users cannot write inside any configuration file. We strongly suggest to define the environment variables AWS\_ACCESS\_KEY\_ID and AWS\_SECRET\_ACCESS\_KEY on your local computer and use the ssh option "SendEnv" to export them on the Leonardo datamovers. .. code-block:: bash AWS\_ACCESS\_KEY\_ID="" AWS\_SECRET\_ACCESS\_KEY="" ssh -xt -o "sendEnv=AWS\_\*" @data.leonardo.cineca.it aws s3 ls s3:// AWS\_ACCESS\_KEY\_ID="" AWS\_SECRET\_ACCESS\_KEY="" ssh -xt -o "sendEnv=AWS\_\*" @data.leonardo.cineca.it aws s3 sync /absolute/path/to/ s3:// AWS\_ACCESS\_KEY\_ID="" AWS\_SECRET\_ACCESS\_KEY="" ssh -xt -o "sendEnv=AWS\_\*" @data.leonardo.cineca.it aws s3 cp s3:// /absolute/path/to/ - \*\*s3\*\* On Leonardo and Galileo100 datamovers it is available also the s3 command from the libs3 system package. Here you can find the git repo, https://github.com/bji/libs3. Since it is not possible to define environment variable on the datamover, it is mandatory to set the environment variable S3\_ACCESS\_KEY\_ID and S3\_SECRET\_ACCESS\_KEY and send these environment to the datamovers, using the ssh option "SendEnv=S3\_\*". Our suggestion is to define this option in the local ssh\_config file. We strongly discourage to define these variable inside a file on the Leonardo filesystem, for security reason. Usage examples: .. code-block:: bash ssh -xt @data.leonardo.cineca.it s3 help S3\_ACCESS\_KEY\_ID="" S3\_SECRET\_ACCESS\_KEY="" ssh -xt -o "SendEnv=S3\_\*" @data.leonardo.cineca.it s3 test s3:// S3\_ACCESS\_KEY\_ID="" S3\_SECRET\_ACCESS\_KEY="" ssh -xt -o "SendEnv=S3\_\*" @data.leonardo.cineca.it s3 put s3:// filename=/absolute/path/to/{file|directory} S3\_ACCESS\_KEY\_ID="" S3\_SECRET\_ACCESS\_KEY="" ssh -xt -o "SendEnv=S3\_\*" @data.leonardo.cineca.it s3 get s3:// filename=/absolute/path/to/{file|directory} .. tab-item:: GridFTP \*\*Introduction\*\* In this section, we shall provide an easy way to transfer data \*\*to\*\* and \*\*from\*\* any CINECA clusters using GridFTP protocol via \`globus-url-copy \`\_ client. GridFTP is a highly efficient and robust protocol designed for transferring large volumes of data, significantly enhancing the standard FTP service by providing faster and more reliable transfers. It is widely used in large-scale scientific projects and supercomputing centers due to its ability to handle very large files securely and efficiently. Key features of GridFTP include: - \*\*Multiple simultaneous TCP streams\*\*: Maximizes bandwidth utilization by allowing parallel downloads from multiple sources or striped/interleaved transfers. - \*\*Partial file transfers\*\*: Enables downloading specific portions of large files, useful for scientific data processing. - \*\*Fault tolerance and automatic restart\*\*: Resumes interrupted transfers from the last successful byte to improve reliability over unstable networks. - \*\*Security integration\*\*: Supports Grid Security Infrastructure (GSI), Kerberos, and SSH-based authentication, encryption, and data integrity. - \*\*TCP buffer/window size negotiation\*\*: Optimizes transfer speed and reliability based on file size and network conditions. - \*\*Cluster-to-cluster transfers\*\*: Uses multiple nodes at source and destination to increase transfer performance. - \*\*Data channel reuse\*\*: Avoids repeated connection setups when transferring multiple files between the same endpoints. - \*\*Third-party control\*\*: Allows secure initiation of transfers between remote sites without the client being directly involved in the data path. The command-line utility used to perform GridFTP transfers is called \`\`globus-url-copy\`\`. Since 2018, the client software \`\`globus-url-copy\`\` can be installed via packages from the Grid Community Forum (GridCF), a global community supporting core grid software. GridCF maintains the Grid Community Toolkit (GCT), an open-source fork of the original Globus Toolkit developed by the Globus Alliance. Although GCT is derived from the Globus Toolkit, it is a distinct project, and GridCF operates independently from the Globus Alliance. Example usage: - User Local PC <==> CINECA HPC Cluster - CINECA HPC Cluster A <==> CINECA HPC Cluster B - CINECA HPC Cluster <==> Other site HPC Cluster \*\*How to install standard client on your local workstation\*\* The following instructions applies to both Debian/Ubuntu users and Windows users running WSL1 or WSL2. .. code-block:: bash sudo apt install globus-gass-copy-progs Otherwise, if you are a RedHat/Fedora user, execute the following command to install the client: .. code-block:: bash sudo dnf install globus-gass-copy-progs For detailed installation guidance, users are directed to the official Grid Community Toolkit documentation at https://gridcf.org/gct-docs/. \*\*Authentication to the service\*\* The authentication process is delegated to SSH, which manages secure user authentication through mechanisms such as public key cryptography and, in CINECA’s case, enhanced with two-factor authentication. For this reason, you have to generate the ssh certificate on your workstation using the step client, via: .. code-block:: bash step ssh login '' --provisioner cineca-hpc For more info please refer to the page :ref:\`general/access:How to configure \*smallstep\* client\` and :ref:\`general/access:How to activate the \*\*2FA\*\* and the \*\*OTP\*\* generator\`. .. note:: For data transfers between a CINECA HPC cluster and an external HPC site, please ensure that the appropriate external access method is verified and properly configured. \*\*Use the standard client\*\* From the workstation with the ssh certificate, you may transfer data from CINECA HPC Cluster A to CINECA HPC Cluster B by using the standard client \`\`globus-url-copy\`\`. .. code-block:: bash globus-url-copy -vb -cd sshftp://@gftp..cineca.it:22/absolute/path/from/directory/ \\ sshftp://@gftp..cineca.it:22/absolute/path/to/ In addition, you may list files in a specific cluster by the command: .. code-block:: bash globus-url-copy -list sshftp://@gftp..cineca.it:22/absolute/path/from/directory/ .. warning:: \*\*Do not switch off the workstation during data transfer!\*\* Client process is hosted on your workstation: switching it off, will kill data transfer process. You may also transfer data FROM/TO local machine TO/FROM a CINECA HPC cluster, via: .. code-block:: bash globus-url-copy -vb -cd /absolute/path/from/directory/ sshftp://@gftp..cineca.it:22/absolute/path/to/ globus-url-copy -vb -cd sshftp://@gftp..cineca.it:22/absolute/path/from/directory/ /absolute/path/to/ where \`\` can be: \*\*g100\*\* or \*\*leonardo\*\*. For more info about \`globus-url-copy\` command please refer to the official guide, or simply use the command line help: .. code-block:: bash globus-url-copy -help man globus-url-copy \*\*GridFTP TCP Port Range configuration\*\* Please note that GridFTP servers on our clusters are configured to use the port range 20000 - 25000 for the incoming and outgoing connections. Endianness ---------- Endianness is the attribute of a system that indicates whether integers are represented from left to right or right to left. At present, all clusters in Cineca are \*"little-endian"\*. --- # Unknown Software ======== | On CINECA clusters, several softwares are already available through the :ref:\`hpc/hpc\_enviroment:The module command\`. | Please check our \`Software Catalog \`\_ page that indicates which softwares and version are available on each specific cluster. | It is also possible to install software by yourself using the available compilers or using the Spack package manager. Many softwares are free of use, but some are covered by a license. CINECA already provides licenses for several softwares available via module, but in some cases the access to these softwares is not automatic and additional actions are requested to the user. For other softwares for which CINECA does not provide licenses, it is possible to configure a connection with your own license server following the procedure described below. Software available with CINECA license -------------------------------------- | Here we provide the list of softwares directly available using CINECA license. | No additional actions are requested. User just needs to load the corresponding module. \* Amber24 \* AMS \* IDL \* Molcas \* Molpro \* Q-Chem \* Totalview Software available using your own license ----------------------------------------- Here we describe the additional actions requested to the user in order to make use of the following licensed softwares: Gaussian ^^^^^^^^ | CINECA provides its own license. | If you would like to use Gaussian, you need to write to superc@cineca.it asking to be enabled to load the corresponding module. VASP ^^^^ | CINECA does not provide license for VASP. User needs to make use of his/her own license. | Please write to superc@cineca.it stating that you possess a VASP license indicating also for which software version. | If you are a collaborator of a research group with a license, please provide the license responsible name and the email you are registered on the VASP portal. | After a check with VASP developer we will enable you to load the corresponding module. MATLAB ^^^^^^ | Thanks to an agreement with MathWorks, \*\*CINECA provides several MATLAB licenses\*\* through its internal license server that can be used on CINECA clusters. | Usage of the CINECA MATLAB licenses is allowed \*\*exclusively for Open Science\*\* (non-commercial) activities. | In case you are interested in using those licenses and you declare us that your activity is devoted to Open Science, please write to superc@cineca.it to be enabled to use CINECA licenses. Crystal ^^^^^^^ | CINECA does not provide a license for Crystal. User needs to make use of his/her own license. | Please write to superc@cineca.it declaring the you or your responsible have a crystal license specifying the type (Basic or Basic+MPP). In the email you have to add in CC the responsible and info@crystalsolutions.eu | After a check with Crystal developers we will enable you to load the corresponding module. How to connect your license server ---------------------------------- In case you are entitled of a software FlexLM license and you would like to use it on CINECA clusters, we need to connect our compute nodes with your license server. Please write to CINECA's staff at superc@cineca.it and provide us the following information: \* the port and host (IP or alias) of the license server where the license is installed. \* the license holder needs to sign a document (:download:\`template <../files/License\_request.odt>\`) in which the holder declares to have a valid license and relieves CINECA of future responsibilities for the usage of that license on CINECA's cluster. \* We will provide the IPs of CINECA'S cluster so that the license server administrators can open their firewall to them. When the aforementioned steps have been completed, your usernames and account(s) will be authorized to use your license running your jobs on CINECA infrastructure. In case you would like to use an academic license, you will also have to indicate us a representative (not necessarily the license holder but with his/her approval) to be contacted by CINECA HPC User Support to allow future requests to use the same license. Advanced Software Specific details ---------------------------------- .. |ico1| image:: img/matlab.png :height: 45px :class: no-scaled-link .. |ico2| image:: img/qe\_logo.png :height: 45px :class: no-scaled-link .. grid:: 2 .. grid-item-card:: |ico1| \*\*Matlab\*\* :link: matlab\_card :link-type: ref .. grid-item-card:: |ico2| \*\*QuantumESPRESSO\*\* :link: quantum\_espresso\_card :link-type: ref .. toctree:: :maxdepth: 1 :hidden: software/matlab software/qe --- # Unknown .. \_known\_issues\_card: Known Issues ------------ This section collects currently known issues affecting CINECA HPC Cloud systems. The list below is intended as a quick reference for users who may experience problems on the system. We strongly encourage all users to report any issues they encounter - whether listed here or not - to the user support team. .. card:: PCI interfaces restrictions on a single VM +++++ \*\*Status:\*\* :bdg-danger:\`Open\` | :octicon:\`calendar\` \*\*Last Update:\*\* 2025-11-05 | :octicon:\`cache\` \*\*Systems:\*\* All .. dropdown:: :octicon:\`info\` \*\*Description\*\* There is a restriction imposed by libvirt which allows a maximum of 28 virtual PCI interfaces used for attaching block devices: 2 of these virtual PCIs are used for server needs (mainly boot device) which leaves 26 virtual PCI interfaces available for block device attaching. For this reason, it is possible to attach maximum 26 volumes to an instance created with the default Ubuntu images provided by CINECA. \*\*SOLUTION:\*\* To avoid this issue the solution is to upload a custom image, editing its metadata from Horizon Dashboard as follows: \`\`hw\_scsi\_model = virtio-scsi\`\` and \`\`hw\_disk\_bus = scsi\`\`. .. card:: Docker MTU issues on virtual machines +++++ \*\*Status:\*\* :bdg-danger:\`Open\` | :octicon:\`calendar\` \*\*Last Update:\*\* 2025-11-05 | :octicon:\`cache\` \*\*Systems:\*\* All .. dropdown:: :octicon:\`info\` \*\*Description\*\* Docker containers built on virtual machines are unable to communicate outside of the vm. \*\*SOLUTION:\*\* To use Docker in your virtual machine please set the MTU value at 1400 in the file /etc/docker/daemon.json. .. code-block:: json { "mtu" : 1400 } .. card:: Creating shares from snapshots +++++ \*\*Status:\*\* :bdg-danger:\`Open\` | :octicon:\`calendar\` \*\*Last Update:\*\* 2025-11-04 | :octicon:\`cache\` \*\*Systems:\*\* ADA .. dropdown:: :octicon:\`info\` \*\*Description\*\* In order to enable the creation of shares from snapshots, the corresponding openstack share-type needs to include the attribute: \`\`create\_share\_from\_snapshot\_support=True\`\`. On ADA, the value of the attribute was set to \`\`False\`\` for both share types (generic, and cephfs) until 03/11/2025. This implies that shares created on ADA \*\*before 4th of November 2025\*\* can generate snapshots, but those are unusable since it is not possible to create new shares from them. .. card:: Share networks creation: differences between dashboard and CLI +++++ \*\*Status:\*\* :bdg-danger:\`Open\` | :octicon:\`calendar\` \*\*Last Update:\*\* 2025-12-15 | :octicon:\`cache\` \*\*Systems:\*\* ADA .. dropdown:: :octicon:\`info\` \*\*Description\*\* When creating a share network, the following two parameters are set with different values, depending whether the network has been created via CLI or Horizon dashboard. .. list-table:: :widths: 1 1 1 :header-rows: 1 :class: tight-table \* - \*\*Parameter\*\* - \*\*Horizon Dashboard\*\* - \*\*CLI\*\* \* - security\_service\_update\_support - false - true \* - network\_allocation\_update\_support - false - true This is significant when trying to deploy manila-csi-plugin in the openstack cloud controller manager for kubernetes. In particular to make the manila CSI provider work, the network shall be created via Horizon Dashboard. References: - https://specs.openstack.org/openstack/manila-specs/specs/wallaby/security-service-updates-in-use-share-network.html - https://docs.openstack.org/manila/latest/user/share-network-operations.html --- # Unknown Environment and Customization ============================= The Software Catalog -------------------- CINECA offers a variety of third-party applications and community codes that are installed on its HPC systems. Most of the third-party software is installed using software modules mechanism (see The module command section). Information on the available packages and their detailed descriptions are organized in a catalog, divided by discipline (\`link \`\_). The catalog is also accessible directly on HPC clusters by using the commands \`\`module\`\` and \`\`modmap\`\` descrived in next sections. The module command ------------------ All softwares installed on the CINECA clusters are available as modules. As default, a set of basic modules are preloaded in the enviroment at login. To manage modules in the production enviroment, the user can execute the command module  with a variety of options. A short description of the most useful module command usage is reported in the following table. .. list-table:: :widths: 35 65 :header-rows: 1 \* - \*\*Command\*\* - \*\*Action\*\* \* - module avail - show the available modules on the machine \* - module load - load the module in the current shell session, preparing the enviroment for the application. \* - module load autoload - load the module and all dependencies in the current session \* - module help - show specific information and basic help on the application \* - module list - show the module currently loaded in the shell session \* - module purge - unload all the loaded modules \* - module unload - unload a specific module \* - module av -a - show also the hidden modules available on the machine. These are modules usable but not guaranteed \* - module load / - to load an hidden module you must specify its version The modmap command ------------------ For an easy reading, the modules are collected in different profiles. Only the \*\*base\*\* profile is automatically loaded at login. \`\`modmap\`\` is a very useful command to look for a specific module in all the profiles at once. It shows at standard output all the modules with the searched name showing in wgicg profile they can be found. For example, suppose you are looking for the lammps software: .. code-block:: bash $ modmap -m lammps Profile: archive applications lammps 20220623--openmpi--4.1.4--gcc--11.3.0-cuda-11.8 Profile: astro Profile: base Profile: bioinf Profile: chem-phys applications lammps 29aug2024 2aug2023 2aug2023--intel-oneapi-compilers--2023.2.1 Profile: deeplrn Profile: eng Profile: geo-inquire Profile: lifesc Profile: meteo Profile: quantum Profile: spoke7 Profile: statistics The output of modmap is showing that several lammps versions are present in the \*\*chem-phys\*\* profile and an old one in the \*\*archive\*\* profile. To load the module is now easy: .. code-block:: bash $ module load profile/chem-phys $ module load lammps/29aug2024 Compilers --------- You can check the complete list of available compilers on a specific cluster with the command: .. code-block:: bash $ modmap -c compilers For \*\*GPU compilation\*\* the available compilers are: \* For \*\*NVIDIA GPUs\*\* cuda-aware \* GNU Compilers Collection (GCC) \* NVIDIA nvhpc (ex PGI) \* NVIDIA cuda For \*\*CPU compilation\*\* the available compilers are: \* For \*\*INTEL CPUs\*\* \* Intel oneAPI compilers (x and classic compilers) \* GNU Compilers Collection (GCC) \* For \*\*AMD CPUs\*\* \* AOCC compilers \* GNU Compilers Collection (GCC) GCC ^^^ .. tab-set:: .. tab-item:: \*\*Serial\*\* The GNU compilers are always available. A GCC version is available on the system (gcc --version ) without the need to load any module. In the module environment you can find more recent version though: .. code-block:: bash $ modmap -m gcc To use a specific version: .. code-block:: bash $ module load gcc/ The name of the GNU compilers are: \* \*\*gfortran\*\*: fully compliant with the Fortran 95 Standard and includes legacy F77 support \* \*\*gcc\*\*: C compiler \* \*\*g++\*\*: C++ compiler The gcc module loading set a specific environment variable for each compiler: \* \*\*CC\*\*: gcc \* \*\*CXX\*\*: g++ \* \*\*FC\*\*: gfortran \* \*\*F90\*\*: gfortran \* \*\*F77\*\*: gfortran The documentation can be obtained with the "man" command after loading the gcc module: .. code-block:: bash $ module load gcc/ On the \*\*accelerated clusters\*\* the available gcc modules support the offloading to the device. For NVIDIA GPUs the target is nvptx. On the \*\*cluster provided with accelerated and non-accelerated partitions\*\* that share the same modules environment the available offloading gcc modules can be used on both. As a result there is one only installation of a specific gcc version that supports the offload-device and you can use also on CPUs partition. .. tab-item:: \*\*MPI wrappers\*\* The \*\*GCC OpenMPI\*\* implementation is always available on accelerated and non accelerated clusters. The version installed for NVIDIA GPUs is configured to support CUDA, but you can use it also for partitions non accelerated of a cluster. In this case, however, it is \*\*highly recommended\*\* to compile with the MPI implementation specific for their architecture (e.g intel-oneapi-mpi module for INTEL CPUs). You can check the list of available OpenMPI modules on a specific cluster with the command: .. code-block:: bash $ modmap -m openmpi To use a specific one: .. code-block:: bash $ module load openmpi/ After loading a specific GCC openmpi module select the MPI compiler wrapper for Fortran, C or C++ codes. \* \*\*mpicc\*\*: gcc compiler mpi wrappers \* \*\*mpic++\*\* \*\*mpiCC\*\* \*\*mpicxx\*\*: g++ compiler mpi wrappers \* \*\*mpif77\*\* \*\*mpif90\*\* \*\*mpifort\*\*: gfortran compiler mpi wrappers e.g. Compiling C code: .. code-block:: bash $ module load openmpi/ $ mpicc -o myexec myprog.c NVIDIA nvhpc ^^^^^^^^^^^^ (ex PORTLAND PGI + NVIDIA CUDA) .. tab-set:: .. tab-item:: \*\*Serial\*\* The NVHPC compilers are always available on the NVIDIA GPUs clusters. In the module environment you can find more recent version though: .. code-block:: bash $ modmap -m nvhpc To use a specific version: .. code-block:: bash $ module load nvhpc/ The name of the NVHPC compilers are: \* \*\*nvc\*\*: Compile C source files (C11 compiler. It supports GPU programming with OpenACC, and supports multicore CPU programming with OpenACC and OpenMP) \* \*\*nvc++\*\*: Compile C++ source files (C++17 compiler. It supports GPU programming with C++17 parallel algorithms (pSTL) and OpenACC, and supports multicore CPU programming with OpenACC and OpenMP) \* \*\*nvfortran\*\*: Compile FORTRAN source files (supports ISO Fortran 2003 and many features of ISO Fortran 2008. It supports GPU programming with CUDA Fortran and OpenACC, and supports multicore CPU programming with OpenACC and OpenMP) \* \*\*nvcc\*\*: CUDA C and CUDA C++ compiler driver for NVIDIA GPUs As of August 5, 2020, the "PGI Compilers and Tools" technology is a part of the NVIDIA HPC SDK product, available as a free download from NVIDIA. For legacy reasons, the NVIDIA nvhpc suite also offers the PGI C, C++, and Fortran compilers with their original names, as follows. \* \*\*pgcc\*\*: Compile C source files. \* \*\*pgc++\*\*: Compile C++ source files. \* \*\*pgf77\*\*: Compile FORTRAN77 source files. \* \*\*pgf90\*\*: Compile FORTRAN90 source files. \* \*\*pgf95\*\*: Compile FORTRAN95 source files. \* \*\*pgfortran\*\*: Compile PGI Fortran The documentation can be obtained with the "man" command after loading the gcc module: .. code-block:: bash $ module load nvhpc/ $ man nvc To enable CUDA C++ or CUDA Fortran, and link with the CUDA runtime libraries, use the -cuda option (-Mcuda is deprecated). Use the -gpu option to tailor the compilation of target accelerator regions. The OpenACC parallelization is enabled by the -acc flag. GPU targeting and code generation can be controlled by adding the -⁠gpu flag to the compiler command line. The OpenMP parallelization is enabled by the -mp compiler option. The GPU offload via OpenMP is enabled by the -mp=gpu option. .. tab-item:: \*\*MPI wrappers\*\* The \*\*NVHPC MPI\*\* implementation is always available on the clusters provided with NVIDIA gpus. The OpenMPI nvhpc version, if installed, is available as \*\*openmpi/\*\* module. The version built-in from NVIDIA is available within nvhpc installation as \*\*hpcx-mpi/\*\* module. You can check the list of available NVHPC OpenMPI/hpcx-mpi modules on a specific cluster with the command: .. code-block:: bash $ modmap -m openmpi OR hpcx-mpi To use a specific one: .. code-block:: bash $ module load openmpi/ OR hpcx-mpi/ After loading a specific nvhpc openmpi module select the MPI compiler wrapper for Fortran, C or C++ codes. \* \*\*mpicc\*\*: nvc compiler mpi wrappers \* \*\*mpic++\*\* \*\*mpiCC\*\* \*\*mpicxx\*\*: nvc++ compiler mpi wrappers \* \*\*mpif77\*\* \*\*mpif90\*\* \*\*mpifort\*\*: nvfortran compiler mpi wrappers e.g. Compiling C code: .. code-block:: bash $ module load openmpi/ OR hpcx-mpi/ $ mpicc -o myexec myprog.c (uses the nvc compiler) Intel oneAPI ^^^^^^^^^^^^ .. tab-set:: .. tab-item:: \*\*Serial\*\* The Intel compilers are the best choice on the Intel CPUs clusters. In the module environment you can find more recent version though: .. code-block:: bash $ modmap -m intel-oneapi-compilers To use a specific version: .. code-block:: bash $ module load intel-oneapi-compilers/ Starting from 2021 version up to 2023 intel-oneapi-compilers module makes available two types of compilers, classic and oneAPI. Intel \*\*classic\*\* compilers: \* \*\*icc\*\*: Compile C source files \* \*\*icpc\*\*: Compile C++ source files \* \*\*ifort\*\*: Compile FORTRAN source files LLVM-based Intel \*\*oneAPI\*\* compilers: \* \*\*icx\*\*: Compile C source files \* \*\*icpx\*\*: Compile C++ source files \* \*\*ifx\*\*: Compile FORTRAN source files \* \*\*dpcpp\*\*: Compile C++ source files with SYCL extensions Starting from 2024 version intel-oneapi-compilers module makes available only oneAPI compilers set and ifort classic compiler which is no longer available from 2025 version. In order to use the Intel classic compilers load: .. code-block:: bash $ module load intel-oneapi-compilers-classic e.g. Compiling Fortran code with oneAPI: .. code-block:: bash $ module load intel-oneapi-compilers/ $ ifx -o myexec myprog.f90 .. tab-item:: \*\*MPI wrappers\*\* The Intel MPI implementation is the best choice on the Intel CPUs clusters. In the module environment you can find more recent version though: .. code-block:: bash $ modmap -m intel-oneapi-mpi To use a specific module: .. code-block:: bash $ module load intel-oneapi-mpi/ This module makes available classic and oneAPI compilers wrappers. After loading a specific intel-oneapi-mpi module select the MPI compiler wrapper, classic or oneaAPI, for Fortran, C or C++ code. Intel \*\*OneAPI\*\* compilers wrappers: \* \*\*mpiicx\*\* (C code) \* \*\*mpiicpx\*\* (C++ code) \* \*\*mpiifx\*\* (Fortran code) Intel \*\*classic\*\* compilers wrappers: \* \*\*mpiifort\*\* (Fortran code) \* \*\*mpiicc\*\* (C code) \* \*\*mpiicpc\*\* (C++ code) Intel \*\*GNU\*\* compilers wrappers: \* \*\*mpifc\*\*, \*\*mpif77\*\*, \*\*mpif90\*\* (Fortran MPI wrapper) \* \*\*mpicc\*\* (C MPI wrapper) \* \*\*mpicxx\*\*: (C++ MPI wrapper) e.g. Compiling C code: .. code-block:: bash $ module load intel-oneapi-mpi/ $ mpiicx -o myexec myprog.c AMD AOCC ^^^^^^^^^^^^ .. tab-set:: .. tab-item:: \*\*Serial\*\* The AOCC compilers are available on the AMD CPUs clusters. In the module environment you can find more recent version though: .. code-block:: bash $ modmap -m aocc To use a specific version: .. code-block:: bash $ module load aocc/ The AOCC compilers allow the development for x86 applications written in C, C++, and Fortran. AMD \*\*AOCC\*\* compilers: \* \*\*clang\*\*: Compile C source files \* \*\*clang++\*\*: Compile C++ source files \* \*\*flang\*\*: Compile FORTRAN source files e.g. Compiling Fortran code with AOCC: .. code-block:: bash $ module load aocc/ $ flang \[command line flags\] -o myexec myprog.f90 AOCC compiler offers target-dependent and target-independent optimizations, with a particular focus on AMD "Zen" processors. You can read more about these in the command line option AMD section https://docs.amd.com/r/en-US/57222-AOCC-user-guide/Command-line-Options .. tab-item:: \*\*MPI wrappers\*\* The \*\*AOCC OpenMPI\*\* implementation is available on AMD clusters. You can check the list of available OpenMPI modules on a specific cluster with the command: .. code-block:: bash $ modmap -m openmpi To use a specific one: .. code-block:: bash $ module load openmpi/ After loading a specific AOCC openmpi module select the MPI compiler wrapper for Fortran, C or C++ codes. \* \*\*mpicc\*\*: gcc compiler mpi wrappers \* \*\*mpic++\*\* \*\*mpiCC\*\* \*\*mpicxx\*\*: g++ compiler mpi wrappers \* \*\*mpif77\*\* \*\*mpif90\*\* \*\*mpifort\*\*: gfortran compiler mpi wrappers e.g. Compiling C code: .. code-block:: bash $ module load openmpi/ $ mpicc -o myexec myprog.c Basic MPI execution ^^^^^^^^^^^^^^^^^^^ To test if your parallel executable works, you can execute it with mpirun on the login node and with a single process: .. code-block:: bash module load mpirun ./myexec To run it in the parallel way you have to allocate the compute nodes via interactive job or sbatch job and execute it with mpirun or srun launcher . \*\*Example:\*\* 2 GPU compute nodes allocation and 2 tasks execution .. tab-set:: .. tab-item:: \*\*via interactive job (salloc):\*\* .. code-block:: bash module load salloc -N 2 --ntasks-per-node=1 --cpus-per-task=1 --gres=gpu:1 -A --time= --partition= --qos= srun -n 2 ./myexec .. tab-item:: \*\*via interactive job (srun):\*\* .. code-block:: bash module load srun -N 2 --ntasks-per-node=1 --cpus-per-task=1 --gres=gpu:1 -A --time= --partition= --qos= --pty /bin/bash mpirun -n 2 ./myexec .. tab-item:: \*\*via sbatch job:\*\* .. code-block:: bash sbatch my\_batch\_script.sh cat my\_batch\_script.sh #!/bin/sh #SBATCH --job-name osu #SBATCH -N2 --ntasks-per-node=1 #SBATCH --cpus-per-task=1 #SBATCH --gres=gpu:1 #SBATCH --time= #SBATCH --account= #SBATCH --partition= #SBATCH --qos= module load mpirun ./myexec or srun ./myexec Totalview ^^^^^^^^^ This document introduces the user how to launch totalview through an :ref:\`general/access:Access via Remote Visualization (\*\*RCM\*\*)\` session. .. Cineca provides the user with an easy tool to establish a graphic session with our systems: RCM. All the software that comes with a graphic user interface (GUI) can be used within an RCM session. In this regard, Totalview makes no exception and can be easily used in conjunction with RCM to establish a debugging session of a parallel code. With respect to other GUIs that can be run on RCM, Totalview is a little peculiar and must be run directly on the nodes that execute the parallel code. In the following, we will detail how to establish a Totalview debugging session through RCM with a SLURM job. .. Please refer to this page for the instructions on how to use RCM; for most of the cases is as simple as: 1) download the tool, 2) launch the executable. Once you have established a connection through RCM with one of our systems, GALILEO100 or Leonardo, please follow the instructions below. .. commented since marconi outdated MARCONI (ahem) ++++++++++++++ Once connected you should have a desktop session open. Now open a terminal following "Applications -> System Tools -> Terminal". When done, a terminal pops-up and you can use it as you do normally with a ssh connection. Now let's go through the operations required to launch a Totalview job. 1. \*\*Get the DISPLAY number\*\* On a terminal session within RCM type the command: .. code-block:: bash $ echo $DISPLAY :8 This will return a display number to use for connecting your totalview job with the RCM session. 2. \*\*Prepare a batch script (job.sh)\*\* .. code-block:: bash :emphasize-lines: 15 #!/bin/bash #SBATCH -e totaljob.err #SBATCH -o totaljob.out #SBATCH -A #SBATCH -N 1 #SBATCH -t 00:10:00 #SBATCH -p skl\_usr\_dbg module load autoload intelmpi module load totalview #set the DISPLAY so as to use the same opened in the RCM session. This is just an example, use your own hostname and display setting. export DISPLAY="r161c001s02:8" totalview srun ./my\_executable Highlighted in the above example, we have told the Totalview user interface to open on the current VNC session (opened automatically by RCM). Please refer to the above section on how to get the correct DISPLAY number. 3. \*\*Submit the job\*\* Now you can submit the above script to the SLURM scheduler. Once it becomes running, the Totalview user interface will pop-up and you will be able to debug your code: .. code-block:: bash $ sbatch job.sh GALILEO100 ++++++++++ .. questa guida era già perfetta, sento il tocco di isa qui .. Once connected on one of our machine via RCM, you should have a desktop session open. Now open a terminal following "Applications -> System Tools -> Terminal". When done, a terminal pops-up and you can use it as you do normally with a ssh connection. Now let's go through the operations required to launch a Totalview job. As in the example above, once connected to GALILEO100 with RCM, open a terminal (start -> terminal). Then follow this set of instructions described below. .. dropdown:: 1) Setup the .tvdrc file - only the first time :name: setup\_tvdrc :animate: fade-in-slide-down :color: light The first time you estabilish a Totalview session, a folder named .totalview will be created in your $HOME (it is not visible with the standard "ls" command, you have to add the flag -a for the hidden directories and files). Inside it, create a text file named .tvdrc, that should contain the following lines documented also in the \`official Slurm manual \`\_: .. code-block:: bash dset -set\_as\_default TV::bulk\_launch\_enabled true dset -set\_as\_default TV::bulk\_launch\_string {srun --mem-per-cpu=0 -N%N -n%N -w\`awk -F. 'BEGIN {ORS=","} {if (NR==%N) ORS=""; print $1}' %t1\` -l --input=none %B/tvdsvr%K -callback\_host %H -callback\_ports %L -set\_pws %P -verbosity %V -working\_directory %D %F} dset -set\_as\_default TV::bulk\_launch\_tmpfile1\_host\_lines {%R} .. dropdown:: 2) Prepare the job (job.sh script) and submit it :name: g100\_job\_script :animate: fade-in-slide-down :color: light Example \`\`job.sh\`\` for GALILEO100: .. code-block:: bash #!/bin/bash #SBATCH -t 30:00 #SBATCH -N 1 #SBATCH -o totaljob.out #SBATCH -e totaljob.err #SBATCH -A #SBATCH -p g100\_usr\_prod module load totalview module load tvconnect srun ./your\_executable Submit the job via: .. code-block:: bash $ sbatch job.sh .. dropdown:: 3) Open a Totalview terminal :name: g100\_open\_totalview :animate: fade-in-slide-down :color: light In the RCM shell, load the module of Totalview and launch "totalview" to open the GUI. When the job starts, you will be asked by a prompt to connect to it and you will see that the tool is trying to debug the "srun" command. .. dropdown:: 4) Launch the simulation :name: g100\_launch :animate: fade-in-slide-down :color: light Press the green "Go" button to launch the simulation. Eventually, a prompt will ask you if you want to stop the parallel job: if you choose "Yes", you will finally see the main code of the executable you want to debug and you can start working on it. Installing packages with python environment ------------------------------------------- In Cineca clusters you can find the available versions for python and py-mpi4py with the command \`\`modmap -m python\`\` and \`\`modmap -m py-mpi4py\`\`, respectively. In case you need to install packages through a python virtual environment you can do: .. code-block:: bash $ module load python/ # In case you need py-mpi4py $ module load py-mpi4py/ $ python -m venv my\_env\_test $ source my\_env\_test/bin/activate $ pip install .. Note:: \* my\_env\_test: choose an arbitrary name for your personal virtual env. \* It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. \* Once you source your virtual environment you will see on your shell (before the login node name), something like this: \`\`(my\_env\_test) \[otrocon1@login02 UserGuideTests\]$\`\` . \* Once you finish to work on your env, you can deactivate it with the command \`\`deactivate\`\`. \* In case you need specific python or artificial intelligence packages optimized for Cineca's clusters you can refer to the section: \*\*Cineca-ai\*\* and \*\*Cineca-hpyc modules\*\*. .. grid:: 2 .. grid-item-card:: \*\*Cineca-ai\*\* :link: cineca-ai\_card :link-type: ref .. grid-item-card:: \*\*Cineca-hpyc\*\* :link: cineca-hpyc\_card :link-type: ref .. toctree:: :maxdepth: 2 :hidden: hpc\_cineca-ai-hpyc SPACK ----- To assist users in customizing their production environment by installing fresh software, we offer a powerful tool named Spack. Spack is a multi-platform package manager that facilitates the easy installation of multiple versions and configurations of software. Below, you will find a step-by-step guide to install software using Spack. For a comprehensive and detailed guide, please refer to the \`official Spack documentation \`\_. Quick usage ^^^^^^^^^^^ .. code-block:: bash $ ml spack $ spack spec -Il # to check current specs $ spack install # to actually install $ ml # load the created module For a fine-grained control, you can select the Spack version (see :ref:\`Loading\_the\_spack\_module\_available\_on\_the\_cluster\`), and you can add specs (see :ref:\`Variants\_and\_dependencies\`) to the \`\`spec\`\` and the \`\`install\`\` commands (see :ref:\`Spec\_command\`). It may happen that the module created by Spack will miss some dependencies, you can create the missing modulefiles via \`\`spack module tcl refresh\`\` (see :ref:\`Module\_command\_and\_Spack\_managing\`). Additional useful steps are: - check beforehand if the package exists in Spack and what is its \*Spack name\* (see :ref:\`Listing\_recipe\`) - check if the package or its dependencies are already installed (:ref:\`Listing\_installed\`) Installing a new package ^^^^^^^^^^^^^^^^^^^^^^^^ .. dropdown:: Loading the preconfigured Spack available on the cluster :name: Loading\_the\_spack\_module\_available\_on\_the\_cluster :animate: fade-in-slide-down :color: light We provide a module to load a pre-configured Spack instance: .. code-block:: bash $ modmap -m spack $ module load spack/ The directory \`\`/spack-\`\` is automatically created into a default space, containing some sub-directories created and used by Spack during the package installation. On GALILEO100, the default area is \`\`$WORK/$USER\`\`, while on LEONARDO is \`\`$PUBLIC\`\`. You will find, for example on LEONARDO: - software installation root: \`\`$PUBLIC/spack-/install\`\` - modulefiles location: \`\`$PUBLIC/spack-/modules\`\` - user scope: \`\`$PUBLIC/spack-/user\_cache\`\` - sources cache: \`\`$PUBLIC/spack-/cache\`\` For GALILEO100 users, please be aware that \`\`$WORK\`\` space will be removed after six months since project expiration. If you want to define different paths for installations, modules, user scope directories, and cache, please refer to Spack manual (a simple workaround is to redefine \`\`WORK\`\` to a different path, e.g. \`\`export WORK=/your/different/path\`\`, before loading Spack module). .. dropdown:: Listing the software that can be installed via Spack :name: Listing\_recipe :animate: fade-in-slide-down :color: light You can check if the software package you want to install is known to Spack via the command \`\`spack list\`\`, which will print out the list of all the packages you can install via Spack. You can also specify the name of the package (or only part of its name): .. code-block:: bash $ spack list $ spack list or .. code-block:: bash $ spack list | grep .. dropdown:: Find already installed packages :name: Listing\_installed :animate: fade-in-slide-down :color: light You will find a suite of compilers, libraries, tools and applications already installed by Cineca staff via Spack. It is strongly recommended you use them to install additional software. Find the already installed packages .. code-block:: bash $ spack find Check if a specific package is already installed or what packages have been already installed to provide a specific \`virtual package \`\_ (e.g mpi) .. code-block:: bash $ spack find $ spack find List the packages already installed and see e.g. the used variants (-v), dependencies (-d), the installation path (-p) and the hash (-l). The meaning of the hash is discussed in the next paragraph. .. code-block:: bash $ spack find -ldvp You can also list the packages already installed with a specific variant .. code-block:: bash $ spack find -l + e.g. $ spack find -l +cuda or which depends on a specific package (e.g openmpi) or a generic virtual package (e.g. mpi) .. code-block:: bash $ spack find -l ^ e.g. $ spack find -l ^openmpi e.g. $ spack find -l ^mpi or installed with a specific compiler .. code-block:: bash $ spack find % .. dropdown:: Add a new compiler to Spack compilers :name: add\_compiler :animate: fade-in-slide-down :color: light The list of all the compilers already installed and ready to be used can be seen with .. code-block:: bash $ spack compilers To add a compiler to the ones known to Spack: .. code-block:: bash $ module load $ spack compiler add $ module unload .. dropdown:: Variants and dependencies :name: Variants\_and\_dependencies :animate: fade-in-slide-down :color: light If the package of your interest is listed by \`\`spack list\`\`, you can inspect its build \*variants\* via .. code-block:: bash $ spack info You can activate (\`\`+\`\`) or deactivate (\`\`-\`\`) variants via .. code-block:: bash $ spack spec -Il +variant\_1 -variant\_2 variant\_3=value $ spack install +variant\_1 -variant\_2 variant\_3=value and also for a dependency .. code-block:: bash $ spack spec -Il ^" +variant\_1 -variant\_2 variant\_3=value" $ spack install ^" +variant\_1 -variant\_2 variant\_3=value" .. \_Spec\_command: Spec and install commands +++++++++++++++++++++++++ In order to install a package with the Spack module, you have to select for it a version (\`\`@\`\`), a compiler (\`\`%\`\`), the dependencies (\`\`^\`\`) and the building variants (\`\`+\`\`/\`\`-\`\`). The combination of all these parameters is the \*spec\* with which the package will be installed. If you don’t select any combination during the installation, a default \*spec\* is selected. Before installing a package, it is strongly recommended to check the default \*spec\* with which the package would be installed: .. code-block:: bash $ spack spec -Il The suggested options to the \`\`spec\`\` command used in the example above are: \`\`-I\`\` (install), which shows the installation status of the package and its dependencies with a symbol preceding the hash of the \*spec\* (\`\`-\`\` not installed, \`\`+/^\`\` installed/installed from another user); \`\`-l\`\` (long) which shows the unique identifier ("hash") of the package installation (e.g. aouyzha). .. Important:: On Cineca clusters it’s recommended to execute always \`\`spec\`\` command before installing a package to make sure its dependencies are satisfied with Cineca installations (\`\`^\`\`) where available. The Cineca installations are optimised and tested for the architecture of the specific cluster. This is especially true for e.g. openmpi. .. Note:: Even when a Cineca installation is available to satisfy a dependency, the default \*spec\* for that dependency may differ, thus a symbol \`\`-\`\` may be shown. If possible, force the \*spec\* to match the one corresponding to the Cineca one (so the symbol will become \`\`^\`\`). A simple way to force this is to force the dependency via its hash: .. code-block:: bash $ spack spec -Il ^/hash $ spack install ^/hash e.g. $ spack spec -Il ^/aouyzha e.g. $ spack install ^/aouyzha Once you select the \*spec\*, a \`\`spack install\`\` is all you need: .. code-block:: bash $ # default spec $ spack install $ $ # custom spec $ spack install @ +/~/ = %@ ^ .. \_Module\_command\_and\_Spack\_managing: Module command and Spack managing +++++++++++++++++++++++++++++++++ You can load the installed software by loading the correspondent modulefile Spack automatically created. To force its creation, you can run: .. code-block:: bash $ spack module tcl refresh --upstream-modules Then you can find and load the new modulefile by adding the "modules" folder to the search path via \`\`module use\`\` (this is done implicitly also when loading Spack), e.g. on Leonardo: .. code-block:: bash $ module use $PUBLIC/spack-/modules $ module av $ module load Please refer to section :ref:\`Loading\_the\_spack\_module\_available\_on\_the\_cluster\` to know the correct path to the modulefiles folder. .. silvia voleva gli env, ma era incerta se fornire quelli di propro in macchina (magari li prendono mentre li stiamo editando), o via git (mi piace di più, ma al momento non ne so nulla, silvia proponeva una call di aggiornamento, aspetto prima di fare cose) .. io userei direttamente $SPACK\_ROOT/../../ccs/spack/env, ma diventa troppo confuso per un utente. ma se non posso riferirmi ai nostri env, non vale la pena .. \_Env: Spack environments ------------------ A Spack environment allows users to group software specs for building them in a coherent and reproducible manner. If you have many packages that needs to share the same dependencies (e.g. the same MPI), then Spack environments may be a valuable option. For a comprehensive guide, please refer to the \`official Spack documentation \`\_. Here, it will suffice --- # Unknown Cluster Specifics ================= In this section, we highlight the specific features of the HPC systems at CINECA, focusing on deviations from the general behavior described earlier. .. |ico1| image:: img/hpc\_icon.png :height: 45px :class: no-scaled-link .. grid:: 3 .. grid-item-card:: |ico1| \*\*Leonardo\*\* :link: leonardo\_card :link-type: ref .. grid-item-card:: |ico1| \*\*Galileo100\*\* :link: galileo\_card :link-type: ref .. grid-item-card:: |ico1| \*\*Pitagora\*\* :link: pitagora\_card :link-type: ref .. toctree:: :maxdepth: 2 :hidden: leonardo galileo pitagora --- # Unknown .. \_serv\_tools\_card: Services and Tools ================== High-Performance Computing (HPC) services typically provide advanced computational resources. These tools and services are designed to support research, simulations, data processing, and other high-performance applications. .. grid:: 3 .. grid-item-card:: \*\*Interactive Computing\*\* :link: interactive\_computing\_card :link-type: ref .. grid-item-card:: \*\*Singularity and Apptainer Containers\*\* :link: hpc\_containers\_card :link-type: ref .. grid-item-card:: \*\*Miniconda\*\* :link: miniconda\_card :link-type: ref .. toctree:: :maxdepth: 1 :hidden: interactive\_computing singularity miniconda --- # Unknown .. \_cloud\_card: Introduction to HPC Cloud ========================= Cloud computing is a crucial paradigm in today's digital era. It encompasses the delivery of diverse services and resources, including computing power, storage, databases, networking, software, and more, via the internet. The \*\*CINECA HPC Cloud infrastructure\*\* integrates and completes the HPC ecosystem, providing a tightly-integrated infrastructure that covers both high performance and high flexible computing. The flexibility of the cloud adapts better to the diversity of user workloads, while still providing high-end computing power. .. toctree:: :maxdepth: 2 :hidden: what\_is\_cloud cineca\_cloud\_model budget\_accounting .. grid:: 3 .. grid-item-card:: \*\*What is cloud computing\*\* :link: what\_is\_cloud\_card :link-type: ref .. grid-item-card:: \*\*CINECA HPC Cloud model\*\* :link: cineca\_cloud\_model\_card :link-type: ref .. grid-item-card:: \*\*Budget and accounting\*\* :link: budget\_accounting\_card :link-type: ref \*\*Section Contents\*\* .. grid:: 3 .. grid-item-card:: :img-background: /cloud/\_img/os\_overview\_icon.png :link: os\_overview\_card :link-type: ref .. grid-item-card:: :img-background: /cloud/\_img/operative\_manual\_icon.png :link: operative\_manual\_card :link-type: ref .. grid-item-card:: :img-background: /cloud/\_img/cloud\_specifics\_icon3.png :link: cloud\_specifics\_card :link-type: ref .. grid:: 3 .. grid-item-card:: :img-background: /cloud/\_img/tenants\_admin\_icon.png :link: tenants\_administration\_card :link-type: ref .. grid-item-card:: :img-background: /cloud/\_img/tutorials\_repos\_icon.png :link: tutorials\_and\_repos\_card :link-type: ref .. grid-item-card:: :img-background: /cloud/\_img/faqs.png :link: cloud\_faq\_card :link-type: ref ------- .. card:: :link: known\_issues\_card :link-type: ref .. figure:: /cloud/\_img/known\_issues.png :align: center :class: no-scaled-link :height: 75px --- # Unknown .. \_operative\_manual\_card: Operative Manual ================ This section collects all the operative guidelines on how to perform the most common operations. The operations are grouped whenever possible according to the respective Components. .. toctree:: :maxdepth: 1 :hidden: compute\_ops/index\_compute\_ops storage\_ops/index\_storage\_ops network\_ops/index\_network\_ops shares\_ops/index\_shares\_ops db\_ops/index\_db\_ops lb\_ops/index\_lb\_ops .. grid:: 3 .. grid-item-card:: |osnova| \*\*Compute operations\*\* :link: compute\_ops\_card :link-type: ref .. grid-item-card:: |oscinder| \*\*Storage operations\*\* :link: storage\_ops\_card :link-type: ref .. grid-item-card:: |osneutron| \*\*Network operations\*\* :link: network\_ops\_card :link-type: ref .. grid:: 3 .. grid-item-card:: |osmanila| \*\*Shares operations\*\* :link: shares\_ops\_card :link-type: ref .. grid-item-card:: |ostrove| \*\*Database operations\*\* :link: database\_ops\_card :link-type: ref .. grid-item-card:: |osoctavia| \*\*Load Balancer operations\*\* :link: load\_balancer\_ops\_card :link-type: ref .. |oslogo| image:: /cloud/\_img/openstack\_logo.png :width: 35px :class: no-scaled-link .. |osnova| image:: /cloud/\_img/nova\_logo.png :width: 35px :class: no-scaled-link .. |osneutron| image:: /cloud/\_img/neutron\_logo.png :width: 35px :class: no-scaled-link .. |oscinder| image:: /cloud/\_img/cinder\_logo.png :width: 35px :class: no-scaled-link .. |osmanila| image:: /cloud/\_img/manila\_logo.png :width: 35px :class: no-scaled-link .. |ostrove| image:: /cloud/\_img/trove\_logo.png :width: 35px :class: no-scaled-link .. |osoctavia| image:: /cloud/\_img/octavia\_logo.png :width: 35px :class: no-scaled-link ------- .. card:: :link: known\_issues\_card :link-type: ref .. figure:: /cloud/\_img/known\_issues.png :align: center :class: no-scaled-link :height: 35px --- # Unknown .. \_cloud\_specifics\_card: Cloud Specifics ================ In the corresponding sub-sections are described the details specific to each HPC Cloud machine in production at CINECA. For the general information on the HPC Cloud infrastructure and service, please refer to :ref:\`cloud/general/general\_info:introduction to hpc cloud\`. .. |ico1| image:: /cloud/\_img/hpc\_icon.png :height: 35px :class: no-scaled-link .. grid:: 3 .. grid-item-card:: |ico1| \*\*ADA\*\* :link: ada\_card :link-type: ref .. grid-item-card:: |ico1| \*\*GAIA\*\* :link: gaia\_card :link-type: ref .. grid-item-card:: |ico1| \*\*MEGARIDE\*\* :link: megaride\_card :link-type: ref .. toctree:: :maxdepth: 2 :hidden: ada .. toctree:: :maxdepth: 2 :hidden: gaia .. toctree:: :maxdepth: 2 :hidden: megaride --- # Unknown .. \_tutorials\_and\_repos\_card: Tutorials and Gitlab repositories ================================= Tutorials for OpenStack dashboard --------------------------------- This section provides you with a list of guides for basic operations. .. grid:: 2 .. grid-item-card:: |tutorial| \*\*Creation of a VM (step-by-step)\*\* :download:\`Download the pdf \` .. grid-item-card:: |tutorial| \*\*Deletion of resources (step-by-step)\*\* :download:\`Download the pdf \` Terraform/OpenTofu/Ansible repositories ---------------------------------------- This section provides you with links to CINECA Gitlab repositories that contain tutorials or modules as examples for the use of :ref:\`cloud/os\_overview/management\_tools/infrastructure\_as\_code:infrastructure as a code\` tools. .. warning:: The GitLab repositories are accessible only if you have an HPC user account - \`Infrastructure as Code (IaC) modules \`\_ - \`Infrastructure as Code (IaC) tutorial \`\_ - \`Customized ansible-scripts for OpenStack \`\_ - \`Slurm mini-hpc cluster on ADA Cloud \`\_ - \`Graphics on HPC Cloud \`\_ - \`RKE2 kubernetes deploy guide \`\_ .. |tutorial| image:: /cloud/\_img/tutorial\_icon.png :width: 35px :class: no-scaled-link --- # Unknown .. \_spec\_users\_card: EUROfusion ========== .. toctree:: :maxdepth: 2 :hidden: gateway .. figure:: ../img/spacer.png :align: center :class: no-scaled-link :height: 20px .. |ico1| image:: img/EUROfusion.png :height: 35px :class: no-scaled-link The |ico1| community has access to the following CINECA HPC systems: \* Leonardo \*\*(until July 31st 2025)\*\* - Booster partition - DCGP partition \* Pitagora \* EUROFusion Gateway (EFGW) .. important:: The general environment defined on our clusters is the same for all the users, so EUROfusion users are invited to refer to the general documentation. Essential links below. For general information regarding the access to the HPC clusters: \* :ref:\`general/getting\_started:Getting Started\` \* :ref:\`general/users\_account:Users and Accounts\` \* :ref:\`general/access:Access to the Systems\` For general information regarding the environment on the HPC clusters: \* :ref:\`hpc/hpc\_intro:Introduction HPC Resources\` \* :ref:\`hpc/hpc\_data\_storage:File Systems and Data Management\` \* :ref:\`hpc/hpc\_scheduler:Scheduler and Job Submission\` \* :ref:\`hpc/hpc\_enviroment:Environment and Customization\` For specific information regarding the HPC clusters used by the EUROfusion community: \* :ref:\`hpc/leonardo:Leonardo\` \* :ref:\`hpc/pitagora:Pitagora\` \* :ref:\`specific\_users/gateway:EFGW Gateway\` Dedicated tutorials ------------------- .. |tutorial| image:: /specific\_users/img/tutorial\_icon.png :width: 35px :class: no-scaled-link A presentation of each supercomputer, and of the access method via two-factor authentication (2FA), have been dedicated to the EUROfusion community. You can find the slides, including a report of the final Q&A session, and the recording at the following links (you should log in through the button :bdg-black-line:\`Access as a guest\`). .. card:: |tutorial| Pitagora: \*Introduction to Pitagora for Eurofusion\* June 23th, 2025 \`Pitagora webinar page \`\_ with slides and recording. :download:\`slides <../files/Pitagora\_EF.pdf>\` .. card:: |tutorial| 2FA: \*Introduction to two-factor authentication (2FA) on CINECA HPC clusters\* June 7th, 2023 :download:\`2FA slides <../files/2FA\_EF.pdf>\` SLURM Partitions ---------------- On :ref:\`hpc/leonardo:Leonardo\` and :ref:\`hpc/pitagora:Pitagora\` \*Job Managing and SLURM Partitions\* sections, you can find the description of SLURM partitions and QOS to submit your jobs. Notice that EUROfusion users have dedicated partitions and QOS: you are allowed to use the \*\*"\_fua\_"\*\* partitions and the related QOS, besides the \*\*"\_all\_serial"\*\* partition which is shared among all users. Low-priority jobs ^^^^^^^^^^^^^^^^^ 1) \*\*If all the budget assigned to your Project Account has been consumed\*\*, you can keep running on Leonardo boost\_fua\_prod and dcgp\_fua\_prod partitions at low priority by requesting in your submission script the \*\*qos\_fualowprio\*\* QOS: .. code-block:: bash #SBATCH --account= #SBATCH --qos=qos\_fualowprio The QOS is \*automatically\* added to your Project Account upon budget exhaustion. 2) You can also request to run low priority jobs, \*\*without having consumed all the budget of yout active Project Account\*\*, by association to the \*\*FUPB1\_LOWPRIO\*\* account on Booster and \*\*FUPA1\_LOWPRIO\_0\*\* account on DCGP (write a mail to superc@cineca.it). You always need to specify also the \*\*qos\_fualowprio\*\* QOS in your submission script. .. code-block:: bash #SBATCH --account= #SBATCH --qos=qos\_fualowprio --- # Unknown .. \_faq\_card: FAQ === This is a page collecting answers to requests arrived to the HPC Helpdesk. Please check the page before sending a specific request. General ------- .. card:: .. dropdown:: \`I still didn’t receive the username and the link for the 2FA configuration?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_I still didn’t receive the username and the link for the 2FA configuration?: You have to do the complete registration on the UserDB page and to be associated with a project (PI has to add you). Once you have inserted all the necessary information and you are associated with a project a new access button will appear, just click on it and you will receive in two mails the username and the link for the 2FA configuration. .. dropdown:: \`I have finished my budget but my project is still active, how can I do?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_I have finished my budget but my project is still active, how can I do?: Non-expired projects with exhausted budgets may be allowed to keep using the computational resources at the cost of minimal priority. Ask superc@cineca.it to motivate your request and, in case of a positive evaluation, you will be enabled to use the qos\_lowprio QOS. .. dropdown:: \`Information about my project on CINECA clusters (end data, total end monthly amount of hours, the usage?)\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_Information about my project on CINECA clusters (end data, total end monthly amount of hours, the usage?): You can list all the Accounts attached to your username on the current cluster, together with the "budget" and the consumed resources, with the command: \`\`> saldo -b\`\` Find more information in :ref:\`hpc/hpc\_data\_storage:Data Occupancy Monitoring Tools\` section. Access and Login ---------------- .. card:: .. dropdown:: \`My new password isn't accepted, with error "Could not execute the password modify extended operation for DN"\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_My new password isn't accepted, with error "Could not execute the password modify extended operation for DN": The error message can be difficult to interpret, but it means that the new password you have chosen does not respect our password policies. Please check the :ref:\`general/users\_account:Users and Accounts\` and choose your new password accordingly. .. dropdown:: \`I receive the error message "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!" when trying to login\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_I receive the error message "WARNING\\: REMOTE HOST IDENTIFICATION HAS CHANGED!" when trying to login: The problem may happen because we have reinstalled the login node changing the fingerprint. We should have informed you through an HPC-news. If this is the case you can remove the old fingerprint from your known\_hosts file with the command \`\`ssh-keygen -f "~/.ssh/known\_hosts" -R "login..cineca.it"\`\` .. dropdown:: \`I keep receiving the error message "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!" even if I modify known\_host file\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_I keep receiving the error message "WARNING\\: REMOTE HOST IDENTIFICATION HAS CHANGED!" even if I modify known\_host file: Please, follow the procedure described below to solve the problem. .. tab-set:: .. tab-item:: Linux/MacOS .. code-block:: bash # LEONARDO mkdir -p ~/.ssh; ssh-keygen -f ~/.ssh/known\_hosts -R login.leonardo.cineca.it; for address in login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it; do ssh-keygen -f ~/.ssh/known\_hosts -R $address; done; for keyal in rsa ecdsa ed25519 dsa; do for address in login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it; do ssh-keyscan -t ${keyal} ${address} | sed "s/${address}/login\*.leonardo.cineca.it/g" >> ~/.ssh/known\_hosts; done; done .. code-block:: bash # G100 mkdir -p ~/.ssh; ssh-keygen -f ~/.ssh/known\_hosts -R login.g100.cineca.it; for address in login01-ext.g100.cineca.it login02-ext.g100.cineca.it login03-ext.g100.cineca.it; do ssh-keygen -f ~/.ssh/known\_hosts -R $address; done; for keyal in rsa ecdsa ed25519 dsa; do for address in login01-ext.g100.cineca.it login02-ext.g100.cineca.it login03-ext.g100.cineca.it; do ssh-keyscan -t ${keyal} ${address} | sed "s/${address}/login\*.g100.cineca.it/g" >> ~/.ssh/known\_hosts; done; done .. code-block:: bash # PITAGORA mkdir -p ~/.ssh; ssh-keygen -f ~/.ssh/known\_hosts -R login.pitagora.cineca.it; for address in login01-ext.pitagora.cineca.it login02-ext.pitagora.cineca.it login03-ext.pitagora.cineca.it login04-ext.pitagora.cineca.it login05-ext.pitagora.cineca.it login06-ext.pitagora.cineca.it; do ssh-keygen -f ~/.ssh/known\_hosts -R $address; done; for keyal in rsa ecdsa ed25519 dsa; do for address in login01-ext.pitagora.cineca.it login02-ext.pitagora.cineca.it login03-ext.pitagora.cineca.it login04-ext.pitagora.cineca.it login05-ext.pitagora.cineca.it login06-ext.pitagora.cineca.it; do ssh-keyscan -t ${keyal} ${address} | sed "s/${address}/login\*.pitagora.cineca.it/g" >> ~/.ssh/known\_hosts; done; done .. tab-item:: Windows .. code-block:: powershell # LEONARDO mkdir -Force $HOME\\.ssh\\known\_hosts; ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R "login.leonardo.cineca.it"; foreach ($address in "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it") { ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R $address }; foreach ($keyal in "rsa", "ecdsa", "ed25519", "dsa") { foreach ($address in "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it") { ssh-keyscan -t $keyal $address | ForEach-Object { $\_ -replace "$address", "login\*.leonardo.cineca.it" } | Out-File -Encoding UTF8 -Append "$HOME\\.ssh\\known\_hosts" } } .. code-block:: powershell # G100 mkdir -Force $HOME\\.ssh\\known\_hosts; ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R "login.g100.cineca.it"; foreach ($address in "login01-ext.g100.cineca.it", "login02-ext.g100.cineca.it", "login03-ext.g100.cineca.it") { ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R $address }; foreach ($keyal in "rsa", "ecdsa", "ed25519", "dsa") { foreach ($address in "login01-ext.g100.cineca.it", "login02-ext.g100.cineca.it", "login03-ext.g100.cineca.it") { ssh-keyscan -t $keyal $address | ForEach-Object { $\_ -replace "$address", "login\*.g100.cineca.it" } | Out-File -Encoding UTF8 -Append "$HOME\\.ssh\\known\_hosts" } } .. code-block:: powershell # PITAGORA mkdir -Force $HOME\\.ssh\\known\_hosts; ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R "login.pitagora.cineca.it"; foreach ($address in "login01-ext.pitagora.cineca.it", "login02-ext.pitagora.cineca.it", "login03-ext.pitagora.cineca.it", "login04-ext.pitagora.cineca.it", "login05-ext.pitagora.cineca.it", "login06-ext.pitagora.cineca.it") { ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R $address }; foreach ($keyal in "rsa", "ecdsa", "ed25519", "dsa") { foreach ($address in "login01-ext.pitagora.cineca.it", "login02-ext.pitagora.cineca.it", "login03-ext.pitagora.cineca.it", "login04-ext.pitagora.cineca.it", "login05-ext.pitagora.cineca.it", "login06-ext.pitagora.cineca.it") { ssh-keyscan -t $keyal $address | ForEach-Object { $\_ -replace "$address", "login\*.pitagora.cineca.it" } | Out-File -Encoding UTF8 -Append "$HOME\\.ssh\\known\_hosts" } } .. dropdown:: \`Windows WSL issue DNS resolution failing\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_Windows WSL issue DNS resolution failing: If the DNS resolution fails with Temporary failure in name resolution or resolution timing out, an automatic change in \`\`/etc/resolv.conf\`\` occured. You can change it manually by replacing the name server value with 8.8.8.8 . This file is automatically generated by WSL: to stop the automatic generation of this file, add the following entry to /etc/wsl.conf: \[network\] generateResolvConf = false. Then, add in your \`\`.bashrc\`\` the following commands for the automatic creation of the name server value in the \`\`resolv.conf\`\` file: .. code-block:: bash if \[ ! -f /etc/resolv.conf \]; then echo "nameserver 8.8.8.8" > /etc/resolv.conf fi .. dropdown:: \`The message "perl: warning: Setting locale failed" appear when I login. How do I solve?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_The message "perl\\: warning\\: Setting locale failed" appear when I login. How do I solve?: This warning is typical of Mac users (but can happen with other OS too). It is actually innocuous and can be safely ignored, but if you want to get rid of it you can add these lines to the \`\`.bashrc\`\` of your workstation, or in general you can execute them before trying to login to our systems: .. code-block:: bash export LANGUAGE=en\_US.UTF-8 export LANG=en\_US.UTF-8 export LC\_ALL=en\_US.UTF-8 if you try to login afterwards, the warnings should have disappeared. .. dropdown:: \`Can I login with ssh inside a compute node?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_Can I login with ssh inside a compute node?: In general is forbidden to login via ssh in a compute node, as the network is local-only restricted, It is always possible to access to compute node by submitting an interactive job with \`\`srun\`\` or \`\`salloc\`\` (check the :ref:\`hpc/hpc\_scheduler:Interactive Job Submission with SLURM\` section), but apart from that, \`\`ssh\`\` is \*\*only\*\* possible if there is a \*running\* job from the user that want to login into compute node. Specifically, in Leonardo, due to recent system updates, the connection via \`\`ssh\`\` can be denied even for that specific case described before. To avoid such limitation is now mandatory to have a pair of SSH keys generated inside Leonardo cluster. The user can create the keys with the command \`\`ssh-keygen\`\` and copy the output , the public key, in their configuration file \`\`~/.ssh/autorized\_keys\`\`. After the generation of SSH Keys, the connection via \`\`ssh\`\` in a compute node will be possible (with the condition of a job running from the user that whant to connect to compute node). .. dropdown:: \`Users using smartcards and GPG agent with SSH support enabled\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_Users using smartcards and GPG agent with SSH support enabled: Step CLI doesn't play well with non-default ssh-agent. The solution is to either: \* Configure the shell to start the default \`\`ssh-agent\`\` on launch (or find the existing one); it can be done on Unix systems or on Windows WSL e.g., by adding to \`\`$HOME/.bashrc\`\` file: .. code-block:: bash if \[ -f ~/.bash\_agent \]; then . ~/.bash\_agent fi steptest=$(step ssh list --raw ''| step ssh inspect | grep "Valid") if \[ -z "$steptest" \] then eval $(ssh-agent) echo "export SSH\_AUTH\_SOCK=$SSH\_AUTH\_SOCK" > ~/.bash\_agent echo "export SSH\_AGENT\_PID=$SSH\_AGENT\_PID" >> ~/.bash\_agent step ssh login '' --provisioner cineca-hpc fi \* Avoid using ssh-agent, downloading your certificate launching the following command in any path of your local PC (we suggest using the ~/.ssh folder): .. code-block:: bash step ssh certificate 'user-email' --provisioner cineca-hpc my\_key You can change \*\*my\_key\*\* with the name you prefer. A passphrase to encrypt the private key is request as input in the shell command line: .. code-block:: bash Please enter the passphrase to encrypt the private key: Both options are explained in greater detail in :ref:\`general/access:How to configure \*smallstep\* client\` section. 2FA --- .. card:: .. dropdown:: \`ERROR: The term 'step' is not recognized as the name of a cmdlet (Powershell)\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_ERROR\\: The term 'step' is not recognized as the name of a cmdlet (Powershell): If running the command step to verify the installation of smallstep you incour in the following error: .. code-block:: bash PS C:\\Users\\user > step step : The term 'step' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + step + ~~~~ + CategoryInfo : ObjectNotFound: (step:String) \[\], ParentContainsErrorRecordException + FullyQualifiedErrorId : CommandNotFoundException check if you find the executable step.exe inside the folder: \`\`C:\\Users\\user\\scoop\\shims\`\` The installation command should have placed it there. If you don't find it, run on your Powershell the command: \`\`scoop install smallstep/step\`\` \*\*ERROR associated to X11 execution\*\* 1. install Xming: https://sourceforge.net/projects/xming/, it will open a window in the background that you won't be able to see but you can see that it's there looking between the icons in the Windows' applications bar (bottom right) 2. follow the steps reported at https://x410.dev/cookbook/built-in-ssh-x11-forwarding-in-powershell-or-windows-command-prompt/ for PowerShell 3. then you can run the command ssh to login on the cluster .. dropdown:: \`undefined method 'cellar' when installing step on MacOS\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_undefined method 'cellar' when installing step on MacOS: You may encounter an error that looks like this: Error: step: undefined method \`\`cellar for #\`\` In this case, the problem is in your homebrew. It may refer to the directories of different processes, e.g. Intel, while you need to make it refer to AMD. You can reinstall homebrew: brew tap homebrew/core and then set the proper paths with simple shell commands: .. code-block:: bash (echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') >> /Users//.zprofile eval "$(/opt/homebrew/bin/brew shellenv)" If this is not the solution of your error, the command \*brew doctor\* should give you an hint about how to proceed in your specific case. Scheduler and Job Execution --------------------------- .. card:: .. dropdown:: \`My job has been waiting for a long time\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_My job has been waiting for a long time: The priorities in the queue are composed of several factors and the value may change due to the presence of other jobs, of the resources required, and your priority. You can see the reason for your job in the queue with the squeue command. If the state is PD, the job is pending. Some reasons for the pending state that could bee displayed: \* Priority= The job is waiting for free resources. \* Dependency= It is depending to the end of another job. \* QOSMaxJobsPerUserLimit = The maximum number of jobs a user can have running at a given time. You can also consult the estimated starting run time with the SLURM command \`\`scontrol\`\`: \`\`scontrol show job #JOBID\`\` or you can see the priority of your job with the sprio SLURM command: \`\`sprio -j #JOBID\`\` .. dropdown:: \`Can I modify SLURM settings of a waiting job?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_Can I modify SLURM settings of a waiting job?: Some Slurm settings of a pending job can be modified using the command scontrol update. For example, setting the new job name and time limit of the pending job: \`\`scontrol update JobId=2543 Name=newtest TimeLimit=00:10:00\`\` .. dropdown:: \`How can I place and release a job from hold state?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_How can I place and release a job from hold state?: In order to place a job on hold type: \`\`scontrol hold JOB\_ID\`\`. To release the job from the hold state, issue: \`\`scontrol release JOB\_ID\`\`. .. dropdown:: \`Error invalid account when submitting a job: Invalid account or account/partition combination specified\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_Error invalid account when submitting a job\\: Invalid account or account/partition combination specified: The error Invalid account might depend on the lack of resources associated to your project or there is an error with the account name in your batch script. Just use the \`\`saldo\`\` command. If the account is correct and valid, are you lunching the job on the right partition? To see which partition is right for your case and account, please consult the :ref:\`hpc/hpc\_scheduler:Scheduler and Job Submission\` section. .. dropdown:: \`I get the following message: "srun: Warning: can't honor --ntasks-per-node set to xx which doesn't match the requested tasks yy with the number of requested nodes yy. Ignoring --ntasks-per-node." What does it mean?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_I get the following message\\: "srun\\: Warning\\: can't honor --ntasks-per-node set to xx which doesn't match the requested tasks yy with the number of requested nodes yy. Ignoring --ntasks-per-node." What does it mean?: This is a message that can appear when using mpirun and Intelmpi parallel environment. It is a known problem that can be safely ignored,since mpirun does not read the proper Slurm variables and thinks that the environment is not set properly, thus generating the warning: in reality, the instance of srun behind it will respect the setting you requested with your Slurm directives. While there are workarounds for this, the best solution (apart from ignoring the message) is to use srun instead of mpirun: with this command, the Slurm environment is read properly and the warning does not appear. .. \_cloud\_faq\_card: HPC Cloud --------------------------- .. card:: .. dropdown:: \`I can't SSH to my virtual machine\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_I can't SSH to my virtual machine: The most common reasons for not being able to login to your VM are related to: - \*\*SSH command\*\*: be sure to use the correct username, address and access key - \*\*Floating IP\*\*: If no FIP is associated to your VM, it is not possible to reach it. (see :ref:\`cloud/operative/network\_ops/fip\_association:Allocate a floating IP\`) - \*\*Security Rules\*\*: In the "Overview" tab, under the section "Security Group" check that the rule for port 22 is defined, and what ranges of IP is allowed to access. (see :ref:\`cloud/operative/network\_ops/secgroups\_create:Security groups: create\`) - \*\*Network issues\*\*: Check that Network, Subnet and Router are set up properly. - \*\*Machine Boot\*\*: Check if the VM booted correctly. On the Horizon Dashboard, enter the VM page and check the "Console" tab. If an error message appears related to "kernel panic" or "no bootable device", then the problem lies on either the specific image or the bootable device used. If no error appears, then check the full Boot Log in the tab "Log". Depending on the error found, could be necessary to perform the rescue of the VM (see :ref:\`cloud/operative/compute\_ops/instance\_rescue:Instance: rescue\`) .. dropdown:: \`I can't create an OpenStack resource\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_I can't create an OpenStack resource: The main reason a user is blocked when creating new resources (virtual machines, volumes, etc.) is that they have reached their project quota. If you need to increase your project quota please contact our user support. .. dropdown:: \`If I resize my virtual machine, will I lose my data?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_If I resize my virtual machine, will I lose my data?: No, you won't lose your data, but you will have to re-partition your disk to use the space you added with the resize operation. .. dropdown:: \`Can I create a virtual machine using a Windows image?\`\_ :animate: fade-in-slide-down :color: warning :chevron: down-up .. \_Can I create a virtual machine using a Windows image?: No, on CINECA HPC Cloud systems users are not allowed to upload and/or use Windows images, even if they own a personal license. --- # Unknown .. \_tenants\_administration\_card: Tenants Administration ====================== This section provides additional information related to the management of cloud projects. In particular, it explains how you can request to associate a DNS name to your instance, and how to safely manage sensitive data. A dedicated section provides guidelines on how to best manage security for your instances to reduce vulnerabilities and risks of attacks. .. |ico1| image:: /cloud/\_img/hpc\_icon.png :height: 35px :class: no-scaled-link .. grid:: 3 .. grid-item-card:: |guidelines| \*\*DNS guidelines\*\* :link: dns\_guidelines\_card :link-type: ref .. grid-item-card:: |guidelines| \*\*Security guidelines\*\* :link: security\_guidelines\_card :link-type: ref .. grid-item-card:: |guidelines| \*\*Store sensitive data\*\* :link: store\_sensitive\_data\_card :link-type: ref .. toctree:: :maxdepth: 2 :hidden: dns\_guidelines security\_guidelines store\_sens\_data .. |guidelines| image:: /cloud/\_img/guidelines\_icon.png :width: 35px :class: no-scaled-link --- # Unknown .. \_os\_overview\_card: OpenStack Overview ================== What is OpenStack? ------------------ In CINECA, we use OpenStack to virtualize and manage cloud computing resources. \`OpenStack \`\_ is an open-source cloud operating system that controls and manages the compute resources of a cloud data center. .. image:: ../\_img/openstack\_software-overview-diagram-new.png :align: center Management Tools ---------------- To manage the resources assigned to a project, the users can interact with OpenStack using in-house tools like Horizon Web Dashboard and the Command Line Interface (CLI) or using Infrastructure as a Code tools. .. grid:: 3 .. grid-item-card:: |oslogo| \*\*Horizon Dashboard\*\* :link: dashboard\_card :link-type: ref .. grid-item-card:: |oslogo| \*\*Command Line\*\* :link: command\_line\_card :link-type: ref .. grid-item-card:: |oslogo| \*\*Infrastructure as code\*\* :link: infrastructure\_as\_code\_card :link-type: ref .. toctree:: :maxdepth: 1 :hidden: management\_tools/dashboard management\_tools/command\_line management\_tools/infrastructure\_as\_code Components ---------- OpenStack is a flexible tool that has a modular structure: it is split up in different components that can be installed and activated independently to address every user's need. .. grid:: 3 .. grid-item-card:: |osnova| \*\*Compute\*\* :link: compute\_card :link-type: ref .. grid-item-card:: |oscinder| \*\*Storage\*\* :link: storage\_card :link-type: ref .. grid-item-card:: |osneutron| \*\*Network\*\* :link: network\_card :link-type: ref .. grid:: 3 .. grid-item-card:: |osmanila| \*\*Shares\*\* :link: shares\_card :link-type: ref .. grid-item-card:: |ostrove| \*\*Database\*\* :link: database\_card :link-type: ref .. grid-item-card:: |osoctavia| \*\*Load Balancer\*\* :link: load\_balancers\_card :link-type: ref .. toctree:: :maxdepth: 1 :hidden: os\_components/compute os\_components/storage os\_components/network os\_components/shares os\_components/database os\_components/load\_balancers .. |oslogo| image:: /cloud/\_img/openstack\_logo.png :width: 35px :class: no-scaled-link .. |osnova| image:: /cloud/\_img/nova\_logo.png :width: 35px :class: no-scaled-link .. |osneutron| image:: /cloud/\_img/neutron\_logo.png :width: 35px :class: no-scaled-link .. |oscinder| image:: /cloud/\_img/cinder\_logo.png :width: 35px :class: no-scaled-link .. |osmanila| image:: /cloud/\_img/manila\_logo.png :width: 35px :class: no-scaled-link .. |ostrove| image:: /cloud/\_img/trove\_logo.png :width: 35px :class: no-scaled-link .. |osoctavia| image:: /cloud/\_img/octavia\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_users\_card: Users and Accounts ================== Usage of CINECA HPC resources is allowed only to users with an \*\*User Account\*\* (or HPC username) and provided with a \*\*Project Account\*\*. UserDB ------ The \*\*UserDB\*\* is the portal containing all User Accounts and Project Accounts and where CINECA users can manage their profile and monitor their computational resources. How to become a User ^^^^^^^^^^^^^^^^^^^^ To obtain a \*\*User Account\*\*, you need to: 1. register on the \*\*UserDB\*\* Portal; 2. get associated to a valid \*\*Project Account\*\*; 3. request to be validated and enabled to access CINECA clusters. Register on UserDB """""""""""""""""" You can reach the portal at \`\`\_. Click on :bdg-black-line:\`Create new user\` and fill in the form. .. image:: img/userdb\_login.png :align: center :height: 450 px :class: no-scaled-link .. image:: img/spacer.png :align: center :class: no-scaled-link .. important:: In case your Name or Surname contain special (non ASCII) characters, please use the corresponding ASCII one Once the registration is completed and you have set the password of your \*\*UserDB credentials\*\*, please go to :bdg-black-line:\`my User\`, click on :bdg-black-line:\`Edit\` and complete your profile: \* upload a valid Identity document (Passport, ID, Italian driving license) in \*\*Documents for HPC\*\* page and sign the CINECA Access policies; \* insert your affiliation in \*\*Institution\*\* page. You can use the \*\*UserDB credentials\*\* to login to UserDB. Alternatively, you can also use your :ref:\`general/users\_account:HPC credentials\` with 2FA by clicking on :bdg-black-line:\`OpenID\` button. .. warning:: Each user can have \*\*only one profile\*\* on UserDB. If the profile already exists or the email is already used you need to recover your previous profile by clicking on :bdg-black-line:\`Request new password\` and inserting the email that you used to register the first time. Write to \`superc@cineca.it \`\_ for any issues. Get associated to a valid \*\*Project Account\*\* """"""""""""""""""""""""""""""""""""""""""""" .. image:: img/registration.png Each user can be either a \*Principal Investigator (PI)\*, or a \*Collaborator\* of a \*\*Project Account\*\*. \* \*\*Principal Investigators\*\*: apply for a project or acquire HPC resources through the processes described below. \* \*\*Collaborators\*\*: request the PI of a project to be associated to it. See :ref:\`general/users\_account:PI and Collaborators\` section. Currently, there are several ways to become \*\*Principal Investigator\*\*: 1. \*\*ISCRA Projects\*\*: dedicated to all scientific researchers affiliated to an Italian research organization. 2. \*\*EuroHPC Projects\*\*: dedicated to European researchers. 3. \*\*Agreements\*\*: For Italian research institutions, contact \`superc@cineca.it \`\_. 4. \*\*General Users and Industrial Applications\*\*: Send a request to \`superc@cineca.it \`\_. You can find on \`HPC Access \`\_ page of our HPC portal a detailed description of each case and useful links. Once your project has been approved, we will create the corresponding \*\*Project Account\*\* on UserDB portal and associate you as \*\*PI\*\*. .. important:: Write to \`superc@cineca.it \`\_ in case the project is approved but you still do not see it in the portal. Submit a request to have a \*\*User Account\*\* """"""""""""""""""""""""""""""""""""""""""" Once the steps above are all completed you can request us to create a \*\*User Account\*\* by going to the page :bdg-black-line:\`HPC Access\` and click :bdg-black-line:\`Submit\`. .. image:: img/submituserdb.png .. warning:: If the :bdg-black-line:\`Submit\` button does not appear, it means that one or more of the above steps are still not completed. In the \*\*HPC Access\*\* page, the portal will indicate you which step is still \*\*not OK\*\*. We will receive your request and after a check of the inserted data we will grant you access to our HPC resources. Within 24 hours you will receive via email the information needed to set your \*\*HPC credentials\*\*: \* your \*\*User account\*\* name (or HPC username) \* a link to setup your \*\*2FA\*\* access (the link will be \*\*valid only for 12h\*\*) You can visit the dedicated page :ref:\`general/access:How to activate the \*\*2FA\*\* and the \*\*OTP\*\* generator\` for the steps to configure the 2FA and use it to access our HPC resources. HPC credentials ^^^^^^^^^^^^^^^ \*\*HPC credentials\*\* are different from the \*\*UserDB credentials\*\*. The former are needed to access CINECA HPC and Cloud infrastructures, while the latter are only to be used inside the UserDB portal. .. caution:: Both \*\*HPC credentials\*\* and \*\*UserDB credentials\*\* are strictly personal and for any reasons cannot be shared with others. PI and Collaborators ^^^^^^^^^^^^^^^^^^^^ Each \*\*Project Account\*\* has a \*PI (Principal Investigator)\*. The PI is usually who have applied for obtaining the budget and is responsible of everything that happens using that account. Each \*\*Project Account\*\* may have one or more \*Collaborators\*. The PI can add autonomously collaborators to its account. .. image:: img/userdb\_collaborator.png The Collaborator needs to be registered on UserDB. To add the Collaborator, the PI needs to go to the page of the Project Account visible in the :bdg-black-line:\`My projects\` page and click on :bdg-black-line:\`Edit\`. In the Collaborators section, insert the \*\*UserDB username\*\* in a blank line and click on the correct name of the collaborator among the ones proposed by the appearing scroll-down menu. .. important:: For any issues in finding the collaborator, please write to \`superc@cineca.it \`\_ .. warning:: If the PI of the project account is registered to UserDB but \*\*is not yet approved to access the cluster (HPC STATUS: Not Configured)\*\*, the project is not loaded on the infrastructure and collaborators \*\*may not be able to access\*\* the cluster. Once the PI is approved, the project account will be activated. Project Accounts ---------------- CINECA users can have access to HPC resources in several ways listed in :ref:\`general/users\_account:Get associated to a valid \*\*Project Account\*\*\` and described in detail in our \`HPC portal \`\_. Each approved grant is uniquely identified by a \*\*Project Account\*\* name. Each Project Account is provided with at least one budget, usually in standard core hours (STDH) for HPC clusters or VCPUs for Cloud resources, on a specific CINECA cluster. A Project Account may have multiple budgets on different CINECA systems. A User Account may be associated as \*Principal Investigator (PI)\* or as \*Collaborator\* to one or multiple Project Accounts. Budgets ^^^^^^^ In the context of HPC (High-Performance Computing) resources, \*\*budget\*\* refers to the allocation of some computational resources granted to a Project Account. The resources differs if the Project is defined on HPC or Cloud systems. .. tab-set:: .. tab-item:: HPC | The utilization of our High-Performance Computing (HPC) resources operates within a "pay-per-use" framework. | Usually short interactive serial jobs run on login nodes for testing incur no charges. | Parallel and Production simulations instead must be executed on compute nodes by submitting jobs to a scheduler. | Compute nodes of a HPC system are usually divided into different partitions. Some partitions may be free of charge. | Please check on :ref:\`hpc/hpc\_clusters:Cluster Specifics\` section for the available partitions and their cost on the cluster where you have budget. On UserDB all the budgets are expressed in \*\*standard core-hours\*\*. On the cluster they are translated into \*\*local core-hours\*\*. The convertion factor depends on the different cost of a single core-hour among the available clusters. Each approved \*\*Project Account\*\* may have multiple budgets on several clusters. In this case the budget name may differ among the clusters. Please check with the command \`\`saldo\`\` the correct name of your budget on the specific cluster. Further information about how to check the availabe \`\`saldo\`\` of a project are reported in the :ref:\`hpc/hpc\_intro:Budget and Accounting\` section. On CINECA clusters a budget linearization policy is enforced. A detailed description is found in the specific :ref:\`hpc/hpc\_intro:Budget Linearization\` section. .. tab-item:: CLOUD Cloud budget is assinged in Virtual CPUs (VCPUs) and Storage Your accounts are in the menu on the top left in Horizon Dashboard. Projects validity and expiration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Project accounts allocated in UserDB have a fixed duration. Once reached the end date, the account enters a so called \*\*expired\*\* status. In the \*\*expired\*\* period, the budget cannot be used to submit jobs, but the data in the project specific areas are preserved. The expiration duration depends on the kind of project. Usually it is \*\*6 months\*\* for HPC accounts and \*\*1 month\*\* for Cloud accounts, but in special cases it may differ. At the end of the expiration delay all the data in the project specific areas are deleted. Please check the :ref:\`hpc/hpc\_data\_storage:Backup Policy and Data Availability\` for further details. Manage your UserDB credentials ------------------------------ In the UserDB portal, the users can manage the following information: \* Changing the UserDB password \* Request a new UserDB password \* Update the affiliation .. note:: It is \*\*not possible\*\* to update the email address directly on UserDB. The change has to be made on SSO portal and the UserDB will automatically update the email (see :ref:\`general/users\_account:Manage your HPC credentials\`). .. dropdown:: How to change the UserDB password :animate: fade-in-slide-down :color: light Login to UserDB portal, then click on :bdg-black-line:\`Edit\`. Insert the Current Password and then below the New Password. If the quality is Good, click on Save at the bottom of the page. .. figure:: img/UserDBpassword.png Please follow the :ref:\`general/users\_account:Policy for UserDB password definition\`. .. dropdown:: How to recover the UserDB password :animate: fade-in-slide-down :color: light If you do not remember the password you can reset it by clicking on :bdg-black-line:\`Request new password\`. You will be asked to insert the email that you used to register on UserDB. The portal will send you an email with a link to set a new password. .. note:: If you insert a different email address, you will \*\*not receive\*\* any link. .. dropdown:: How to update the affiliation :animate: fade-in-slide-down :color: light Any time you chnge your affiliation if you still make use of CINECA resources, we kindly ask you to update your affiliation. .. figure:: img/Institution.png Login to UserDB portal, click on :bdg-black-line:\`Edit\` and then on :bdg-black-line:\`Institution\` and update it. Policy for UserDB password definition ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you change the password on UserDB portal here you can find the password policies: - Password must not match last 3 passwords. - Password must not contain the username. - Password must be at least 10 characters in length. - Password must contain at least 2 digits, 4 letters, 2 lowercase and 2 uppercase characters. - The following special characters are allowed \*\*(!"#$%&'()\*+,-./:;<=>?@\[\\\]^\_\`{|}~)\*\* Manage your HPC credentials --------------------------- Thanks to the new Identity Provider, \`sso.hpc.cineca.it \`\_ , users can now manage their HPC authentication credentials independently. This includes: \* Changing the HPC password \* Updating the email address associated to their User Account \* Recovering the OTP generator and re-generating recovery codes For any issues with these procedures or if you have any questions, please contact superc@cineca.it. .. dropdown:: How to change the HPC \*password\* :animate: fade-in-slide-down :color: light \* If you have already configured the 2FA but do not remember the password \* by clicking on :bdg-black-line:\`Forgot password\` while authenticating on https://sso.hpc.cineca.it .. figure:: img/pwd\_1.png \* specify your \*HPC Username\* or the \*email\* address used to register on :ref:\`general/users\_account:UserDB\` portal. You will receive on that email a link to reset the password. .. figure:: img/pwd\_2.png \* click on the link and insert a \*new password\* according to the :ref:\`general/users\_account:Policy for HPC password definition\` and click on :bdg-black-line:\`submit\` .. figure:: img/pwd\_3.png \* If you \*\*remember the password\*\*, but just want to update it on SSO portal \* login on the \`SSO \`\_ portal. \* choose \*\*Account security\*\*. .. figure:: img/key\_2.png \* select :bdg-black-line:\`Update\` in \*\*my password\*\* section. Choose a \*new password\* according to the :ref:\`general/users\_account:Policy for HPC password definition\` .. figure:: img/key\_3.png .. warning:: The \`\`passwd\`\` command has been disabled. If you still need to configure your 2FA but you don't remember the password, or your password is expired, the above solutions \*\*will not work\*\*. Please write to superc@cineca.it in order to solve the issue. .. dropdown:: How to change the \*email\* address :animate: fade-in-slide-down :color: light \* login on the \`SSO \`\_ portal. \* select \*\*Personal Info\*\*. .. figure:: img/key\_2.png \* edit the field \`\`email\`\` and \`\`save\`\`. .. figure:: img/key\_1.png Within 24 hours the email will be updated also on UserDB portal. .. warning:: Users cannot change their email address directly on the UserDB portal. If you encounter issues while attempting to update your email address, please contact superc@cineca.it for assistance. .. dropdown:: Recover OTP One-Time Code Generator :animate: fade-in-slide-down :color: light If you lost access to your OTP generator, you can configure a new one using the recovery authentication codes saved during the initial OTP setup. After inserting username and password on SSO login procedure, the page will request the One-Time Password. Click on :bdg-black-line:\`Try another way\`. .. figure:: img/otp\_recover\_1.png \* choose :bdg-black-line:\`Recovery Authentication Code\` .. figure:: img/otp\_recover\_2.png \* Insert the requested \*\*Recovery Code\*\* (obtained during the first configuration of the OTP). .. figure:: img/otp\_recover\_3.png .. note:: The system requires a specific recovery code of the list of available codes. It is identified by the "#" on top of the Recovery code field. In the above example it is asking the first (#1) recovery code of the list. .. important:: \*\*Renew the recovery codes\*\*: It is also possible to generate new recovery codes by clicking on :bdg-black-line:\`Set up Recovery authentication codes\` in the section \*\*Recovery authentication codes\*\* of \*\*Account Security\*\* page. Policy for HPC password definition ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you change the password on the portal \`sso.hpc.cineca.it \`\_, it will be automatically changed on all the clusters (the propagation can take up to one hour). Here we report the HPC password policies: - The new password has to be at least 10 characters long and contains at least 1 capital letter, 1 number, and 1 special character \*\*(!"#$%&'()\*+,-./:;<=>?@\[\\\]^\_\`{|}~)\*\* - The password has a validity of 3 months. You will receive a reminder 10 days before the expiration when you login. - The new password has to be different from the previous 5 ones. - Any password change will be notified to the user by email. --- # Unknown .. \_access\_card: Access to the Systems ===================== Accessing any section of the Cineca HPC systems requires activating two-factor authentication (\*\*2FA\*\*) for each user account. This enhanced security measure verifies a user's identity by requiring a second, independent factor in addition to the account password. Even if the correct account password is used, \*\*2FA\*\* ensures that unauthorized access is prevented, significantly improving the system's overall security. This access modality operates seamlessly for users, who continue to utilize standard protocols such as the SSH client. Before connecting to the cluster, users must request an SSH certificate from our Identity Provider (IP) via the \*smallstep\* client. Upon making the request, a web page will automatically open in the browser, prompting users to authenticate with our IP by entering a one-time password (\*\*OTP\*\*). Following successful authentication, the server will issue a time-limited certificate valid for \*\*12 hours\*\*. This certificate allows users to connect to CINECA systems via \*SSH client\*. .. important:: For the \*\*First-time Access\*\* and the activation of \*\*2FA\*\*, a user must complete the following steps: \* Registration on :ref:\`general/users\_account:UserDB\`. \* Correct configuration of 2FA and OTP (:ref:\`general/access:How to activate the \*\*2FA\*\* and the \*\*OTP\*\* generator\`). .. important:: \* If you are a \*\*Cloud\*\* user only, \*smallstep\* installation is not required. \* For \*\*HPC Resources\*\* you must use \*smallstep\* client to allow \*\*ssh\*\* protocol (:ref:\`general/access:How to configure \*smallstep\* client\`). How to activate the \*\*2FA\*\* and the \*\*OTP\*\* generator ----------------------------------------------------- Follow the instructions below to activate the 2FA authenticator method and configure the OTP generator. .. dropdown:: 2FA activation and OTP configuration, step-by-step guide :animate: fade-in-slide-down :color: light \* Step 1 - Activate the \*\*2FA\*\* \* point the website https://sso.hpc.cineca.it \* Sign in using your CINECA cluster username and password (received by email). \* \*\*At first login\*\*, you'll be prompted to verify your email, change your password, and configure your OTP (One-Time Password). .. figure:: img/2fa\_1.png .. important:: New users will automatically receive the link. Once it is set up, you'll use the OTP alongside your password for logging into the cluster. If it's your first time logging in and you haven't received a valid access link, contact superc@cineca.it. if you already have access to the cluster via username and password but you haven't yet done your first login on the new sso portal and you haven't configured the 2FA, you need to write at superc@cineca.it to receive a valid link for the access. \* Step 2 \* After your first login, you will receive an email from CINECA with the subject "CINECA HPC Single Sign-On: Verify Your Email." This email will be sent to the address you registered with on the UserDB portal. \* The email contains a verification link labeled \`\`Click on the link to proceed\`\`. Once you click it, you will be prompted to define a new \*password\*. Please ensure that your chosen password complies with the :ref:\`general/users\_account:Policy for HPC password definition\`. .. figure:: img/2fa\_2.png \* Step 3 After the definition of a new valid password (that will replace the password used to login to CINECA clusters), you will be asked to configure the 2FA following few simple steps. .. figure:: img/2fa\_3.png \* Step 4 - OTP Applications \* Authentication codes can be generated using either \*FreeOTP\* or \*Google Authenticator\*, or other compatible apps. If you don't already have one of these apps, download it onto your smartphone. \* Once the app is installed, use it to scan the QR code displayed on the setup page. The OTP will be automatically configured on your authenticator. \* As a final step, enter the \*\*6-digit\*\* code that appears in the app to verify the correct configuration. If you have multiple OTPs in the app, the correct one will be labeled \*\*"CINECA HPC: "\*\*. \* After verifying the correct configuration, the following page will display your \*\*Recovery codes\*\*. Save these codes by downloading, printing, or copying them into a text file. .. figure:: img/2fa\_4.png .. warning:: \*\*Recovery codes\*\* are requested to the user in case of problems with your management of the OTP codes (for example, issues with the app or smartphone lost), so saving them somewhere is very important. \* Step 5 \* At this point, your 2FA is active and your OTP configuration is done. In case you are a user of HPC resources (not Cloud), you can follow the guide below to setup the smallstep client for the authentication certificate and to connect to the HPC clusters via ssh protocol. .. important:: If you need to regenerate the OTP, please refer to the procedure outlined in the :ref:\`general/users\_account:Manage your HPC credentials\`. Users that want to get access \*only\* to \*\*Cloud\*\* resources can skip the next steps. How to configure \*smallstep\* client ----------------------------------- Once 2FA is enabled as authentication method for CINECA clusters, you will need to install and configure a 2FA-compatible program on your PC, to download the temporary certificate locally. At CINECA, we recommend using the \*\*Smallstep\*\* client. .. note:: \*\*smallstep\*\* client is not required to access to cloud resources. To obtain and install the Smallstep executable, visit the Smallstep \`website \`\_ and follow the installation instructions for your operating system. Alternatively, you can download the executable directly from the \`GitHub repository \`\_. After installation, you must configure Smallstep according to your operating system's requirements. If you encounter any issues or need further assistance, please contact superc@cineca.it. .. dropdown:: Linux/MacOS Systems :animate: fade-in-slide-down :color: light .. warning:: Users with \*\*Ubuntu Linux\*\* operating systems (also for other Linux distributions) should not run the command \`sudo apt install step\` because this will install a different software that will give errors when following the rest of the instructions. \* \*\*Step 1\*\* Configure \*smallstep\* on your system by running the following command line instructions in your shell: .. code-block:: bash step ca bootstrap --ca-url=https://sshproxy.hpc.cineca.it --fingerprint 2ae1543202304d3f434bdc1a2c92eff2cd2b02110206ef06317e70c1c1735ecd the command output should be the following: .. code-block:: bash The root certificate has been saved in /.step/certs/root\_ca.crt. The authority configuration has been saved in /.step/config/defaults.json. .. note:: If you have a previous version of smallstep installed and configured on your system, the client will ask if you want to overwrite the existing configuration. To save a copy of a previous version of smallstep installed and configured on your system, make a copy of the directory \*.step\*. \* \*\*Step 2\*\* To use the certificate created in \*\*Step 1\*\*, the user should activate the \*\*ssh-agent running\*\* with the following command: .. code-block:: bash eval $(ssh-agent) .. note:: If the agent is already active, this step is not required. \* \*\*Step 3\*\* To use the certificate for the authentication, use the command: .. code-block:: bash step ssh login '' --provisioner cineca-hpc The command will prompt an output as in this figure: .. figure:: img/ca\_linux.png \* \*\*Step 4\*\* Once the certificate is created, a webpage will automatically open in your default browser. You will need to \*\*sign in\*\* using your cluster credentials (username and password). Afterwards, you will be prompted to enter a temporary code generated by your OTP application to complete the process. .. figure:: img/otp.png Once authenticated, you will see a \*\*Success message\*\* on your browser, meaning that the certificate has been generated and it is available on your PC. .. figure:: img/success\_ca.png .. note:: the certificate is valid for \*\*12 hours\*\* !!! If you reboot your PC, the certificate is lost and you need to download a new one (repeating step 3 step ssh login ... ) ! Windows users have multiple options: .. dropdown:: Windows Powershell :animate: fade-in-slide-down :color: light \* \*\*Step 1\*\* \* Open the \`\`Powershell\`\`. Windows O.S will show the standard prompt .. figure:: img/powershell\_1.png \* \*\*Step 2\*\* - Download and install \`step\` \* From the \`\`Powershell\`\` prompt type the command: .. code-block:: shell winget install Smallstep.step and type "Yes" or "Y" if prompted. After the installation, close and reopen Powershell so that the alias for step will become available. \* \*\*Step 3\*\* - Configure \`smallstep\` client \* Initialize \`smallstep\` client with the command: .. code-block:: shell step ca bootstrap --ca-url=https://sshproxy.hpc.cineca.it --fingerprint 2ae1543202304d3f434bdc1a2c92eff2cd2b02110206ef06317e70c1c1735ecd A successful configuration will show the message: .. code-block:: shell The authority configuration has been saved in C:\\Users\\m.rossi\\.step\\config\\default.json. PS C:\\Users\\m.rossi> Get-Service -Name ssh-agent \* \*\*Step 4\*\* - Activation of the \`\`ssh-agent\`\` \* On the O.S. Windows 11, the \`\`ssh-agent\`\` may not be active by default. You can verify the activation status with the command: .. code-block:: shell Get-Service -Name ssh-agent The output on \`\`Powershell\`\` should be: .. code-block:: shell Status Name Display Name ------ ---- ------------- Running ssh-agent OpenSSH Authentication Agent \* If the service is not in \`\`running status\`\`, it can be activated with: .. code-block:: shell Start-Service -Name ssh-agent .. note:: If the service will not start after the comamnd \`\`Start-Service -Name ssh-agent\`\`, the user can force the activation by opening a \`\`Powershell\`\` with administration privileges (from the control panel of Windows) and use the following commands: .. code-block:: shell Set-Service -Name ssh-agent -StartupType Auto Start-Service ssh-agent \* \*\*Step 5\*\* - Check the certificate \* Run the following command to get the timed certificate: .. code-block:: shell step ssh login --provisioner cineca-hpc \* Please, refer to the :ref:\`general/access:How to manage authentication \*certificates\*\` section for furter information. .. dropdown:: Windows Subsystem Linux (\*\*WSL\*\*) :animate: fade-in-slide-down :color: light To proceed, open a shell and install \`\`Step\`\` by carefully following the installation instructions for the Linux environment. \*\*WSL\*\* does not support separate tabs, and any new window will not recognize a previously issued Step certificate by default. To avoid generating a new timed certificate for each session, we recommend adding an automatic certificate verification process in your WSL .bashrc file. This process can utilize variables initialized by the \`\`eval $(ssh-agent)\`\` command and redirect them to an appropriate text file. For example: .. code-block:: bash if \[ -f ~/.bash\_agent \]; then . ~/.bash\_agent fi steptest=$(step ssh list --raw ''| step ssh inspect | grep "Valid") if \[ -z "$steptest" \] then eval $(ssh-agent) echo "export SSH\_AUTH\_SOCK=$SSH\_AUTH\_SOCK" > ~/.bash\_agent echo "export SSH\_AGENT\_PID=$SSH\_AGENT\_PID" >> ~/.bash\_agent step ssh login '' --provisioner cineca-hpc fi .. dropdown:: Clients SSH/SFTP under Windows :animate: fade-in-slide-down :color: light There are many SSH or SFTP Clients available for Windows, that are of common usage but are not automatically configured for working with the new 2FA system. It is possible to login with them by exploiting the OpenSSH agent forwarding that can be set by taking advantage of another tool installable on Powershell, that is \`WinSSH-Pageant \`\_. \* \*\*Step 1\*\* - as a prerequisite, complete the configuration for \`\`Powershell\`\` as explained in the dedicated paragraph above. \* \*\*Step 2\*\* Download WinSSH-Pegeant .. code-block:: bash winget install winssh-pageant after downloading, you should find a new executable in the following path: \`\`C:\\Users\\$Env:UserName\\AppData\\Local\\Programs\\WinSSH-Pageant\\winssh-pageant.exe\`\` The variable \`\`$Env:UserName\`\` will be specific for your personal workstation. \* Create an alias to simplify the execution of \`\`winssh-pageant.exe\`\` .. code-block:: bash New-Alias winssh-pageant C:\\Users\\$Env:UserName\\AppData\\Local\\Programs\\WinSSH-Pageant\\winssh-pageant.exe .. important:: Keep in mind though that Powershell keeps an alias alive only until the shell is closed. An easy permanent solution would be to copy the program winssh-pageant.exe to another folder, for example \`\`C:\\Users\\$Env:UserName\\scoop\\shims\`\` that has been already included permanently in the \`\`PATH\`\` variable by the previous installation of step and is therefore recognized by \`\`Powershell\`\` without the need of expliciting the full path. \* \*\*Step 3\*\* \* Launch the WinSSH-pageant with the command: \`\`winssh-pageant --sshpipe\`\` \* Check if \`\`winssh-pageant\`\` is active and works properly .. code-block:: bash Get-Process | Select-String pageant You should expect an output like: .. code-block:: bash System.Diagnostic.Process (winssh-pageant) \* \*\*Step 4\*\* - Create a new certificate .. code-block:: bash step ssh login --provisioner cineca-hpc At this point you can \*\*connect\*\* via SSH/SFTP client by using your preferred client. In the following, we report the proper settings for the most popular clients (Putty, WinSCP, Filezilla, MobaXterm). \*\*Putty:\*\* In the login window, check the category “Connection --> SSH --> Auth” and be sure that the boxes “Attempt authentication using Pageant” and “Allow agent forwarding” are ticked. .. figure:: img/putty\_1.png \*\*WinSCP:\*\* In the login window, from the Advanced settings go to “SSH--> Authentication” and tick the boxes “Attempt authentication using Pageant” and “Allow agent forwarding”. Be sure that the file protocol is set to “SCP”. .. figure:: img/wscp\_1.png .. figure:: img/wscp\_2.png \*\*Note\*\*: It is possible that if you try to edit an already saved site, the ssh-agent won’t be recognized. If this is the case, we recommend to create a new site from scratch and configure it accordingly. The new site can then be saved and will keep working. \*\*Note\*\*: In certain cases, we noted that the procedure may not work at first try, and you will get an error at login even if everything is in order. In most cases, a simple reboot of your workstation solves the problem and the issue will not occur again. \*\*FileZilla:\*\* In your site configuration, be sure that the Protocol is set to “SFTP - SSH File Transfer Protocol” and the Logon type is set to “Normal”. .. figure:: img/filezilla.png \*\*MobaXTerm:\*\* In the upper menu bar with the general options, make sure that in "Settings" → "Configuration" → "SSH" the box "Use external Pageant" is ticked (it should be by default). .. figure:: img/moba\_1.png After that, opening a simple ssh session should be enough. .. figure:: img/moba\_2.png Other SSH/SFTP clients don’t seem to be working with this method and are currently not supported by CINECA (for example BitviseSSH), or haven’t been tested yet. We will keep updating the Userguide when other clients will be proven compatible. How to manage authentication \*certificates\* ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Managing authentication certificates in HPC environments involves ensuring secure access to resources. Users typically generate certificates via an Identity Provider or a local client, like \*Smallstep\*, for secure session authentication. Key operations include: \* Certificate generation: Performed during initial setup or when required by the system. \* Re-generating certificates: Necessary if the current certificate expires or becomes invalid. Users typically log in to the portal or use a client tool to request a new certificate. \* Certificate renewal: Ensures continued access without disruptions, often automated if supported by the system. For issues or detailed procedures, users should consult the documentation or contact: superc@cineca.it. .. dropdown:: How to re-generate the certificate :animate: fade-in-slide-down :color: light .. code-block:: bash $ step ssh login '' --provisioner cineca-hpc .. dropdown:: How to check the presence of a valid certificate :animate: fade-in-slide-down :color: light It is possible to check for the presence of a valid certificate either via ssh-agent or via step with one of the following commands: .. code-block:: bash $ ssh-add -L ecdsa-sha2-nistp256-cert-v01@openssh.com AAAAKGVjZHNhLXNoYTItbmlzdHAyNTYtY2VydC12MDFAb3BlbnNzaC5jb20AAAAgYjJfSnpeTTNrMHB4Lm9yX3YjZWNxXyRxcHM9blRzU1gAAAAIbmlzdHAyNTYAAABBBAJRZ11/PIo0VJknlFMDa5BIaJp/w0OWd95ueZbWlQ4uG92aSZ+K8aKgkyDiOGla3x7l+saVT/pIR+x3zBgvwgkLrbmYufPPVAAAAA EAAAAUbS5tb3Jnb3R0aUBjaW5lY2EuaXQAAAAMAAAACG1tb3Jnb3R0AAAAAGILhpwAAAAAYgv3HAAAAAAAAACCAAAAFXBlcm1pdC1YMTEtZm9yd2FyZGluZwAAAAAAAAAXcGVybWl0LWFnZW50LWZvcndhcmRpbmcAAAAAAAAAFnBlcm1pdC1wb3J0LWZvcndhcmRpbmcAAAAAAAAACnBlcm1pdC1wdHkAAAAAAAAADnBlcm1pdC11c2VyLXJjAAAAAAAAAAAAAABoA AAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAE3K7f5piMLWXDm9c6kd+VAJmBClKXkQ9i/8E1UA9DcBFofX+r9JyBOULZSDkGtr84oqpNX0fa5DMCar3AQp1YAAABkAAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAABJAAAAIDg33ohPQ6BgzV1ATGsSVSbRwrbYa8LprV2EEHk4mMgWAAAAIQCkd8QKYS+zbeyD1nXeuRAXVWJXJeoxMScgDVx2 qqu2Mg== $ step ssh list 256 SHA256:x+QEW8xmDBtRjVRtAukc7v7zKEHef/9joyFP9n/gZtk (ECDSA-CERT) .. dropdown:: How to examine the validity of the current certificate :animate: fade-in-slide-down :color: light .. code-block:: bash $ step ssh list --raw '' | step ssh inspect -: Type: ecdsa-sha2-nistp256-cert-v01@openssh.com user certificate Public key: ECDSA-CERT SHA256:TdhIpD5KFZD37roGYcDstS7180TruOnNgNJeS8eJJPk Signing CA: ECDSA SHA256:e0ZF6AnnUzi0g7Db9nOaXxkEjRq9D6Ka4tV04XqiIgM Key ID: "" Serial: 841532770994081620 Valid: from 2025-05-12T11:55:24 to 2025-05-12T19:55:24 Principals: Critical Options: (none) Extensions: permit-X11-forwarding permit-port-forwarding permit-pty .. dropdown:: How to create a certificate in file format :animate: fade-in-slide-down :color: light If it is necessary to avoid using ssh-agent, you can download your certificate launching the following command in any path of your local PC (we suggest using the ~/.ssh folder): .. code-block:: bash step ssh certificate 'user-email' --provisioner cineca-hpc my\_key You can change \*\*my\_key\*\* with the name you prefer. A passphrase to encrypt the private key is request as input in the shell command line: .. code-block:: bash Please enter the passphrase to encrypt the private key: .. note:: \*\*Three keys\*\* will be generated in the path where you executed the above command. To use the keys to access the cluster you can place the three files in the \`\`~/.ssh folder\`\` (default path), or you can specify \`\`-i \`\` within the ssh command, and enter the passphrase you selected in the previous step: .. code-block:: bash $ ssh -i /path/my\_key @login..cineca.it Enter passphrase for key \`\`my\_key\`\` Access via Secure Shell (\*\*SSH\*\*) --------------------------------- SSH is commonly employed for remote access to a machine, allowing users to execute commands (remote console), run programs, and transfer files securely. On Linux and Mac systems, the SSH client is typically pre-installed. However, on Windows systems, users need to download and install an SSH client. Some popular SSH clients for Windows include \`Powershell \`\_, \`openSSH \`\_, \`Putty \`\_ or \`Tectia \`\_. Connection adopting 2FA procedure does not require to provide password. The access is done via one of the following commands: .. code-block:: bash $ ssh @login.marconi.cineca.it $ ssh @login.g100.cineca.it $ ssh @login.leonardo.cineca.it $ ssh @login.pitagora.cineca.it You can use the option \`\`-X\`\` to enable \*\*X11\*\* display forwarding. .. important:: \* After \*\*12 hours\*\*, a new certificate must be generated using the smallstep client (:ref:\`general/access:How to manage authentication \*certificates\*\`). \* You will login to our systems with one of the two shells: \*\*bash\*\* or \*\*tcsh\*\*. Contact the HPC support (superc@cineca.it) if you want to change your default login shell. \* Login is prevented on systems in which you don't have a budget account. \* We have identified a potential issue for local PC with openssh 8.6 (check with the command \`\`ssh -V\`\`). The solution can be found here in our :ref:\`faq:FAQ\` page. .. dropdown:: Troubleshooting - WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! :animate: fade-in-slide-down :color: warning :chevron: down-up The DNS listed above to access our clusters are aliases pointing at different login nodes. When accessing multiple times on the same cluster, if you end-up on a different login node than the previous session you may get the following error: \*\*"WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!"\*\* the reason is a possible mismatch between the DNS and the node's fingerprint saved in your known\_hosts file. To solve this issue, we sugget the following steps. .. tab-set:: .. tab-item:: Linux/MacOS and Windows WSL Open your terminal, get the 2FA certificate, and run the following command: .. code-block:: bash ssh-keygen -f ~/.ssh/known\_hosts -R ; for keyal in ssh-rsa ecdsa-sha2-nistp256; do for address in ; do ssh-keyscan -t ${keyal} ${address} | sed "s/\\b${address}//g" >> ~/.ssh/known\_hosts; done; done where \`\`\`\` is the domain name system of the cluster, and \`\`\`\` is a wildcard pattern derived from the cluster DNS used to generalize host entries in the known\_hosts file. An example for Leonardo cluster: .. code-block:: bash ssh-keygen -f ~/.ssh/known\_hosts -R login.leonardo.cineca.it; for keyal in ssh-rsa ecdsa-sha2-nistp256; do for address in login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it; do ssh-keyscan -t ${keyal} ${address} | sed "s/\\b${address}/login\*.leonardo.cineca.it/g" >> ~/.ssh/known\_hosts; done; done .. Note:: You can find the previously configured one-line command for each cluster in our :ref:\`faq:FAQ\`. To generalize you can use the following: .. code-block:: bash # Define the main cluster DNS name cluster\_dns=login.leonardo.cineca.it # Define an array of explicit DNS names for each login node clustes\_explicit\_dns=( login01-ext.leonardo.cineca.it login02-ext.leonardo.cineca.it login05-ext.leonardo.cineca.it login07-ext.leonardo.cineca.it ) # Generate a generic DNS pattern by replacing the first '.' with '\*.' # (e.g., "login.leonardo.cineca.it" → "login\*.leonardo.cineca.it") cluster\_generic\_dns="${cluster\_dns/./\*.}" # Remove any existing SSH key entries for the main cluster DNS from known\_hosts if \[ -f ~/.ssh/known\_hosts \] then ssh-keygen -f ~/.ssh/known\_hosts -R ${cluster\_dns} fi # Loop over the key algorithms to scan (e.g., ssh-rsa, ecdsa-sha2-nistp256) for keyal in ssh-rsa ecdsa-sha2-nistp256 do # Loop over all explicit login node DNS entries for address in ${clustes\_explicit\_dns\[@\]} do # Retrieve the SSH public key for the node using ssh-keyscan # Replace the explicit node address with the generic pattern in the output # Append the result to the known\_hosts file ssh-keyscan -t ${keyal} ${address} | sed "s/\\b${address}/${cluster\_generic\_dns}/g" >> ~/.ssh/known\_hosts done done .. tab-item:: Windows Powershell and other SSH clients Open your \*\*Powershell\*\* terminal, get the 2FA certificate, and run the following command: .. code-block:: powershell ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R ""; foreach ($keyal in "ssh-rsa", "ecdsa-sha2-nistp256") { foreach ($address in ) { ssh-keyscan -t $keyal $address | ForEach-Object { $\_ -replace "\\b$address\\b", "" } >> "$HOME\\.ssh\\known\_hosts" } } where \`\`\`\` is the domain name system of the cluster, and \`\`\`\` is the list of the explicit DNS of the login nodes. An example for Leonardo cluster: .. code-block:: powershell ssh-keygen -f "$HOME\\.ssh\\known\_hosts" -R "login.leonardo.cineca.it"; foreach ($keyal in "ssh-rsa", "ecdsa-sha2-nistp256") { foreach ($address in "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it") { ssh-keyscan -t $keyal $address | ForEach-Object { $\_ -replace "$address", "login\*.leonardo.cineca.it" } >> "$HOME\\.ssh\\known\_hosts" } } .. Note:: You can find the previously configured one-line command for each cluster in our :ref:\`faq:FAQ\`. To generalize you can use the following: .. code-block:: powershell # Define the main cluster DNS name $cluster\_dns = "login.leonardo.cineca.it" # Define an array of explicit DNS names for each login node $clusters\_explicit\_dns = @( "login01-ext.leonardo.cineca.it", "login02-ext.leonardo.cineca.it", "login05-ext.leonardo.cineca.it", "login07-ext.leonardo.cineca.it" ) # Generate a generic DNS pattern by replacing the first '.' with '\*.' # (e.g., "login.leonardo.cineca.it" → "login\*.leonardo.cineca.it") $cluster\_generic\_dns = $cluster\_dns -replace '\\.', '\*.', 1 # Path to known\_hosts $known\_hosts\_path = "$HOME\\.ssh\\known\_hosts" # Remove any existing SSH key entries for the main cluster DNS from known\_hosts if (Test-Path $known\_hosts\_path) { ssh-keygen -f $known\_hosts\_path -R $cluster\_dns | Out-Null } # Loop over the key algorithms to scan $keyal\_list = @("ssh-rsa", "ecdsa-sha2-nistp256") foreach ($keyal in $keyal\_list) { foreach ($address in $clusters\_explicit\_dns) { # Retrieve the SSH public key for the node $keyscan\_output = ssh-keyscan -t $keyal $address 2>$null # Replace the explicit node address with the generic pattern $updated\_output = $keyscan\_output -replace \[regex\]::Escape($address), $cluster\_generic\_dns # Append the result to the known\_hosts file Add-Content -Path $known\_hosts\_path -Value $updated\_output } } Access via Remote Visualization (\*\*RCM\*\*) ----------------------------------------- Remote visualization has become popular as an HPC service since it allows to: \* \*\*visualize\*\* the data produced on HPC infrastructure (scientific visualization) \* \*\*analize\*\* and \*\*inspect\*\* data directy on the HPC infrastructure. \* \*\*debug\*\* and \*\*optmization\*\* of codes via GUI tools directly on the HPC infrastructure. All the aforementioned use cases can take advantage of use applications on the \*\*server side\*\*. The requirements to access to this service are the same for the access to HPC infrastructure. The remote visualization service at Cineca is provided through the Remote Connection Manager (\*\*RCM\*\*) application. Using this tool you can graphically inspect your data without moving them to your local work station. .. important:: It can be used by any user with valid credentials to access CINECA clusters. .. toctree:: :maxdepth: 1 ../rcm/rcm Continuous Integration (\*\*CI/CD\*\*) at CINECA -------------------------------------------- Continuous Integration, continuous delivery and deployment, also known as \*\*CI/CD\*\*, is the practice of a constant monitoring of the code development through the activation of an automatic pipeline. Every time a developer applies a change to the code, the automatic pipeline validates it by building the code and running some simple tests (typically unit tests). CINECA has activated a new service where to run your Continuous Integration (CI/CD) pipelines on CINECA clusters, based on CINECA GitLab service. On \`Gitlab website \`\_ there is a detailed documentation about CI/CD. This service is active on Galileo100, and is at the moment in an experimental phase. .. dropdown:: How to use it :animate: fade-in-slide-down :color: light You can activate our CI/CD service in projects defined into our \`GitLab instance \`\_. If you are already a CINECA HPC user, you can access the CINECA GitLab using the same credentials. If you are interested and you are still not an HPC User you can find \`here \`\_ the instructions on how to get access. Once logged to CINECA GitLab, you can activate the CI/CD service by enabling shared runners that pick up and execute your CI/CD pipeline on our cluster. They can be enabled as in the following: 1. From your project's web page, select "\*Settings\*" and then on "\*CI/CD\*", from the menu on the left; 2. Once in the web page, expand the section "\*Runners\*"; 3. Activate the switch under "\*Enable shared runners for this project\*" on the right, in the "\*Shared Runners\*" right column. The \*shared runners\* are listed in that section along with blue labels specifying the \*tags\* associated to them. Now shared runners are available to your CI/CD pipeline. The CI/CD pipeline has to be specified inside the \`\`.gitlab-ci.yml\`\` file through tags (see \`Gitlab documentation \`\_ for how to create and manage pipelines). \*\*IMPORTANT\*\*: There are \*\*two different kind of runners\*\*. You have to identify \*\*which runner\*\* you would like to run your pipeline by \*\*specifying one or more tags\*\* summarized in the table at the bottom of the page. \*\*IMPORTANT\*\*: If you do not select any tag, the pipeline \*\*will never start\*\*. We set a \*\*time limit\*\* for the execution of each single job of a given pipeline that cannot last for more than \*\*10 minutes\*\*. .. dropdown:: Runners description :animate: fade-in-slide-down :color: light All shared runners are based on \`docker images \`\_, so in your CI/CD pipeline you can optionally choose in which \*\*container image\*\* your pipeline job will run. You will find 4 distinct shared runners, consisting of: - \*\*2 CPU-only\*\* runners, with access to up to 24 CPUs each. Jobs are executed in concurrent execution. (specific tags: \*\*x86\_64, cpu, docker\*\*) - \*\*2 CPU+GPU\*\* runners, limited to run 1 CI job each at the time. Each runner has access to a dedicated GPU. \*\*No concurrent execution\*\* is allowed on these runners. (specific tags: \*\*x86\_64, docker, nvidia-sm70, nvidia-volta, nvidia-cuda\*\*) All shared runners run on a dedicated node of Galileo100 with \*\*Intel x86\_64 architectures\*\* (2 x CPU Intel CascadeLake 8260 processors with 24 cores each, 2.4 GHz, 384GB RAM). GPU runners make use of \*\*Nvidia\*\* V100 GPUs. .. dropdown:: Summary :animate: fade-in-slide-down :color: light Below we summarize the runners and the tags needed to select the correct one. .. list-table:: :widths: 30 45 45 :header-rows: 1 \* - \*\*Runners\*\* - \*\*Tags\*\* - \*\*Notes\*\* \* - 2 CPU-only - x86\_64,cpu,docker - Up to 24 cpus each. Concurrent execution \* - 2 CPU+GPU - x86\_64, docker, nvidia-sm70, nvidia-volta, nvidia-cuda - Each runner has a dedicated GPU. No concurrent execution. \*\*IMPORTANT\*\*: \*\*If you do not select any tag\*\*, the pipeline \*\*will never start\*\*. Access via X.509 certificate ---------------------------- An X.509 certificate is issued by a trusted Certificate Authority (CA), which verifies the user's identity and ensures the certificate is both valid and associated with a real individual. It is used as an authentication mechanism, serving as an alternative to traditional username/password credentials, and helps avoid the need to replicate user accounts across systems. When connecting to a service, the user's certificate is mapped to a local account, under which all commands and operations are executed. .. important:: \*\*Access to CINECA clusters no longer requires or supports X.509 certificates for authentication.\*\* However, X.509 certificates may still be necessary for accessing certain external services or resources, such as data repositories, collaboration platforms, or grid computing infrastructures. The following describes the procedure for obtaining an X.509 certificate and generating a proxy certificate for temporary use. \*\*How to get your X.509 certificate\*\* - Users in need of an X.509 certificate can visit the \`HARICA website \`\_. Academic users can select \*Academic Login\* and authenticate using their institutional credentials. - In the dashboard's left-hand menu, click on \*IGTF Certificate\*, then select \*GÉANT Personal Authentication\*. - Review and accept the terms and conditions, then click \*Submit Request\*. .. figure:: img/X509\_1.png - Under \*Ready Certificates\*, click \*Enroll your Certificate\*. Choose your preferred algorithm (RSA or ECDF), then click \*Enroll Certificate\* again. After enrollment, click \*Download\*. .. figure:: img/X509\_2.png - A \`\`.p12\`\` file (containing your personal certificate and private key) will be downloaded. \*\*How to Use Your X.509 Certificate (Browser and Command Line)\*\* Once you have downloaded your \`\`.p12\`\` certificate file, you can either: - Import it into your browser for web-based authentication. - Convert it into PEM format (\`\`cert.pem\`\` and \`\`key.pem\`\`) to use it with command-line tools such as \`\`grid-proxy-init\`\`, \`\`voms-proxy-init\`\`, or \`\`globus-url-copy\`\`. .. dropdown:: Using the .p12 Certificate in Your Browser :animate: fade-in-slide-down :color: light \*Firefox\*: - Go to Settings → Privacy & Security → Certificates → View Certificates. - Open the Your Certificates tab. - Click Import, then select your .p12 file. - Enter the password used to protect the certificate. Your certificate is now ready for web authentication. \*Chrome / Edge / Safari\*: These browsers use the system certificate store. - On Linux (GNOME): Use the Passwords and Keys application (Seahorse) → Import your .p12 file. - On macOS: Double-click the .p12 file to open Keychain Access → import it into the login or system keychain. - On Windows: Double-click the .p12 file → follow the Certificate Import Wizard, choose "Current User", and confirm the installation. .. dropdown:: Converting .p12 to cert.pem and key.pem for Command-Line Use :animate: fade-in-slide-down :color: light To use your certificate with command-line tools, you’ll need to copy the \`\`cert.p12\`\` file in your \`\`$HOME\`\`, create the directory \`\`$HOME/.globus\`\` and finally extract the certificate and private key from the \`\`.p12\`\` file using OpenSSL: \*\*Extract the private key:\*\* .. code-block:: bash $ mkdir $HOME/.globus $ openssl pkcs12 -nocerts -in cert.p12 -out $HOME/.globus/userkey.pem \*\*Extract the User certificate:\*\* .. code-block:: bash $ openssl pkcs12 -clcerts -nokeys -in cert.p12 -out $HOME/.globus/usercert.pem \*\*Protect your keys:\*\* .. code-block:: bash $ chmod 600 $HOME/.globus/userkey.pem $HOME/.globus/usercert.pem --- # Cineca-hpyc and Cineca-ai modules — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Environment and Customization](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html) * Cineca-hpyc and Cineca-ai modules * [View page source](https://docs.hpc.cineca.it/_sources/hpc/hpc_cineca-ai-hpyc.rst.txt) * * * Cineca-hpyc and Cineca-ai modules[](https://docs.hpc.cineca.it/hpc/hpc_cineca-ai-hpyc.html#cineca-hpyc-and-cineca-ai-modules "Link to this heading") ====================================================================================================================================================== Cineca-hpyc and Cineca-ai are collection of python and artifitial intelligence packages, respectively, optimized for Cineca’s clusters. The cineca-hpyc module[](https://docs.hpc.cineca.it/hpc/hpc_cineca-ai-hpyc.html#the-cineca-hpyc-module "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------- CINECA-HPyC is a collection of Python scientific packages optimized for CINECA HPC clusters. List of principal packages includes: > * numpy > > * scipy > > * pandas > > * numexpr > > * mpi4py > > * Cython > > * pythran > > * joblib > > * matplotlib > > * IPython > > * notebook > To see the complete list of all the python packages available and check their versions: $ module load cineca-hpyc/ \# list all available installations $ python \-m pip list \# To use a specific pakage $ python \-c "import " \# To install an additional package $ pip install Copy to clipboard Once you loaded the cince-hpyc module, some of the main common packages of HPC enviroment are automatically loaded (e.g. cuda, openMPI). Here there is an example about how to import and use pandas package present in cineca-hpyc module: $ module load cineca-hpyc/ $ python \-c "import pandas" \# You can start a python interactive section and use pandas $ python Python 3.8.12 (default, Jul 29 2022, 16:25:49) \[GCC 10.2.0\] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pandas as pd >>> df \= pd.DataFrame({"col1": \["sap", "hi"\], "col2": \[3, 4\]}) >>> print (df) col1 col2 0 sap 3 1 hi 4 Copy to clipboard Note * On Galileo 100 you need to use autoload in order to load the cineca-hpyc module `module load autoload cineca-hpyc/`. If you wish to install additional python packages to use together with the cineca-hpyc suite, you can create a personal virtual environment and install what you need in the following way: $ module load cineca-hpyc/ $ python \-m venv my\_env \--system-site-packages $ source my\_env/bin/activate $ pip install \# Once you finish to work with your env $ deactivate Copy to clipboard Note * my\_env: choose an arbitrary name for your personal virtual env. * It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. * the –system-site-packages flag gives the virtual environment access to the system site-packages directory (otherwise you cannot access the cineca-hpyc environment). The cineca-ai module[](https://docs.hpc.cineca.it/hpc/hpc_cineca-ai-hpyc.html#the-cineca-ai-module "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------- The bulk of the “cineca-ai” package, provided by the deeplrn profile, includes (for example) Tensorflow, Pytorch, XGBoost, and other related packages and dependencies. This module has been personalized by CINECA AI experts. You can find different cineca-ai modules in profile/deeplrn. For a complete list load the module and launch the “python -m pip list” command. The CINECA AI project can be used in several ways, depending on the method more suited to your needs and on the availability of conda/pip packages. The way to use the installations of the cineca-ai environment goes through the loading of the module: $ module load profile/deeplrn \# To see the available versions $ module av cineca-ai \# Select a verion and load it $ module load cineca-ai/ \# list all available python installations of cineca-ai $ python \-m pip list \# use a specific package $ python \-c "import " Copy to clipboard If you need to use a package not included in the list of those provided by the cineca-ai modules, you can always rely on the cineca-ai environment for the dependencies and install what you need within a personal virtual environment and/or a conda environment. **Installing additional packages within a python virtual environment** If you wish to install additional python packages to use together with the cineca-ai suite, you can create a personal virtual env and install what you need in the following way : \# create the virtual env loading cineca-ai module $ module load profile/deeplrn $ module av cineca-ai $ module load cineca-ai/ $ python \-m venv \--system-site-packages \# activate the created virtual env to install your python packages. $ source my\_env/bin/activate $ pip list $ pip install \# Once you finish to work with your env $ deactivate Copy to clipboard Note * my\_env: choose an arbitrary name for your personal virtual env. * It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. * the –system-site-packages flag gives the virtual environment access to the system site-packages directory (otherwise you cannot access the cineca-ai environment). * to test the installation launch: python -c “import ”. * to use the installed package, just source your env (source /bin/activate): you will access your packages AND those of the cineca-ai environment. **Installing additional packages within a conda virtual environment** If you wish to install additional conda packages to use together with the cineca-ai suite, you can create your conda virtual environment and install what you need in the following way (we strongly suggest using a python venv if possible) : \# create the conda virtual env loading cineca-ai module $ module load anaconda3/deeplrn $ module load profile/deeplrn $ module av cineca-ai $ module load cineca-ai/ $ conda create \-p /my\_env \-c conda-forge \--override-channels \# activate the created conda virtual env to access cineca packages and install your conda packages. $ conda activate /my\_env $ python \-m pip list $ python \-m pip install \# Once you finish to work with your env $ conda deactivate Copy to clipboard Note * my\_env: choose an arbitrary name for your personal virtual env. * It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. * the –system-site-packages flag gives the virtual environment access to the system site-packages directory (otherwise you cannot access the cineca-ai environment). --- # Matlab — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Software](https://docs.hpc.cineca.it/hpc/hpc_software.html) * Matlab * [View page source](https://docs.hpc.cineca.it/_sources/hpc/software/matlab.rst.txt) * * * Matlab[](https://docs.hpc.cineca.it/hpc/software/matlab.html#matlab "Link to this heading") ============================================================================================= The following guide describes how to use MATLAB to submit jobs on CINECA clusters, retrieve results and debug errors. MATLAB is available on [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo) and [Galileo100](https://docs.hpc.cineca.it/hpc/galileo.html#galileo100) clusters. Pre-requisites[](https://docs.hpc.cineca.it/hpc/software/matlab.html#pre-requisites "Link to this heading") ------------------------------------------------------------------------------------------------------------- To use MATLAB on CINECA HPC’s environment, please check the following pre-requisites: > 1. You and your collaborators have **a valid account** defined on HPC cluster, see [How to become a User](https://docs.hpc.cineca.it/general/users_account.html#how-to-become-a-user) > . > > 2. You have access to a **valid MATLAB license** to be used on CINECA HPC clusters. > Thanks to an agreement with MathWorks, **CINECA provides several MATLAB licenses** through its internal license server that can be used on CINECA clusters. Usage of the CINECA MATLAB licenses is allowed **exclusively for Open Science** (non-commercial) activities. In case you are interested in using those licenses and you declare us that your activity is devoted to Open Science, please write to [superc@cineca.it](mailto:superc%40cineca.it) to be enabled to use CINECA licenses. For all the other cases, or in case you would like to use your personal/department/university license, we need to connect your license server, where the Flex-LM license is installed, with the CINECA compute nodes of the cluster. Detailed instructions can be found in [How to connect your license server](https://docs.hpc.cineca.it/hpc/hpc_software.html#how-to-connect-your-license-server) . Configuration[](https://docs.hpc.cineca.it/hpc/software/matlab.html#configuration "Link to this heading") ----------------------------------------------------------------------------------------------------------- It is possible to configure MATLAB to submit jobs on CINECA clusters directly from your local MATLAB installation (Running MATLAB from your Desktop) or from login nodes of CINECA clusters (Running MATLAB on the HPC Cluster). ### Running MATLAB from your Desktop[](https://docs.hpc.cineca.it/hpc/software/matlab.html#running-matlab-from-your-desktop "Link to this heading") It is possible to submit MATLAB jobs to the compute nodes of a CINECA cluster directly from your local MATLAB installation. This setup needs to be done once per cluster, per version of MATLAB installed on your machine. **Supported versions:** R2022b, R2023a, R2023b, R2024a, R2024b, R2025a Please check that your local MATLAB installed version **matches one of the supported versions**. Moreover, you need to to have the “**Parallel Computing Toolbox**” installed on your computer. To check for it, you can run the following commands on MATLAB: \>> license('test','distrib\_computing\_toolbox') \>> ~isempty(ver('parallel')) Copy to clipboard if answer is 1 for both, it means that the Parallel toolbox is correctly installed. Otherwise you need to install it. **Download** the CINECA MATLAB support package from here ([`cineca.Desktop.zip`](https://docs.hpc.cineca.it/_downloads/0d6ffeca1acdc69a948bead574f8df88/cineca.Desktop.zip) ) (**Last update: 01 August 2025**). It contains a set of scripts (Integration Scripts) needed to configure MATLAB to launch jobs remotely. Unzip the file in the location returned by the MATLAB command: \>> userpath Copy to clipboard For different solutions you can refer to this [MathWorks dedicated User Guide page](https://it.mathworks.com/help/matlab/matlab_env/what-is-the-matlab-search-path.html) . **Only in the case you are going to use your personal/department/university license**, you will also have to modify the following files inside the cluster folder you find in the Integration Scripts: `communicatingSubmitFcn.m` and `independentSubmitFcn.m` by adding a MLM\_LICENSE\_FILE line indicating the port and the IP of your license server as in the following: 'PARALLEL\_SERVER\_DEBUG', enableDebug; ... 'MLM\_LICENSE\_FILE', '@'; ... 'MLM\_WEB\_LICENSE', environmentProperties.UseMathworksHostedLicensing; ... Copy to clipboard This step is **not needed** if you are going to use CINECA licenses. Finally open MATLAB and create a new cluster profile launching the command \>> configCluster Copy to clipboard Submission to the cluster requires SSH credentials. You will be prompted for the cluster, your username and password. Jobs will run on the cluster rather than on the local machine. Important You can access only clusters where you have an active budget account. To manage the local cluster configuration in the top menu select “Parallel”, then “Create and Manage Clusters…” A window will be opened where you can modify the Additional Properties of your configuration based on your needs (See [Configuring Jobs](https://docs.hpc.cineca.it/hpc/software/matlab.html#configuring-jobs) Section about a description of the Available Properties). ### Running MATLAB on the HPC Cluster[](https://docs.hpc.cineca.it/hpc/software/matlab.html#running-matlab-on-the-hpc-cluster "Link to this heading") Alternatively to Remote submission, you can also launch MATLAB jobs directly from login nodes of CINECA clusters. Log-in to the cluster and load the MATLAB module: $ module load profile/eng $ module load matlab/ Copy to clipboard There may be available more than one MATLAB version. You can check for it through [The modmap command](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#the-modmap-command) section. Take care to select the last valid release for your license. Configure MATLAB to run parallel jobs. This only needs to be called once per version of MATLAB and once per user. $ configCluster.sh Copy to clipboard in alternative you can start MATLAB without desktop $ matlab \-nodisplay Copy to clipboard then launch the command \>> configCluster Copy to clipboard A new profile will be created (i.e. ‘galileo100 R2024b’ on Galileo100). Jobs will run across multiple nodes on the cluster rathen than on the host machine. You can check the list of available profiles: \>> \[ ALLPROFILES,DEFAULTPROFILE\] \= parallel.clusterProfiles ALLPROFILES = 1x2 cell array {'galileo100 R2024b'} {'local'} DEFAULTPROFILE= 'galileo100 R2024b' Copy to clipboard Please check that the `DEFAULTPROFILE` is not set to ‘local’. The ‘local’ profile is not allowed on our cluster, so don’t use it. If it is set to ‘local’ you have to set for example \>> DEFAULTPROFILE\='galileo100 R2024b' Copy to clipboard on Galileo100 and similarly on Leonardo. Configuring Jobs[](https://docs.hpc.cineca.it/hpc/software/matlab.html#configuring-jobs "Link to this heading") ----------------------------------------------------------------------------------------------------------------- Prior to submitting the job, various parameters have to be specified in order to be passed to jobs, such as queue, username, e-mail, etc. Note Any parameters specified using the below workflow will be persistent between MATLAB sessions if saved at the end of the configuration. Before specifying any parameters, you will need to obtain a handle to the cluster object. \>> % Get a handle to the cluster \>> c \= parcluster; Copy to clipboard You are now **required** to specify an Account Name, a Queue Name and the Wall Time (visit [Budget and Accounting](https://docs.hpc.cineca.it/hpc/hpc_intro.html#budget-and-accounting) to see how to retrieve your Budget Account Name using the saldo command) \>> % Specify an Account to use for MATLAB jobs \>> c.AdditionalProperties.AccountName \= 'account\_name'; \>> % Specify a queue to use for MATLAB jobs \>> c.AdditionalProperties.Partition \= 'partition-name'; \>> % Specify the walltime (e.g. 5 hours) \>> c.AdditionalProperties.WallTime \= '05:00:00'; Copy to clipboard On Leonardo cluster there are two partitions: ‘boost\_usr\_prod’ to access GPU nodes and ‘dcgp\_usr\_prod’ to access CPU nodes. You can find additional info on the [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo) dedicated pages. For Galileo100 cluster the main partition is ‘g100\_usr\_prod’. In [Galileo100](https://docs.hpc.cineca.it/hpc/galileo.html#galileo100) dedicated page you can find other possible Partitions and QOS available allowing for different combinations of nodes, walltime and priority. You can specify other **additional** (not-mandatory) parameters along with your job. \>> % Specify QoS \>> c.AdditionalProperties.QoS \= 'name-of-qos'; \>> % Specify processor cores per node. Default is 32 for Leonardo GPU nodes and 112 on Leonardo CPU nodes; 18 for Marconi and 48 for Galileo100. \>> c.AdditionalProperties.ProcsPerNode \= 18; \>> % specify the number of GPUsPerNode. Valid only on Leonardo GPU partition \>> c.AdditionalProperties.GPUsPerNode \= 1; \>> % Specify memory to use for MATLAB jobs, per core (default: 4gb) \>> c.AdditionalProperties.MemUsage \= '6gb'; \>> % Require node exclusivity \>> c.AdditionalProperties.RequireExclusiveNode \= true; \>> % Request to use a reservation \>> c.AdditionalProperties.Reservation \= 'name-of-reservation'; \>> % Specify e-mail address to receive notifications about your job \>> c.AdditionalProperties.EmailAddress \= ‘test@foo.com’; \>> % Turn onthe Debug Message. Default is off (logical boolean true/false). \>> c.AdditionalProperties.DebugMessagesTurnedOn \= true; \>> % Specify the tmpfs dimension, for Leonardo CPU partition (default: 10GB) \>> c.AdditionalProperties.Tmpfs \= '20G'; Copy to clipboard To check for the values of the current configuration options, call the AdditionalProperties without semicolon \>> % To view current configurations \>> c.AdditionalProperties Copy to clipboard To clear a value, assign the property an empty value (‘’, \[\], or false). \>> % To clear a configuration that takes a string as input \>> c.AdditionalProperties.EmailAddress \= ‘ ’; Copy to clipboard To save a profile, with your configuration so you will find it in future sessions \>> c.saveProfile; Copy to clipboard Serial Jobs[](https://docs.hpc.cineca.it/hpc/software/matlab.html#serial-jobs "Link to this heading") ------------------------------------------------------------------------------------------------------- ### Interactive Jobs[](https://docs.hpc.cineca.it/hpc/software/matlab.html#interactive-jobs "Link to this heading") To run an interactive pool job on the cluster, continue to use parpool as before. Note This is valid ONLY when running MATLAB on the cluster \>> % Get a handle to the cluster \>> c \= parcluster; Copy to clipboard Rather than running a local pool on the host machine, the pool can now run across multiple nodes on the cluster. \>> % Run a parfor over 1000 iterations \>> parfor idx \= 1:1000 \>> a(idx) \= rand; \>> end Copy to clipboard Delete the pool when it’s no longer needed. \>> %Delete the pool \>> pool.delete Copy to clipboard ### Independent Batch Jobs[](https://docs.hpc.cineca.it/hpc/software/matlab.html#independent-batch-jobs "Link to this heading") Use the batch command to submit asynchronous jobs to the cluster. The batch command will return a job object which is used to access the output of the submitted job. See the MATLAB documentation for more help on [batch](https://www.mathworks.com/help/parallel-computing/batch.html) .. \>> % Get a handle to the cluster \>> c \= parcluster; Copy to clipboard Submit job to query where MATLAB is running on the cluster \>> j \= c.batch(@pwd, 1, {}); Copy to clipboard Query job for state: queued | running | finished \>> j.State Copy to clipboard If state is finished, fetch results \>> j.fetchOutputs{:} Copy to clipboard or \>> fetchOutputs(j) Copy to clipboard Display the diary \>> diary(j) Copy to clipboard Delete the job after results are no longer needed \>> j.delete; Copy to clipboard To retrieve a list of currently running or completed jobs, call parcluster to retrieve the cluster object. The cluster object stores an array of jobs that were run, are running, or are queued to run. This allows us to fetch the results of completed jobs. Retrieve and view the list of jobs as shown below. \>> c \= parcluster; \>> jobs \= c.Jobs; \>> % Get a handle to the second job in the list \>> job2 \= c.Jobs(2); Copy to clipboard Once we’ve identified the job we want, we can retrieve the results as we’ve done previously. `fetchOutputs` is used to retrieve function output arguments; if using batch with a script, use load instead. Data that has been written to files on the cluster needs be retrieved directly from the file system. To view results of a previously completed job: \>> % Fetch results for job with ID 2 \>> j2.fetchOutputs{:} Copy to clipboard Note You can view a list of your jobs, as well as their IDs, using the above c.Jobs command. If the job produces an error view the error log file \>> c.getDebugLog(j.Tasks(1)) Copy to clipboard Note When submitting independent jobs, with multiple tasks, you will have to specify the task number. Parallel Jobs[](https://docs.hpc.cineca.it/hpc/software/matlab.html#parallel-jobs "Link to this heading") ----------------------------------------------------------------------------------------------------------- ### Interactive Jobs[](https://docs.hpc.cineca.it/hpc/software/matlab.html#id1 "Link to this heading") To run an interactive pool job on the cluster, you can use parpool. Valid only on login nodes. \>> % Get a handle to the cluster \>> c \= parcluster; Copy to clipboard Open a pool of 64 workers on the cluster \>> pool \= c.parpool(64); Copy to clipboard Rather than running local on the local machine, the pool can now run across multiple nodes on the cluster. \>> % Run a parfor over 1000 iterations \>> parfor idx \= 1:1000 a(idx) = rand(); end Copy to clipboard Once we’re done with the pool, delete it. \>> % Delete the pool \>> pool.delete; Copy to clipboard ### Batch Jobs[](https://docs.hpc.cineca.it/hpc/software/matlab.html#batch-jobs "Link to this heading") Users can also submit parallel workflows remotely from their local MATLAB installation on their PC with batch. The following example are available at the following directory available after loaded the module CIN\_EXAMPLE\=/cineca/prod/opt/tools/matlab/CINECA\_example Copy to clipboard **Parallel\_example.m** Let’s use the following example for a parallel job. function t \= parallel\_example(iter) if nargin\==0, iter \= 16; end disp('Start sim') t0 \= tic; parfor idx \= 1:iter A(idx) ) idc; pause(2) end t \= toc(t0); disp('Sim completed.') Copy to clipboard We will use the batch command again, but since we’re running a parallel job, we’ll also specify a MATLAB Pool. \>> % Get a handle to the cluster \>> c \= parcluster; \>> % Submit a batch pool job using 4 workers for 16 iterations \>> j \= c.batch(@parallel\_example, 1, {}, ‘Pool’, 4); Copy to clipboard For more info on the batch commands, please see the [MATLAB on-line guide](https://www.mathworks.com/help/distcomp/batch.html?s_tid=doc_ta) . \>> % View current job status \>> j.State \>> % Fetch the results after a finished state is retrieved \>> j.fetchOutputs{:} ans = 15.5328 \>> % Display the diary \>> diary(j) The job ran in 15.53 sec. using 4 workers. Copy to clipboard **Note that these jobs will always request N+1 cores for your job**, since one worker is required to manage the batch job and pool of workers. For example, a job that needs eight workers will consume nine CPU cores. We’ll run the same simulation, but increase the Pool size. This time, to retrieve the results at a later time, we’ll keep track of the job ID. **NOTE:** For some applications, there will be a diminishing return when allocating too many workers, as the overhead may exceed computation time. \>> % Get a handle to the cluster \>> c \= parcluster; \>> % Submit a batch pool job using 8 workers for 16 simulations \>> j \= c.batch(@parallel\_example, 1, {}, ‘Pool’, 8); \>> % Get the job ID \>> id \= j.ID Id = 4 \>> % Clear workspace, as though we quit MATLAB \>> clear j Copy to clipboard Once we have a handle to the cluster, we’ll call the `findJob` method to search for the job with the specified job ID. \>> % Get a handle to the cluster \>> c \= parcluster; \>> % Find the old job \>> j \= c.findJob(‘ID’, 4); \>> % Retrieve the state of the job \>> j.State ans = finished \>> % Fetch the results \>> j.fetchOutputs{:} ans = 6.4488 \>> % If necessary, retrieve output/error log file \>> c.getDebugLog(j) Copy to clipboard The job now runs 6.4488 seconds using 8 workers. Run code with different number of workers to determine the ideal number to use. **hpccLinpack.m** This example is taken from $MATLAB\_HOME/toolbox/distcomp/examples/benchmark/hpcchallenge/ Copy to clipboard It is an implementation of the HPCC Global HPL benchmark \>> function perf \= hpccLinpack( m ) Copy to clipboard The function input is the size of the real matrix m-by-m to be inverted. The outputs is perf, performance in gigaflops Start to submit on 1 core, with m=1024: \>> j \= c.batch(@hpccLinpack, 1, {1024}, 'Pool', 1) Data size: 0.007812 GB Performance: 1.576476 GFlops Copy to clipboard Repeat on one full node on Marconi \>> j \= c.batch(@hpccLinpack, 1, {1024}, 'Pool', 35) Data size: 0.007812 GB Performance: 0.311111 GFlops Copy to clipboard Increase the size of the matrix, \>> j \= c.batch(@hpccLinpack, 1, {2048}, 'Pool', 35) Data size: 0.031250 GB Performance: 2.466961 GFlops \>> j \= c.batch(@hpccLinpack, 1, {4096}, 'Pool', 35) Data size: 0.125000 GB Performance: 47.951919 GFlops \>> j \= c.batch(@hpccLinpack, 1, {8192}, 'Pool', 71) Data size: 0.500000 GB Performance: 86.003520 GFlops .. .. \>> j \= c.batch(@hpccLinpack, 1, {16384}, 'Pool', 35) Data size: 2.000000 GB Performance: 356.687648 GFlops Copy to clipboard Debugging[](https://docs.hpc.cineca.it/hpc/software/matlab.html#debugging "Link to this heading") --------------------------------------------------------------------------------------------------- If a serial job produces an error, we can call the getDebugLog method to view the error log file. \>> j.Parent.getDebugLog(j.Tasks(1)) Copy to clipboard When submitting independent jobs, with multiple tasks, you will have to specify the task number. For Pool jobs, do not deference into the job object. \>> j.Parent.getDebugLog(j) Copy to clipboard The scheduler job ID can be derived by calling schedID \>> schedID(j) ans = 25539 Copy to clipboard To learn More[](https://docs.hpc.cineca.it/hpc/software/matlab.html#to-learn-more "Link to this heading") ----------------------------------------------------------------------------------------------------------- To learn more about the MATLAB Parallel Computing Toolbox, check out these resources: > * [`Hands-On Wokshop@CINECA`](https://docs.hpc.cineca.it/_downloads/d9422bafe3d85efe3431ca9e9ee804b8/MATLAB_PCT_handsOnWorkshop.pdf) > > * [`Exercises Workshop Day 1`](https://docs.hpc.cineca.it/_downloads/e9d67653963fb2c96eebb800010e75d6/Matlab_PCT_Workshop.7z) > > * [`Exercises Workshop Day 2`](https://docs.hpc.cineca.it/_downloads/a6c530363d371d918067eff3a1169ded/MATLAB_exercise.7z) > > * [Parallel Computing Coding Examples](http://www.mathworks.com/products/parallel-computing/code-examples.html) > > * [Parallel Computing Documentation](http://www.mathworks.com/help/distcomp/index.html) > > * [Parallel Computing Overview](http://www.mathworks.com/products/parallel-computing/index.html) > > * [Parallel Computing Tutorials](http://www.mathworks.com/products/parallel-computing/tutorials.html) > > * [Parallel Computing Videos](http://www.mathworks.com/products/parallel-computing/videos.html) > > * [Parallel Computing Webinars](http://www.mathworks.com/products/parallel-computing/webinars.html) > Parallel Computing Benchmark and Performance[](https://docs.hpc.cineca.it/hpc/software/matlab.html#parallel-computing-benchmark-and-performance "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > * [Benchmarking Parfor Performance Using a Simple Example: Game of Blackjack](https://it.mathworks.com/help/distcomp/examples/simple-benchmarking-of-parfor-using-blackjack.html) > > * [Resource Contention in Task Parallel Problems](https://it.mathworks.com/help/distcomp/examples/resource-contention-in-task-parallel-problems.html) > > * [Benchmarking Distributed Jobs (Task Parallel Applications on the Cluster)](https://it.mathworks.com/help/distcomp/examples/benchmarking-independent-jobs-on-the-cluster.html) > > * [Benchmarking Parallel “" Operator (Ab)](https://it.mathworks.com/help/distcomp/examples/benchmarking-a-b.html) > > * [Profiling Load Unbalanced Distributed Arrays in Data Parallel Applications](https://it.mathworks.com/help/distcomp/examples/profiling-load-unbalanced-codistributed-arrays.html) > > * [Profiling Explicit Parallel Communication while using Message Passing Functions in MATLAB](https://it.mathworks.com/help/distcomp/examples/profiling-explicit-parallel-communication.html) > --- # QuantumESPRESSO — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Software](https://docs.hpc.cineca.it/hpc/hpc_software.html) * QuantumESPRESSO * [View page source](https://docs.hpc.cineca.it/_sources/hpc/software/qe.rst.txt) * * * QuantumESPRESSO[](https://docs.hpc.cineca.it/hpc/software/qe.html#quantumespresso "Link to this heading") =========================================================================================================== The following guide describes how to load, configure and use QuantumESPRESSO @ CINECA’s cluster. QuantumESPRESSO is available on [Leonardo](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo) and [Galileo100](https://docs.hpc.cineca.it/hpc/galileo.html#galileo100) clusters. Relevant links[](https://docs.hpc.cineca.it/hpc/software/qe.html#relevant-links "Link to this heading") --------------------------------------------------------------------------------------------------------- * QE repository: [https://gitlab.com/QEF/q-e.git](https://gitlab.com/QEF/q-e.git) * MaX benchmarks: [https://gitlab.com/max-centre/benchmarks-max3.git](https://gitlab.com/max-centre/benchmarks-max3.git) * JUBE xmls: [https://gitlab.com/max-centre/JUBE4MaX.git](https://gitlab.com/max-centre/JUBE4MaX.git) * spack recipe: [https://gitlab.com/spack/spack/-/blob/develop/var/spack/repos/builtin.mock/packages/quantum-espresso/package.py](https://gitlab.com/spack/spack/-/blob/develop/var/spack/repos/builtin.mock/packages/quantum-espresso/package.py) Modules[](https://docs.hpc.cineca.it/hpc/software/qe.html#modules "Link to this heading") ------------------------------------------------------------------------------------------- CPU-based and GPU-based machines deploy QuantumESPRESSO with different software stacks, to fully exploit the underlying hardware. In particular: * **Intel/Oneapi** compiler and MPI implementation on G100 and Leonardo DCGP, plus **MKL** for FFT, BLAS/LAPACK and SCALAPACK * **NVHPC** compiler and **OpenMPI/HPCX-MPI** on Leonardo Booster, plus **OpenBLAS** and **FFTW** libraries. Installations based on gcc compiler do not provide performance, and are provided for postprocessing executables. Alternative Installations[](https://docs.hpc.cineca.it/hpc/software/qe.html#alternative-installations "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------- If you wish installing your own version of QuantumESPRESSO, we suggesting using CMake and the options provided in the [Wiki of the official repository](https://gitlab.com/QEF/q-e/-/wikis/Developers/CMake-build-system) for the CINECA cluster in use. Parallelization strategies[](https://docs.hpc.cineca.it/hpc/software/qe.html#parallelization-strategies "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------- QuantumESPRESSO supports different parallelization strategies. * R&G (\-npw or no options) processes to distribute real/reciprocal spaces * pools (\-nk) to distribute k-points * images (\-ni) to distribute irreducible representations or q-points in a dispersion * band processes (\-nbnd) to distribute the Kohn-Sham states * linear algebra processes (auto) to distribute diagonalization, via scalapack or custom algorithm. For GPU installations, the diagonalization is done on a single GPU (scalapack are not used We suggest the following for optimal performance on Leonardo Booster: * prioritize pools over R&G , in particular for workloads with hundreds of planes or less in the z-direction, also for intra-node distribution. * The minimum number of k-points per pool (kunit) in PWSCF is the number of k-points (kunit=1), while in phonon is usually kunit=2, except the following cases: (i) lgamma and not noncolin or domag: kunit=1 (ii) not lgamma but noncolin and domag: kunit=4. * Images implements independent calculations but they might be affected by imbalanced workload, so a mixture of images and poolsusually provides best performances. More detailed information about parallelization strategies can be found on this [link](https://qe-on-leonardo.readthedocs.io/en/latest/usage.html) GPU performance considerations and troubleshooting[](https://docs.hpc.cineca.it/hpc/software/qe.html#gpu-performance-considerations-and-troubleshooting "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ### Mapping and binding[](https://docs.hpc.cineca.it/hpc/software/qe.html#mapping-and-binding "Link to this heading") If you are using the node in exclusive mode, distribute the resources among MPI tasks (usually 1 MPI task per GPU) as follows 1. Set SLURM options to ask for 8 cpus per task #SLURM --nodes= #SLURM --ntasks-per-node=4 #SLURM --cpus-per-task=8 Copy to clipboard 2. Launch the MPI application by mapping the tasks over the full node with 8 cpus per task. The following code snippets show the command line for mpirun and srun: mpirun --map-by node:PE=$SLURM\_CPUS\_PER\_TASK --rank by core pw.x Copy to clipboard srun --cpus-per-task=$SLURM\_CPUS\_PER\_TASK --cpu-bind=cores pw.x Copy to clipboard 3. MPI-GPU binding is done by source code, so an external binding with CUDA\_VISIBLE\_DEVICES is not needed. ### Multi-node multi-GPU runs[](https://docs.hpc.cineca.it/hpc/software/qe.html#multi-node-multi-gpu-runs "Link to this heading") If your workload requires **multi-node distribution** due to memory constrains on GPUs, we suggest testing the following environment variables to improve performnaces. **Memory issues during SCF loop (OOM)** Your code crashes after some iterations steps in the SCF loop. Instead of increasing the number of nodes, try adding the folliwing environment variable in your jobscript. export UCX\_TLS=^cuda\_ipc Copy to clipboard This error is due to handles automatically created by the MPI library when calling Isend+Irecv\*Waitall. Note Do not export this environment variable if using a number of R&G processes less or equal to 4. **Increase multi-node BW** If your code distributes FFTs across multiple nodes, the MPI installations might not use all the NICs available for inter-node communications. Try interposing this script between the mpi launcher and the executable #!/bin/bash # Replace with OMPI\_COMM\_WORLD\_LOCAL\_RANK if using mpirun case $(( ${SLURM\_LOCALID} )) in 0) export UCX\_NET\_DEVICES=mlx5\_0:1 CUDA\_VISIBLE\_DEVICES=0 ;; 1) export UCX\_NET\_DEVICES=mlx5\_1:1 CUDA\_VISIBLE\_DEVICES=1 ;; 2) export UCX\_NET\_DEVICES=mlx5\_2:1 CUDA\_VISIBLE\_DEVICES=2 ;; 3) export UCX\_NET\_DEVICES=mlx5\_3:1 CUDA\_VISIBLE\_DEVICES=3 ;; esac echo Launching on $UCX\_NET\_DEVICES Copy to clipboard Note The environment variable depends on the mpi launcher, srun (SLURM\_LOCALID) or mpirun (OMPI\_COMM\_WORLD\_LOCAL\_RANK) **Improve multi-node scaling with FFT distribution** If you need to distribute FFTs over multiple-nodes and achieve the so called ‘eager’ regime, with small messages exchanged among processes, try reducing the threshold for the rendez-vous algorithm, which can be more efficient on GPUs export UCX\_RNDV\_THRESH\=8192 Copy to clipboard --- # Pitagora — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Cluster Specifics](https://docs.hpc.cineca.it/hpc/hpc_clusters.html) * Pitagora * [View page source](https://docs.hpc.cineca.it/_sources/hpc/pitagora.rst.txt) * * * Pitagora[](https://docs.hpc.cineca.it/hpc/pitagora.html#pitagora "Link to this heading") ========================================================================================== Pitagora is the new EUROfusion supercomputer hosted by **CINECA** and currently built in the CINECA’s headquarter in Casalecchio di Reno, Bologna, Italy. The cluster is supplied by Lenovo corp. and is composed of two partitions: a general purpose partition cpu-based named **DCPG** and an accelerated partition based on NVIDIA H100 accelerators named **Booster**. The specific guide for the **Pitagora** cluster contains unique information that deviates from the general behavior described in the HPC Clusters sections. Access to the System[](https://docs.hpc.cineca.it/hpc/pitagora.html#access-to-the-system "Link to this heading") ------------------------------------------------------------------------------------------------------------------ The machine is reachable via `ssh` (secure Shell) protocol at hostname point: **login.pitagora.cineca.it**. The connection is established, automatically, to one of the available login nodes. It is possible to connect to **Pitagora** using one the specific login hostname points: > * login01-ext.pitagora.cineca.it > > * login02-ext.pitagora.cineca.it > > * login03-ext.pitagora.cineca.it > > * login04-ext.pitagora.cineca.it > > * login05-ext.pitagora.cineca.it > > * login06-ext.pitagora.cineca.it > Warning **The mandatory access to Pitagora is the two-factor authetication (2FA)**. Get more information at section [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems) . Note **Even-numbered login nodes** have the same architecture of **Booster** parition’s compute nodes while **odd-numbered** have the same architecture of **DCGP** parition’s compute nodes * **login-boost.pitagora.cineca.it** will allow users to log on one of the **even-numbered login nodes** in a round robin fashion. * **login-dcgp.pitagora.cineca.it** will allow users to log on one of the **odd-numbered login nodes** in a round robin fashion. System Architecture[](https://docs.hpc.cineca.it/hpc/pitagora.html#system-architecture "Link to this heading") ---------------------------------------------------------------------------------------------------------------- The system, supplied by Lenovo, is based on two new specifically-designed compute blades, which are available throught two distinct SLURM partitios on the Cluster: * **GPU** blade based on NVIDIA NVIDIA H100 accelerators - **Booster** partition. * **CPU**\-only blade based on AMD Turin 128c processors - **Data Centric General Purpose (DCGP)** partition. The overall system architecture uses NVIDIA Mellanox InfiniBand High Data Rate (HDR) connectivity, with smart in-network computing acceleration engines that enable extremely low latency and high data throughput to provide the highest AI and HPC application performance and scalability. ### Hardware Details[](https://docs.hpc.cineca.it/hpc/pitagora.html#hardware-details "Link to this heading") Booster | **Type** | **Specific** | | --- | --- | | Models | Lenovo SD650-N V3 | | Racks | 7 | | Nodes | 168 | | Processors/node | 2x Intel Emerald Rapids Xeon Gold 6548Y+ 32c 2.5 GHz | | CPU/node | 64 | | Accelerators/node | 4x NVIDIA H100 SXM 80GB HBM2e | | Local Storage/node (tmfs) | | | RAM/node | 512 GiB DDR5 5600 Mhz | | Rmax | 27.27 PFlop/s ([top500](https://www.top500.org/system/180348/)
) | | Internal Network | Nvidia ConnectX-7 NDR200 | | Storage (raw capacity) | 2 x 7.68 GiB SSDs (HW RAID 1) | DCGP | **Type** | **Specific** | | --- | --- | | Models | Lenovo SD665 V3 | | Racks | 14 | | Nodes | 1008 | | Processors/node | 2x AMD Turin EPYC 9745 128c 2.4 GHz - Zen5 microarch | | CPU/node | 256 | | Accelerators/node | (none) | | Local Storage/node (tmfs) | | | RAM/node | 768 GiB DDR5 6400 Mhz | | Rmax | 17 Pflop/s ([top500](https://www.top500.org/system/180348/)
) | | Internal Network | Nvidia ConnectX-7 NDR SharedIO 200Gbit/s | | Storage (raw capacity) | Diskless nodes | Job Managing and SLURM Partitions[](https://docs.hpc.cineca.it/hpc/pitagora.html#job-managing-and-slurm-partitions "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------- In the following table you can find informations about the SLURM partitions for **Booster** and **DCGP** partitions of the production environment. Please note that the slurm email service is not active yet. See also Further information about job submission are reported in the general section [Scheduler and Job Submission](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scheduler-and-job-submission) . Booster | **Partition** | **QOS** | **#Nodes/#per job** | **Walltime** | **#Max Nodes/#per user** | **Priority** | **Notes** | | --- | --- | --- | --- | --- | --- | --- | | boost\_fua\_prod | normal | max = 16 | 24:00:00 | 32 | 40 | | | boost\_qos\_fuabprod | min = 17 (full nodes) max = 32 | 24:00:00 | 32 | 60 | runs on 96 nodes (GrpTRES) | | boost\_fua\_dbg | normal | max = 2 | 00:30:00 | | 40 | runs on 2 nodes (GrpTRES) | DCGP | **Partition** | **QOS** | **#Nodes/#per job** | **Walltime** | **#Max Nodes/#per user** | **Priority** | **Notes** | | --- | --- | --- | --- | --- | --- | --- | | dcgp\_fua\_prod | normal | max = 64 | 24:00:00 | 64 | 40 | | | dcgp\_qos\_fuabprod | min = 65 (full nodes) max = 128 | 24:00:00 | 128 | 60 | runs on 640 nodes (GrpTRES) | | dcgp\_qos\_fualprod | max = 3 | 4-00:00:00 | 3 | 40 | | | dcgp\_fua\_dbg | normal | max = 2 | 00:30:00 | 2 | 40 | runs on 8 nodes (GrpTRES) | ### Processes/Threads Binding/Affinity[](https://docs.hpc.cineca.it/hpc/pitagora.html#processes-threads-binding-affinity "Link to this heading") **Processes Binding** * By default, srun (SLURM launcher) performs an automatic binding. For multi-threaded application request the proper –cpus-per-task and bind the processes to cores (srun –cpu-bind=cores). * By default, OpenMPI libraries (mpirun launcher) bind processes to core. For multi-threaded applications this causes the cpu overallocation. Ensure that you are either not bound at all (by specifying –bind-to none) or bound to multiple cores using an appropriate binding level or specific number of processing elements per application process (–map-by socket:PE=$SLURM\_CPUS\_PER\_NODE). * By default, IntelMPI libraries (mpirun launcher with hydra process manager) performs a correct binding. If you opt for IntelMPI mpirun as launcher, unset the I\_MPI\_PMI\_LIBRARY (meant for using Intelmpi with srun) defined when loading the module to avoid the verbose warnings. **Threads Affinity** All present compilers (gcc, nvhpc, aocc, intel) by default don’t bind threads to cores. You can act on the threads affinity with the standard OMP\_PLACES/OMP\_PROC\_BIND variables. Known Issues[](https://docs.hpc.cineca.it/hpc/pitagora.html#known-issues "Link to this heading") -------------------------------------------------------------------------------------------------- This section collects currently known issues affecting PITAGORA. The list below is intended as a quick reference for users who may experience problems on the system. We strongly encourage all users to report any issues they encounter - whether listed here or not - to the user support team. Internode GPUDirect Communication: UCX GPUDirect RDMA Error **Status:** Open | **Last Update:** 2025-07-31 | **Partition:** Booster > **Description** > > The **UCX GPUDirect RDMA** feature is currently not functioning for point-to-point communications. This is likely due to an incompatibility between **Intel UPI** (intersocket connection) and **UCX**, as referenced in [NVIDIA issue 2235234](https://docs.nvidia.com/networking/display/hpcxv216/known+issues) > . > > **Case 1**: The mpi job fails with errors like the following one: > > > \[r310c04s01:2918358:0:2918358\] ib\_mlx5\_log.c:179 Local protection error on mlx5\_0:1/IB (synd 0x4 vend 0x51 hw\_synd 0/2) > > \[r310c04s01:2918358:0:2918358\] ib\_mlx5\_log.c:179 RC QP 0xb232 wqe\[20\]: SEND s-e \[inl len 10\] \[va 0x14986e800000 len 1 lkey 0x63a2\] \[rqpn 0xc526 dlid\=2992 sl\=0 port\=1 src\_path\_bits\=0\] > > \[r310c03s04:1011253:0:1011253\] ib\_mlx5\_log.c:179 Local protection error on mlx5\_0:1/IB (synd 0x4 vend 0x51 hw\_synd 0/2) > > \[r310c03s04:1011253:0:1011253\] ib\_mlx5\_log.c:179 RC QP 0xc526 wqe\[25\]: SEND s-e \[inl len 10\] \[va 0x14fa6a800000 len 1 lkey 0x688b\] \[rqpn 0xb232 dlid\=5592 sl\=0 port\=1 src\_path\_bits\=0\] > > \==== backtrace (tid:2918358) \==== > > 0 0x00000000000129b0 uct\_ib\_mlx5\_completion\_with\_err() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/ib\_mlx5\_log.c:179 > > 1 0x00000000000279ec uct\_rc\_mlx5\_iface\_handle\_failure() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:235 > > 2 0x00000000000279ec uct\_rc\_iface\_arbiter\_dispatch() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/rc/base/rc\_iface.h:455 > > 3 0x00000000000279ec uct\_rc\_mlx5\_iface\_handle\_failure() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:238 > > 4 0x0000000000013a25 uct\_ib\_mlx5\_check\_completion() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/ib\_mlx5.c:477 > > 5 0x0000000000028d97 uct\_ib\_mlx5\_poll\_cq() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/ib\_mlx5.inl:148 > > 6 0x0000000000028d97 uct\_rc\_mlx5\_iface\_poll\_tx() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5.inl:1891 > > 7 0x0000000000028d97 uct\_rc\_mlx5\_iface\_progress() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:127 > > 8 0x0000000000028d97 uct\_rc\_mlx5\_iface\_progress\_cyclic() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:132 > > 9 0x000000000004dc4a ucs\_callbackq\_dispatch() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/ucs/datastruct/callbackq.h:215 > > 10 0x000000000004dc4a uct\_worker\_progress() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/api/uct.h:2813 > > 11 0x000000000004dc4a ucp\_worker\_progress() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/ucp/core/ucp\_worker.c:3033 > > 12 0x0000000000031f73 opal\_progress() ???:0 > > 13 0x0000000000054955 ompi\_request\_default\_wait\_all() ???:0 > > 14 0x000000000009ce07 MPI\_Waitall() ???:0 > > 15 0x0000000000403e33 main() ???:0 > > 16 0x0000000000029590 \_\_libc\_start\_call\_main() ???:0 > > 17 0x0000000000029640 \_\_libc\_start\_main\_alias\_2() :0 > > 18 0x0000000000404cc5 \_start() ???:0 > > \================================= > > > > Copy to clipboard > > > > **TEMPORARY SOLUTION:** > > > > UCX GPUDirect RDMA has been disabled by default through the following environment variable, set automatically in all OpenMPI modules: > > > > export UCX\_IB\_GPU\_DIRECT\_RDMA\=no > > > > This can be verified by inspecting OpenMPI modules. In the example below, unrelated lines have been omitted for clarity. > > > > Copy to clipboard > > > > \[@ ~\]$ module show openmpi/4.1.6--gcc--12.3.0 > > ------------------------------------------------------------------- > > /pitagora/prod/opt/modulefiles/base/libraries/openmpi/4.1.6--gcc--12.3.0: > > > > module-whatis {An open source Message Passing Interface implementation.} > > \[...\] > > setenv UCX\_IB\_GPU\_DIRECT\_RDMA no > > \[...\] > > append-path MANPATH {} > > ------------------------------------------------------------------- > > > > Copy to clipboard > > > > Important > > > > * **No action is required by the user to apply this workaround.** > > > > * If users wish to test UCX GPUDirect RDMA manually, they can unset the variable after loading the module to re-enable the feature. > > > > * **No significant performance degradation has been observed** in synthetic benchmarks (e.g., OSU) or selected real-world applications. > > > > **Case 2**: Pytorch with NCCL backend jobs hang. > > > **TEMPORARY SOLUTION:** > > > > Disable NCCL GPU Direct RDMA as follows: > > > > export NCCL\_NET\_GDR\_LEVEL\=LOC > > > > Copy to clipboard IntelMPI Provider/Fabric Compatibility on AMD Processors **Status:** Open | **Last Update:** 2025-12-18 | **Partition:** DCGP > **Description** > > **DESCRIPTION:** > > The **Mellanox (MLX) provider for IntelMPI does not work correctly with several applications and libraries on the AMD-based DCGP partition**, resulting in job crashes or significantly reduced performance. This is related to the UCX Transport Layer behind the communication, expecially to the TLS “rc” (Reliable Connection) that presents some bugs. > > **Known affected codes:** > > * STARWALL > > * ASCOT5 > > * PETSc-based software > > * GENE > > > **TEMPORARY SOLUTION:** > > The **rc TLS has been automatically discarded for IntelMPI**. This is done automatically when loading the IntelMPI module by exporting the following environment variable: > > export UCX\_TLS\=^rc > > Copy to clipboard > > This can be confirmed by inspecting the IntelMPI module. In the example below, unrelated lines have been omitted for clarity. > > \[@ ~\]$ module show intel-oneapi-mpi/2021.12.1 > ------------------------------------------------------------------- > /pitagora/prod/opt/modulefiles/base/libraries/intel-oneapi-mpi/2021.12.1: > > module-whatis {Intel MPI Library is a multifabric message-passing library that implements the open-source MPICH specification. Use the library to create, maintain, and test advanced, complex applications that perform better on high-performance computing (HPC) clusters based on Intel processors.} > conflict intel-oneapi-mpi > \[...\] > setenv UCX\_TLS\=^rc > \[...\] > setenv MPIFC mpiifx > ------------------------------------------------------------------- > > Copy to clipboard > > Important > > * **No action is required from the user to enable this configuration** as it is now the default for IntelMPI modules. > > * For some applications (e.g., ONIX), this configuration may be cause of slowdowns. In these cases, it is recommendable to test other provider and TLS choices: in the case of ONIX, switching the provider to Verbs may improve the performance (it is not case for most other applications). **To use Verbs**: > > > > export FI\_PROVIDER\=verbs > > > > Copy to clipboard > > * If switching providers manually, ensure that you export the variables **after** loading the IntelMPI module. > --- # Leonardo — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Cluster Specifics](https://docs.hpc.cineca.it/hpc/hpc_clusters.html) * Leonardo * [View page source](https://docs.hpc.cineca.it/_sources/hpc/leonardo.rst.txt) * * * Leonardo[](https://docs.hpc.cineca.it/hpc/leonardo.html#leonardo "Link to this heading") ========================================================================================== Leonardo is the _pre-exascale_ Tier-0 supercomputer of the EuroHPC Joint Undertaking (JU), hosted by **CINECA** and currently located at the Bologna DAMA-Technopole in Italy. This guide provides specific information about the **Leonardo** cluster, including details that differ from the general behavior described in the broader HPC Clusters section. Access to the System[](https://docs.hpc.cineca.it/hpc/leonardo.html#access-to-the-system "Link to this heading") ------------------------------------------------------------------------------------------------------------------ The machine is reachable via `ssh` (secure Shell) protocol at hostname point: **login.leonardo.cineca.it**. The connection is established, automatically, to one of the available login nodes. It is possible to connect to **Leonardo** using one the specific login hostname points: > * login01-ext.leonardo.cineca.it > > * login02-ext.leonardo.cineca.it > > * login05-ext.leonardo.cineca.it > > * login07-ext.leonardo.cineca.it > Warning **The mandatory access to Leonardo si the two-factor authetication (2FA)**. Get more information at section [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems) . System Architecture[](https://docs.hpc.cineca.it/hpc/leonardo.html#system-architecture "Link to this heading") ---------------------------------------------------------------------------------------------------------------- The cluster, supplied by EVIDEN ATOS, is based on two new specifically-designed compute blades, which are available throught two distinc Slurm partitios on the Cluster: * X2135 **GPU** blade based on NVIDIA Ampere A100-64 accelerators - **Booster** partition. * X2140 **CPU**\-only blade based on Intel Sapphire Rapids processors - **Data Centric General Purpose (DCGP)** partition. The overall system architecture uses NVIDIA Mellanox InfiniBand High Data Rate (HDR) connectivity, with smart in-network computing acceleration engines that enable extremely low latency and high data throughput to provide the highest AI and HPC application performance and scalability. The **Booster** partition entered pre-production in May 2023 and moved to **full production in July 2023**. The **DCGP** partition followed, starting pre-production in January 2024 and reaching **full production in February 2024**. ### Hardware Details[](https://docs.hpc.cineca.it/hpc/leonardo.html#hardware-details "Link to this heading") Booster | **Type** | **Specific** | | --- | --- | | Models | Atos BullSequana X2135, Da Vinci single-node GPU | | Racks | 116 | | Nodes | 3456 | | Processors/node | 1x [Intel Ice Lake Intel Xeon Platinum 8358](https://www.intel.com/content/www/us/en/products/sku/212282/intel-xeon-platinum-8358-processor-48m-cache-2-60-ghz/specifications.html) | | CPU/node | 32 | | Accelerators/node | 4x [NVIDIA Ampere100 custom](https://doi.org/10.17815/jlsrf-8-186)
, 64GiB HBM2e NVLink 3.0 (200 GB/s) | | Local Storage/node (tmfs) | (none) | | RAM/node | 512 GiB DDR4 3200 MHz | | Rmax | 241.2 PFlop/s ([top500](https://www.top500.org/system/180128/)
) | | Internal Network | 200 Gbps NVIDIA Mellanox HDR InfiniBand - Dragonfly+ Topology | | Storage (raw capacity) | 106 PiB based on DDN ES7990X and Hard Drive Disks (Capacity Tier)

5.7 PiB based on DDN ES400NVX2 and Solid State Drives (Fast Tier) | DCGP | **Type** | **Specific** | | --- | --- | | Models | Atos BullSequana X2140 three-node CPU blade | | Racks | 22 | | Nodes | 1536 | | Processors/node | 2x [Intel Sapphire Rapids Intel Xeon Platinum 8480+](https://www.intel.com/content/www/us/en/products/sku/231746/intel-xeon-platinum-8480-processor-105m-cache-2-00-ghz/specifications.html) | | CPU/node | 112 cores/node | | Accelerators | (none) | | Local Storage/node (tmfs) | 3 TiB | | RAM/node | 512(8x64) GiB DDR5 4800 MHz | | Rmax | 7.84 PFlop/s ([top500](https://www.top500.org/system/180204/)
) | | Internal Network | 200 Gbps NVIDIA Mellanox HDR InfiniBand - Dragonfly+ Topology | | Storage (raw capacity) | 106 PiB based on DDN ES7990X and Hard Drive Disks (Capacity Tier)

5.7 PiB based on DDN ES400NVX2 and Solid State Drives (Fast Tier) | File Systems and Data Managment[](https://docs.hpc.cineca.it/hpc/leonardo.html#file-systems-and-data-managment "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------- The storage organization conforms to **CINECA** infrastructure. General information are reported in [File Systems and Data Management](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#file-systems-and-data-management) section. In the following, only differences with respect to general behavior are listed and explained. **$TMPDIR** * on the local SSD disks on login nodes (14 TB of capacity), mounted as `/scratch_local` (`TMPDIR=/scratch_local`). This is a shared area with no quota, remove all the files once they are not requested anymore. A cleaning procedure will be enforced in case of improper use of the area. * on the local SSD disks on the serial node (`lrd_all_serial`, 14TB of capacity), managed via the Slurm `job_container/tmpfs plugin`. This plugin provides a _job-specific_, private temporary file system space, with private instances of `/tmp` and `/dev/shm` in the job’s user space (`TMPDIR=/tmp`, visible via the command `df -h`), removed at the end of the serial job. You can request the resource via sbatch directive or srun option `--gres=tmpfs:XX` (for instance: `--gres=tmpfs:200G`), with a maximum of 1 TB for the serial jobs. If not explicitly requested, the `/tmp` has the default dimension of 10 GB. * on the local SSD disks on DCGP nodes (3 TB of capacity). As for the serial node, the local `/tmp` and `/dev/shm` areas are managed via plugin, which at the start of the jobs mounts private instances of `/tmp` and `/dev/shm` in the job’s user space (`TMPDIR=/tmp`, visible via the command `df -h /tmp`), and unmounts them at the end of the job (all data will be lost). You can request the resource via sbatch directive or srun option `--gres=tmpfs:XX`, with a maximum of all the available 3 TB for DCGP nodes. As for the serial node, if not explicitly requested, the `/tmp` has the default dimension of 10 GB. Please note: for the DCGP jobs the requested amount of `gres/tmpfs` resource contributes to the consumed budget, changing the number of accounted equivalent core hours, see the dedicated section on the Accounting. * on RAM on the diskless booster nodes (with a fixed size of 10 GB, no increase is allowed, and the `gres/tmpfs` resource is disabled). Job Managing and Slurm Partitions[](https://docs.hpc.cineca.it/hpc/leonardo.html#job-managing-and-slurm-partitions "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------- In the following table you can find informations about the Slurm partitions for **Booster** and **DCGP** partitions. See also Further information about job submission are reported in the general section [Scheduler and Job Submission](https://docs.hpc.cineca.it/hpc/hpc_scheduler.html#scheduler-and-job-submission) . Booster | **Partition** | **QOS** | **TRES Limits per Job** | **Walltime** | **MaxTRES per User** | **Priority** | **Notes** | | --- | --- | --- | --- | --- | --- | --- | | lrd\_all\_serial

(**default**) | normal | Max = 4 cores

(8 logical cores) | 04:00:00 | 1 node / 4 cores

(30800 MB RAM) | 40 | No GPUs, Hyperthreading x 2

**Budget Free** | | boost\_usr\_prod | normal | Max = 64 nodes | 24:00:00 | | 40 | | | boost\_qos\_dbg | Max = 4 nodes | 00:30:00 | 4 nodes / 128 cores / 16 GPUs | 80 | Max 1 job running and/or pending

per User Account | | boost\_qos\_bprod | Min = 65 full nodes

Max = 256 nodes | 24:00:00 | 256 nodes | 60 | | | boost\_qos\_lprod | Max = 8 nodes | 4-00:00:00 | 8 nodes / 32 GPUs | 40 | Max resources per Project Account | DCGP | **Partition** | **QOS** | **TRES Limits per Job** | **Walltime** | **MaxTRES per User or Proj. Account** | **Priority** | **Notes** | | --- | --- | --- | --- | --- | --- | --- | | lrd\_all\_serial

(**default**) | normal | Max = 4 cores

(8 logical cores) | 04:00:00 | 1 node / 4 cores

(30800 MB RAM) | 40 | Hyperthreading x 2

**Budget Free** | | dcgp\_usr\_prod | normal | Max = 16 nodes | 24:00:00 | 512 nodes per Prj. Account | 40 | | | dcgp\_qos\_dbg | Max = 2 nodes | 00:30:00 | 2 nodes / 224 cores per User Account

512 nodes per Prj. Account | 80 | Max 1 job running and/or pending

per User Account | | dcgp\_qos\_bprod | Min = 17 full nodes

Max = 128 nodes | 24:00:00 | 128 nodes per User Account

512 nodes per Prj. Account | 60 | GrpTRES = 1536 nodes

Min is 17 FULL nodes | | dcgp\_qos\_lprod | Max = 3 nodes | 4-00:00:00 | 3 nodes / 336 cores per user Account

512 nodes per Prj. Account | 40 | | Network Architecture[](https://docs.hpc.cineca.it/hpc/leonardo.html#network-architecture "Link to this heading") ------------------------------------------------------------------------------------------------------------------ **Leonardo** features a state-of-the-art interconnect system tailored for high-performance computing (HPC). It delivers _low latency_ and _high bandwidth_ by leveraging **NVIDIA Mellanox InfiniBand HDR** (High Data Rate) technology, powered by [NVIDIA QUANTUM QM8700 Smart Switches](https://nvdam.widen.net/s/zmbw7rdjml/infiniband-qm8700-datasheet-us-nvidia-1746790-r12-web) , and a **[Dragonfly+ topology](https://ieeexplore.ieee.org/document/7885210) **. Below is an overview of its architecture and key features: * **Hierarchical Cell Structure:** The system is structured into multiple _cells_, each comprising a group of interconnected compute nodes. * **Inter-cell Connectivity:** As illustrated in the figure below, cells are connected via an all-to-all topology. Each pair of distinct cells is linked by 18 independent connections, each passing through a dedicated Layer 2 (L2) switch. This design ensures high availability and reduces congestion. * **Intra-cell Topology:** Inside each cell, a non-blocking two-layer fat-tree topology is used, allowing scalable and efficient intra-cell communication. * **System Composition:** * 19 cells dedicated to the _Booster_ partition. * 2 cells for the _DCGP_ (Data-Centric General Purpose) partition. * 1 hybrid cell with both accelerated (36 Booster nodes) and conventional (288 DCGP nodes) compute resources. * 1 cell allocated for management, storage, and login services. * **Adaptive Routing:** The network employs adaptive routing, dynamically optimizing data paths to alleviate congestion and maintain performance under load. ![../_images/leo-net-all2all.png](https://docs.hpc.cineca.it/_images/leo-net-all2all.png) ![../_images/spacer2.png](https://docs.hpc.cineca.it/_images/spacer2.png) Cell Configuration and Intra-cell Connectivity Booster Each Booster cell is composed of: * **6 × Atos BullSequana XH2000 racks**, each containing: * 3 × Level 2 (L2) switches * 3 × Level 1 (L1) switches * 30 compute nodes — each equipped with 4 GPUs, each connected via a dedicated 100 Gbps port **Total per Booster cell:** 18 L2 switches, 18 L1 switches, and 180 compute nodes. #### Connectivity Overview **Level 2 (L2) Switches:** * **UP:** 22 × 200 Gbps ports connecting to L2 switches in other cells * **DOWN:** 18 × 200 Gbps ports connecting to L1 switches within the cell * **Oversubscription:** 0.8:1 **Level 1 (L1) Switches:** * **UP:** 18 × 200 Gbps ports connected to all L2 switches in the cell * **DOWN:** 40 × 100 Gbps ports connected to GPUs across 10 compute nodes * **Oversubscription:** 1.11:1 [![../_images/leo-net-booster_cell.png](https://docs.hpc.cineca.it/_images/leo-net-booster_cell.png)](https://docs.hpc.cineca.it/_images/leo-net-booster_cell.png) DCGP Each DCGP cell is composed of: * **8 × Atos BullSequana XH2000 racks**, each containing: * 3 or 0 Level 2 (L2) switches * 2 × Level 1 (L1) switches * 78 compute nodes — each connected via a dedicated 100 Gbps port **Total per DCGP cell:** 18 L2 switches, 16 L1 switches, and 624 compute nodes. #### Connectivity Overview **Level 2 (L2) Switches:** * **UP:** 22 × 200 Gbps ports connecting to L2 switches in other cells * **DOWN:** 18 × 200 Gbps ports connecting to L1 switches within the same cell * **Oversubscription ratio:** 0.8:1 **Level 1 (L1) Switches:** (divided into two groups): * **9 switches with 40 downlinks:** * UP: 18 × 200 Gbps ports connected to all L2 switches in the cell * DOWN: 40 × 100 Gbps ports connected to compute nodes * Oversubscription ratio: 1.11:1 * **9 switches with 38 downlinks:** * UP: 18 × 200 Gbps ports connected to all L2 switches in the cell * DOWN: 38 × 100 Gbps ports connected to compute nodes * Oversubscription ratio: 1.05:1 [![../_images/leo-net-dcgp_cell.png](https://docs.hpc.cineca.it/_images/leo-net-dcgp_cell.png)](https://docs.hpc.cineca.it/_images/leo-net-dcgp_cell.png) ### Advanced Information[](https://docs.hpc.cineca.it/hpc/leonardo.html#advanced-information "Link to this heading") Network Topology - Map > The topology is presented in a table format, where each row corresponds to a compute node. For each node, the table specifies the associated L1 switch and cell, providing a clear overview of the physical and logical network layout within the cluster. > > [`Network Topology - Map`](https://docs.hpc.cineca.it/_downloads/b3e4d490b726ab3df2335843480b2537/ntopology.dat) Network Topology - Distance Matrix > The attached compressed CSV file contains the distance matrix of all compute nodes in the cluster. The matrix uses the following metric to represent the network distance between any two nodes: > > * **0** – Same nodes > > * **1** – Same L1 switch, same cell. > > * **2** – Different L1 switch, same cell. > > * **3** – Different L1 switch and different cell. > > > This matrix can be used to analyze communication locality and optimize node selection for distributed workloads. > > [`Distance Matrix`](https://docs.hpc.cineca.it/_downloads/958475284937277d554ff9cb0fcdca7b/ntopology-dst_mtx.tar.bz2) Switch Naming Format > isw > > Copy to clipboard > > where `` is a 5- or 6-digits number varies based on the location and type of the switch. > > Specifically: > > * `RR` = region number (1 or 2 digits) > > * `rr` = rack number (2 digits) > > * `SS` = switch id (2 digits) > > > Note > > If `SS` is an even number, it refers to an L1 switch; if it is an odd number, it refers to an L2 switch. Documents[](https://docs.hpc.cineca.it/hpc/leonardo.html#documents "Link to this heading") -------------------------------------------------------------------------------------------- * Article on Leonardo architecture and the technologies adopted for its GPU-accelerated partition: CINECA Supercomputing Centre, SuperComputing Applications and Innovation Department. (2024). “LEONARDO: A Pan-European Pre-Exascale Supercomputer for HPC and AI applications.”, Journal of large-scale research facilities, 8, A186. [https://doi.org/10.17815/jlsrf-8-186](https://doi.org/10.17815/jlsrf-8-186) * Details about new technologies included in the Witley platform with Intel Xeon Icelake contained in the Leonardo pre-exascale system ([link](https://urldefense.com/v3/__https://software.intel.com/content/www/us/en/develop/articles/third-generation-xeon-scalable-family-overview.html__;!!P1tgJ-3e!TrmMus5wzdLQ963vkc3yfy0BlhC1Hu8vOoce4SgltsTbkSSDrX2p1zTXPCIrpPm3$) ) * Additional documents ([link](https://urldefense.com/v3/__https://software.intel.com/content/www/us/en/develop/articles/xeon-performance-tuning-and-solution-guides.html__;!!P1tgJ-3e!TrmMus5wzdLQ963vkc3yfy0BlhC1Hu8vOoce4SgltsTbkSSDrX2p1zTXPKZ5awkS$) ) Some tuning guides for dedicated enviroments (ML/DL or HPC Clusters): * [`Tuning Guide`](https://docs.hpc.cineca.it/_downloads/5a3e534740b0c9c9ee5fec0649ba4230/Tuning_guide.pdf) * [`Deep Learning`](https://docs.hpc.cineca.it/_downloads/29dbbb88e01d5ab22853da137556bb11/Deep_learning.pdf) --- # Galileo100 — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Cluster Specifics](https://docs.hpc.cineca.it/hpc/hpc_clusters.html) * Galileo100 * [View page source](https://docs.hpc.cineca.it/_sources/hpc/galileo.rst.txt) * * * Galileo100[](https://docs.hpc.cineca.it/hpc/galileo.html#galileo100 "Link to this heading") ============================================================================================= Galileo100 is a new infrastructure co-funded by the European ICEI (Interactive Computing e-Infrastructure) project and engineered by DELL. It is the national Tier-1 system for scientific research and is available to the Italian public and industrial researchers since September 2021. It also features 77 cloud computing servers and was expanded in November 2022 with 82 additional nodes. **Galileo100** is used for high-end technical and industrial HPC projects, as well as meteorology and environmental studies. The specific guide for the **Galileo100** cluster contains unique information that deviates from the general behavior described in the HPC Clusters sections. Access to the System[](https://docs.hpc.cineca.it/hpc/galileo.html#access-to-the-system "Link to this heading") ----------------------------------------------------------------------------------------------------------------- The machine is reachable via `ssh` (secure Shell) protocol at hostname point: **login.g100.cineca.it**. The connection is established, automatically, to one of the available login nodes. It is possible to connect to **Galileo100** using one the specific login hostname points: * login01-ext.g100.cineca.it * login02-ext.g100.cineca.it * login03-ext.g100.cineca.it Warning **The mandatory access to Galileo100 is the two-factor authetication (2FA)**. Get more information at section [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems) . System Architecture[](https://docs.hpc.cineca.it/hpc/galileo.html#system-architecture "Link to this heading") --------------------------------------------------------------------------------------------------------------- ### Hardware Details[](https://docs.hpc.cineca.it/hpc/galileo.html#hardware-details "Link to this heading") | **Type** | **Specific** | | --- | --- | | Models | Dual-soket Dell PowerEdge | | Nodes | 630 | | Processors/node | 2xCPU x86 Intel Xeon Platinum 8276/L 2.4GHz | | CPU/node | 48 | | Accelerators/node | 2xGPU Nvidia V100 PCIe3 with 32 GB Ram on 36 Viz Nodes | | RAM/node | 384 GiB (+ 3.0 TiB Optane on 180 fat nodes) | | Peak Performance | 2 PFlop/s (3.53 TFlop/s in single node) | | Internal Network | Mellanox Infiniband 100GbE | Disks and Filesystems[](https://docs.hpc.cineca.it/hpc/galileo.html#disks-and-filesystems "Link to this heading") ------------------------------------------------------------------------------------------------------------------- The storage organization conforms to **CINECA** infrastructure. General information are reported in [File Systems and Data Management](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#file-systems-and-data-management) section. In the following, only differences with respect to general behavior are listed and explained. Job Managing and SLURM Partitions[](https://docs.hpc.cineca.it/hpc/galileo.html#job-managing-and-slurm-partitions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- | **Partition** | **QOS** | **#Cores per job** | **Walltime** | **Max jobs/resources per user** | **Max memory per node (MB)** | **Priority** | **Notes** | | --- | --- | --- | --- | --- | --- | --- | --- | | g100\_all\_serial

(default) | noQOS | 4 cores | 04:00:00 | 4 cores

120 submitted jobs | 31,200

(30 GB) | 40 | on two login nodes

**budget free** | | g100\_all\_serial

(default) | qos\_install | 16 cores | 04:00:00 | 16 cores

1 running job | 100 GB | 40 | request to [superc@cineca.it](mailto:superc%40cineca.it) | | g100\_usr\_dbg | noQOS | 2 nodes | 01:00:00 | | 375,300

(366 GB) | 40 | | | g100\_usr\_dbg | qos\_ind | Depending on the specific agreement | Depending on the specific agreement | | 375,300

(366 GB) | 90 | Partition dedicated to specific kinds of users. | | g100\_usr\_prod

_g100\_usr\_smem_

**g100\_usr\_pmem** | noQOS | min = 1

max = 32 nodes | 24:00:00 | 100 running jobs

120 submitted jobs | 375,300

(366 GB) | 40 | runs on thin and persistent memory nodes

_runs only on thin nodes_

**runs only on persistent memory nodes** | | g100\_usr\_prod

_g100\_usr\_smem_

**g100\_usr\_pmem** | g100\_qos\_bprod | min = 1537 (33 nodes)

max = 3072 (64 nodes) | 24:00:00 | 100 running jobs

120 submitted jobs | 375,300

(366 GB) | 60 | runs on thin and persistent memory nodes

_runs only on thin nodes_

**runs only on persistent memory nodes** | | g100\_usr\_prod

_g100\_usr\_smem_

**g100\_usr\_pmem** | g100\_qos\_lprod | min = 1

max = 2 nodes | 4-00:00:00 | 2 nodes

100 running jobs

120 submitted jobs | 375,300

(366 GB) | 40 | runs on thin and persistent memory nodes

_runs only on thin nodes_

**runs only on persistent memory nodes** | | g100\_usr\_prod

_g100\_usr\_smem_

**g100\_usr\_pmem** | qos\_special | \> 32 nodes | \> 24:00:00 | | 375,300

(366 GB) | 40 | request to [superc@cineca.it](mailto:superc%40cineca.it) | | g100\_usr\_bmem | noQOS | 25 nodes | 24:00:00 | 100 running jobs

120 submitted jobs | 3,036,000

(3 TB) | 40 | runs on fat nodes | | g100\_usr\_interactive | noQOS | max = 0.5 node | 8:00:00 | 100 running jobs

120 submitted jobs | 375,300

(366 GB) | 40 | on nodes with GPUs

–gres=gpu:N (N=1) | | g100\_meteo\_prod | qos\_meteo | | 24:00:00 | | 375,300

(366 GB) | 40 | Partition reserved to meteo services, **NOT opened to production.**

Runs on thin nodes | Dedicated Services[](https://docs.hpc.cineca.it/hpc/galileo.html#dedicated-services "Link to this heading") ------------------------------------------------------------------------------------------------------------- ### Interactive Computing[](https://docs.hpc.cineca.it/hpc/galileo.html#interactive-computing "Link to this heading") Galileo 100 resources are also accessible via web browser on a Jupyter-based interface at the following link: [https://jupyter.g100.cineca.it/](https://jupyter.g100.cineca.it/) Further details are reported at the following [Interactive Computing](https://docs.hpc.cineca.it/services/interactive_computing.html#interactive-computing) . Please note that the service is considered in pre-production, thus the resources are not accounted from the budget and the service is provided with no warranty. --- # Miniconda — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Services and Tools](https://docs.hpc.cineca.it/services/services_and_tools.html) * Miniconda * [View page source](https://docs.hpc.cineca.it/_sources/services/miniconda.rst.txt) * * * Miniconda[](https://docs.hpc.cineca.it/services/miniconda.html#miniconda "Link to this heading") ================================================================================================== Miniconda is a minimal installer for Conda, a popular package manager for Python and other languages. Unlike the full Anaconda distribution, which includes hundreds of preinstalled packages and tools, Miniconda provides only the Conda package manager and Python. This lightweight installer allows users to create customized environments by installing only the packages they need. Miniconda is ideal for: * Users who want a smaller installation footprint. * Environments where storage space or bandwidth is limited. * Developers and researchers who prefer full control over package versions and dependencies. With Miniconda, users can: * Create and manage isolated environments with different Python versions. * Install packages from multiple channels such as conda-forge or bioconda. * Ensure reproducibility and compatibility across systems. How to Install Miniconda[](https://docs.hpc.cineca.it/services/miniconda.html#how-to-install-miniconda "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------- Important **Cleaning Up Anaconda3 previous Configuration from the Home Directory** Sometimes, previous Anaconda installations may interfere with the correct installation of Miniconda. For this reason, it is recommended to perform the following cleanup steps before proceeding with the installation: > * **Delete the Conda configuration file**: > > rm -f $HOME/.condarc > > Copy to clipboard > > * **Delete the Conda data directory**: > > rm -rf $HOME/.conda > > Copy to clipboard > > * **Remove Anaconda initialization from your shell configuration file**: > > > Open your `$HOME/.bashrc` and remove all lines related to Anaconda3. These lines are usually located at the end of the file and enclosed between the following markers: > > > > \# >>> conda initialize >>> > > ... > > \# <<< conda initialize <<< > > > > Copy to clipboard > > > > You can safely delete this entire block. To [install Miniconda3](https://docs.anaconda.com/miniconda/install/#quick-command-line-install) , you have to download the installation script by running: wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86\_64.sh Copy to clipboard Execute the downloaded script with: bash $HOME/Miniconda3-latest-Linux-x86\_64.sh Copy to clipboard During the installation, reply **“yes”** or **“ENTER”** to all prompts. At the end of the installation, reload the bash session to apply the modifications introduced by the installer: exec bash Copy to clipboard This command will start a new bash session in which the `(base)` Conda environment will be automatically activated. ### Configure Channels[](https://docs.hpc.cineca.it/services/miniconda.html#configure-channels "Link to this heading") Conda channels are configured at multiple levels: * Global configuration: `~/.condarc` * Environment-specific configuration: `/conda-meta/.condarc` To configure Conda to work correctly: * Disable automatic activation of the base environment: conda config \--set auto\_activate\_base false Copy to clipboard * Reload the bash session again: exec bash Copy to clipboard This will start a new session where the `(base)` environment will no longer be automatically activated. * Enable strict channel priority: conda config \--set channel\_priority strict Copy to clipboard * Add the conda-forge channel: conda config \--add channels conda-forge Copy to clipboard * Edit the following configuration files: * `$HOME/.condarc` * `$HOME/miniconda3/.condarc` In both files, comment out (by adding a `#` at the beginning of the line) any lines containing: \- https://repo.anaconda.com/pkgs/main \- https://repo.anaconda.com/pkgs/r Copy to clipboard Note In some versions of Conda, you may instead see: \- defaults Copy to clipboard In that case, comment out the `- defaults` line. Removing defaults from your Conda configuration will not break Conda. Conda will continue to function as expected. You can use conda-forge as your only channel without issues. These last three steps ensure that only the conda-forge channel is used, and disable the default Anaconda channels which may cause timeout errors or access errors when downloading packages on our HPC systems. For full documentation on this, see: [https://conda-forge.org/docs/user/transitioning\_from\_defaults/](https://conda-forge.org/docs/user/transitioning_from_defaults/) [https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually](https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually) ### Create a New Environment[](https://docs.hpc.cineca.it/services/miniconda.html#create-a-new-environment "Link to this heading") To create a new environment: conda create \-n new\_env Copy to clipboard Activate the environment: conda activate new\_env Copy to clipboard ### Install Packages from Conda-Forge[](https://docs.hpc.cineca.it/services/miniconda.html#install-packages-from-conda-forge "Link to this heading") If the desired package is available in the conda-forge channel, install it _inside_ the environment: conda install \-n new\_env Copy to clipboard Example: conda install \-n new\_env pytorch Copy to clipboard To check availability: conda search pytorch Copy to clipboard --- # What is Cloud Computing — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Introduction to HPC Cloud](https://docs.hpc.cineca.it/cloud/general/general_info.html) * What is Cloud Computing * [View page source](https://docs.hpc.cineca.it/_sources/cloud/general/what_is_cloud.rst.txt) * * * What is Cloud Computing[](https://docs.hpc.cineca.it/cloud/general/what_is_cloud.html#what-is-cloud-computing "Link to this heading") ======================================================================================================================================= Cloud computing is a **virtualization-based technology** that allows users remote access to a pool of virtual resources (typically CPUs, memory and storage) completely isolated one from another that can be used to create and operate one or more virtual machines depending on the needs. **HPC cloud**, or High-Performance Computing cloud, integrates high-performance computing resources and capabilities with cloud computing infrastructure. It combines the computational power and scalability of traditional HPC systems with the flexibility and on-demand nature of cloud services. Features of HPC Cloud[](https://docs.hpc.cineca.it/cloud/general/what_is_cloud.html#features-of-hpc-cloud "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------- HPC Cloud users are able to exploit the computational power of HPC machines with the added benefit of extreme flexibility. Users can organize their pool of computational resources how they see fit for their personal workflow, creating a single virtual machine that harnesses the full capability of all the resources at once, or smaller virtual machines that can communicate and interact with each other. More in details, within HPC Cloud the users can: * decide how to exploit computational and storage resources, creating their personal virtual infrastructure (**flexibility**) * manage their virtual machines, for example by deciding the operative system and software environments (**flexibility**) * scale the computational resources based on their needs allowing to handle varying workloads efficiently (**scalability**) * optimize the use of the resources without idle time (**resources optimization**) * access the resources remotely (**accessibility**) ### What to apply for: HPC or Cloud computing?[](https://docs.hpc.cineca.it/cloud/general/what_is_cloud.html#what-to-apply-for-hpc-or-cloud-computing "Link to this heading") | | HPC | Cloud Computing | | --- | --- | --- | | **Performance** | Target the highest possible. | Depends on workload, but generally, virtualization has a small impact. | | **User access** | HPC site staff authorization. | Once a project is granted, it is managed by the user. | | **Operating System** | It is chosen by HPC site staff given the HW constraints. Security updates are managed by HPC site. | Selected by the user. Security patches and updates are managed by the user. | | **Software Stack** | Mostly installed by HPC site staff. Users can install their own without _root_ privilege. The environment is provided _as is_. | The user is root on the VMs and can install all the required software stack. Users can modify the environment to suit their needs. | | **Snapshots of the environment** | Cannot be done | User can save snapshot images of the VMs. | | **Running simulations** | Users are provided with a job scheduler (SLURM) and have to wait for resources to free in order for their job to run | Users are able to run serial or parallel jobs how/when they need. | | **Very large simulations** | Users are provided with a job scheduler and have to wait for resources to free in order for their job to run. | Users can plan their workloads execution. | ### Service Models[](https://docs.hpc.cineca.it/cloud/general/what_is_cloud.html#service-models "Link to this heading") Cloud computing resources can be provided using different service models: * **Infrastructure as a Service (IaaS)**: Provides virtualized computing resources over the internet. Users are able to deploy those resources how they see fit. * **Platform as a Service (PaaS)**: Offers a platform allowing developers to build, deploy, and manage applications without worrying about the underlying infrastructure. * **Software as a Service (SaaS)**: Delivers software applications over the internet, accessible via a web browser, without the need for local installation or maintenance. In CINECA, we provide resources using the IaaS model (see section [CINECA HPC Cloud Model](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#cineca-hpc-cloud-model) for more information). --- # Singularity and Apptainer Containers — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Services and Tools](https://docs.hpc.cineca.it/services/services_and_tools.html) * Singularity and Apptainer Containers * [View page source](https://docs.hpc.cineca.it/_sources/services/singularity.rst.txt) * * * Singularity and Apptainer Containers[](https://docs.hpc.cineca.it/services/singularity.html#singularity-and-apptainer-containers "Link to this heading") ========================================================================================================================================================== On CINECA’s HPC clusters, the containerization _platforms_ available can be eighter **Singularity** or **Apptainer**. Both containerizaion _platforms_ are specifically designed to run scientific applications on HPC resources, enabling users to have full control over their environment. Singularity and Apptainer containers can be used to package entire scientific workflows, software, libraries and data. This means that you don’t have to ask your cluster admin to install anything for you - you can put it in a Singularity or Apptainer container and run. The official Singularity documentation for its last release is available [here](https://docs.sylabs.io/guides/latest/user-guide/) while the official Apptainer documentation for its last release is available [here](https://apptainer.org/docs/user/latest/) . Differences between Singularity and Apptainer[](https://docs.hpc.cineca.it/services/singularity.html#differences-between-singularity-and-apptainer "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- In this section basic information about the history of the Singularity project are provided in order to help users which have no prior experience with both Singularity and Apptainer to better understand the differences between those platforms. > * The Singularity project first begun as an open-source project in 2015 form a team of researchers at Lawrence Berkeley National Laboratory lead by Gregory Kurtzer. > > * In February 2018, the original leader of the Singularity project founded the Sylabs company to provide commercial support for Singularity. > > * In May 2020, Gregory Kurtzer left Sylabs but retained leadership of the Singularity open source project: this event cause a major fork inside the Singularity project. > > * In May 2021 Sylabs made a fork of the project and called SingularityCE while in November 30, 2021 when the move into the Linux Fundation of the Singularity open-source project has been [announced](https://apptainer.org/news/community-announcement-20211130/) > the Apptainer project born. > Currently, there are three products derived from the original Singularity project from 2015: > * Singularity: the commercial software supported by Sylabs. > > * SingularityCE: the open-source, community edition software also supported by Sylabs. > > * Apptainer: the fully open-source Singularity port under the Linux Fundation. > From a user perspective, quoting the announcement of the beginning of the Apptainer project: “**Apptainer IS Singularity**”. The Apptainer project makes no changes at the image format level. This means that default metadata within Singularity Image Format (SIF image) and their filesystems will retain the Singularity name without change ensuring that containers built with Apptainer will continue to work with installations of Singularity. Moreover, `singularity` as a command line link, is provided, and maintains as much of the CLI and environment functionality as possible. Important As a direct consequence of all the information previously reported, during the rest of the documentation all the command examples always use the `singularity` command. How to build a Singularity or Apptainer container image on your local machine[](https://docs.hpc.cineca.it/services/singularity.html#how-to-build-a-singularity-or-apptainer-container-image-on-your-local-machine "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- In this section the building procedure of container images with one between Singularity or Apptainer on your local machine is explained. A Singularity container image can be built in differnt ways. The simplest command used to build is: $ sudo singularity build \[local options...\] Copy to clipboard the `build` command can produce containers in 2 different output formats. Format can be specified by passing the fllowing `build option`: 1. **default**: a compressed read-only **Singularity Image Format (SIF)**, suitable for production. This is an _immutable object_. 2. **sandbox**: a writable **(ch)root directory** called _sandbox_, used for interactive development. To create those kind of output format use the `--sandbox` build option. The build `spec target` defines the method that `build` uses to create the container. All the _methods_ are listed in the table: | **Build method** | **Commands** | | --- | --- | | Beginning with library to build from the [Container Library](https://cloud.sylabs.io/library) | `sudo singularity build library://path/to/container_img[:tag]` | | Beginning with docker to build from [Docker Hub](https://hub.docker.com/) | `sudo singularity build docker://path/to/container_img[:tag]` | | Path to an existing container on your local machine | `sudo singularity build --sandbox ` | | Path to a directory to build from a _sandbox_ | `sudo singularity build ` | | Path to a [SingularityCE definition file](https://docs.sylabs.io/guides/latest/user-guide/definition_files.html) | `sudo singularity build ` | Since build can accept an existing container as a target and create a container in any of these two formats, you can convert an existing .sif container image to a sandbox and viceversa. Some experienced Docker users may be in possession of Docker images not available on any container registry (_e.g._ custom container images): those users will take benefits from the possibility to convert Docker images into Singluarity image files. Convert Docker container images into Singularity image files Important Before following this procedure, ensure that both Docker and one between Singularity or Apptiner are installed on your local system: the installation instructions can be found on the [Docker official documentation](https://docs.docker.com/engine/install/) and on the [Singularity official admin guide](https://docs.sylabs.io/guides/latest/admin-guide/installation.html) or on the [Apptaniner official admin guide](https://apptainer.org/docs/admin/main/installation.html) respectively. 1. Verify that the image exist by looking at the output of the docker command line utility. sudo docker image ls Copy to clipboard Remember to annotate the ID of the desired image 2. Generate a `.tar` archive form the desired image using the following command: sudo docker image save .tar Copy to clipboard 3. Build a Singularity image file starting from the freshly generated archive with: sudo singularity build .sif .tar Copy to clipboard At the end of a successfull building process, if not needed for other purposes, remove the Docker image archive. 4. Change the ownership to the Singularity image file to be able to move it to a remote host without any permissions related issues. sudo chown $(id \-nu):$(id \-ng) .sif Copy to clipboard For further informations on how to move files between local systems and one of the CINECA’s clusters, visit the [Data Transfer](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#data-transfer) section. ### Understanding Singularity or Apptainer definition file for building container images[](https://docs.hpc.cineca.it/services/singularity.html#understanding-singularity-or-apptainer-definition-file-for-building-container-images "Link to this heading") The Definition File (or “def file” for short) is like a set of blueprints explaining how to build a custom container image including specifics about the base Operative System to build or the base container to start from, as well as software to install, environment variables to set at runtime, files to add from the host system, and container metadata. A definition file is divided into two parts: > > | **Parts** | **Purpose** | > | --- | --- | > | Header | Describes the core operating system to build within the container.

In the Header all the information to configure the base operating system features needed within the container are reported.

_e.g. Linux distributions with its specific version or a base container image_ | > | Sections | The rest of the definition is comprised of sections.

Each section is defined by a `%` character followed by the name of the particular section.

All sections are optional, and a def file may contain more than one instance of a given section. | A definition file may look like this: \# This is a comment \# -- HEADER begin -- Bootstrap: docker From: ubuntu:{{ VERSION }} Stage: build \# -- HEADER end -- \# -- SECTIONS begin -- %arguments VERSION\=22.04 %setup touch /file1 touch ${APPTAINER\_ROOTFS}/file2 %files /file1 /file1 /opt %environment export LISTEN\_PORT\=54321 export LC\_ALL\=C %post apt-get update && apt-get install \-y netcat NOW\=\`date\` echo "export NOW=\\"${NOW}\\"" \>> $APPTAINER\_ENVIRONMENT %runscript echo "Container was created $NOW" echo "Arguments received: $\*" exec echo "$@" %startscript nc \-lp $LISTEN\_PORT \# -- SECTIONS end -- Copy to clipboard For further informations on how to write a custom definition file, users are strongly encouraged to visit the dedicated page both on [the official Singularity user guide](https://docs.sylabs.io/guides/latest/user-guide/definition_files.html) or [the official Apptainer user guide](https://apptainer.org/docs/user/main/definition_files.html#) . [![../_images/change_formats.png](https://docs.hpc.cineca.it/_images/change_formats.png)](https://docs.hpc.cineca.it/_images/change_formats.png) A quick outline over [SPACK](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#spack) , a package management tool compatible with Singularity, which can be used to deploy entire software stacks inside a container imageis provided. Advanced Singularity or Apptainer container build with Spack Spack ([full documentation here](https://spack.readthedocs.io/en/latest/) ) is a package manager for Linux and macOS, able to download and compile (almost!) automatically the software stack needed for a specific application. It is compatible with the principal container platforms (Docker, Singularity), meaning that it can be installed inside the container and in turn be used to deploy the necessary software stack inside the container image. This can be utterly useful in a HPC cluster environment, both to install applications as a root (inside the container), and to keep a pletora of ready-available software stacks (or even application built with different software stack versions) living in different containers (regardless of the outside environment). Getting Spack is an easy and fast three steps process: * Install the necessary dependencies, eg. on Debian/Ubuntu: `apt update; apt install build-essential ca-certificate coreutils curl enviroment-modules gfortran git gpg lsb-release python3 python3-distutils python3-venv unzip zip`. * Clone the repository: `git clone -c feature.manyFiles=true https://github.com/spack/spack.git`. * Activate spack, eg: for _bash/zsh/sh_: `source /spack/share/spack/setup-env.sh`. The very same operations can be put in the `%post` section of a Singularity definition file to have an available installation of Spack at the completion of the built. Alternatively, one can bootstrap from an image containing spack only and start from there the built of the container. For example: sudo singularity build \--sandbox docker://spack/ubunty-jammy Copy to clipboard **Spack Basic Usage** $ spack install openmpi@4.1.5+pmi fabrics\=ucx,psm2,verbs schedulers\=slurm %gcc@8.5.0 Copy to clipboard Generally speaking, the deployment of a software stack installed via spack is based on the following steps: 1. Build a container image. 2. Get spack in your container. 3. Install the software stack you need. In practice, and if foresight of building an _immutable_ SIF container image for compiling and running an application, one can proceed as follow: 1. Get `sandbox` container image hodling an installation of spack and open a **shell** with `sudo` and writable privileges (`sudo singularity shell --writable `). 2. Write a `spack.yaml` file for a spack environment listing all the packages and compilers your application would need (more detaile [here](https://spack.readthedocs.io/en/latest/environments.html) ). 3. Execute `spack concretize` and `spack install`, if the installation goes through and you are application can compile and run you are set to go: 1. either transform your sandobox `.sif` file fixing the changes to a conteiner image. 2. or, for a clean build, copy the `spacl.yaml` file in the conteiner in the specific `%` in a `*.sif` file fixing the changes to a conteiner imagefile section, activate spack and execute `spack concretize` and `spack install`. Following, a minimal example of a Singularity definition file: we bootstrap from a docker container holding a clean installation of ubuntu:22.04, we copy a ready made `spack.yaml` file in the container, get spack therein and use it to install the software stack as delineated in the `spack.yaml` file. Bootstrap: docker From: ubuntu:22.04 %files /some/example/spack/file/spack.yaml /spacking/spack.yaml %post \### We install and activate Spack apt-get update apt install \-y apt install build-essential ca-certificates coreutils curl environment-modules gfortran git gpg lsb-release python3 python3-distutils python3-venv unzip zip git clone \-c feature.manyFiles\=true https://github.com/spack/spack.git source /spack/share/spack/setup-env.sh \### We pretentiously deploy a software stack in a Spack environment spack env activate \-d /spacking/ spack concretize spack install %environment \### Custom evironment variables should be set here export VARIABLE\=MEATBALLVALUE Copy to clipboard Bindings[](https://docs.hpc.cineca.it/services/singularity.html#bindings "Link to this heading") -------------------------------------------------------------------------------------------------- A Singularity container image provides a standalone environment for software handling. However, it might still need files from the host system, as well as write privileges at runtime. As pointed out above, this last operation is indeed available when working with a sandbox, but it is not for an (_immutable_) **SIF** object. To provide for these needs, Singularity grants the possibility to mount files and directories from the host to the container. * In the default configuration, the directories `$HOME` , `/tmp` , `/proc` , `/sys` , `/dev`, and `$PWD` are among the system-defined bind paths * The `SINGULARITY_BIND` environment variable can be set (in the host) to specify the bindings. The argument for this option is a comma-delimited string of bind path specifications in the format `src[:dest[:opts]]` where src and dest are paths outside and inside of the container respectively; the dest argument is optional, and takes the same values as src if not specified. For example: `$ export SINGULARITY_BIND=/path/in/host:mount/point/in/container`. * Bindings can be specified on the command line when a container instance is started via the `--bind` option. The structure is the same as above, eg. singularity shell `--bind /path/in/host:/mount/point/in/container `. Enviroment variables[](https://docs.hpc.cineca.it/services/singularity.html#enviroment-variables "Link to this heading") -------------------------------------------------------------------------------------------------------------------------- Environment variables inside the container can be set in a handful of ways, see also [here](https://docs.sylabs.io/guides/latest/user-guide/environment_and_metadata.html) . At build time they should be specified in the %environment section of a Singularity definition file. Most of the variables from the host are then passed to the container except for `PS1`, `PATH` and `LD_LIBRARY_PATH` which will ne modified to contain default values; to prevent this behavior, one can use the `--cleanenv` option, to start a container instance with a clean environment. Further environment variables can be set, and host variables can be overwritten at runtime in a handful of ways: | **Scope** | **CLI Flag/Host-side variables** | | --- | --- | | Directly pass an environment variables to the containerized application | `--env MYVARIABLE="myvalue"` | | Directly pass a list of environment variables held in a file to the containerized application | `--env-file
` | | Automatically pass host-side defined variable to the containerized application | `export SINGULARITYENV_MYVARIABLE=myvalue` _on host machine_

results in `MYVARIABLE=myvalue` _inside the container_ | With respect to special `PATH` variables: | **Scope** | **Host-side variables** | | --- | --- | | Append to the `$PATH` variable | `export SINGULARITY_APPEND_PATH=
` | | Prepend to the `$PATH` variable | `export SINGULARITY_PREPEND_PATH=` | | Override the `$LD_LIBRARY_PATH` variable | `export SINGULARITYENV_LD_LIBRARY_PATH=`



**NOTE**

By default, inside the container the `LD_LIBRARY_PATH` is set to `/.singularity/libs`.

Users are strongly encouraged to inlude also this path when setting `SINGULARITYENV_LD_LIBRARY_PATH` | As a last disclaimer, we point out two additional variables which can be set in the host to manage the building process: | **Scope** | **Host-side variables** | | --- | --- | | Pointing to a directory used for caching data from the build process | `export SINGULARITY_CACHEDIR=` | | Pointing to a directory used for temporary build of the squashfs system | `export SINGULARITY_TMPDIR=` | Note All the aforementioned variables containing the `SINGULARITY` word can be interpred and correctly applied by Apptainer. However, Apptainer may complain about using those variables instead of using the Apptainer’s specific ones: to do so, users have to simply replace the occurance of `SINGULARITY` with `APPTAINER`. Containers in HPC environment[](https://docs.hpc.cineca.it/services/singularity.html#containers-in-hpc-environment "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------- In this sections, all the information necessary for the execution of Singularity or Apptainer containers along with all the container _platform_ flags are reported to perform their execution on CINECA’s clusters. In order to move locally built SIF images on CINECA’s clusters, consult the “Data Transfer” page under the [File Systems and Data Management](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#file-systems-and-data-management) section. However, Singularity allows pulling existing container images from container registries as the one seen in the third section. Pulling container images from registries can be done on CINECA’s cluster via the following command synthax: singularity pull registry://path/to/container\_img\[:tag\] Copy to clipboard This will create a SIF file in the directory where the command was run allowing the user to run the image just pulled. Parallel MPI Container The MPI implementation used in the CINECA clusters is OpenMPI (as opposed to MPICH). Singularity offers the possibility to run parallel applications compiled and installed in a container using the host MPI installation, as well as the bare metal capabilities of the host such as the Infiniband computer networking communication standard. This is the so called _Singularity hybrid approach_ where the OpenMPI installed in the container and the one on the host work in tandem to instantiate and run the job, see also the [documentation](https://docs.sylabs.io/guides/latest/user-guide/mpi.html) . Note Keep in mind that when exploiting the _Singularity hybrid approach_, the necessary MPI libraries from the host are automatically bound above the ones present in the container. The only caveat is that the two installations (container and host) of OpenMPI have to be compatible to a certain degree. The (default) installation specifics for each cluster are here listed: | **Cluster** | **OpenMPI version** | **PMI implementation** | **Specifics** | **Tweaks** | | --- | --- | --- | --- | --- | | Galileo100 | 4.1.1 | pmi2 | `--with-pmi`

`--with ucx`

`--with-slurm` | | | Leonardo | 4.1.6 | pmix\_v3 | `--with ucx`

`--with-slurm`

`--with-cuda` \* | `export PMIX_MCA_gds=hash` \*\* | | Pitagora | 4.1.6 | pmix\_v3 | `--with ucx`

`--with-slurm`

`--with-cuda` \* | `export PMIX_MCA_gds=hash` \*\* | \* only available in `boost_*` partitions. \*\* suppres PMIX WARNING when using srun. Note Even if the host and container hold different versions of OpenMPI, the application might still run in parallel, but at a reduced speed, as it might not be able to exploit the full capabilities of the host bare metal installation. A suite of container images holding compatible OpenMPI versions for the CINECA clusters are available at the [NVIDIA catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nvhpc/tags) , on which we dwell in the next section. GPU Aware Container To run GPU applications on accelerated clusters on first has to check his container image holds a compatible version of CUDA. The specifics are listed in the following table: | | **Driver Version** | **CUDA Version** | **GPU Model** | | --- | --- | --- | --- | | Galileo100 | 470.42.01 | 11.4 | NVIDIA V100 PCIe3 32 GB | | Leonardo | 535.54.03 | 12.2 | NVIDIA A100 SXM6 64 GB HBM2 | | Pitagora | 565.57.01 | 12.7 | NVIDIA H100 SXM 80GB HBM2e | while the [CUDA compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/) table is: | **CUDA Version** | **Required Drivers** | | --- | --- | | CUDA 12.x | from 525.60.13 | | CUDA 11.x | from 450.80.02 | One can surely install a working version of CUDA on his own, for example via Spack. However, a simple and effective way to obtain a container image provided with a CUDA installation is to bootstrap from an NVIDIA HPC SDK docker container, which already comes equipped with CUDA, OpenMPI and the NVHPC compilers. Such containers are available at the [NVIDIA catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nvhpc/tags) . Their tag follows a simple structure, `$NVHPC_VERSION-$BUILD_TYPE-cuda$CUDA_VERSION-$OS`, where: 1. `$BUILD_TYPE`: can either take the value devel or runtime. The first ones are usually heavier and employed to compile and install applications. The second ones are lightweight containers for deployment, stripped of all the compilers and applications not needed at runtime execution. 2. `$CUDA_VERSION`: an either take a specific value (e.g. ) or be a `multi`. The multi flavors hold up to three different CUDA version, and as such are much heavier. However, they can be useful to deploy the same base container on HPC with different CUDA specifics or to try out the performance of the various versions. In the following we provide a minimal Singularity definition file following the above principles, namely: bootstrap from a develop NVIDIA HPC SDK container, install the needed applications, copy the necessary binaries and files for runtime, pass to a lightweight container. This technique is called multistage build, more information available [here](https://docs.sylabs.io/guides/latest/user-guide/definition_files.html#multi-stage-builds) . Bootstrap: docker From: nvcr.io/nvidia/nvhpc:23.1-devel-cuda\_multi-ubuntu22.04 Stage: build %files \### Notice the asterisk when copying directories /directory/with/needed/files/in/host/\* /destination/directory/in/container /our/application/CMakeLists.txt /opt/app/CMakeLists.txt /some/example/spack/file/spack.yaml /spacking/spack.yaml %post \### We install and activate Spack apt-get update apt install \-y build-essential ca-certificates coreutils curl environment-modules gfortran git gpg lsb-release python3 python3-distutils python3-venv unzip zip git clone \-c feature.manyFiles\=true https://github.com/spack/spack.git . /spack/share/spack/setup-env.sh \### We pretentiously deploy a software stack in a Spack environment spack env activate \-d /spacking/ spack concretize spack install \### Make and install our application cd /opt/app && mkdir build cd build cmake \-DCMAKE\_INSTALL\_PREFIX\=/opt/app\_binaries .. make \-j make install ########################################################################################### \### We now only need to copy the necessary binaries and libraries for runtime execution ### ########################################################################################### Bootstrap: docker From: nvcr.io/nvidia/nvhpc:23.1-runtime-cuda11.8-ubuntu22.04 Stage: runtime %files from build /spacking/\* /spacking/ /opt/app\_binaries/\* /opt/app\_binaries/ Copy to clipboard ### Execute containerized application in an HPC environment[](https://docs.hpc.cineca.it/services/singularity.html#execute-containerized-application-in-an-hpc-environment "Link to this heading") As explained in the previous section as well as in the [documentation](https://docs.sylabs.io/guides/latest/user-guide/mpi.html) , if the MPI library installed in the container is compatible with that of the host system, Singularity will take care by itself of binding the necessary libraries to allow a parallel containerized application to run exploiting the cluster infrastructure. In practical terms, this means that one just need to launch it as: mpirun \-np $nnodes singularity exec Copy to clipboard In comparison, the following code snippet will launch the application using _MPI inside the container_, thus effectively running on a _single node_: singularity exec mpirun \-np $nnodes Copy to clipboard Regarding launching containerized applications needing GPU support, again Singularity is capable of binding the necessary libraries on its own, provided a compatible software version in the container and host has been deployed; full documentation is available [here](https://docs.sylabs.io/guides/latest/user-guide/gpu.html) . To achieve this, one just need to add the `--nv` or the `--nvccli` flag on the command line, namely: mpirun \-np $nnodes singularity exec \--nv Copy to clipboard Important In most recent versions of both Singularty and Apptainer, the `--nv` flag used for NVIDIA GPUs, has been replaced by the `--nvccli` flag. Note Similarly to what said about the _Singularity hybrid approach_ in the”Parallel MPI Container” tab, for GPU parallel programs, the necessary CUDA drivers and libraries from the host are automatically bound and employed inside the container provided the `--nv` or `--nvccli` flag is used when starting a container instance. e.g. `$ singularity exec --nv `. ### Cluster specific tweaks[](https://docs.hpc.cineca.it/services/singularity.html#cluster-specific-tweaks "Link to this heading") In this section the specific version of Singularity or Apptainer installed on each CINECA’s cluster are reported along with some useful information to help users properly executing their containerized applications. Galileo100 On Galileo100, [Singularity 3.8.0](https://docs.sylabs.io/guides/3.8/user-guide/) is available on the login nodes and on the partitions. Beware that, for the Galileo100 cluster, nodes with GPU are available under both the Interactive Computing service and by requesting the `g100_usr_intercative` Slurm partition with one main difference: | **Platform** | **Maximum number of GPUs per Job** | | --- | --- | | Interactive Computing service | 2 | | `g100_usr_interactive` Slurm partition | 1 | The necessary MPI, Singularity and CUDA modules are the following: > * `module load profile/advanced` (profile with additional modules) > > * `module load autoload singularity/3.8.0--bind--openmpi--4.1.1` > > * `module load cuda/11.5.0` > Note The `module load autoload singularity/3.8.0--bind--openmpi--4.1.1` command automatically loads the following modules: * `singularity/3.8.0--bind–openmpi–4.1.1` * `zlib/1.2.11--gcc–10.2.0` * `openmpi/4.1.1--gcc--10.2.0-cuda–11.1.0` The following code snippet is an example of a Slurm job script for running MPI parallel containerized applications on the Galileo100 cluster. Notice that the `--cpus-per-task` option has been set to **48** to fully exploit the CPUs in the `g100_usr_prod` partition. #!/bin/bash #SBATCH --nodes=6 #SBATCH --ntasks-per-node=1 #SBATCH --cpu-per-task=48 #SBATCH --mem=30GB #SBATCH --time=00:10:00 #SBATCH --out=slurm.%j.out #SBATCH --err=slurm.%j.err #SBATCH --account= #SBATCH --partition=g100\_usr\_prod module purge module load profile/advanced module load autoload singularity/3.8.0--bind--openmpi--4.1.1 module load cuda/11.5.0 mpirun \-np 6 singularity exec Copy to clipboard Leonardo **Necessary modules and Slurm job script example** On Leonardo, [Singularity PRO 4.3.0](https://docs.sylabs.io/guides/4.3/user-guide/) is availabe on the login nodes and on the partitions. The necessary MPI, Singularity and CUDA modules are the following: > * `module load hpcx-mpi/2.19` > > * `module load cuda/12.2` > The following code snippet is an example of a Slurm job script for running MPI parallel containerized applications on the Leonardo cluster with GPU support. In order to equally and fully exploit the **32 cores** and **4 GPUs** of the `boost_usr_prod` partition, one needs to set `--ntasks-per-node=4`, `--cpu-per-task=8` and `--gres=gpu:4`. As a redundant but necessary measure, we also set the number of threads to eight manually via `export OMP_NUM_THREADS=8`. #!/bin/bash #SBATCH --nodes=6 #SBATCH --ntasks-per-node=4 #SBATCH --cpu-per-task=8 #SBATCH --gres=gpu:4 #SBATCH --mem=30GB #SBATCH --time=00:10:00 #SBATCH --out=slurm.%j.out #SBATCH --err=slurm.%j.err #SBATCH --account= #SBATCH --partition=boost\_usr\_prod export OMP\_NUM\_THREADS\=8 module purge module load hpcx-mpi/2.19 module load cuda/12.2 mpirun \-np 6 singularity exec \--nv Copy to clipboard As explained above, provided the container and host OpenMPI share a compatible pmi, the application can be launched via the srun command after having allocated the necessary resources. For example: salloc \-t 03:00:00 \--nodes\=6 \--ntasks-per-node\=4 \--ntasks\=24 \--gres\=gpu:4 \-p boost\_usr\_prod \-A export OMP\_NUM\_THREADS\=8 srun \--nodes\=6 \--ntasks-per-node\=4 \--ntasks\=24 singularity exec \--nv Copy to clipboard Pitagora _tab under constuction_ On Pitagora, [Apptainer 1.4.0](https://apptainer.org/docs/user/1.4/) is available on the login nodes and on the partitions. --- # CINECA HPC Cloud Model — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Introduction to HPC Cloud](https://docs.hpc.cineca.it/cloud/general/general_info.html) * CINECA HPC Cloud Model * [View page source](https://docs.hpc.cineca.it/_sources/cloud/general/cineca_cloud_model.rst.txt) * * * CINECA HPC Cloud Model[](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#cineca-hpc-cloud-model "Link to this heading") ========================================================================================================================================== This page describes how CINECA provides HPC Cloud resources to its users. Note CINECA HPC Cloud infrastructure is certified ISO 27001 since 2022 for **“Servizi informatici HPC in cloud per la ricerca in ambito life science”** and since 2025 for **“Erogazione di servizi IaaS per ricerca e innovazione su HPC Cloud”**. Details can be found [here](https://www.cineca.it/it/chi-siamo/cineca-oggi/certificazioni) . CINECA Service Model[](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#cineca-service-model "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------- CINECA HPC Cloud infrastructure is provided via an **Infrastructure as a Service (IaaS)** model. In IaaS model, the Cloud Provider administrates the hardware and virtualization layers of the infrastructure and provides both computing resources (virtual CPUs, storage, network, GPUs…) and high-level APIs (dashboards, command line (CLI) tools) that users can employ to control the resources they were granted. HPC Cloud use cases[](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#hpc-cloud-use-cases "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------ Cloud computing means **paramount flexibility**. With a cloud IaaS model, users are able to setup their project environment as they see fit, using all the infrastructure tools and resources and with the support provided by CINECA to meet their specific needs. CINECA users rely on the HPC Cloud infrastructure to address different use cases. The list below is not meant to be exhaustive, but to provide examples of scenarios where HPC Cloud can be particularly useful. Hosting of data processing and analysis services (typical Infrastructure as a Service, IaaS).​ Hosting of HPC mini-cluster with adequate performance.​ Hosting of data management services receiving or exposing data from/to web.​ Hosting of data management services receiving or exposing data from/to internal CINECA HPC infrastructure.​ Hosting of workload processing sensitive data.​ Bridging HPC Infrastructure, e.g. hosting front-end services for management of workloads on CINECA HPC system.​ Flexible and automated deployment via Kubernetes on top of OpenStack of containerized workflows.​ Collaborative infrastructure deployment within a user tenant (Infrastructure as Code, IaC).​ Everything that requires performance and flexibility (respect to the HPC cluster).​ Responsibilities[](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#responsibilities "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------ While CINECA is responsible for the provisioning and maintenance of the hardware and virtualization layer (OpenStack), the users are responsible for anything they set up and install on their project (e.g. network setup, OS and applications on virtual machines, access to services and VMs). A clear separation of roles in using the service is represented in the scheme below: ![../../_images/cloud_model.png](https://docs.hpc.cineca.it/_images/cloud_model.png) | | | | | | --- | --- | --- | --- |Roles and responsibilities[](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#id1 "Link to this table") | **Name** | **Description** | **Role** | **Responsibilities** | | --- | --- | --- | --- | | **CINECA** | Cloud provider | * Administers physical infrastructure

* Provides virtualization layer and API tools | * Maintaining hardware and virtualization layer

* User support | | **User Admin** | * Users with granted budget on CINECA HPC cloud

* Project PIs and collaborators in [CINECA UserDB](http://userdb.hpc.cineca.it/) | * Create and manage cloud resources via the provided APIs (dashboard or CLI)

* Responsible for all the resources they create (VMs, storage, networks,…) | * Administer of the resources

* Maintain VMs for which they have admin privileges

* Implement security measures

* Backups/snapshots of resources during the project and at the end of the validity period | | **User** | Users with granted access to the project VMs by User Admins. | Can utilize VMs and services they have been granted access to by User Admins. | Maintain the VMs for which they have admin privileges | Any user (_“User Admins”_ or _“Users”_) with administration privileges on IaaS resources (VMs) has the responsibility to maintain the security (security patch, fix) on those resources. In particular, they have the responsibility to perform VMs and volume data backups. See the dedicated page for [Security guidelines](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#security-guidelines) information. Warning Currently snapshots and backups of resources are stored in the same HPC Cloud infrastructure. From the project management perspective, CINECA will interact only with _“User Admins”_. At the end of the project validity, the _“User Admins”_ will receive communication from CINECA staff that the project as expired with the date by when the resources will be removed. It is _“User Admins”_ responsibility to make copy of the necessary VMs or data before that date. --- # Unknown 1 © 2018 The MathWorks, Inc. Accelerating and Parallelizing MATLAB Code on HPC Infrastructure Francesca Perino –Sam Marshalik-Sergio ObandoQuintero Application Engineering Team 2 Choose a Parallel Computing Solution ▪Do you want to process your data faster? ▪Do you want to offload to a cluster? ▪Do you want to scale up your big data calculation? 3 Practical Application of Parallel Computing ▪Why parallel computing? ▪Need faster insight on more complex problems with larger datasets ▪Computing infrastructure is broadly available (multicore desktops, GPUs, clusters) ▪WhyparallelcomputingwithMATLAB ▪Leveragecomputationalpowerofmorehardware ▪Accelerateworkflowswithminimaltonocodechangestoyouroriginalcode ▪Focusonyourengineeringandresearch,notthecomputation 4 Parallel Computing Paradigm Multicore Desktops Core 3 Core 1Core 2 Core 4 Worker Worker Worker Worker MATLAB multicore 5 Cluster Parallel Computing Paradigm Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker WorkerWorkerWorkerWorkerWorkerWorker 6 Migrate execution to a cluster environment MATLAB MATLAB Distributed Computing Server GPU Multi-core CPU Parallel Computing Toolbox GPU Multi-core CPU 7 Cluster Computing Paradigm ▪Prototype on the desktop ▪Integrate with existing infrastructure ▪Access directly through MATLAB User Desktop Headnode Compute Nodes Parallel Computing Toolbox MATLAB MATLAB Distributed Computing Server 8 Parallel Computing Paradigm NVIDIA GPUs Using NVIDIA GPUs MATLAB Desktop (client) GPU cores Device Memory 9 Steps for writing a MATLAB parallel code Technology / Product 1.Best practices in programming ▪Identify bottlenecks (e.g. Profiler, Code analyzer) ▪Vectorization& pre-allocation 2.Better algorithms ▪Different algorithmic approach to solve the same problem ▪Themostrecent MATLAB release 3.More processors, cores, and GPUs ▪Utilize high level parallelconstructs (e.g. parpool, parfor) ▪Scale to clusters, grids, and clouds web(fullfile(docroot, 'matlab/matlab\_prog/techniques-for-improving-performance.html')) 10 Example: Block Processing Images ▪Calculate a function at grid points ▪Take the mean of larger blocks 11 Best Practices ▪Profile your code ▪Minimize file I/O ▪Reuse existing graphics components ▪Avoid printing to Command Window 12 Access Multiple Files to Import Specific Columns 13 Access Multiple Files to Import Specific Columns 14 Steps for writing a MATLAB parallel code Technology / Product 1.Best practices in programming ▪Identify bottlenecks (e.g. Profiler, Code analyzer) ▪Vectorization& pre-allocation 2.Better algorithms ▪Different algorithmic approach to solve the same problem ▪Themostrecent MATLAB release 3.More processors, cores, and GPUs ▪Utilize high level parallelconstructs (e.g. parpool, parfor) ▪Scale to clusters, grids, and clouds web(fullfile(docroot, 'matlab/matlab\_prog/techniques-for-improving-performance.html')) 15 Exercise: Birthday Paradox ▪What is the probability that in a group of 23 randomly selected individual, at least two of them will share the same birthday? 16 Exercise: Birthday Paradox Implementation ▪Profile runBirthdaySum.m ▪EditrunBirthdayUnique1.m –TODO: without a FOR loop create a list with a random birthday for each member in the group ▪EditrunBirthdayVec.m –TODO: try a different algorithmic approach based on |sort| to solve the same problem 17 © 2018 The MathWorks, Inc. Parallel and Distributed Computing with MATLAB 18 Steps for writing a MATLAB parallel code Technology / Product 1.Best practices in programming ▪Identify bottlenecks (e.g. Profiler, Code analyzer) ▪Vectorization& pre-allocation 2.Better algorithms ▪Different algorithmic approach to solve the same problem ▪Themostrecent MATLAB release 3.More processors, cores, and GPUs ▪Utilize high level parallelconstructs (e.g. parpool, parfor) ▪Scale to clusters, grids, and clouds web(fullfile(docroot, 'matlab/matlab\_prog/techniques-for-improving-performance.html')) 19 Programming Parallel Applications ▪Built-in multithreading –Automatically enabled in MATLAB since R2008a –Multiple threads in a single MATLAB computation engine ▪Parallel-enabled MATLAB Toolboxes –Enable parallel computing support by setting a flag or preference ..., ‘UseParallel’, true) 20 Parallel Computing T OOLBOXES Worker Worker Worker Worker Worker Worker T OOLBOXES B LOCKSETS 21 Parallel-enabled Toolboxes (MATLAB ® Product Family) Enable acceleration by setting a flag or preference Optimization Estimation of gradients Statistics and Machine Learning GPU-enabled functions, parallel training Neural Networks Deep Learning, Neural Network training and simulation Image Processing Batch Image Processor, Block Processing, GPU-enabled functions Computer Vision Bag-of-words workflow Signal Processing and Communications GPU-enabled FFT filtering, cross correlation, BER simulations Other Parallel-enabled Toolboxes 22 Programming Parallel Applications ▪Built in support –..., ‘UseParallel’, true) ▪Simple programming constructs –parfor, batch Ease of Use Greater Control 23 Embarrassingly Parallel: Independent Tasks or Iterations ▪No dependencies or communication between tasks ▪Examples: –Monte Carlo simulations –Parameter sweeps –Same operation on many files Time Time 24 Mechanics of parforLoops a = zeros(10, 1) parfori = 1:10 a(i) = i; end a a(i) = i; a(i) = i; a(i) = i; a(i) = i; Worker Worker Worker Worker 1 2 3 4 5 6 7 8 9 10 1 23 4 5 6 7 8 9 10 25 Example: Estimate 흅using the Buffon-Laplace method 26 Factors Governing the Speedup of parforLoops ▪No speedup because computation time too short ▪Execution may be slow because of •Memory limitations (RAM) •File access limitations ▪Implicit multithreading •MATLAB uses multiple threads for speedup of some operations •Use Task Manager or similar on serial code to check on that ▪Unbalanced load due to iteration execution times •Avoid some iterations taking multiples of the execution time of other iterations. 27 Programming Parallel Applications ▪Built in support –..., ‘UseParallel’, true) ▪Simple programming constructs –parfor, batch ▪Full control of parallelization –spmd, parfeval Ease of Use Greater Control 28 DatatypeMemory Location Use case tall Disks Pre-processing, statistics,machine learning distributed Cluster Sparse and dense numerics gpuArray GPU GPU computations Datatypes for Scaling Data Represent data not in “normal” memory 29 Distributed Arrays MATLAB and Parallel Computing ToolboxMATLAB Distributed Computing Server parpool(‘local’) x = A\\b; % prototype with small A,b % A,bare distributed arrays parpool() x = A\\b; % For largeA,b % A,bare distributed arrays Develop applications once, change run environment by changing the profile 112641 122742 132843 153045 163146 173247 203550 213651 223752 30 Example: Estimate 흅using the Buffon-Laplace method ▪We want to speed up the estimation of 흅for 10 9 trials –Define a 10^9-by-1 codistributedarrays, distributed by columns with a uniform partition scheme. –Create on xworkers ›x0 = a \* rand(nNeedles,1,codistributor); ›y0 = b \* rand(nNeedles,1,codistributor); ›phi= 2 \* pi \* rand(nNeedles,1,codistributor); 31 Offloading Computations T OOLBOXES B LOCKSETS Scheduler Work Result Worker Worker Worker Worker 32 Offloading Computations ▪Send desktop code to cluster resources –No parallelism required within code –Submit directly from MATLAB ▪Leverage supplied infrastructure –File transfer / path augmentation –Job monitoring –Simplified retrieval of results ▪Scale offloaded computations MATLAB code Cluster Computer Cluster Scheduler 33 Migrate to Cluster / Cloud ▪Use MATLAB Distributed Computing Server ▪Change hardware without changing algorithm 34 MATLAB Desktop (Client) Offloading Serial Computations with batch ▪Offload the computation to a workstation targets compute-intensive applications Result Work Worker batch(...) 35 Offload and Scale Computations with batch with a Parallel Pool ▪batch jobs are particularly suitable when you are working on a compute cluster. MATLAB Desktop (Client) Result Work Worker Worker Worker Worker batch(...,'Pool',...) 36 Estimate 흅using the Buffon-Laplace method 37 Use MATLAB Distributed Computing Server MATLAB Desktop (Client) Local Desktop Computer Profile (Local) MATLAB code 1.Prototype code 38 Use MATLAB Distributed Computing Server 1.Prototype code 2.Get access to an enabled cluster Cluster Computer Cluster Scheduler Profile (Cluster) 39 Use MATLAB Distributed Computing Server 1.Prototype code 2.Get access to an enabled cluster 3.Switch cluster profile to run on cluster resources MATLAB Desktop (Client) Local Desktop Computer Profile (Local) Cluster Computer Cluster Scheduler Profile (Cluster) MATLAB code 40 ▪Offload computation: –Free up desktop –Access better computers ▪Scale speed-up: –Use more cores –Go from hours to minutes ▪Scale memory: –Utilize tall arrays and distributed arrays –Solve larger problems without re-coding algorithms Cluster Computer Cluster Scheduler Take Advantage of Cluster Hardware MATLAB Desktop (Client) 41 Summary ▪Easily develop parallel MATLAB applications without being a parallel programming expert ▪Speed up the execution of your MATLAB applications using additional hardware ▪Develop parallel applications on your desktop and easily scale to a cluster when needed 42 Parallel Computing with MATLAB –Beyond PARFOR Well-known features ▪parallel-enabled toolboxes ▪parfor/parsim ▪gpuArray Full spectrum of support ▪batch submission, jobs and tasks batch, createJob, createTask ▪asynchronous queue for feval parfeval ▪parallel support for big data tall, mapreduce ▪distributed arrays (“global arrays”) distributed, codistributed ▪message passing labSend, labReceive tutorials 43 Some Other Valuable Resources ▪MATLAB Documentation –MATLAB Advanced Software Development Performance and Memory –Parallel Computing Toolbox ▪Parallel and GPU Computing Tutorials –https://www.mathworks.com/videos/series/parallel-and-gpu-computing-tutorials- 97719.html ▪Parallel Computing on the Cloud with MATLAB –http://www.mathworks.com/products/parallel-computing/parallel-computing-on-the- cloud/ 44 45 46 47 Scheduling Jobs and Tasks Scheduler Job Results Worker Worker Worker Worker Task Result Task Task Task Result Result Result MATLAB Desktop (Client) 48 Example: Scheduling different solvers on the same ODE system 49 © 2018 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc.See www.mathworks.com/trademarksfor a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders. © 2015 The MathWorks, Inc. --- # Budget and accounting — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Introduction to HPC Cloud](https://docs.hpc.cineca.it/cloud/general/general_info.html) * Budget and accounting * [View page source](https://docs.hpc.cineca.it/_sources/cloud/general/budget_accounting.rst.txt) * * * Budget and accounting[](https://docs.hpc.cineca.it/cloud/general/budget_accounting.html#budget-and-accounting "Link to this heading") ======================================================================================================================================= Following the [CINECA HPC Cloud Model](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#cineca-hpc-cloud-model) , the users manage autonomously the Cloud projects (tenants), one or more, to which they are associated. To access Cloud resources, you need to get a CINECA HPC account. For detailed instructions on creating an account, please refer to the [How to become a User](https://docs.hpc.cineca.it/general/users_account.html#how-to-become-a-user) section. Each tenant is then composed by a pool of virtual resources (project quota/budget) which are defined in terms of: * Number of vCPUs * GB of RAM * GB of storage * Number of public IP addresses (floating IPs) On request, you can be also granted * Number of GPUs * Additional storage for shares Upon accessing the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) with HPC credentials, within the assigned tenants, you can autonomously create and manage all components of the virtual infrastructure (VMs, Networks, Security Policies, Load Balancer Policies, and so on) in compliance with the Access Policies accepted at the time of registration on UserDB (see the [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html#operative-manual) for specific operations details). When resource consuming operations, such as virtual machine creation, are performed, the request is validated against the maximum quota permitted for the current project. The users can also autonomously monitor the usage of the assigned resources in the OpenStack Horizon Dashboard, see [Instance: manage and monitor](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_manage.html#instance-manage-and-monitor) page. --- # EFGW Gateway — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [EUROfusion](https://docs.hpc.cineca.it/specific_users/specific_users.html) * EFGW Gateway * [View page source](https://docs.hpc.cineca.it/_sources/specific_users/gateway.rst.txt) * * * EFGW Gateway[](https://docs.hpc.cineca.it/specific_users/gateway.html#efgw-gateway "Link to this heading") ============================================================================================================ EFGW is the new EUROfusion Gateway system hosted by CINECA in the headquarter Casalecchio di Reno, Bologna, Italy. The cluster is supplied by Lenovo Corp. and is equipped with 15 AMD nodes, including 4 nodes with fat memory (1.5 TB), and 1 node with 4 H100 GPUs and local SSD storage. How to get a User Account[](https://docs.hpc.cineca.it/specific_users/gateway.html#how-to-get-a-user-account "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------- Users of the old EUROfusion Gateway were migrated on the new system with the same username. For new users: to get access to EFGW the following steps are required: > * Register on [UserDB portal](https://userdb.hpc.cineca.it/) > > * complete the registration filling your affiliation in the **Institution** page and uploading a valid Identity Document in **Documents for HPC** page. > > * Download the [`Gateway User Agreement`](https://docs.hpc.cineca.it/_downloads/730f4c3650065029eba04a3ecdd7adef/eurofusion_gateway_user_agreement_26_10_2022.pdf) > (GUA) > > * Fill and sign the GUA, send it via email to EUROfusion Coordination Officer Denis Kalupin (Denis.Kalupin-at-euro-fusion.org) > > * After the GUA is signed by the EUROfusion Coordination Officer, the user will receive an email from CINECA with final instructions. > Access to the System[](https://docs.hpc.cineca.it/specific_users/gateway.html#access-to-the-system "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------- The machine is reachable via `ssh` (secure Shell) protocol at hostname point: **login.eufus.eu**. The connection is established, automatically, to one of the available login nodes. It is also possible to connect to **EFGW** using one the specific login hostname points: > * **viz05-ext.efgw.cineca.it** > > * **viz06-ext.efgw.cineca.it** > > * **viz07-ext.efgw.cineca.it** > > * **viz08-ext.efgw.cineca.it** > Each login node is equipped with two AMD EPIC 9254 24-Core Processors. An alias hostname pointing to all the login nodes in a round-robin fashion, will be set-up in the next weeks. Warning **The mandatory access to EFGW is the two-factor authentication (2FA) via the dedicated provisioner efgw**. Get more information at section [Access to the Systems](https://docs.hpc.cineca.it/general/access.html#access-to-the-systems) . **Please note**: EFGW users have to obtain the ssh certificate from the **efgw** provisioner. * In the section [How to activate the 2FA and the OTP generator](https://docs.hpc.cineca.it/general/access.html#how-to-activate-the-2fa-and-the-otp-generator) use the [step-CA EFGW client](https://sso.hpc.cineca.it/realms/EFGW/account) in the place of the step-CA CINECA-HPC client * In the section [How to configure smallstep client](https://docs.hpc.cineca.it/general/access.html#how-to-configure-smallstep-client) **Step 3**, obtain the ssh certificate from the efgw provisioner step ssh login 'username' \--provisioner efgw Copy to clipboard * In the section [How to manage authentication certificates](https://docs.hpc.cineca.it/general/access.html#how-to-manage-authentication-certificates) , use the efgw provisioner in all the key _step_ commands (certificate re-generation, certificate creation in file format) How to access EFGW with NX[](https://docs.hpc.cineca.it/specific_users/gateway.html#how-to-access-efgw-with-nx "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------- 1. Get a ssh key with _step_ and put it in your $HOME/.ssh folder step ssh certificate 'username' \--provisioner efgw ~/.ssh/gw\_key Copy to clipboard 2. Configure a NX session as follows: > [![../_images/nx1.png](https://docs.hpc.cineca.it/_images/nx1.png)](https://docs.hpc.cineca.it/_images/nx1.png) > [![../_images/nx2.png](https://docs.hpc.cineca.it/_images/nx2.png)](https://docs.hpc.cineca.it/_images/nx2.png) > [![../_images/nx3.png](https://docs.hpc.cineca.it/_images/nx3.png)](https://docs.hpc.cineca.it/_images/nx3.png) > [![../_images/nx4.png](https://docs.hpc.cineca.it/_images/nx4.png)](https://docs.hpc.cineca.it/_images/nx4.png) You can freely install software on your NX desktop using the “flatpak” package. Please refer to the [official documentation](https://docs.flatpak.org/en/latest/using-flatpak.html) for instructions on how to use it. **Please keep in mind that you need to use it with the –user flag.** System Architecture[](https://docs.hpc.cineca.it/specific_users/gateway.html#system-architecture "Link to this heading") -------------------------------------------------------------------------------------------------------------------------- The cluster, supplied by Lenovo, is based on AMD processors: > * 10 nodes with two AMD EPYC 9745 128-Core Processors and 738 GB DDR5 RAM per node > > * 4 nodes with two AMD EPYC 9745 128-Core Processors and 1511 GB DDR5 RAM per node > > * 1 node with two AMD EPIC 9354 32-Core Processors, 4 H100 GPUs, and 738 GB DDR5 RAM per node > File Systems and Data Management[](https://docs.hpc.cineca.it/specific_users/gateway.html#file-systems-and-data-management "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------- The storage organization conforms to **CINECA** infrastructure. General information are reported in [File Systems and Data Management](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#file-systems-and-data-management) section. In the following, only differences with respect to general behavior are listed and explained. The storage is organized as a replica of the previous Gateway cluster with the data of **/afs** and **/pfs** areas copied on the new lustre storage system (no afs available, only the data were copied). Please notice that the path **/gss\_efgw\_work**, linked to the /pfs areas on the old Gateway, does not exist on the new Gateway. The TMPDIR is defined: > * on the local SSD disks on login nodes (2.5 TB of capacity), mounted as `/scratch_local` (`TMPDIR=/scratch_local`). This is a shared area with no quota, remove all the files once they are not requested anymore. A cleaning procedure will be enforced in case of improper use of the area. > > * on the local SSD disk on the GPU node (850 GB of capacity, default size 10 GB ) > > * on RAM on all the 14 cpu-only, diskless compute nodes (with a fixed size of 10 GB) > On the GPU node, a larger local TMPDIR area can be requested, if needed, with the slurm directive: > $ SBATCH \--gres\=tmpfs:XXG > > Copy to clipboard up to a maximum of 212.5 GB. Environment and Customization[](https://docs.hpc.cineca.it/specific_users/gateway.html#environment-and-customization "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------- The main tools and compilers are available through the module command when logging into the cluster: > $ module av > > Copy to clipboard To have all the modules of aocc, gcc, and OneAPI stacks installed from CINECA staff, you need to load the “cineca-modules” module and execute “module av” command: > $ module load cineca-modules > $ module av > > Copy to clipboard For getting information about “module” usage, compilers, and mpi libraries you can consult the [The module command](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#the-module-command) and [Compilers](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#compilers) You can install any additional software you may need with [flatpack](https://docs.flatpak.org/en/latest/using-flatpak.html) or [SPACK](https://docs.hpc.cineca.it/hpc/hpc_enviroment.html#spack) . ### How to make your $HOME/public open to all users[](https://docs.hpc.cineca.it/specific_users/gateway.html#how-to-make-your-home-public-open-to-all-users "Link to this heading") In order to configure on the new EFGW your $HOME/public as it was on the old EFGW afs filesystem, please add the proper ACL to your $HOME directory as follows: > $ setfacl \-m g:g2:x $HOME > > Copy to clipboard Job Managing and Slurm Partitions[](https://docs.hpc.cineca.it/specific_users/gateway.html#job-managing-and-slurm-partitions "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------ In the following table you can find informations about the Slurm partitions on the EFGW cluster. | **Partition** | **QOS** | **#Cores per job** | **Walltime** | **Max jobs/res. per user** | **Max memory per node** | **Priority** | **Notes** | | --- | --- | --- | --- | --- | --- | --- | --- | | gw | noQOS | max=768 cores | 48:00:00 | 2000 submitted jobs | 735 GB / 1511 GB | 40 | Four fat memory nodes | | qos\_dbg | max=128 cores | 00:30:00 | Max 128 cores | 735 GB / 1511 GB | 80 | Can run on max 128 cores | | qos\_gwlong | max=256 cores | 144:00:00 | Max 128 cores, 2 running jobs | 735 GB / 1511 GB | 40 | Four fat memory nodes | | gwgpu | noQOS | max=16 cores/1 gpu / 188250 MB | 08:00:00 | 1 running job | 735 GB | 40 | | Note In the new Gateway the debug partition has been replaces by a QoS. How to request support[](https://docs.hpc.cineca.it/specific_users/gateway.html#how-to-request-support "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------- For general support: Please write a mail to [superc@cineca.it](mailto:superc%40cineca.it) specifying EFGW in the Subject. For problems related to IMAS-ITER software and installations: please refer to the ACH-04 (PSNC) support: [https://confluence.eufus.psnc.pl/spaces/PSNCACH04/overview](https://confluence.eufus.psnc.pl/spaces/PSNCACH04/overview) --- # DNS guidelines — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Tenants Administration](https://docs.hpc.cineca.it/cloud/tenant_adm/index_tenants_administration.html) * DNS guidelines * [View page source](https://docs.hpc.cineca.it/_sources/cloud/tenant_adm/dns_guidelines.rst.txt) * * * DNS guidelines[](https://docs.hpc.cineca.it/cloud/tenant_adm/dns_guidelines.html#dns-guidelines "Link to this heading") ========================================================================================================================= DNS name[](https://docs.hpc.cineca.it/cloud/tenant_adm/dns_guidelines.html#dns-name "Link to this heading") ------------------------------------------------------------------------------------------------------------- It is possible to ask CINECA for a DNS name association to the virtual machine by sending an email to [superc@cineca.it](mailto:superc%40cineca.it) . In CINECA DNS, it is necessary to comply with the following rules: * The **reverse** of the Floating IP (PTR record) must be set to the hostname of the VM, with the following naming convention: > * for external users: .ext.cineca.it > > * for CINECA staff: .cineca.it > * The **record A** in the DNS is set accordingly to the previous point. * If the service should be exposed with a different name, you can ask to set the **CNAME** with the chosen different name. If no other information is provided, only the record A will be set. Additionally, you might set up a CNAME with your DNS provider of choice. It is not possible to set the **PTR record** in CINECA DNS, if the record A has been set on an external DNS. --- # Load Balancer — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Load Balancer * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/os_components/load_balancers.rst.txt) * * * Load Balancer[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/load_balancers.html#load-balancer "Link to this heading") ====================================================================================================================================== [OpenStack Octavia service](https://docs.openstack.org/octavia/latest/reference/introduction.html) enables the deployment of load balancing solutions in OpenStack projects. A **load balancer** functions as a traffic intermediary, directing network or application traffic to multiple server endpoints. It helps manage capacity during high traffic periods and enhances the reliability of applications. The main components of a load balancer are the following: * **Listener**: The listener is a component that defines how incoming traffic is received. It listens for connection requests on a specific port and protocol (e.g., HTTP, HTTPS), and directs this traffic to the appropriate backend pool * **Pool**: The pool is a collection of backend servers (also known as members) that receive and process the incoming traffic distributed by the load balancer. The pool determines the load balancing algorithm and health check policies to manage traffic distribution effectively * **Members**: Members are the individual servers within a pool that handle the actual processing of the traffic. Each member represents a single endpoint (server) that performs the required tasks or services requested by the client. A load balancer determines which server to send a request to based on a desired algorithms (e.g., Round Robin, Least Connections, Random). The choice among the load balancing algorithms depends on the requirements of the specific use case. Note The Octavia service is available but it is not enabled by default to all HPC Cloud projects. If you want to use it please ask access sending an email to [superc@cineca.it](mailto:superc%40cineca.it) . --- # Security guidelines — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Tenants Administration](https://docs.hpc.cineca.it/cloud/tenant_adm/index_tenants_administration.html) * Security guidelines * [View page source](https://docs.hpc.cineca.it/_sources/cloud/tenant_adm/security_guidelines.rst.txt) * * * Security guidelines[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#security-guidelines "Link to this heading") ======================================================================================================================================== This list of security guidelines is not meant to cover every possible case or scenario, but to serve as a starting point for keeping everyone secure: please read it carefully. A complete description of roles and responsibilities is provided in [Responsibilities](https://docs.hpc.cineca.it/cloud/general/cineca_cloud_model.html#responsibilities) . Additional information for the management of sensitive data is provided in [Store sensitive data](https://docs.hpc.cineca.it/cloud/tenant_adm/store_sens_data.html#store-sensitive-data) . Important Concerning security in particular: * Users are responsible for the security of the virtualized resources under their control. This includes, but it is not limited to, virtual machines, network configuration, user accounts, disk volumes * If you discover a critical security flaw or believe that your machine has been compromised, please contact us immediately at [superc@cineca.it](mailto:superc%40cineca.it) * CINECA **reserves the right to suspend connectivity for the IPs affected by security breaches**, when necessary to protect the infrastructure or mitigate potential risks. Account and Credentials management[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#account-and-credentials-management "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- Carefully manage the credentials that provide access to the service. ### General account guidelines[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#general-account-guidelines "Link to this heading") * Set up additional user accounts to access the virtual machines using a SAML provider whenever possible. * Enable Multi-Factor Authentication (MFA) for enhanced security. * Always apply the principle of least privilege when assigning roles. * Utilize SAML accounts as much as possible and reserve the use of the Virtual Machine Admin for cases of actual necessity, * Promptly close any accounts to the service that are no longer necessary. * Periodically verify (at least once a year) that all active accounts are still necessary. ### Password management[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#password-management "Link to this heading") All users, especially system administrators, must comply with the following guidelines to ensure password security: * General Best Practices: * Choose strong, high-quality passwords as described below. * Keep passwords confidential: * Do not store them in plain text on local or remote systems. * Avoid writing them down unless stored in a locked, secure location. * Change passwords immediately if compromise is suspected. * After receiving a new account, perform the first login promptly and change the temporary password provided by the administrator. * Never share your password with anyone. * Rules for Strong Passwords * Minimum length: 8–12 characters. * Must include: Uppercase and lowercase letters, Numbers, Special characters. * Avoid: Common dictionary words, Personal information (name, birthday, phone number, tax code, car plate, relatives’ names, pet names, or work-related keywords), Simple sequences like 123456, abcdef, or repeated characters. * Secure Password Transmission * Passwords must never be sent on the same channel where they were requested (e.g., ticketing systems). * When sending initial credentials via email: > * Use a generic subject line (e.g., “Requested Information”). > > * Include only the password in the email body. > > * Do not reference usernames, tickets, or account details. > * Password Renewal * For accounts managing personal or sensitive data, users must change passwords before the expiration date imposed by the system to keep accounts active. ### User Account Management[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#user-account-management "Link to this heading") Regularly review the user accounts enabled in your system. Some applications create default accounts which are unnecessary or even directly insecure. Recommended setup: * **root** with ssh disabled and no password. This is the default in the images provided on CINECA HPC Cloud for the different OS (i.e. Ubuntu, Centos,..). * **account for a sysadmin** that can only be accessed via ssh keys and has sudo access. CINECA HPC Cloud VM images provide this user pre-configured as well, the name of the user depends on the distribution (cloud-user, centos or ubuntu), see the [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics) of the system you are using for specific information. * **user-level accounts** that run a single service and have no login possible, neither remote nor local access. Do not enable password login, **use SSH keys instead** (see [Key Pair: create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/keypair_create.html#key-pair-create) ). Passwords can be, with enough time and compute power, guessed with brute force. The average SSH server deals with thousands of such attacks every week. When using SSH keys, challenge-response authentication is used instead. This means that for each login a different challenge is asked and a different response is the correct one. No secret (password or key) ever travels across the network. * Password protect your SSH keys and make sure your key never leaves the hardware where it was created. * Do not store public keys (much less private) on the image used to create the VM. Network[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#network "Link to this heading") ---------------------------------------------------------------------------------------------------------------- It is very important to keep your network configuration as secure as possible, as it is the gate any intruder will use to enter in your system. It is relatively simple to apply some good practices that will give a good extra security layer. Here below few strategies are advised. ### Restrictive firewall (white listing)[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#restrictive-firewall-white-listing "Link to this heading") Your Virtual Machine instances should be configured so that they allow the minimum required access to run your application. By default, virtual machines have no external access (default security group rules in CINECA HPC Cloud), this means no single port is opened by default to the public Internet. In order to connect to them, or to provide any kind of service, access has to be explicitly granted. It is important to open only the ports that are needed and open them only for the least amount of IPs possible. Every virtual machine running in CINECA HPC Cloud comes with default Security groups. [Security Groups](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#security-groups) are the easiest way to apply a set of complex firewall rules to a set of virtual machines. This is an example of a security group that gives access to port 22/SSH to only 2 subnets (which could be the 2 public ranges that your organization uses in its office network): ![../../_images/sec_guidelines.png](https://docs.hpc.cineca.it/_images/sec_guidelines.png) Security groups are easy to configure and easy to visualize in the Horizon Dashboard under the Network tab or in each virtual machine’s instance page (see [Security groups: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/secgroups_create.html#security-groups-create) ). ### Disable unneeded services[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#disable-unneeded-services "Link to this heading") Do not run unnecessary services on your VM, even if they are not accessible from the outside. The more services you run, the more potential attack surfaces you have that top intruders might exploit. ### Use secure protocols[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#use-secure-protocols "Link to this heading") Wherever possible, use encrypted and secure communication protocols to avoid man in the middle attacks; this is when someone get access to your communications and can read the data going through like in a public WIFI. For example: do not use HTTP, use instead HTTPS. Do not use FTP to transfer files, use instead FTPS, SFTP or S3. If you need a web certificate for your VM, we suggest to use the service provided by [Let’s Encrypt](https://letsencrypt.org/) . ### Use intrusion detection software[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#use-intrusion-detection-software "Link to this heading") Tools such as [denyhosts](https://github.com/denyhosts/denyhosts) or [Fail2ban](https://en.wikipedia.org/wiki/Fail2ban) can be used to analyse log files and ban IP addresses that are attempting to make brute-force attacks to your application. They are very powerful tools, but they have to be used with care as they can lead to false positives, i.e. Banning IPs that should not be banned. These tools are a best practice to provide 24/7 services, while may not be necessary for single user VMs. Software[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#software "Link to this heading") ------------------------------------------------------------------------------------------------------------------ Running secure software is also very important. It is not a trivial task to develop fully secure software, but there are some simple strategies that will help with the task. ### Automatic software updates[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#automatic-software-updates "Link to this heading") All operating systems have the ability to apply updates automatically. If you run regular updates, you are less exposed to known security problems. It is common that the fix is available before the security problem is published. In Centos 8 and newer, you have `dnf-automatic`: sudo yum install dnf-automatic \-y systemctl enable \--now dnf-automatic-install.timer Copy to clipboard For Centos 7, you have `yum-cron`: sudo yum install yum-cron \-y sudo systemctl enable yum-cron.service sudo systemctl start yum-cron.service Copy to clipboard For Ubuntu, you have `unattended-upgrades`: sudo apt install unattended-upgrades Copy to clipboard Each OS version will have its own way to activate this. **Kernel updates**: Some updates, such as kernel upgrades, require rebooting the virtual machines. Please schedule this into your regular maintenance. If your use case does not support automatic updates, which is common for highly available setups, please make sure to schedule regular maintenance windows where the software upgrade is scheduled. **Subscribe to security announcements for your OS**, if there is a security problem in your operating system, you need to find it out as soon as possible. You can subscribe to an appropriate mailing list, RSS feed, … to keep an eye out for anything that requires urgent action. ### Only install from reputable sources[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#only-install-from-reputable-sources "Link to this heading") Be mindful of the sources for the software you install. Only install software from reputable sources. If possible, use the distribution’s package manager (yum, dnf, apt, …). Packages managers make it easy to install software, keep it updated, and uninstall it. If the desired software is not available in the distribution package manager repository, an official source must be used. Follow the instructions on the official website of the software you need. If more than one source is offered, think about using the one that provides an easier life-cycle (install/update/uninstall/…), like [snap](https://en.wikipedia.org/wiki/Snap_(software)) or [flatpak](https://en.wikipedia.org/wiki/Flatpak) . ### Security for databases[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#security-for-databases "Link to this heading") If you have databases in your VM please make sure that these: * are not open to the whole internet (0.0.0.0/0) * are password protected * information is transferred via a secure connection. ### Keep logs of your applications[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#keep-logs-of-your-applications "Link to this heading") Use the best practices for logging: * Make sure that the services are logging to a secure location, that is as tamper-proof as possible. * Keep the logs for a reasonably long amount of time. * Consider logging to a remote server as well. Images[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#images "Link to this heading") -------------------------------------------------------------------------------------------------------------- The main reason behind the **prohibition of uploading community images**, to be used for instances creation, is the fact that they are visible and can be uploaded by any user on the cloud platform. CINECA HPC Cloud infrastructure administrators have thus no control on community image content. The use of a community image implies an acceptance of the risk that the image owner has uploaded, unconsciously or not, an image containing malicious software or vulnerability like **backdoors**, **keyloggers**, **viruses** or **malware**. Even if, the use of community images is not a guarantee of vulnerability, keeping in mind the aforementioned risks, the CINECA HPC Cloud infrastructure administrators have chosen to inhibit the possibility to upload images with such visibility. More information[](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#more-information "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------- If you are interested to learn more about security in cloud application, we advise to read the material provided by [NeCTAR](https://support.ehelp.edu.au/support/solutions/folders/6000203455) . **Acknowledgements**: CINECA Team would like to acknowledge the following source of information for this page: [https://docs.csc.fi/cloud/pouta/security/](https://docs.csc.fi/cloud/pouta/security/) --- # LoadBalancer: troubleshooting — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [LoadBalancer operations](https://docs.hpc.cineca.it/cloud/operative/lb_ops/index_lb_ops.html) * LoadBalancer: troubleshooting * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/lb_ops/lb_troubleshooting.rst.txt) * * * LoadBalancer: troubleshooting[](https://docs.hpc.cineca.it/cloud/operative/lb_ops/lb_troubleshooting.html#loadbalancer-troubleshooting "Link to this heading") ================================================================================================================================================================ Load Balancer in provisioning\_status = `PENDING_CREATE` or `PENDING_UPDATE`[](https://docs.hpc.cineca.it/cloud/operative/lb_ops/lb_troubleshooting.html#load-balancer-in-provisioning-status-pending-create-or-pending-update "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- When a load balancer is in provisioning\_status `PENDING_UPDATE` or `PENDING_CREATE`, any action on it is blocked from OpenStack. If the load balancer remains stuck in this state it can’t be modified, recovered or deleted. In this case, the loadbalancer has to be deleted by our sys admins. To solve the issue please write to our support team ([superc@cineca.it](mailto:superc%40cineca.it) ). Load Balancer in provisioning\_status = `ERROR`[](https://docs.hpc.cineca.it/cloud/operative/lb_ops/lb_troubleshooting.html#load-balancer-in-provisioning-status-error "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ When a load balancer is in provisioning\_status `ERROR` something has failed in one or more of its components (i.e. the amphora machines): if the error occurred on only one of the load balancer’s amphora machines, the load balancer itself may be still operative but any modification operations (including deletion) performed by the user will be prevented. In order to solve this issue, the load balancer failover operation can be carried out by OpenStack admins. To solve the issue please write to our support team ([superc@cineca.it](mailto:superc%40cineca.it) ). Load Balancer in operating\_status = `DEGRADED`[](https://docs.hpc.cineca.it/cloud/operative/lb_ops/lb_troubleshooting.html#load-balancer-in-operating-status-degraded "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ When a load balancer shows its operating\_status equal to `DEGRADED` this means that an error has occured to one or more of its pool member. This state does not necessarily compromise completely the loadbalacer, it just means that the loadbalancer is not operating at maximum capacity. The reason could be either: * at least one of the members is in `ERROR` state * all the members are in `ACTIVE` state but there is some configuration or network error inside the virtual machine Those problems should be solvable by the user since the problems depend on the inner workings of the member and users have access to the member via ssh. If the member is stuck in `ERROR` state, please write to our support team ([superc@cineca.it](mailto:superc%40cineca.it) ). --- # Store sensitive data — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Tenants Administration](https://docs.hpc.cineca.it/cloud/tenant_adm/index_tenants_administration.html) * Store sensitive data * [View page source](https://docs.hpc.cineca.it/_sources/cloud/tenant_adm/store_sens_data.rst.txt) * * * Store sensitive data[](https://docs.hpc.cineca.it/cloud/tenant_adm/store_sens_data.html#store-sensitive-data "Link to this heading") ====================================================================================================================================== Warning Following CINECA access policies, you must inform CINECA in case the activity requires the loading and processing of data that may fall under the GDPR (personal data), to identify the appropriate security level; in any case, **sensitive or personal data shall not be loaded and processed with CINECA resources without CINECA written authorization**. If your application or workflow is processing sensitive data, besides getting the required authorization and signing with CINECA the Data Processing Agreement (for the appointment of the Data Processor), you need to take the necessary technical precautions to safeguard the data from unauthorized access. On CINECA HPC Cloud infrastructure, sensitive data can be stored on special **encrypted Cinder Volume** of type LUKS. By using the OpenStack Horizon dashboard, every user can create such volumes and then attach them to a virtual machine. Due to a limitation of the crypto library, the **maximum size of each volume is 15 TB**. Since LUKS are encrypted volumes, the time needed to create one can vary greatly in association to the size of the volume (most of the time is needed to encrypt the data). Here are some indicative times for the creation of different sized LUKS volumes from the dashboard: * 1 TiB: 15 minutes * 7 TiB: 2 hours * 10 TiB: 3-4 hours The user can access the data stored in such LUKS volumes by login into the corresponding virtual machine. Only the users with authorization to login into the virtual machine will access the data “in clear”, even if it is encrypted by key. The keys used by the OpenStack volume encryption feature are managed by Barbican, the official OpenStack Key Manager service. Barbican provides secure storage, provisioning and management of secret data. This includes keying material such as Symmetric Keys, Asymmetric Keys, Certificates and raw binary data. Note CINECA HPC Cloud infrastructure is certified ISO 27001 since 2022 for **“Servizi informatici HPC in cloud per la ricerca in ambito life science”** and since 2025 for **“Erogazione di servizi IaaS per ricerca e innovazione su HPC Cloud”**. Details can be found [here](https://www.cineca.it/it/chi-siamo/cineca-oggi/certificazioni) . --- # Unknown 7z��'�ے<�$��ӣ���\]3I��a��aʪ)e"a>t����B�>ͩn\*�d��<����\_�i�r���\`�1$��"�1�lvՎ��Иؼ�˹Z�����D�Qt,�|Y\[�ix�J�����I��d;9P\]#x�r��wSr�:�u6/�'�$�N\[Me-��'؋��� Z ځ<�T�i?�!��s��ŀ����8��N�Oˉ�I �b��9$�8,w方�V�8�/��\]0��>ns�|���y��vaZ�+���I�~B(�����A�h�69�l��Ĩ;5�O }�R�\[�����!a���E�O��{\`�/ݔ��-x� +�/���\[��;�y�<���d\`�i��3��no״>+�TK�T�nf���^��y��Zk�����r��m@������:�C � \_�R�5��y�����a�ߡ xJB��a'^��K�S"��U;@:1��)�{A�C�����^�|�|�o}�$����+I�#ư��/R�?����Y��{�h\\�.��KQ��";a,F � 8ٚ��e&���Y���GZ.��0����q@�&��?e��F��(����(�U�˪��>&E��Զ.��9��3P�h�W��{4u#�N$<맫�TH� �:��ڪ��L�r��+�� ��gs����MwP�xש��V�� j8M�&�jzk2n�Jܵ=�ʾ�w�� ���������+r��Q�VZ' ԗ'�����; �5�@��틄O��^o���Y-K\*�t.rq� ��9��#,8��\_!҃�c�&�|Qa�-\]����bBH�e��>֜�B��ф���<�= )�b��0�E���I��\`p�ג �~� ?��g��A�x c�� ��1��Z5 ��� 7?JLr)��1q�B�ŋ��ʯ��Ƌk�g��&�k�|dGVMr^6̬�y��a91�\[%Ɇʄ S Ϥ�L ����������� +)h�E�y�In��F\_$?"p�GY� ����bc#c��lJ\_$�pbTµ�j����\[E 3�@8apޑ���P�;I&���\`,)V���# � 1,�E���K�B{�������\\��~�-s �X�G�s�oͻ��\\9�x��P&� 3R ���+1g��"��慄i<���8H~h#�iƿ�҄N����Ę�}��f��e��&r��x\]�pg�>z��!� ��K���jx�oG��Zd-�X�7�gI����3��N��+��5T�%���܏N���4t�\*�'{�P� /�\_��#��Mm<��J �$frD{2\*� �@��G��� \[u�F\_yO����W~��.ݝj\_���2��v3dB/�݉���S|���f&J �G��xf)�^4U�Ʌ$��O��w p�{u����K5����Xn�K�L�HH�k�yq�>Cͺ�(�vw�q��U���ڜ��ɝ�lEy'��>��t�����Saā�ͻ׳��e�RDC�1'tLJݸ5 ���2 t�d�h��8�7K�i�1�E\_��� ~( ��}6YԤWv�تZTT��'�p\\��ӊx�O={�$���+�8���>������=8V/�I� ��Dh�U���o�� K �>�^<���R���f��O���W������^�J�L�2��z^)�������T\`73O��%l���5�g,i�S/���/�~�i�tn�D2")����m�IWG�k&���d�w�밼{� J#�Q'�n����!��x ���'��5��C"\`D�;I���א\]N�@�g �N6xQ�H�ڻ��%�Ѭ^||~rp}�g+<鳅\[�o�r�'��6Sd��Sٴ���6�N��U�&��Jn���X\[S�����n�oV�p��A�#��p�M{K�f��K�u�9x@w���w��Z}���tv2�O��D����� yQ���C�K�@sA�)EmЇ�|d� ��B ��4�=�'k=�5�gNgg�$f�d2�GG�򆪸;n�=�N�W��HJ�;��Z3d�f� �۫6 f#j�b�J~�ER6�� �4�� ��\[ MзId�\\P��A#�v͜�V��8q\_�'�) Ь��O���"�W>��v81�\`���S���/�~m����ȩN�Q� � ܂�l�G� F�գ^{��gV���|�+\\��q�ʨ���z�����b�ma��h�E�Ɓ��U|#�ж,�,��7����9oDJX���bYh**x�O$�p(��o�����ח����tq�婘�U'����W��G$����C=n����ҏ2Q!9�I�0j1���Ieio6w\*�����u��\[��\_L�� �����C��� � i�� uO��Ջ�X4}��BdvI���� ���9g8ꊔ\*�f��=j0�K�8ΨS�f�H��瑢����R��%�=��kn.7�� ��aA��δ{'�y@yD�"���Y,��o�d˭���K,/�^��0|�z���z2,v��^��鳿�@���� �Ea�\]F 2"�.�n�e \\>nÁ�44HF�. ������ ž{Ž��#�oM�l$ ��sЗg���\[���� B)f�ğ5L��XL�Z�e�{9��4=4\\ �͠�xb�9 R�O�g���B������rE��#8�+�w�AK�m���5��SW�tq�B6�T��ЈRߜu,\[�Z��hG��s�\]$�\[0K33 �Y �)��N�d�Y'�PjJ�ϧ֫��<�����j���SŨ���FaOGdU�l�'%sAA=�u��<� Cµ��T^�cs������A{IS�裰���Џ�)ո������1+�� �Kπ.C�p��r�xq㍩�>\[b>�%�B+L���c�©��(�ݛ�0����"�#q��pT� ~�a��΀�M��̔���� ��s���ʾ��+"�c�L#��� ���FF��Ah0�̸!yn�(BT��s�k���&6�ͻ����qK�xg�@����M��K�@��u���l���<�K�qouX���) ��/e�)���i vF�gX7��~CjO��kH{�oM��#�m�f�7� ~�Р��{�Ձ9�70��9M,�o��\]����W-xt���X��\]۲��;|��,^$��fo��aK���z��� YQ�J�ݛ=� Cy�����ͣ�\_O��x��m�������9�>/��DQ�\*� �%5�b� -/uD��ێ�zfT�6DB��?"-G� �ZP\\�EgJ��:Lv�=F�BC���ԞIC�����;��/Xm��Z�κ���:��:��{om}������vv�,��g��Ja�J�k�P�@�2@�ݗ�a�w����CiȅL���}�Ez/xht魤r��5\[���s-Q0j�G�}���M\]O{�}���p;�l�Ѝ3 GH��Lh7j ݼ�j\[=T���\*�f!�'g��sbS3�iy΢������7�{Z���s�����\`����\]$���p�O� �h ȄnO�'l#KD�LI�:5ˊ}|AH̽�|,��p'u���L��ͭ��JX��Wny�5ǔp{���WU �\]'\`.� ����æ���m���y"�U�-�9%8-��\_�7 ���I3 ҂��CQ}}�M6Ux� O�oX���"|R���o�ԇ"�0\`������A!�4�<�8��Y��\\�����n�t�����}�1�g��ɯs�/��d\_񇓷#�}����ϧ�n"�Lt呃p������A�A"W����&g8����Ijɇō�u�� �,��'Kj��.){��”��\_G$�Pd��^�%��gs�bsN^��|��O���u���j��Ks,aɇ���P�MH�3��rQ�X�^��/$L Du\\�U��fRX�����2$�q�u�����Ƚ�N�!hs�h��\[ ��n�@�s��Q�+�N�u�R����l���Ķ�4!�CNCk����%� ��=^Q���G��<�p/1����F<�+k{�N>\\zrme {(�֒�#uF�ܔ�U٢��V�LM��9����6F�f|�>y��c������\_�U��B4'�T��lϼ��&�{aSx�Y\`P0g �$X�m� ��� ���Ozu�\\�룿�F��J�Y�$��:�+ ��B��V.�5:�8�zpE�/\[�( (AB���˜��f�z�y�@c���Jm�n�7��AV�����ՙЛ���+s#(�2\\���}{��!�-(�,=�ǃ�U\_A�=�� K�B4�M1v����O�J ���'�E������’CDmْ���(��\`% �r�I.#V��\_�T�ŖUH8�G2���Q̇�����F��1��^��w��M˖���,I��=R8�/��\_��ę�U�i5��=viP����ցHj>7����-�H�6}� �R~m�)�\]" ~��>�����P#�I���N:��u6���(i-���� �\*O�\_�ur��&��B�ڒ�H���G�. �o��g�3�+:byI,'D���8��Yr� v�������\*�����n���~�T@��S76��V3�;k@��́OII.6W�lV�:��Z�U��K2D꣆��~<���16Q�ݼ�,o����q��F�Iy>��I��A��SU� >fV�j����%U���V8t=lcMA�y9���x&dj����� ��R��4�EE�06?�-��C=��g�Nav���~nx��͜^����vҰڲA��2u�y��h���\_\[������K���=t�$6\\�Z;��ϡ1R�N��ә�#���Y׏�-w�\[8,����w~y>��'7.{? =X|����1V�\`�m��T�\[�i�fk� �"!|�U-U�@�{��9��d'+%C-���}\*�4r�����T˱Q�2�I����� /��R�͌�Rg:�W2.�cR��r�SP|����jg$E�KC.��n�\\�\\���q\[�t�vu��'�1�b�t���L�x�e<�����6|�$}�,��<@�����q���ő#�Oj�vLQ�lɤe��Y��F��4�П�p��v��Q������S�a\*��fd%\_tv�q?\]c�lQF�a,��N\_�����H&��q+5�B����O���\[A�ĩ�n���Շ I ׍\]�o�Yf�D5�D����m!�^r>O"�c���+H�/;�؜�/���I"Ծ��������A�p����I3��—��鿄Il\\������� 4�8��k��e�̬ۇ�Б-޴���j��Mc5�W�Dh���Xv��jW�|���줰�ZE�^�vv �\*���a�j5�6L&�t�t�L3�J+���&�����s$S��� �5f�nH�=�3��'�5D�r;�W�$ �c.���Uk�%A\\����L�@��Ä+�=.�\_%�|�7�A�ܥ�"���=�&�� p��E����i���C��s��E�,��juw�"Ɛ� �.E��$%{>Z�y ��"t�Ʊw\_!�HD�fT"�1R�/cQlN�NnZy%��ja%{��pNGZ�7�:-��GX. Ta\*�w~x�&� ne}�Y3�V�S�Js�ڼ��C�2,1���)tG}����7Xu� 0�5�uv\\铌��e�X���<���v/��:a ��7gP���LG�A\[\\8ǩ)6n��|�\*� ��n�@�fX��0-36N؍����FFm�ɢB�/m+�+��9����P�6��0�!'\[e��J/��� �4��ߋ'5ޘ���B�zm �s�6oݬ~e�����m��� �07 �q��)�V\[����Of��.\\57��7������S\_�~�\*��u;��fVi�q�lPֺ����M�;.�rTS�jJk4��Ӽʞ�����\]��H�jܱ��b&2���=<��k$0ψ����s Aj�G~kb7O������"��L�?$�r��8W��ْ�����\\�r�v�I��lz(����l�)"�=ͬ�w&��a�B�����5����^��O�����mڵ�Xe  �\`��ɂ�ZF�R&.d� ��#6"�2���I�6�q� w� �^9���2� �LY�&��Q��{n1�Ċ�|��J�bbS�g�$ $'\_䐇+J�Apڐ�yC��+��F�h�I5;�+�c+X\*~�q ��Yҳ�� �\*\*�i~���Mw���+�\_�} >� �������9{�M�l� LB��R���Wr����Fh}\[Kܯ�=�|�^�A�Jj���a(� l��%9� xv:���>ɟ�f�2�����\]=�h�mHfT������6��Q�2��Lr�c� N��j'?6pG�ʹ�i�~�1�M/�=Z؂a��D$;���$�S�R�\]duQ�pqrm��r����vf����x�Ȝ^�Z� 1�I��.�����K��ú��\`�����w��b�A:D�ܤ�<����G��}��Ӻ�2�T�àm�����6&��n7��2��/�E ħQ��o ݧ8���2,�{����tv$�������O.��I&��\*> HU,����c��BT@;~K6 �;�ˡ�����"�� G��I����p� ���ѻ�7\_������̴a�F� ��������c���\[\_k��p�t\\���Z���%+I#W㕮�ǵ��;%������\]����79����s��q<��9E)az�e�\]���RUu��Q�t\[�Hy��\[)6�ά\\�=�FR�i?������Ur�'�sf�B����K��j��Q�A�}��P9v�qq\\C�?�\]f�jK7B��V � 8���n���T �\] -�� ������i\*aD���p��G}��x��$��)�c�Ǵ��x�nDh߂"�o)�x��u m�зm��YD��!q�j�9��\*�?�L�!Jޣ�F�jgq�t; �#���3A�$��W&3/'ن%���ܺı�dm8�{y'u�-9V���S Y�' �A��p���>Ti6�K0�����m�)�q\[ؕ��\_���G�w�Oȃ�5��HQ�MEV�a����U>.�H���Q���^��i4�i��2�Qٞb�?<���K�e���˻���\]7�z׭���'$R���t��@0�X����� Dگ�Id9Y��z�������&ȧ��ՎUs�E��M��x�$��8My��4��q�f ��i\]Nw"8�ME QĶ���rE;'\\��!S�W��$\*�F� ���e��;Dro��6.|hG���n8+�E�Fa$�&�zԤ�О�{���N���k^ɯ%��캓:�ܓ�.}}�GG�1� O<�U���L���&H�yb�/8�x^�m~��\[��rCU�\`�y�U����Zʬz�Eq�w �6Q@�d0AH�T�$� �����YD��� ~��(cR6S���7 \]Wu:}�@^�>��}Mj���#F�~�\*��H�x�j7�a���W�\_v5���������g��m��ĵX\\��p�ņ6� ;��"�R�l� ����OK�3hZ�tM�n�PpPY59mh����K�g�|�E��lT��!��\`�w6��m��L:.b\] \\!,���\_u�#X�ts��ʧ�E ����+;���M#�j��&7����\]���!\\j.�\*�Ї��ܫ��˙-+�KԻ��=Q��Z��#oJ�����\\��I�2���jXbjI.#�h/�5�ؿ���f1��Ʀ�k�D�s�T���nvFH;�ib�ye�V�����"�ٿt\*|s��dI;�kL�wǹ��xq�����XC\[���3���2;��v�����=�I� �~�F�k !�Kÿ��d;&�0k�b��WJ�oByT�i����5l�����i�� 8�C4��ɞ���dL����$ڑ�o���^��\_L^���91/��D�´���'�2��Y"����� ,\];�6�I�!���L��UT����Zv mD�}���N~�mC�G�$j�Ԁ�D�ѫ81߼Tv:��M�+^��R�=,3��!�e�K�� 4%7V��ب�d�ЁZa��̛􊛪�����œj�?;�\`�Sdq�֢^�^ Lq��X�&��?Ҽ�W��vSa�4�;�x@:' %>)�)��\\��;�5���%2M7cUӀw��ޠH�c���Y�(�Z�O8�퀨�� ��k�4\\\_��\\�v ��^}��o�Ќr������AGY���x�#4�\_0����ł�����P� w�v���(�8�c2�Y�X4���Sj$q��n%^�ö�-�^���� Ԝ��>ϵ�H}�4K�kLe�^� �g���~Ѧ��#Q.5׫�F���̙���(���<����v۟�NM��f��������IJ9ё�k� � �ܑ>����w{f�3�UQ!V�t\[��8q��i�IVoڲ}�o\`gBY�L��o����x�撳ʰ\`K��͖Jr��XU�����{� J������+���24�x��z;��,�\_#����5q�w�6&,����x��� ��S~Y�3i�t��Z��pOX�%�=�L�� �B�y'�#�eI/�� ( ��j�̷-�����(��>�����oG�(&�Yj�;����b�,�WAX�ŏ�5� 3>K��.ܛ�!Es|b�Lu����ׇ�u�dw��B<;>y� a�\`E���Fv�9����x���xP��W�1�\`F\\=2 +��n����e� \\�-6~�i�j�)Vc��i�)w�� j�Z��NI ���G�\]�C^�y�軵���{z&����Z  ���O�\*mxh"h�0,�e���!�\_�1� (�h��K?�s�4�Ŗ(�F4vi���L���.�d��Pʺ� Xd��D\]?+dSEPT��kW���❇��j�:�3L\[��.F��v�����g���K�8x���'�\`�ڰ�⟣羳�(�g��������l\[�4G^�O���y�c���w~Có'b�\` ��N���B����$�Q�h���o��5�0b\`kP�pU�\`��%��t�v�Ά�1���x�\*�D:���/�qT���^��� /\\���ǐ ����;7��N�Hb9�N̺D��K1?4�w�R ����\`��J�������"\`�&��e:26riH����v���N$ ��W��b���#Y{\]>#��W�;�����\[��'��P�IaY� ��\`�E��gċ�~��������(�⛪�z�o�\]�ҟ�WsͰ �p&L�g�tXR�IRa�&��,� aΐh��r�>���4S�,\`#Nl�ޕ.�~�AYr�8��cs9<�Y႐X@:����p�C�}��(���GI�|�kO'�=+�����)����7����~ƨ�.����N�M%tNL|r?�� ltZ��^��=X��7�!~98��x�����V߬�I��:��&{��̒�:U��,N��r�ㄗ\_��br��q}�ɼ^-=�8 XUZ�Y�œ�RZ��׸�K�����$�Aۋ�\[#�)�����,h �IC�;���P8}��\_Y�:��8�R�� T XHN����� ��Vp��zϚ��|6�Eʚ���\`�\[���YB��췜E6�yË�G�w�����ݓ�(\`�-�p�7/�- ����SmҸ@H�����S��!�U�C �MVSR��I�L�i�ZΕ>fqO�Z�����O�jW�dZ��n���v�gԟ;bU�d�nϕp���5f��:���){c�U�Z�mb4˱�s��—�Mӽ6\]��@���f��'����۸ڑgL�� �,mz�>�.œ� �|Y��ٖ\]�5L��-RW�-�tĩ��v�tM������ �m��70?>�qD�2��CQKsݢh�4�ʩ��K��W���"v��r �c�ìK䗉N3v�0@��1����\\kޙM'�ǓhA{�T�������0�. '���G���E�Mc��3�~�hg��ϑ��{Q잭� ��ɬ�\_L��so�x,�PR2,�8��L�۸���)d%>�1��ztZ�ϯA)c��Z5�� �w8o�$T�1��T�����Y��u�Y�3��i!�3��H�����ybJ�-c)��&��:5�Ey)Bݭ�f��{1�R�9�p� �I�Ώ�������W�4!ɖ �a�T�6��r����X�o'�W���^TР������P{\]���Q\`�u�R4�� �  �%T��K�o�N��c(������Wi���j������v�D΃���<�=)@-L���"<�{W�m!G�ej�)�y���s�,��{c�p�����e� ��Cf�U��=68\\+M�h��A&�v�D�?2�\*�٧�}~\]w����&Q9 �|�dn�Y��\`>4��W>����ZG���?��n��T<����m��.��֚Г2�=Dd���f� ̛�� �� ��?�o�0D�}F���ס(�-\]^#��cy�a�TAT�?(���"i�����-������'ٍ��!--I��د��'o|Y�@�j�@�"���ɀ���J\[����#�svag�=�5�H�ώ��Xo����o\]t�+��x�o��k��ט�������T�Օ��t��+x���6+�pxcA�W\\q#�8��ī���k\`s9fÏ%O���N��q&{X�hC��q���ghUO\\���������{�0+�c���xǼ1����x�}�7�̹p^�\\�O��WT@\]ŹTj-5���������n��j�Ճ�Β�� w���(��B�-Ȭ��� � ���)%��83;�/��x\\�;��?9��U\_��B�)�R@������Yc��^�L��QD���N���e'J%���6\]�r:���:������ۃN�E����T��P�6/g֤�z2K;���2^��� ���Y(�ש��3�w��h���|T��\]�����|(��Z}m��X�L;�Mvu��CB�~.����A�h�Ʌ0��c3o�!��,ڱ�<���V��6J6��;��P�v?Og�)a��;�L-�MEi&��aLg����\`—"zm��L�|\\JFYUC5n%t�B\]�Z���W~���O��0�i�g������~�M�q�����P�k ٞ{��a�۶�.��BY�J!%C{� �'��yӤ�����L�k��i����\_°������>�i��$���Q�F�ܻ-�N�����õ<�km�5ЦBi.�fe�����^�)P>2��(�B� e)=e�m;Ӟ,l��M̾7� ��=��~��}Y��������{\`��ef^SS�ʚ+�¯|<��hX�Z'�������Y#qBv�3#�(�m�@c�$,N �S��oS=YD���@\_� ��Avᙅ#b� �ud�\[��va��S't ��L������E xj��8 �dʺ�� ���~�� x6�u$�����D�����Fo��o�W�#gԗ#�����\\v���0z����D��3���QO#��iU�HU����c��!C���|31U��5燈�\_���0's�U2f����8���%�3jԿ�aFpX�3n�f;n��f烱��{��BaH��0|���A�w��;D�����@ �SH Xsd�B��:�'"����"%�589l�$5�ݯ���?�h��iJ�����s�u���?f\*6���K�r�Kw���T أ�Oڊ@\_$��r'\*f���&��JE��1��C���K��/��O�F\`e���~�=��\[� �edn쩪�\_��\_ ���������v�@\`�V\*�{S۝㮒���AW��G�!�L�nsfG������Q4��H^�����6�+��ʦ��$kjY�Q\\ i��oỳ5�ԅΚ���H&�Ĩ�Y�F͒����.tvgc#I��"%�A���U���J;9�d���bQ�Aǚ���TB)��+��9�Z�%���\`��L/N#�O'�V\*�����5�f��"����;2z1a�9��"~�¿!�)�XqECj�s����G��\[�e��!��-{��L=���F��LZD�� ��8�G���gI�1� �'D�R�������M�5� l���PL�F�Wn�8z�l��Kh'�J~v{�Kr����R|?���������J^�\*펃�^�L��P�^�P\_����pJ\*3�Q�����lQ�9M¤D��Rd�e�\\�� ޒ��j����ք����?������D��\]���4���1q��.0��U�L���+޾J����)�-F�\]bƢt�13���m|�-�P!sʧC2N��n�dL� ������G�y��|(��ԮZ��0a�)������T��^4����Nj�#�C�^b���N���\[������x�R��\*�%��ͼ��e7d�^�oƦo�b\]I�����̇0I^����?Y�հ�"�L����>Q#�dtc��� ��?(e��'N��\]f�h��{IS���T"�ʓ֬�L&�A�Ѫ3A�P��N@F�@�Tq{�J\]��/���5n�Y�f���K�ژRSՎ �e�Y�&\`F=��n�(�ϯ��>��uAl��NP�y���;sV�L���c:J�o�e�^l\[��� �+�>sS�ᵫ \* Ty#4�@�RsmS����! p�p�ouL�{(��\_�n\\���ZW�Πڪ�U\\b���gD��EQ�D���2 UH��G�"����/~����L\`q�-��y���,�N�k� �����nͶB����Kt^i��� �{����Ѳ0�!��#����� @���p~�˹�N�N���a�w����C#3�\`Z�}8ڰvh3�;DD#3�y�!sQW��9�d���u�𚞄�>QԼq\[�Mh,�A&�C���3�-���}��C��׮��E��Q�QL��M��P �;�mἓ���|8^��dz|#!+��� \`� .�b3���VD��ay��lG��?�� gY�Ɠl-z��¾�kI�scp|��I�)���t��\\J4ꃃ~t0�#A���H 8 i�t�$J�޽�����>�h��}a���k��G7;Ԣ%�����N喣62���rDic>\`���0�-�4��t���7}Z�Lt��dvt�Y���Ų����J�pJG1��\\1�\\���=V����h�)���O�9 �-Ւ�\_C,7�TW޻ ��\`�ެ��|��-�x�k���}Lδhi8Ι�n9OiQ2�'�% =��&�l3���ة�!�H ����HC�{)�\[�k� �L����3��%�7� +���&� ���AHT5�����fp�˻UO:� �cx��e�\]|�u�1| �\*W����ZO��;Ɛ���P}o��k�~҇ �T %��#��o\`M�2I�\*�e�hS5��핺��҆��Z&^\`���S������y����9�l3�Oo\_��m���E\*�Ǘ��FOp���3���ӷ�X6ǭ\]?\_��93D�Q�����{�ɒꀈ,>&/� �\\у ��W�y�\\�� �'�|c�XN�F � Ȓv�� iVT\_�W�4WBl��mC��Q����?L��U'V��\_� �ژ��)���O����o�YL���\[ݗr�� q����{�B\`̤S†}H��䯈��{pDʱR���:lm�R �'q�C����@��!��P��6Zi�g��� �Z䕴?�Y�d\*S��)�GExi�\]�ڬ��E�ݘ�Ó�"�xS��v;<� r�n+.��p.�7\`{l��y�hDu|^�Ů����u�eW’\]�ڂ��m�\`��mk-�/���9Բi���ξ��T�$ F���&�1\`�|� �~q#�nY���ף9a!�\*D�q;�.�+ư�Yz�Zۚ�Ɣ��r��H�o���C��.@�9?!��9/��B�!���v��3וay#�C =��;���k����'+D�#���,m}\]"6"DŽ?~Aف:oN��M��M妕@\`��U@'w��zϦڴS1Tt���v��2����}a�9Q��s��BG%�'����k����l�~�&��#f k��G�sw�\_����:12ǣt-��n\*>�A�D��6Ki��b�d4zt��7�<��ot�d�O�oĬ!=4Γ����;T���a�G����R&��h���PJ���-�th �t+Q@/R���5ɀ>MI�� \`���=���j�>}I$�p�x����vp��"�r-AÙh��+��^����KcX'F蚺��1�hXI�l�᯦ox��wχ� l�S�w�+Κ��;,XET����#ΚIh���w” <�TkN� ����-Y|X�\[<��u(�ZƗ�N��6���z8���,�Y\\4��k �s#��X�V�{�t�������� �?��tV�\\δ!��Q\\:���D3��mZ�8k/?� �;�&�%=��yx�ٖ��㖶�뙻��}�� L�RZ;H�����h��䙶�ǃz<8cs��X|�+�S{}�:��^$\*����7G�^;˙����=��\]ڹ����a��L����<��~�"�(�|�ĭ� X?�' ��|����Ɵ����ka����\_�+E�1嚛s2}�v�3�Tw'��M(�\\���H������BU�g� ^}����\*����N� ��ށ�o�?\_����EU�%�pW:��Y!4�ﱟ���NL�{����Dt\`� &�G�s5�ubH�����\]�L�-Pm����W� ~�۰�\_�w��o� A����\]��2�5���� 'm 1E� 0��|����/� ?�B�.�=��S�f/�X����o$��, q! ���X@��La6)�\`�3���r\`VT\]{\\���N��?gJ���AB�¨�\_i�Y���&p��t�Z��v^\*w �쯦|�fGjK�+�B�P���r)�{��B<@�f-Xl�2:\\,~:N��ewKOݠ��F���\[t �4�#��ao��m�8C\\S�@��Fֶ�g��'N�? {�L�W�AJ��2����������\`�Hb���� ԥU�:S�(�� ��}3c'V@�Q�W�z����:(�ي��|�XA�0���V�\`�\`?�W��i�$�����Dn���E��V�+�lw$�l���;�ƴ�HעK �^KvGN:��� �<�x.u1t�k/Mt֗"�a쳎U��̷W��^Z�.����n��U�L��\`�b�Hô�(�C��vq�c\\���\[��B�w���kɪ��'�xs|��0�#����V�\`9�����=k�ٌ��̟UXR��|1���ʹ��+;��X�"�“��0X���%��ӽa�K�z\`R��4��x���s:gcfgR�v\_x�E6( ƥ��!���o�����RS������������N\]�b��ԀqF�Xa�����0�,���m?��5�7�K����AE��w.�ۤ���Z�1�L�����������ڙ��I����z�<Ҹ�9ÿ�,L�)�ͳ��6)�r�)΀{w܃�y���s ��'l�? ��סOT ��ō'1A�����B�?J��}��jDAco��MlF���U����p���1Ͻ DH�1)���#�W�t����z�W .�\\���KT�yVWˁ<�n��@D>Gq����K���WR�;^���a�܁��Q �q�BXRgf �1���Y�j \\�U�ӊ�xXq^DlѰl��V�6�:���Kύ����g����(\[\\�ba06J��=zX\`w^D\]ZET,��0G��6�2Mzc��2s��Y!1�q�� ���w\_�Udv��p��Q�WW7 T��������¥c�R�(�\*��J��N׉5����'�e��\*^�lՂD;.8��Yϝ�-�c�jy�\`1�G� �,'a�m'�!��9M���9��\`�ʠ� ����9!y{�G���\`�ðKӑ�FXDK'��T���(g�cD^���Px%keĶ��yؠܿ������{ ��k�� g���5�tirt����3�2o>t)0�F8����\\��A���g�k��)aM�1���i����X#��ă=�F��˟1�/�p�g�}�Q �Xc�7�����W�8�oc��^FÈ2�s��^;�޽� ֆ���fc�@�����Bڃ��� vB�$Ao�L�Ш2���nŒ���\_^�/�o�����I#��2�UQh��j�6�Y��G \[Ź��>1��2��ֽ� s�}�Y��̽w�כ�Ab�N��Dt�!��4��VIJ���ܰ���\`�����d�i�JE���0�ݟ�ISYo��.���J�ʡw�����2)3�j,�ͳ���u�Š�ݱ��U\_:� �7n��,�1#��V�V���0^ >pC�\_F�$L��RQY�����:��q#���i��\[6�5�����w�eM��'ç���NJ RX��ʲ���;!4a%�1g�-.vܬ�\`��Ӭ��&�����\`��\\P�;�%�E\*�p��n��� L�3�T��j�B��?��v���ҍ� U�\*\_'���x��E����f{y�lCo�d���C� Z�x�)|����\]�D�y-w��:G#쫱�t��nֿ��Ydv!�����|��@ �#�r�o��J�Z�%� �����j�� $~WG�TlD�s�����e��?z�faR��$�;Q�Xr��u�R-�6\[$) ��n���ڀu��Л\\o���7=����zz�;.y=�ǒ�����gU\\�T��؅�}FQ�XQG�&Nå��w�����^$ӽ�:���G�g@�%-E&���� �A���m�/�y\` nHR��FZ��b��� ::ʘߴ��v�Ր#Ļ�\`} �Е�4� �{^nd��{�0H�ۂU\\?'m\\�\`� 7�~����\_��1�\_-��3H'g��$X��&h�@fJ��\_�T&-M�)�#w6�0u9TA9'E��8���H�S�a��8 ��S����耊��Py�?������#�D�p�}Ci�t;9��}>��=�q��N�\[�LP�wA�-l�pn\]�$���Sr��iJ�e��^��Ƚ��lfc����~��TqV���+ �˻۾�}�� L� �d��r�{w�ֈ�w��N����ˉ�1Xu�ޙ��-9���I�5�\_� ��!�x�ܟ�8t���W%Ṽ�=��L�eNް8��� PE����/���ɷQ:�d��MWs��@������r�)qy��������^����������7��2~�B\*�s���|�#Ss�ry�\*ϸ���Ae}�m8i�Ք5Z٥����������Dazsp��N�G2�M7��<6�q F�����x%�a�2@fl\]�X���cl����Jh�\]MaP^~��Z�Ko �;�,�.�8|���◹��P��z!�� n>����ʱ¸~A��4E�d���h Z���B���R��6����k�����'�F �#lm���'�%����E���<%9��m��.v�\\�\]��mxc�.y�J��%wZ�x������p,�t���G)t"������\]L#���s��QK�5�B?��~I$�\]x��E����\`�����\`���K��t'l�X˰���>E�10\[�h��'��(��Q��6-Q���Eds�k�jX��ۯ \]�\_�Xc!t�AX��X�砓Jj�\*���bώ���c�}���;�����2�6Ԯ�+� �<C��ŏE���ZP.«������D}�� T���(:Za��\_|I�������& ����{w��"�\`|p\*�5z��Xͽ& =�H���O<�9k� Pv����cT�\_S �B}t�\*2r��!�s!��&�RM��u�-a%� �;|�bF�x����Q"p�7�Z�iҬ��ڎ���Y��z���K$� ���qW�돭�����PcSu��j���V���ޥ�ׯ j�,�Pq�!���t, ''�j47�����\*C�N�p�h#d�|��7l�.^߃G+��{�����U�񦧌���A�v�\_��� ���' ���~'��������Xv�\]y�&gj&v��{�\_�~��wR���2x-H{�\[r���v�xtGW���O\\�>�n��\\�y��Go8���4��;P\`w��=��7T峭3����N3l�R�L̻)�yF�����1�5u�3>D>\\�€�Ȟ�^���<������}��U�z.����-i��qg\]A��| �q%n���@�����dp�d�>��y��ڻ�RX�j�;#O����SMm�lq�q;�\\$�\`����< �/�?�3�@����H�oBݧ1�62'�N��e�;�u�c)@��e�vM�TǯB-�W=�ܤ6�I|0��L(\`���uv�\*:��Tv����I#��\*���l��� <זbVLᡎg��M�ҙk�2�<��������~whZjMĕ��D�aV�� �rg���B$�Xk\]Mi5l�ZP+:!,���8���,@�#�V�N7�q���\[�݃k�hz�����&F��-}i �M�"K�:w R6d��Yy=;�"��C�HUv�!��� �R�(�#�͏����h���p#��wx9��B�������SG�I�P�6\*b�6��~c�ݥ�m�E��{;V��� \*~$@�FU�$Xv�8e7��K��b�N֒k����X�t���( �y0K����1�0�)\*�Kܚ��Z\\���.�5��� 0��P'�Wu ҃r��.�M��{�@��<��}\*k� �v<΢>��˨~��9B�K��oF\[ J���BN9��K�?����aw��\]���q$���E��ם-�/�uV��s�c�������/\\��-+K���c�6V�:�XE%��;}�n=�J)��n���Ά���a�>�~%�M��< �PBO&�\`K'�H��\]a�%2d���'\\~����ʬ�O��!\_=X�$dX��e�sU�q#>�� �v��f���LY\`b��#i�f�h�O�% f�~��\`�������H 亸��p�4�WТ���������EP��\\N��'�B�L:���8 �uH6�~�4,�%��=4�Dhr�,Ot��|�B����Q�i�D�v>CtM;B\]o�����Ϩ�Z1/���'�\\ fL)��ΙbdQ9�9$ ��ap��,N��Պi �f�Z^�B�L�wx�{��6\_����8�\_֭8o��O��۴͔tӷ3�S�Ǐ�.��x5��5NP2Q�����|�\*�#��J��p�.B^r����.��Xo��߁(P���<���FQ�!x�?��b?f6���4������^ �P� ����D�ѐf�A>Q��k ןӡc��=6�9��C6w��w\]�60ڦlh�-��v������������z����лЀ�aW�6��N�Od��G��t�v�V�8˛�<�:ļPL%9��nk{m��4����;��%j:=����>Ġ�一�R���ie����(���n&h�-Cě� i� ڱ�:��S���?��| ��{�@x1�\*M��\_e�\\�@���A����U������T��/��N�������zQ}��>yq���V���\]���0��i�����Al��g(w��X|����y�5�~n|9��O\]�n��p��r��k���L>̪��\]�ʀ!۱9����K��q�|#��U\\&�l8��eTi�ŇY���}�7@�e�2�ۜR%���91�\_iP����5�#� i:�E)����QI9.�B���"n�H���מ���r�F�6��F�镲��Y����R�viU�|��R��ׇ�%H�SI�m���9W�' �Iއ�kX���ɪŨ{�� c�j��}rQ'��~��h�.����ә�?؝���߻޶А�q�q�|z�7�mn����1�-�-���$⍼�g�ׁ�@.��?�5.�3�%ݢ�vr������\]�{\*)�t��N��\\POtץO����o �z�����b5Cn�!�"�����f4�da�<9�;�5p�z��j�)4��#��;\]�0��n�J���c&�Gm"��EЎ.c�s�B��B� MY畸�2e�{{XP�>a� �G,�h�b����i1������\]?������c.� !{/@lg�\]���=��&n5~ �Y:��e��2��D�%���\\��-Nf\]w?��3��.-��mՁ��Ƣ)}M{-�������gfL�i������ �� ��M��SrgI�+�x"���䗚\[�l�C���Ŕ�xϬ�e�� ��|E�Ca�W w �8�VPNA��m�u\_�. �bܭN�v�N�u6��/n�ã�M�X���"Cbnr�����\*h�3t���zi�#��#�|țG��X��������ZAe�N+^�E4��Mz�b�Q\`&3�ߑ��$Sb� ��r�9�?���q0�4�U\[I~���J���\*�C���T��c��73��ʫ� ��z�q��.\_?T Ŷ��U��U��6��#O��u����${Dz�'dM��;�3<�b �{�^� ��Ƅ\_�(�Ts���L��!8\*\[��\[�L��. ���� c� v\]��DL%p\\�� ��G��3"��bZf,^� ��.��\*(��m"�,���<��x���1�$j�M�V\[�{�Mzh�:C���ܼ Y�}n̙�O�%�W�H.��;��' �b��!���!R �V��޽����\[t�R\*r��X�W��Gs� ���7�<�2�{ u�W6uqd�\_z"��b(�y;� ����$�3 �Y���q@����c�zt\\�|ɓX��r�?�&E�\[��fG!�l��yV�6 ��9���SL��AnD�8��|��i�E-uEԔ����J�V�t��OA D�t2m,��� ��a��X(Bǻ��$Ԇ%q ��<5��Yx����F� \*�/��'�\\�\\5�"6�M�3� �Mk���>�N� A�A$���x�r��hWf�+��\_�^NQ��3�DM��P�j\]$ӧp����w6)������H2\[�J=�2�������r���S�8�vh�EZo�JT��6���f��Rj\`뭹�\_�G0\_�,\[Z�U䄤Q� ֎���n�\\�\\f�2� ��e�$u��e"%�~��b�KA݆�<�t��G�nw4�,�U�SH�L{ԯL��x�����}�84B\*�X 2gh홑0ѫ-�u1ay{g���mB5�\]��G隣u�����ᰊ�ݴ�t��zD�� 0�~��\*)h�̴��T���yh9/\`L���ӡ� !4���� +��6�/�}�4� �5 G}M����<�\`���I�����O\]W˨��-@t��^� �j�r5�?�"� ��������J5\_�+��j.� K�^�8�e$A��@0�K(���P�s-�8�>�=͘�|�Q�@Q����oI� Z�@�F�W!���^!\\%T^�ҽ�WM\] �wT�����AR���?�8x7�g}Q�F�<�L脡>lsq8�@Jj�rW,�{H�f�V�'�NȄ��th-��A�2���溸AK�T�\*��z����oB\]4t춳�����z�H�<�< �<�~b���lK��fE���������y9G����䌠���\\��z�$�eC�O3N�m�#ᅌt����r�%V�8G2��9�\\X��@�.#���&D�����& '��(�<�z�}~��4���������!Z��g ���iE)��DQJv\`���G�#�F�����{Y��IC\`��ܳ��Pf��K0�{LGb�/�+�d�#�:���4%����D�����b\]�捀P��"Z��L/>�r6���OQ�8�y=�qӝ���9U.���?Xo)����ЍO\_5j⹛QFu\[$"��F���3���#�Mb���-�-u�N����������b4���rj��� ,r�n��Y����F�1WMe���= ��E��̥6߽�)¶h�5���ap��ˠy\[ʇ��u��k���&�Z��{0���Y�N&��v��qK+ Ti�kɧ��ؐ�"����p��~��T�T�/����/$?�9����,/��9ǸO�'P����(xЯ�aN�u/��n.\[w}���i��)��q'��t��˼1� ��� ��,/���������~�nj�K�Q�� ��e|Q���/�a?g�����p��g���L��x�"�������L.�%:����d��UF�;�Ҳ���9)�yGKq�����m��d��I#��zM2���Έ��B�I~X��c ɜ�Ȳ�:y�8Pm@�L� ^x������;����N��u5����?���|=e�|�o�K�@�'I��5�R&+�� �6.� �EacxI��m�<o@h�ChV t ���i��� tKӀ�����.���c���;��\*d�5�oCAǙ����/���l�w���;�$�Qf�O�e�.^�\_F"�~�M�/ڢ9�-�̈́c��8��iɄ"��蕐�z'����SM�4�㱨���N��o ���4�q���V��M�x�R�^K���Ї�ľ��'@����q��L!�J�����!��|b�>Ei��Dn\_}g�QS�{�#������{��b��&�J�xɃ��P��3��7HY�n���3�V������޾1�%lK;�\[�=��+�!�Ta��jƋ�%T�m���c�M4J��� �:���' �~����˧4h�����߼O�93�l\\�SX�a����� k3H���ה�?O�7��-1~�Y3�1��p����yf�،9��{�>�&0�\\�U\\�p��,+��1{;r"�x�Bg�o��MO0q�R�{Mb.!�W�(���Jkӄ���ΰJ�����~����'B=���ZS�5'�E����ί��E���M�����E�J��Φ8�/c�G���PXKW�)�^�1rrs���������yhz�)Ct͟��Ok^!���=�Q~&���w�~�i��%x�T$�5�p&�F�bl�B�Ɏnzr�f��\]��3٦�y>݂��������v��!��u�o-�b�� ��ɊNG��6�G� hGk;x�a\_�\]楍��󿷍��nn�1/� SF�E���^�5F" ���G^�<\\�mjk����g����W��M��@\]�2������'&�8b��؃��޻�GK������� .�mY��ݬ14M�Q��i���"#k����S3�Vof\*,����S��-K��o/\\�����@?j�m ��QwO�� .Ub�/���#bݻ�f�ڻ2G��& �s�?{��K� ���eccq��8T�'�������m.�!r�:ߕ i��w>#w��"BקJ⦈��d� �˨��QH����� �D�G\`Dx�F�5:���\]��4�&�wmօ8��K�'��E��b�,e#\_�d����\*���b������u�@�#�-G� ��d��;�\*� ����«7��꾶)g�:��UP�(�8Ӗ�c9����"U��V��Tɼso� �&��Ƣ�͗s>>�Ԩ��D:@ �P�d���+^�L}p�8�k�Ư(�h�|�\\�!�i&޹���j���!�F������;�����ڲ����>��F%�\]�m�$�x�QR��P�9WgGX���� ��y�\*l��y��x�}"tЇ� �b�)! (U�5�~�ju�A��s�F��C/e���r��9�e���\\��7� c��=L'��E�W6z -���A���D��.r�9F �;�����^1�R�&�9-º\*-ㆨpLߙ^g�\\}�cpp������~�y�1�-��!�+�t�{�(��J����V:G��������<�Sc\_��j�I"�U�P� !���B�|� /q���> ���%�$b3Mf!ۺD5�{ƉKF�t���693��ǭj���8����<����=��.���H-x���!��jE��v�N��k��XQ� ��. ��j��U֛��\\c�jX�ė2 ���%�p�B����Qx�\*j��Do?���C>�U ��0�5������ �ѠY;6� �o@Y�HS �0I2��E���C��mo�0�@}�� K�֯����k���j��Nゔ�E��n�E��\`��C�Ԩ�mk���;X�� ��\[���P����U���hj+9ZR2V#���h��p�z���W�p0�:�w���1�|E�ڮ}W����\` ����<�.�z�-c� �<����v�'�%�5\_�����r�C �Ġ��#�e�U��A?�h8�wbq��>�8�/�K�(H���u\_�=�o�����\[A��\\@��N��~�=�Ê�m�����"�^\_� z6�~+ ��sL�dd��P�l�-�� �I.�L�����o �f�<+Q��Q�wj@.r�C�C�w ��Dݗ�ܴyD��$G�8u�՟���\]Ȧ�s�\*� 5�O�c��H��+z&z���x �퉽���\*/"�\_����$�^�(۝o0 \]i+e���Z� ��)�D��+FG뀻 �������&�0�/���޿�rlj��r�PǴ�\\�J+��󯪈��g�h��L��������N�4���zgd��H��AT? =��ƴ�Ƕ3U� }����碾�#͙�6�����b�^p�\[��q!��߃� ������$&)p�N���0���8s�ߑ�B�6�=Z�ϪjWmwd�U۾ۑ7�iy< p}���\*V̵�|>?��wOCC�"^3�!��A��w��{!� � 즦S�4$���n�VI��X�V���qwn��wrê� ��?t9��9Ԭǘ�p�E�W�$��:��9>b ��D�ᔇ>g0�D�y� a�deSV|ǘ�U�5�����c��3�e��/e���o��p�~K)���0&\]W��)��>N6\]�/���kt���Q��}� ���gi޵k���^��;�Mx�>�g5���0���i��x��B \[� .�hٹ�4�΅-ľF�s�g���1NJ���N�5��!��pQ��s�e��Ȩ��>��o$��?��g�������<�%��S6�D������w�۬���>6���q7�Җ߂j���?\]�����nʩ�R9}�\`�b����~G^��P����A·����\\�Վ���J+�)\_�NC�����Y�227�E?\\�跴��ѧ2Z���X�U��#�Ep� ����RP�M�u�JSB~\[���{ڴ�ʥD��� o:�0��P���q3�(pG%� ڵ@8�VӦ4�����ކ�s��B� #���u�q������z �����.䕝�T~v #���q/Q��I� %��۰ᪿ��;5�\`\\���&�6������\_���p�E����e�Ә�cN\[y�����һ�W�v���r�O.Y�;���U���=-�ZáL��.� {�#ܛ̆��G��k{�yu�>�<���bH�����=ܟ(���s���\_ �m��\*�1��Au��b�g �� �\*�� V�e���,?D{P>�PP�\[�j ������?��:�A�őa��6����ħ���)���)c�x�łĞ�����3�.CWfETw��s�В���afz�OE��z 8�7�r���~����F����\`h~�U���C��\[\[F7\_��a�\_�����z��6�m�z �ހ��b���W�գѣp��'���� �G�œ&\]%��+�����fI�)�f6;��33Y�˯�Yٸ81�aE�����'4����^ �\_�t?QbR����vz�G�6��|�U�QWC lQ:��XJ\*��q�-X�/ְDE��SVj�űI���� ��X&����jſ���{�����\\ƚ�g9H��x��ց+L��P3c�Tt��c�+�ޯ u�� N��򺨪��d� F�wd sϮǞS��L�� �<��k=A�� �7��( ш^��n�ql���h���F��,��U�%��n�@ܪKW �ޱI-I�5(���D�U\\�rH^p@ B�g��^� �j���m��\_,͛~'�H@p��\_��E�Ap������ˎ�@$\*|�E�aC{;0���������P�k +�|?#\`A�%�=�+����u�w��Z��KJލt�E��#̎��n�j��5v�閶C5�.y��-4�HI��\_�79ݫi(n�E챚���\\C2}/��EF,7���"� �����A��; ���IX��re���q;/?#�{�\*�������OHQ��e� Թ��TZ�= ��zz�lW�� �}@���e��Q���R\]� �~�\_��n�?C������fh0 �㓶쑛�u�6��#�J�5���T�Ee8�����ڷp���rʔ�@����Z����8W�ߢ�m8�2.��f:�A�Yψ4�A7?\`\]��ӓ��5��4�K��O���\`\\nM"4H3JA��%7�� | ��zWC��Z�Bg�/��Hɟ:�whR������ڮ�Q�-�4BL,�,�)s��L\]��{�����)y��q,U�1@C#K۝�� �$=��ު �V�qN��&>@�(6����T�X����仙�J\`��\]�P�?�fHQ���3n v�G��?t�JP�\[���Z�O�,7y �����,l�� ������h{u�Bs�b����D�Ng}���u\[�Q���� 솨���w��Y<�u�2�Ψ�$+�߰r��·�����tq��Y����7'�� ޯ�c��� �svU ��2�W:��x������\`:\]�7�z�gB�?$�48S\*��� �%^4�y)��:���.E �{�od24o䛼�:����)�L�H4ZH!f5�\[ �h�ҎX ��U/����&�,n����ʛ�Й���Z�Z�����'|p#�2}���g�-������\\Nק�8�(��/��O�z��7&�f ��2�g���$Fφ��CC�����8�gK�w~wP�����a�|7�<�W�������ѥQ����8l� NP���.xJ����A�6H{��Rt�4EflC ��U�eX�\*@���� ��V�%dv/��|��r{&|�o2�o�O�́�:�c;zr����\`��C�{\`�v ��b���/�v5��9BE��̑�m�Qd�N\]�n��{��ʚ�C�h�h�d2�6?\]��Km�?�m�~��^�|r��f+�� s�Y�9@���'֭�C���\_�a\`��.��~aKUX� ���$7�+/�exM���I�>�C.r�؄'�Ђ)� -ArW\*d�o�\[\[r��7���D&2�k� �P��y$SBi-�u��f7�������u�(�} �0�g�|\*���fB7�iK�~1���Z|4ܶ�5ڣf� n2�d��\`�7.Qx���\*W1�-$��4�1h�C�}��k4� G\]7��$ũ�Mr�H�s�������F������@���>V�h@�Adu/���I��(�Z��M��=�����㑀����(�\_�Bл7��,t�E�O�?�7(c2�X�qޑ���9��X�xм���P A\]�L�Q�S��ap̰yA��8�1P�Ӑ#�5�ۥ�y������/�&WC��S����\]�����!=6\[�;���Q5,67I�\_�ez%�ƨ�FL.�5��������p�G��Lo�qn��1/:�,Xo"�690��!�e֡<�D���$��Eѡ�,8� ��C��\*r�wxǪp��a����k�)�H��S8�O�� �4 !�D��g����2,a@a!��E��E���P+������⭃�Ӿ�=L �Ц5�?us��h�a�e)#8G�2p��vu��,���3����p7�!j�E��t�Ҳ�ҹ�?؞��,��U�L�M,�>��j�6�W�08��T�D��7+���Od��/)8�w�D3��I��N��0�( ��۝�\*���)�+��f����a�� �W�� N�3k���%���#"���)%qGR~�-�����^�v���Q��^�9ql��2� �^�"\]\`�X�yi�6��/t��!F��xg���H�\[#á��Aȃ{F�g�c��M�,b|�%H;�� r�|K���F=Q: �rS��.���c��h<'�1�L�!��;� �$��èvb-�m<Ϸ��� pD����Ɏ�V��\\{��D�����kZf�V&b��� m�@/J� GM�'�\[�"T�;���)��q��=8�y3��}q����r��� �,<�رAGPѳc���Aů���{��rߓg�9���P|��n������ E|%t v.jx��&��t��� � �� d�Y�+Gs�\\M!'edV��Ki�B��j�9AX�)r��!��+QB7ܸx�L�3�ٮ�s8vYwPT�f�Tn&���}�խ���4��{�ȉ�v�\[��89�+�ߵ䖚���yph\]�\[k;u��! " 3W�@�zMv�\\~sFch�h�8n�P;�g!U\`\*�@ύ�ݙR���L76�9Ր�w��A'�Lꢃ���K�����V���,Ι�X�\`�j:"�O�GW���2�\\b}�>b'����5����cx5�P�oĽ.R�K(|����ǩ\]����V��ewV�,�$�\*M�)f�y�Uz.|���>I �eO\*���e�ܤ��C4R)!5ǵN��T�V��HrA�eC=�K���c�o��X�E�0sk"h�Ӹ(��#��x�1:?By{/�us���db!�6d�w�Pz��L��!��vE\*C@z�pK:�W�b�\_W40K�# К�#s �\]�h\\���49#����.�g���5Z��!F+\[��+��Z<�sZ��}�8\\1ٺL=:� \]wW�Lu�Q��~�'��\[� �X<#�\*+��l���� ���毃�F��$ �o>d�}L7��t�\_�������ۡ��t��N�;&s#����钟e�������p+g�����\\b���p4�(�NP�!o�ăG���)uL� ������T�€l�9��+��Z��� �ġ����7�9��\*�0����&?ҭ#����b٬��+ /Y<,�! �n��R�3H(�&�lw,�6�2Z������¬ �G�ae�堏�J�ItP�|)��vH��h6�뺸&�%���Ĥ��e@$qĄ8��#h�,�d��x�rۊn�KEӿ�2�Es��Ρŭ��P���'�0�g�m�o��%����SKE�;^l�L���;�� 6+V%�Ѩ���8�l�oMMJT/5Jj�!L��pP)��H�F��∈��cgs��%FZ���� L�&'�4����7���#\_���\`8�\\� �;��Zn���!��}�oɯ{��b�I�!6��M"7�� ��wZ^è�G�'7�u�^:�T����&�m��cAM)i��/5�J� ���� � �F�:c9�+ �\_���\[�­��p� ��u�k��w�;�����91�"�S� }��9n�xޭ�zjH"�c�g^.���t�v� f1�Ϋ���bcShn�|9�!ß�uT\*iwW8�Z�^!� �CLs��q¨GID)R��(Z�gx \`���N�\]\]��\]��B��I��c���{� �p j���rr����ngs-��n�w��p2e�N\_�¬q|f�b�mN�f< k3���(�{� �dBw�-<�Nj�KY\\ �IU�\\��A�s������do�y��G�eHF�jg� LS� \`����Q�} ��8Y~kQ��Cf�~�n!�M�So�كz:������V։���c)�����W.�k �>�U��F�̉S� �i07"��>&$a�;�BYcA W��\]��@�i�U؂��✪Oy��E�\_�IP���\]�6�,�ˣ�'pK=Ă4r,k ��92��Bx��bt��a� Z=����wk�ET�R7�kJ{4��6rv�ۛ1��"��^+�z�L��\\��1�3�}jj�bY��s�omo: �x�DƔo�e�7��=��%� Dq�� 0/eH�%�qh����t˦#=?q�\`�}�|��P%Oq<�(��+�N���;i�2��iȆUhSN����#�2z����Q���+wׅڃ�)E���we�ɝ�M�τ�#�& ĭ\\ �u��a��G W���I���T��<�� PƼ����)�IV�M�K�I� �ay���2�aޚ��1�V��+Ť5i�O.f̚0�����e��ͽ5����MC?�� {�����DŽ0��� �k��k�B���C٤D�{���/v2�t"�-�ˬ�C��x鶞E ����$���N-k��z��;# 2��tb�!������z�4��f����H�4�ƝZX�&����\[ڛk��X\`:b;XMs�!���~�5h)5�ᤄ�G,���q� Y�O���y �y��<$��:�-�$�\`\]���T\`�Q\\���m ��y;��C� ����DpR�A��^�~o ��\\��������\[�'4�� �ÑE�Eq�um �6�h �y�\_�۰�w�d�?W��߽�Q5�̓i�:̲����5�?@s�u�׳����ؗ�H^��v�(����$A�@T�湊��hՄo q|p?�\\���>���p"��g����O 8���Ư�\[\_.r�i\_Bh��F��R1B�ǰ�4�L�pXk�� E�Y�0�yEE |��p�8��4��\*<0+�j\`�t�8wڒfy�ӎX�Qv�w6 H��mL�= �y�6���Z�&� �T~�U��ػ$�L�NU+j1���x~���I�=s��Od��ILDF\]e���ZO�|��6�X,�U�>�O�=���x��$�\*�oQ���'mÍ�n9��,��վ�T4"c�/; ��i?���Ӹ�a�,��N�>���OL'QgO��s��J{ֱ�d���))\\��V(U�?94Bk�!{������#8�7�"�l<�.��2�%��"v�Ⱦ����K?�Zw��OB�B��-0�Ɂ������� �I��D���瞗�r�YɣM���\_=�%7 �u�ڬ�)�P�����:S����{-���M�gh:��w4����b�s"L���moM����Oj � Kơv���<�g�x;�\`)s��󁇿��g�5�8�\_������O�����0\_��K�)D�cyKs��˒,���2;����6���x��cɢ�W���jw���~��榍���h��bhI"�+�=�mY��S��N��ϕn0�~$���%vq?�@¾� 졀��aN ��<�Q�� �&u��H��v.K�p� �4�2��W��/�\[��m�Cm�B=D�����\*xf�<�k�Jn�ыm� rz�9�.�ZEbN��$��0�uV���r(S.�=���^���MS97�/U��W @f�s�Rf���5��p$r����0����;x9�^;B9q)\\�\* Yk\\���e\*�t��(����������X˓� �I6�Q.߫Vx��~�Lǭ���j��߯A��T\`�O�i���L��֏(�L�9�o}egLK�OO�\_j�~Fh�� ��=g��{�}�M��ͤ���\`���!R 8�/d��I��XQ�>I8��u�?~z̾Q<�����(�� ��&�$9\[V;9ö�7 �f�w~�+��g<����d������3���X��ś�b(�7F)fF��y���C��)�����Z� c��VNPU� �l� �Ɍ�Y/p4j������#�Dc�^���A� ,smr��5��'���(��z��8� ���P?�ЂI<�V��Vd)�'̄��}/I�KՌ��\`�"�D��Y����8�����a�)�E!�D堩p���;X��j�DЌf\_^"��n�4�\_�c����񱹨��r�(׳����n��k������q�5V,��+\_|E�b1{�Vo ��'/�w|�W�.˱���2E��̊3�c����=�\]:���������9�̶m�-{�������Jl,���Q'��Wm�����׳�8�C�Z�3{R̗j���hF|�- B�X�R�����t�uŹ�?�Q��Q�\*���$>)^ZP=���B��:�OL�=f��W�/\[�A'�V�3Y\\<I6i=y�\]p{vu77l8���\];���N �\\N��1�߯��|��}�\_\[2���A+�Q�%>4���Կ��������eǬ �Li��9u�h�����W'òU��\[��d��HuNR�:�\*j�"TN�E�g��Lw��,Y|f)�=g�������rs���\]��Hw��r"��qs�8�'�q�o���d�؀\*ޥ�D�m�d /\]-/6���W�vo�DKT��j�݃���Č��)���S�^¿��&O=��K�к���\[\]f�v�TyY�jڊ�\\9>oa=\\,Hښ�/�����47,|�6Q|���Q�Ae��9ZX��Ҹ��Gr}�  ��8���hB�pK��Bz���BB�����5�k�ڰi����1\[�L\[�;�Cl�lC�����CϥJ0�g�S%��+K�����Fu�w��:�����+���F�q��%4�!�(�%��\[w���{ˆa���a�2�R�\_�ʳ�ua�C찚�FHD���2��O��r8�d�9��&׍����o\_e�h�t���^�p�����=jzڋ%Ӧ�Dq��nޠeI- ���a0� 4�eK��<3x���X&�ѭo��D�ͩe�:&+������%Mv+%���{e!!��y2���D ǟĊG�A<�q0�W^Rw ĭ��x�|��A�WYB;~�€΂>���d����)\\.��7@;l�gu�o�It�zF��U��pd��ќ4l�Ց.�ȕF?�Dt>%��ٲ\]c�h7;v � \\O��F�IY��<��\[�'�6y$��M�VZ�\[���\`�(n�OdcJ&�2�l�5�\]�d󭎯���ha/<�Ԍ��e�U�Zzu�f��@���:n��L��ȅD�\_��nʹ�-�O�ǫ����l��K��\_b\_�Y����N'| � �=)H�l:������\\��n�QGBJK�\*H���GA�ߢr'f9�ޞ�}36���6��m�2���I�q��Q��u�yiٌ�'�\`���P�"��鿾�� �� G��n����mmI��ҼsT�u(E�!\]�c�F����^\`�k\*�\]�L�� <�k�Bg����L�E������l���\_�tIUt�^dK�v�Β�oJ%jh�vg����i�������7:?3��&�O�0���È#G\*� ��(X����������oK��\`����r�-$¦�5K��k�,UZ&�X��^D��N�  y�=\]��+&�Q��h���#"������W��Kk�\_��~�w�LJ�,\[�Q��{��X��'�\]�&@E.���i{�5-��{�g�#̾���\[l��� W�yc�� P�$#�\]1�V� �l���/ p�� �?��g�+����\\��:!$V�<����~��˻2� )��8�J&;�$��r�{��"��L�F� ��f��3��\[��(��;\`���;+��~/p��j���Oi�!�R��rPP�$%UF�Uֆa��,���s��1��.������ �w��-.���C��R�� �UA����\_4~ �a��#�������u�2֡6� ��Z���a�x�L׸�1���(���뮊 \*�OeQ<\\E�\\ϵӵ��ۈ�plT�"���Y�e�Q:)ˁliY#�祭-��r���$e�����N��'����R3���ދ�zd�dh�N4�? GW�7\*TB/�L ��)��K�H�\[�~Ժ�%Ĭ���4� �.a5\[~G��C%���X0����u̳���J�,�� ���.���A/����sy���wLy��N;"�q�"��U�6(�N��X�k:�B�x����ˣ�L2�\\�:����, ٞI�����iy��g^�MJ�i�Rt�쭐t�ɚb�h ;�֚ �;Z��a��Pœ��s�ܟ��UK\\�+�0-��\*\]qr�Զ�Lu��t�-J�\[&+|ab�W�3�l�j����&�ӱ�\\�8D��I����8 �o�>~svu���׽��΢��Q8���\`+�9���X�QkR\[M�\[B�(+׽ �,���8�P � 0\_ଵ�:c�g협��/�ъly�ǽ�5a@Ha�����U�b�/4��3bk '�0�g'�p�w3���з�UDW� ��z�K��-�#g҆���66�C<�� <��:yu���J��G�cVz��s�k��e��Fob'�L�k��Wq )��w��U��s9�������\]�B>����;��+F�b�>�eX�\*�m}|�� ���?��{��pbuW�&���|+\]�xM��Yf%�^f+}>�����%HLk#�p�7�ǝ�2�����S0Y��DI��䬣Vh�����Lt��ʟ���{A��\]-sD���Cق.���V�����i����!��|�e�7�1lX��~��s"��2h�q�bz�X�I��@��ڏQ�6xqcZx1�K76�3��G��6�i����@쫑���ZS�z� ����f߰�\_����~U�û%�Q��WҞ���L���ݝ��5(��I(K�t�k� �Fv�5��&�t��M�?i�Q\`���8E!�\_A2���.c����6�vON(��W�Nαyxѽ����KKMˮ������{f���� ��Q���1u�� #F��# �� � ix�~n�N^��|�c��Y�wA( �l�da�Lvx,�/��+��q��Ë^QOr\[&8QA��a���2�r�X\]����M�$rIo�\\�0���9�zl �C�ʗ��nl�z���Z��{4J�V�J���A8LB��㊐3SdX��좮��Z N. I\`WFA;�0��\_lۼ=����W)��=�PK�x�e��O��RT7Tl� : ͒q�m3��/o��Z��c�͂�����i\_�' AJ�R D.��A��6��xI��q�\_K��TV��>��W۵"�!w�aj9xJݣ�rcA�ؖi�i�<��}�{�\[ɱ���Sd̼ؑ��u�<���}��Ĩ��6�G�l��@TAɁP�"���Dn�Q�n2����P�;Ã�Z���0� ���y:�۹��^�� �Jh?I���}�-�K���NJF6F\\��/��}��\`\*��ʰ|����/j��q\[4P������� KH��8ѻph���6(3�>+P�sW \_�6����Q�+�l�n��?� ��2�� l�@ �\]9(M����ǻH d�F�$�\_���j��i�6����-�����F��\[��0yt�R(hh{���XKЏ򅮩VO��ޛ�U�iǵޒ1:�\[��|%��?�����it?n�\\��=�l�����\\�(M��= R� �ռ)nbnr\\�|w| p9�q������� �7�AN�L+�� �\\��L���K���{>����� ���̭c��h�� ��>5���Oƛ�W�j�u�\]C���Vm�҂�e �3�Tx�tE�w �ã"��F A?u�Q�ٛ51wԩY�� wO~e�{v��u�Lyt���,�l�g����ub���� v����"�b �̞�ЧI�i��k��J轴<���: &I��W��b'ʋ���M�2���X��R�k�?�@B�s���P��.d�V�g� ��!�\_����\[ʤ��6��+�+�'��\[|�ј \]WE �,WV{��Z'��O�����h��\`)��e\_-�6g<�\]p� P�20�EW��\`>� ׽�Iٔ���� iͺT�����ɩ5� x-fÌ��, �NLH\]U��p,��}f��A��j|6�֍׎���q�CE��/#�~CɉqR<��P�m��=s\_ Ɉ�Xt�9s�m�������F �ˆ��j���I � ���Sj�t���E�^�5��2�4�"��S(Y KS�mjt�ZP��iR� �u"ܘ���g��~�c r}t�Ay�F��%��0Iv���pi�������� t� B�����|IwM9\_�)؆��X�o�#j.�U�w�:z�x^�^6Bm�\]k)��K�Q>��=ٷe�\`4\\�z�"��\`a�N�T�g&�p���e��ra�\*e$\\ܩ���b'@��(�o\\lCUn}�a�; <�L���&"q��&B�L��?<�O<\*��oZUz��k:'=� ��C-,7������^6��S���\]����bb��jY�\`^O�̽gG^ǃ�4�Z,����1�򹦟p�\[�"���fGO�$l.����,dj��������S��ɊnF|�����q��yOխ����ge�bb���4D�G�ϛ�� ��s$�>�� /�þ�F��J�v>rνT���f5hu�������Ӻ7k�5(Y��/i^�\*����.ɦ����xk�P���H��h����s�y�5ȃ������gT�mN;k����\]����\].ԃ�\]\_~Z7�p�p��2���x�W���\_s$�ʕ ��������vz1�1�D��<�U��r�/�ēh�pi|rd?�}IV�Xe���Y�o��?�ќᒨ+B��nGS��Y�.�N�}���v�1��(�����5/Eު Z�U�;Y�5J�ٯ|n���u:G�M3J�Й{�$g��6�U�b׾a�(P2e�s��r\[�������� ���u-f�쟭\`�O�RM����oSC���7n�o\[ʄSaAW<Î��J^"�RM��-��2�#%��� �F�6� i�R\\�3��� HN�/q�%��w�/�lƙc:t�鳒�{���tBX%�B�Djc@'�C&+.ʞ��Bgǔ� }\[O}��s�%A�C#(�/ח�΋#���O�EAp ����\`��ղ\[N�6 u�5f�f��J39��9�U"��RU�r��{�T\\FZJ�h=�G|N���NI'�ZfE����7�&?Hnv��g2�)6C+M���˰�Y }���Mv�!w�z�I� �4�^�muh�98���S/��U�7�<,\` �6�w�4�'�;N�\\�~��\*W�2���te�6�;�ƶ��Ƹ�j�ɑTKT���u�̇��gy���\_TC�����+�Q5h����A�΅ :��(L�:x���T�@05��F<&¼�eY\\.�ԫ� զ%�T�� ����\`cQ0���b�z����"NX� QLh~\`8 UP�E>�� �&�y"Y��3�l9?�����t7�ƹh�|��&�����RGZ�w�;T2���n��Z0^��)3��/l'�S�w\`�lw���'�����̏�A�LJ���:�^�N��S�FIv>��K�E��c�� b$�2��$��\*�/}�\\��)q���DT��� �(�iY\]�)�;s2�}�/��������G+� y� ��"� X{q��5� +Cc��m�!�#oYoocɂ\`&����tD�;lY�)Ҝ���b�⚹oX՜�\[�@\`N X�VӉ���������u����� �4������wu>�DM�T\`�o�.g�;� c\\�n�0b���h�<���S��i���"�p4�߬�� S X@�RZVi�1o��w����;?&£5/؟������'qu�+���®�lC+A�v�3� .w��6�)�#�ya &���Ձ��M�U9�q�eΧ�d����Df���,�Ѐ�����FmƆ�L��Z�T��M��.��f�� / 4�V�2 � whL ��U��� ��̩ʣ�Zy� S\*� ��؏^�mk�d ���Z1{ǘ+���Z,W£|B�+~5�cR�\*�������F Nr�?Q�m#e�4���:������%o����t&%i����9HN�iUuʸd4K�\_��%�(䡼���� !�K{�^4ڼR�SY0����v�,�e$S���KMd���D���'Ej�k� aO"藌q�(Z����=O��I�\\�5Q��2YyK�njy?C�� �6�ek "VVÿ�t����v��B@��݁N��\[���y�L���o�l�i�Qj�1mw�@��l/RK�Z.�7X�B�:���x��wd�ZT��R�읎�Յ�r\*It�U &�{�^��l�$�6���� g�9��4�I�Z�yGMA4��AF��37��ck(�-A�E1,����M�Va N��F7}��qR��ҭ?����a�vv����� i�\_/�����=P\\�6�԰�;��VhSb���1Ż�V0zrs���0�E�\`1 vuA>�\*꯶��ʝ�!e�\[���@���w���X�P�&G���z�{\*/����)�F�I�+\\�Q#�Q ��,'�U�2V�ŝ��� 1�ep����8����U�� �O���pM�hu�\[&ҤE5����� b���5�K�"�����n��5�z�X�2+b���lJ��I,Ϻ� �AW�$�F�������W\\��\[�q8��{��&���7�d�v{ � \\��e�\_�Q���;����~���\\�1gY.�;�q#lٖ������v%(�P�E��g�Vj�\`�b�q"M�\[���� ��)�TK7�����,\_��z\`ǃȨgYU�k��d�s�l(\_��� �=�+U� �E-������jeB�L�jv췡�G�t��r(�i��n䱼�j�܆о��LH{��K�r3�0 ��<�U�S����Z^�̈́�{.f���ć�߇e.�}�?7 ���#���ر�M\*YYv�(.�p�g"�t���f����׋�\`ǽ=�rcz���wCC),�YD�� R���߸��|W'��j�Xֲ�D��� ������\*�A�ˉ\`�G�W���N>�{W��q�ܪʰ�\_Kx}� ���Z+ǀ��X��4�����Ľi�(�CpxD������a�lD�nZ�7~D˲�t���Qa����U�\]�L)����sЫDt�~�{�t.ä�"��Z���!d���&�n������%\\&h�CbE� C<���p����j���v���0�d������/�L����2��Aw�G4��n�r�pKakHB�ښ�N@�K��m�鴓r����\]V,��Vc��6h��YU���ϙn)��TEZ�h�6�,�6e��K�&�P(~�8�i?�e�5^��F�~��Ѫ���vfP��%\*�\_Fރ���0��7��VW�{��rG��9By1}18��y���dw�� &�z΋!qSKP�jsψfu);7C%���W��v:5�d�1��:��$v4 >� �b��Gg�8T���z#go!�a�Kq�-mt�p�\_��˚w�.�\\P(��Ћ,�3Q��{9��2��;��ʼ�v� va�\`ۨ|2s$c\*��Z>!�c�&�r!�,�����捨� ��e��1݊��\["g0 �I����-�T���m��W!�sl,���;VA|��!�k� ��9QMUf��C\`�'m�Բ\_�TS����ץ�ĆU�$�ΓZ\\$���K�1у��YJZ��^=�� Y����\\��g'�J��9���"G���ge��2�uȪ�Z�R�K����ˆ�'#2�o��)l��7��y�&��5Fi �C��I��Ǯ�o۶�5�e�h18\_���y��6�;g�UIW�D <^���u�&��Ȧ-�c�^�����zt��܂EY�xx�92��vSZ��ܬ>��3K���!�?⛬��Hs@��Z벣���y������\]�ޢ�.\`�T�g�L�����q�G�ȅ�PR��\]O��<+15�\_h�I����np��ɷ���đȍIxZ�XzP��:L�LJ)����҇���M�%��֯.&�جq�Z���M����T5��Y1i ���l;~>�Q\[���h�̇��#��M�hv����LN��5�7�w �����0X}=:�%CO�0���$�j7�R�D���DZI��O�tk'��g�e����Z�;r$�{��-�� &/�ӧ�s�U:�' &�����iD�.>��7r��;<�Wm@&���}}F�'���^㒏����{d��|����Èu�~I40{���^Бl���\*��n32��6�Go/8��ms��w�H�k�F��^�/����^�1������B-��t�U�bǙ��7���t��+>%G�e� ��� �jש{� �n�=d&a�\_���zRV��%+�6 Z�)�Z|����ɖ��J;�H!�DAF����1�|=��Ac\\�����6ޅqijژ�P����&Yv���.,^�θ����S��h��I��Bv�7\`���Ƞw ���>�����-P��wp�^%=�>W�BbNj~�i#VM�o5�S �b�Z��+\_��������&I�lu�: @�#5��QiT��ع��j�7��m�e �� �u ��)0�ָ=�ӥ��=r�ѱ(�� ��8@��lz;���εc�h/�A?\[9<{��=�a\`��Q�Ao��$� /\]���V��!��i����xu��@4h΁��z=)�!��'fwYB&���UU���K�%ґ��TRYv�;��^��#w�\`�m����M���,� H �1�aՌ�Jk&�GJ����yQ��{����%�Qu���:\_��}W�xs��b����&�z���Z��!�S��}�L�uz�����:Nu\`~�$ZC�V9�Hg�����^�(H}.0t��E�xQ����km���Ҡ�ӟ�tC������w^��� ��R ��Z,�S����/#�o�a�u�!�E;T�;5�y�5"�$�nȖ�ʒ�;���p� u�J�9v��Ry�v��V��J4�F��+��\\6���I��.vʑ�O��.�&s Q����8��~{=�˳���Vn���{�����PI��x�H����Y��t��$�oԶ��Wx����(��)�\_��g��mzl����-\`W>�J���N��40����Q���-io2\[��l1,p�"31\`k�n����-+0G�h�k�Ӱ�I���a�I,5�X���7�7�D.���pX-+ q��9�lvK���O�ѥ3�<|c�������Y�T��!K\`}�;B�Y�?��@Xw��� Z|�|(͈����M�5�hF{dQń3�����y+&��� yVG��8p �>.i�C�� Pf�t������P\_I�s�&�XPZ\*�Cs�1�LQ�)!���%������u��peX�ǭ���hL.t�����/��� �7C���Q�O�zu�Y��X�b�\\�w�IPo�S+7SF0)�A��zve2�&>�?����J���\]��$>�E��-,1�K+)���؍�Y�������+��D�T���P#\[A�6�Ӝ�����ߒ���?O����e�FE��\[�t�ʟ���-l��W����!!�F�ef��L� �n�pZ�������;�R�\[��C�Q��R�!䠺��0X�EmY�V/o����R��rܻ,�F,B��h��\[^�\\���F�Q�ׂl��)a�~��'� ؁ �6F������A�\\(qZ�z����چ6�����0v\[�a��)įJ��%���b��u���-z���גq���&��E\_�3I�z�(ҥ��i�=ju?2�����Is�l �\[��SOR�ܔ�����2�kYþ�1����3���PN�V�D�k�:z��d��I9t'�n®���DK��&iS���4��\_Hj��\*%�?�t�5R-�{^�c"A�O|��d�K��v�fY�b��hӃ8�y0�� ��v��9�AcMȩ��)j�����G���2T���0�we\*R�& 3V �5��I��l���7�m2����y�8�<#Ӛ� ,�N�THk'�Cl��N�1;x�d�{v��E�V�I�^��T��\\׃� �٣Ѹ)1��)��8��}V����9�5�\\H��f�ӡ�;��5�9?���:9��V�ق�bq�V�Q�EY��9@���r� 6�D��C1�B�A�y0��rלZ�DF�RI�D�Hg&��.w�s����R��oMVu0�vK���2{)8B(g �ồR���S"�>���Xؑ,$� �~�IQ����D�������6I�� 9B\]O��9�S��t׼�W4 u����=���yf�щ�=�Nm2��'�e�Y�׸5�{p6�$�� 9c}E��q�>� ��c��(��˲u� A�q���F�K���Z���R~ӏwP�\] �\*/,��'�3'�L��HQ�}$�Ҭ|��2��BZ�k��X:��������I<@�#����� � ��N�!pr �B����'?W��5W+v��i5��^��ҵ�os�K>�����L�R1ޢ�Jj�!Ѝ�s�|ku���J��|�0o��U�\\+v5 h#�wYr�u\`��.4�fq�L+�i 䌟����������&HE��~�/v �^^�묊�U��; ��:h����FJC�����\]FJ��\_�/�z��c��AP��w>YOt���N�O)��y�(�"��\_�-�o����ߧ�2��6Wk�\[�)�ib&.�1��C����tb�ϸU46�R��\[�5\\�#� ���'s6u8y�� 'r+�Jo�/��"��)D�-���#؅7�G�D��\]���4����h�s6Š�\]#1h�v�xKC��=������g�k�O�•�S2&>+��ajK�B�1�������\*/e���L���F��w5)=� �) (�B�K+���lo�M3�X��l�h�}ιh��K�����l/e<����K$��(�\]� �(s�X6����~��r���J�;f��́�m�ʆf��=�P�aX�\[ �p��ĵ�D$<���;�����6-�h ;�� ��~e�8覥�N1!\`~� ����0q@m����g ��G�BË�n�Y\*4�7�P(\`6�>Pn�$wQ����,?/��mf��Rvqe��o�N��=���x9r���n��pH�b��?�n���)>��s¥��yz0���t ��F0� I����7@��t|\`����8�e�l�x�j���F����F�Q\`)�Q� �w'�x��&wΥ��E$�R�����C�\*�6)���׺dMm���v� �������%�KRL��1i��J�:�f\]�&;�H.ߡ#�ޏ�=���~ݰ�� W�\[�u�����A�M�|��� ��� �Z����|��� L��\]#e�����x�eKRv8S�Y���W��;�K�U\`d�:�#\[�e.�$�q��?W�T�՗����Q�7�$��lX������w�6��"n�Sy@Z\\7�'k�;A���ǝ�LROk9���)�Ipw���p�}D@34���;����T�eH�>�Y�@L�?X���ʣP8.���y�~��#�'�CY����8���$�?(�C��ق�ܞ���,rR��Dў"XY�S5���JM��2iN�+Lq��Q���gsX�d��p��>ch�Sw+�ꫀ��'�\[�ºߠ\*)���BwuZ��I����}\`Y����\*�;\*r�A��qW�-� ۟,5SM0��P@A��"ɇ�4ԸG�K���-��W�w0A�Y�ى/�F������|�����b�����a(�sb5��\\���V��d���$&��U;}�va= ����'\[�B ����t8�Xӄ��w'\\D�I F�,�T�%՝E�� ;Pڋ@@�F�\*�\\�i�\]�����Ʀ�t� \[�x.��.-ޫ���NoE�>��{5���x�T�?45?d���.�нuX��r)�n\_ �jȪ�4��^�g����N%R��%���f:mzQ@��ޞ�sS1XQ�b�S\\Dm��g�O=NƄ2�9���H�J� ~ � \`�\_���4\]������� z<���P?���ٚ�����¬��񝔃�2���<=v�Acd ��qZ�",Q 0�|E��@�)%w��ۡ�#���{���k H@ٽ � ��!\]O����gl�����g�%R+�\_�j� �6��6�����Dx�=I\*x;Z� �\` �Fv�>yxHy���jR\*c\\s���=���;�1Y�D:�7s�rQ�\*9W��{���X���I�-��1=�.���z�m�<�����9�p�x�s�0xUP͹�yd�a�n4��Fu�#:���;�V��K\`A�K��+x�?�1Z\`��<��0�g�x}\`��U�Q���"�Tg�Za�����������&�oxk.�ʪ�����/�"@Q�@�7�$��/��+�%��#�AY4d\]��z�&w�� �9-��^1su��}h֥@O��^�sD"s�q�Gc�R\*d�nB� 6����ӻ�W��N���?|�D6^�u��2И������;Phu^B/1��ɡl �f�ˈX!��w�"�;�~$�e�+D���.�O�fdJ�ÕIV߸��F�Ƥ1��ᡝ�b�A$�bi��� pM�<\_��8y?H�x�;$��cCw��?� 7����� h|~�p�h/LI!�{�kJ��Li68.�W�YҲn�#��W����ͨ���!v�kA@:AroxB�q��#���n���3���=W�����"?vs�p���f.�^j���Y�Q����\_=b�+9%o̫��/�R&� <\_Dk s!��Q � ���lR�|� ι�aI�g��T\_\_���㞹^�U�s\\�PDE����G�žE�� bjIc�EgO��|�v�Q\]�U�W�\_K��VKy�;V=���r�s�ֽ:/cE �d7����� �ңXP��#��D1O�Ɠ���� s}���J��z�x��ʓ�l� +yg���?9��ggxo)������<����rԦC�3Ւ�Wg�!Y(�^�/.�|�I�R��>�Z�͘J� s�"�t�F��M�\_.�-�CbS�B�8a�I\_3N�&��!2v�g���Y,�m�@= � �):�7ݝn������~hA\`j�v�ʢ% j���4ǖ�G��٥-6�@<�.��֨B�K��ܢ��\]�y���2a"")'����Q�(�Ҥ˸�Y;���Ԙ��L��̐Y�@��jnDS H�42HR8�1����� A���O ��8��W�+ݮ�\`�<�C�O�v.Qj�/�o���� �\`ٻw�I������0���FP\\Wٴ\[����d��@�8:��Y\_ \`\\��%��e���<���n-�u��ʂA�jy�By��˧r�W���@�+x�)�ŭ8�z��Y�:l���x���\*�t���JM pNAf��S�/̵潾�O���m��\]�hM#I'F�:U �Jo���/���ҾK>\\���� mf��;V����\\����7�\\�:��>���ۅ�8�Y��Dh@CO<�#AJ�f�����D������d�χ��q�\[�.�)�,cZ;c��;��9G��@h��Ob|��KI��|o�Г\]� A���A7^�Af�q�r�Q�"�ܚ���aүۥ�!LoG��RgP�T����fK�n�������0��鳻���t���GQ6��LV"v�������p?s�+�AO� ���?�G� ���3qt��9�3 ��bk��xs���oMqA�\\�H�T��w��W�.4;ل�J�4Ɓ�fZd��,�<����‹a{��'�>�:Z��\`\[�&��j�t�9��fO��R���F��{�R~#��A����o�;��� 3Bm\_4! {/�9��玱�r����沙j٩ ����\_��#H�T3�c�-����X3��+'J��f����OELvܗ˓J��хb�z��L���ո�Y3L�.���/Wn�\]���8�.�t�1�}:ɳ(ۗ���� rT�5E�Ue�C-�.7zG%�������D���w���l/���W���Z>9Zq����uE Np�M�)���\\��H5��1�7}��)\\?��Y���GmsF�jʼn��$�P�%͞cU��d�n�U����t .��b ��>�4�6"8�\[�n�G�𾂸���agN ����P�� �d,.��erJdi��\`��;�-�K�>�>��I�hm,\_�V��K�w���� g�acz��s������ �>���-L��Hϧ��a�����%�"�L�oYƮH@�k�w��2ᙽ+j�ψ����Ґ�,�0�����{����\`c�֕7�B6ѿd��� A��� 0 w����2�L�קfr��!�p�y��ҍ��\[�UK��j�9O�9�$��^��K�\\��j����; �ш!��w}Y9'�f���k�o�?pP/��V����|'γe��:~փ?�����ۡkIMM���Ǩ�/�Y����C�V�^q#nf!�îcf��c/���I�V�����c���iւCڴ��}Gk�q�s����!���@�\\2Nt��X �����-�^�� �ބ1;��ǙR�m����׆��t5DyE�V����W�vFO�^�ޕ� Er�ԁ�)%p��Z����x�6��B(��j��99���xYk+&�} ����sGN܎<};>Q,��6~ { z\[���\`��ԕhu� ��w�=��o���ۙ2��;>R@��\`���J\*���sC�9�ȼ����w�.��ڋ�Z/�����Yr�ׄ׳�7��:��jF,����LQo�A�s��Vik 쮵GN�g�b&iv����9x\`BZ���k\_O9\\,h��,���\[l$c���1����=�۬��T���yEc �#=�{$ fD ��@���a��i�������(t���3mdB���w����1�D�H�'jd޴�#�1��t�vU ��\_B���6�A;����N��4s ����=#�àaB�F��X����@��w�H4���<&�8(�/��F0Խ�� +y~3��I� 믳��֢����4�2�Z�N�'6�\]�k��֐�ʪ�y7�I}.����0��xFf����{T;�C��Lj�biJP���\`���X��fWo���>7�5b�\]���Io�����Q�cRt\\� �/�/P�NO f#�?H�?��&�e������%3�n���p�x�;'��)��SpIF ��acR���� %��\*�Z5����h�\\N�)up �r�Q�\*�ݢ�f��3�34��p�X+�8<��gxp���@M�������O���{SK�$㷟��A��y�c��ŶvNjG�#p�ߩ�)���U�"B��:�\\%aB�5�N#{�S����՝���\]�X�P���)\\ ��s�2+�i�����e�@�~�,(����u�l�(��"a�w�:(!I�i G�|����'���5f�e&!���ES���Lm��kd��S��y�H/��Ýw�B��7�E��L�@S}��� ��\_<=��N�)ys��U<��4���%���m���q\*�D�ޗ i'��2���A~��|/��=�5Q��\]�/�7����F��o\_Dz; A\`�8��Er�@�4z�ӺU�qb�M�S+��(f���wu����N��\[ >ds��+�a�:�e���ǯZ�Y��K��C�I� � ��� ڑg.�v�+\\�xd�;�m?\`&�"$��@���U?�x5�L~�"Qi���q���A�\[��tӌ^� ��Ѩ(YQ�}A�Fz��\\��E�P�E��ɕ¦+H\`LFsXw���?���R"�O+oh!�1���O�6V�,�O������i����\_i/��χi����4\\�d ���5� |���V����{�+����{�L̲h�Դ7���l���O5�rN$Ov5� ��DOKX�O=����pM��p�P���-�6mǴ�?h T�n$P���jP�&:l�4'��2��Z����1�z1����,���  �1�HȢSǬC�%�D>\`.��,>۝�j��^S��%��u�H�Gx������G ��+�����9z��|$��+�T}�������Lc�5�\[�F��a�/m h F����g���V��(䍪 ��ͮ�7����~I�^�Yl�:L�jH�B� ��LD���u����$���@$�j���V�0��%�lE���揬�h�/��8�����/�����X�}U&�h)!(E\_yl�!��d��3�Cg��S�r,�p}��.W �('���HON|�\[c-lx���o�t�c�}\]ɥ�ݪ|H((�z�ikYc�T�ިhї��B\]r�j8�'+���۲e�1���.��A���U8;%�� ��5�����C\\����,h��ޖ��װlD�H�sp�\`ȣ�)���!����o�͠k�W ���%SK �v�SNs)�i�� QX���ߵp��e�l&��V�i�Q̫�.�-����4�O����6����% �~�} Ѱ�e��2."�g� H�y�t�VRB��>��˜տ&uS���/s�s\_G��@ϳ�n2��� e�i@�.n��Fnp�.��Sg��v�%����\`�Ro��P� a�������ɚ�bW>�����m͏9�Nj���!��˜�\[�o�\[\[>����B����\_���7�6�{d�����1��O-T�������\\ݤ0#Ү���>e��e���:s|ɦ�i�U����}jwך�G���!U��,><$ �M�;�\_ƤJM�H@J�݃p�����C��!~�z�>�!\_�?���7k�^sͽB��\[6�m��5Q����.�T���|��V�)"cIPT�6\\�s���.�J�����W��B+�)zU�� ��3���c�~G궶 �\*��9?��O� =�d�/�-��!" 3Pa/$F% (�� u�2�1u��-g\[qe6s:����8c��ly�>� �ƛ+~�$�>���̿������d���~?������|b��wc�K��JQŵ���%F�j�� E����!�7\`���K9�b��Zrx��V��7-��Vm�K�rs�ZS�g�,�!�1|��#{�����M�ˏ����׉R�O��&�-�a��u�6�%���u��,�?u����?���Ö�ɢ}�C���6�(�c���D^�S1(��9��m�����U�9婐��L'�>w HV:bs�����~�u�s��^C�8����g3�U����\]� X\`2�� 5��f�2�~�bB��jV��>���:!�n���\] Ӈ���jA^F1 �� .\[0���\\��� j��b\*�� +��B)�?� +��W���1������!I�#\]}��A�\\3�ܟ��+�\`a.T �( 4��7��4;US��>�Λ�E:�����R�T�����J��L��(�%��.(bD};����'��z�Ά��3/��gw�\_����6�1i��䚷���+Z#�&�3y�7׽l� bt3qQQf�"���^ۃpC��ԧ����\_Hď���&�r�����@ \`���Ի|q�=� �����fq��H�ɇ�� &-��������o5��ɳ{N� �\[����w���D�w\*R�"����~��iS�6�9P��٩ Wu�.2g!+��0J�d�l;�Q���-6�op��a�D�|y��ʠ�y��ڬ5z PG�' e��b��|Yh���fT�ʰ�y2^n�\\.�غ٣jJ�������5���D7��w��o�l{��e(�4 �ʱ��n>�\*o~�(0�$\_��B%b��\[ 2X�Q�G�?!��,�י��u��Z7x��3S�T�9j�v�<bQvg�BQ䇯����|���^�;q���������/W��mx�J�Oc�6��t� G�Rޱ-8F>އtN�T{�V�ɒ��Y6x��;���?�&�~��ѳ4�U{�s9�z �\\���=}�+3��A����� S68a:֩���"�%���:��\_�"$�I\_M�i �f����#5��='�\*US�b�j���q�61�U�݅cUJ�SBM���MH���=n���3ұ�〖:��ͧ��=e3�u�lU���翴�.�"���N�� =S�&$ Dv�}A5�|�"�����P�Ϗ�+v�Mjdߘ����\]�����b��7}�j���� �!E3P:� \`�9�؅�f�q}����8�ȞP{}/\*�����Y�/!-�����Sl|fI���� r��{�-kf�ۑ��l�s���o닷f�J�zi�e��M�d�D���v��1�ΐY��ҕ�0jҠ������i����޼��: �R�5!5��p 4�F�a�5���UQO���l��v;&\]� ���ujZl������:hxԉ���u�����:��0\\�\*� ��n��Y�S�K����3N��f=�{�5 T�v-��Rх��T�(���"j5��iVs�C0ɞ��ݺ�����Y����B)a��%h"$w\*���,!�GȜ�Q�k9��(�1mߩ��L,Kϳ\_��-q���ON��-�H�%p��AS�%��Tr iI"���>3�֚��-��-c��Ua�<�0��6\\���̶Z@���CFk'�-=�C�W@&ⓢe��iL\\W�\]�(��o�-�'��1V����.��1����d����9b֣�ߥ �\_P�\\���#�E��� 6ׯ�@\]G(�i+ڐ%2jh��=+p�Yz�y$d 2��6k^,J�mv�jnT5ơ�fA�ǃ7��z��r,�+t�P�/�!�B%o��%�r-)��z�sO"��܎�3�^Q�����% 5�9nތ������M��&��5�P�h��6�ʺ�w�Pf�u�<�m@���������YH����O����O O�&o���\]�U��B����{F�����c����'��"����,��i�Y� ���8��p��{�3E���R�?˧ay�����B(�F��6>���a���ަ喲�Ϗz���@g��N} {'�"�O���<���腘�"Y�\_�N�8�Y9���O�\`!}k�z;���t� ��K��?͐{����J�=P�U�ګ���VG���qOdT62@T��"E�#����F��\`@9JG�����d�;���l��Qu�K�w�$�3@���ʽ7\[N0��.��g��\*7g�F}��g^�.��I�w@"����6��;1c�E�r(�/�P�+�vcA���lS�s���0���u��$f�\\��nw7�$B�gP=��} Axot�(�/ +Pƭ����= ���l-��\]�����V�����G��5��Y 4��f�\*����&?������cTO3��e/���W�) �̈́(0}$��z��'�N$�l}G���s����=!j��2�p�w���\`t�\]�cMp�"�U��1�\_��,�H��'�Z�}٤��X�}�mldD�A�D�p�������z��a��m��v�>��&DJS��1'�6\]�B��}aH�C^�T�"3�P;q��l�\]����1��& }��\`83ɈDӁ�s;��K/�m��1�=id �$�Š�mV��~IZ��9QR�L\*ْ�B��j��L�eAk?���v\_H&���Ah���><���ľ6��-nY��'< ��il�d��� Y$��ކ�F���Ft\]1Z�(}��1,������9�1�F:X�w ��c̠�h ���z�JuϽ��&�?�0�����-lcX�X�,K������au ut�rC\`��-$\_=4Ň��(�ԍ��M���3�>�}�"�~\[�����%�W�4�����P�4����N�w���������ʓ�SwBk������^��iCc^+J��&bU������@����=�7p�y�I�����m�Wѐ�=�1�\`6�s �y���Q��8�C�O�<@L46�h�V�����.�O���\]C$�0P9dƞA �NiN\*�7i�I�HR�r �eЄ��ۺeF�{���jO����LHFbջ�n�-���lv�����K8$��#ahl�h ��=���T б+�{Q$�5wU�Պ����%�,\[b\*�R��h\]x�p�p�/vN��ޭ(s��d��lj7�|\_���ע����Cn�(���T�~Rt����{x�/7�h؟�����q�qj�{� ������ C�ꘜ�"-��)��+3��5����Q���\*Q�k�"iW �6��%d�!�U�=\]�^�����u��yؤ�� � €TI\[4I��,dі�)�V�jr�w�z�K�d���� y�Y�r�:Tj���v-ڝ\*�Dh�;��\\z��6��e\[�j��T��բBr��r�z���� �ޞl�\]Ù���h�� �y�Z!\*�R�Uv��Mw3�N�hDþ�9I#7����f@?m��E�42�5�H� ����E�\_=2\\�;D�4������g�3�쳧2������u�U���bǍo��N/վUi\`�ȋ�E6�Q�z��\]�ڼz3�R�Y�,�����Ĕ�;��Q$�D��ߎ���T�,p���j�����Z� �=�ڭ�mo��Ի\_�xm�W�#���؃��o�8uy��֡4�<�|ao�z�<�VB��fX�� ;=D:b�����O���Tb�?�Ϲu8�I�T��\`�������~�~��A��R�f��D"^����+"c\_i�Ԁ�n��3�y9}�n9/��@GBb���\_6ؒ�&�z@6��d}5T�3a�'�֑��w��� � ڈ�Y��n�үdt�B����\[|}�w<�ӹT�k!���ԗ��孫k�:�����=B�N�X�=C.kuɐ��W\]\`�:IJ���>�2Vg�������8���)-�p' j�W\\w�ཻ��zUAR�ON��=����8�m�\]�~ ��G�o���)��a 1�����75C�i8�E��cא�.았D���t>Y�Fó����|S�>��e�yO ��}-��B��X�'��/X� ,�H�\*BL0����d��ؑ�H�M����BXߐ"��\]�E��AQ/\`�\]�lzr� �Z��-8w���U�\]Aj�Y�s������(�����Mp%pm-�p-{ �g�g���S| ��W;�U\]שX�!��Z�@�&B�J:8�3��Y���%5��2 �; �0:+����N�Bg����)���񊿯�,!�1�ap�ެ^��Զ���W����>�B �����F�x�"�r�:ģ ���~��>�Em�9yW�4���HVRp�u!Y�C�q�BC�U\\U��R��ERC���\[^\`�΁�jC�����\]n��RB�����Q3>"J ۑ�������q�����Rn\`̇k9�ԁ�L���&����j�\]0i�\[����z���ҵ�?�8�}8� \\W?AH��6|�����NJk)8+�-��?ԦO��Z@�M�§Ź���!���V�����\]ۃjhg�l�T�Q�£���\]�L�����p�)I�.�Lr"v�p� �i�om��4���В�7 �V\[5��Pe�k����U�������XuP�� ɞ�1tT� ����7ſ+�{kF��"Us��Y ��N��cS4n��b62T��; r�jd���T���ĩ��&� GŴN\\L���uIN�o��f���"D0"����=B��YL���R���z \`�j\\����״\[k�u����W�u}��� �L����y�)۰j�T���"v�,\*=�+�͆)��+��^q��е1�PsN%���q�ޣ��+� � ��������. ���n0׾F�{�|.��G����>1����}B��(�cN��t{�\*�:zƩ��#RA���$P���̊��D�;�e��m&@^&�s���N�Ch�8\[�p�s�)��{X�����\]��v|���oB+'�q���!,���X���e�^�a-ĦF��������숕p���A����<�V\_^�L�� ������U������v���0����:���F����rG� ��4i����S��%�{5 ���/�b�סRe�>�� ��z���B�cr�=\\؈���v#\_t\[���̞�q�~�Ax�h ��k}�9���~5��f�$L�\_.U:�\*�1S��ص 4tp�3��6�%��6���9-���kj��xpF �.Y�D���,���.� ��괺������%��C=W \]�����fcq���'�)#����8��)H:Za�B�����崽�b�W���Jw�Ӳ������\]���7�M߃|�l�h���\_^��� M�c����M�0�K��%gzb��2�ױ7��2����@-��!1>��g�5���h�%Gܐ�L����-�c�L�Xr �\[��cS%�L9j�1 '�'���U^�O��E�7�9r�����\_������rA��H�~-�N�G\]Zr�Uc@(��+��̏�� ���0R\]�2��)/��>������7Hɗ�nZ�/�tD��y���\]cN�m�X��@/�M@�Ć�v�22x�+{ ����Pg�h�ŧ�|^�Z�\]� X�o�S67�&��� ۩ �C��P-�S���)P��ad�H���Ȱ^��ʡ\]9><6^������i�g�����1���9j�S�e ��\\�6r�B|�\]mxG ��n4!\*d�k#�v����Ws��Yw �x4o����L$�{���K�gر��8X�E��j�(J��2�y��5����Z �O+$������������E��a'mN��/��Ǩ-�ؑ?���cy�g3~ɹ���lى)�o�lU{Z}��3ʠJMo��;���6}�T����ܨ�/Q��j�� 4�t�tZ�U+�&�M�)U�\`�St3��g�3�73q��E��&ɵ?M�󟵩�ާ �SߜKpC4mg-����o�@�T�V�˫r�e�1{�m�U�\`�� �=}p�#�Pv�+X�n�g;l\`~���G&�}|�^^@!�K��;0�.ۨ�fn�MQ�d� �jw�N?�� خj�����\\�vM!��Yv3��)&�����@R�~�~inx��aT�|!"Ͼ�=xl~�\_�D:c˨�D���!Y��\\ &&,�mNikϊ���!%����� ��:\_�\[����#��S溺?����������&�� �(8� ����5�C� ����U�E���n�p�� �j��D�����4S����u���L ��X���zSe����a�����S+��q:��ߤ�3x�?�i���u�j� ��=>�� WB��\[j2h��9~-�}�֤��Z%�ej�)Z��^�$�J�t"���ߐ ff�@�e��l��2Vb���^\_A�\\#wE�7G4m��Pei?M}���{��O�%�%�Q\_2t K��4�\\W���c��r~�l�A/P��m�h\*2\\Be�e����qe�)B�� L��M�M��eE��8��8za��o\*,º~ݘ �卟�n�� ?yb�<�a���ND�#��{ި�0��V2x� � ���ۚ:r������߂��p��\\6�g�h�~�6p\[����'��A����g��m6�\*��Exh�����"�|�2�����?i2����hw�yTR��"F?b\`nsid}ۗ�И��/! \`�M9Oښ�a���c��C�gl喢R'=� �xԎ��y���a��v��\]ԕ��3qm��y�F�h�a�=�\\���.��l��L�(�\]h����I�z������.Z8���Н��ͧ�ڇ��c5�z �\[����r��>d��/�. ��IcW�1N�IH\` �B��\[hc��nƯ�y��x,��������pQZ2+�+D�P��Q��:�Eİw�ǐ�����8���=%�=�K� �� ;�\]\[�ǵ0>=�w��Fأ �#�XN��…P��\\��Wo'-O��\[���za��\`��\_�m��e�kOs=�c��E������j:���A�T!�)��8 mT��կ�!v��s)��㋭���2�+��C�F�F��y�z}0���d���#\\�������\[��3��}���!��+�������±�4�?���A��+ @p ��U�\_V�\`jLz�N2�k�C�\`^x����\*�E=A ,\_h�̊99\[�d���\`v��&aب ��:��5v{ײ$Mu�t���V�WT�W�������(\_(�0�8�V\_k�b���^�aZv�N�Q�H���^����aEA�~���-���j8�4%��f?v�Pb�0�z6�&td�6�c�}�!� �J{����x���I�1^m�.�Jn��hT�@Lת�bͿ�=�o�{�43 80K�纓��ʺ�z��X�D�����Gl�7�?7Q������{�k����ڪ�}(1˾�q�M�Ƶ��>4�H�y�s��En�����i>fKf1��K�� � �Y�g �e"��  ��rR�;B~�VK֮n��@HvχI��,)��?4K��+wc�Ѯ0��,�o�V���s��O��G7���W�AO�o�U���jlyO a'�1b@8:Lh.�3\`�qp6I\_����l�&0����@��T�΢sfi�3���c��#����p7"\*S\[�Ϻ��?�a�ғ���>��N���'�E�X\]h.�ZK� \\E>?#�OZ@(�� ��� r�K7ߨ5��p.��|Krţ��i���� 8�Ls��w���X���Lw�O�8\]jޡ��˛D�ץ�� ��<$��\\弙À�q��G�N������ז嘤c�eL|��\[+�\[����ytL�0���S�� �;m�0��+�q�::��;��H��7 Pֲ@fu(��p���&l�߼�|�"T־3u9D��Qe,�B{��׈�2t�r�֝�N�j��� ���a�qzn|\_ͬ,i9�dg �1:���.��u�5s1�j)�I),�/'�NJ�,�����u'�Э��\_����TO��!-���ޔ��:��r�a /�I4}";�� xU�s\[ok���P��n�}����\`^�l�E�j��$�b\\=��G�\\�֜GI���^}72:B�NK�\*�e�ζ1�C���Vי��=������4����{���%ǜǜ���ٱ}��\*���k��Dd��=p�\`pK��$u�;8f\`�M�y�-����|Ք ��p�8��A�H��jC��Q�\_���r��V'\*�<�� �� ���s�v��!�:9� �k�����Io �\]���2��S�s}�}��\_Q�{�\`ҙ��� #�����Z�:g�He��^N�v��"���a��!C.�H���z�L�픺�2?1�Fmޞ�4���y�@��t��Y�Y�{�͐� �Y���\]j�k�O^����E\\�zn?%^ �>�\[�O�P\`��i� �8P��� ���"P�! ����+8�ѭ�l��H4qr�@�&����W��q�J�P�$ ��\]@촪����kY,Jj�q�UZ�  ��>lZ\*��} c-���uڇ1��b�|<�r|d�(A�<��A �W��� v�q.��r�1��� �zn��|�!�/�{yɋ�G�{:q� W^�jw���>Sk)�d�E� "k�\`����ɤ<9�w��G�o"��j�0�VX�Z�F�k���)�)��˶��Un���"@���ֲ���u=� �|B%���B"N'�\]���� ���5�\_"9�̌�� 8�P��S�1�:��+Q��2�9��g�@���>�~2E�/b�1@��l�C���V�$�4\*��oI7;���� tѡ+��I��ƨ���(�V℟'�q) E �3zC ����W��ӥ:.���6or���1>���M�B����Pm�4Z0{��l3i�����ɜ|^D�EO���{��7u�q����y0X�1ӯ��4�(��ʣO:�S8>ͩ�Z2�� �R �,0R"�;{�!�r�aK�\_c7}�0�UI�أ ���f�b��E��}�6��fR��k�V���ĉ|�ۃ�a���)&�E���͂��!aۥztB9�:��;��A&��X�\[Z����E.�s${��e�v#�� !� }u�é�\]P�Ӂ֍d -������?���j���ю���P,t0���t���e��I�\`u��KmK�t@�S��#�\]������/ t6��ݱկ\\��>�9����h�E��q��\_���D#�X�\]y�p��KO�:͘�D������'��P���K�=�(Ǝ f��s�!\_SA�xMB1G����G(ƀ ��� ��vOcTT¡�\`�{3�l �xmdo�\]����uhwH'��x�o2EMp�H�~.��T<)��66��Pz)�4@�a�\[+��\\|Mꏊ� o�~\\����6���K^ٔgpY�G;K<�h&���\]�R��6������+� ��˧�;P�'��x�h9n��/����$u��Ns���fad��;:��kaU4?�0�9f5\`Jk�-FY� S9� q\_߀���b�}�<�.�����ǽ�h�\`T6�t�)}P��#�(�#Js(����� �tL��ꅋ�����jWp���2V��)؏�n��f�\`�^K R# ��T�K��la���Ɋ�u�uh̖\]� ��� �'0.�o!�\*تh �k} $R���� ih�m8C\\��Pf� /(�O�֑�'\*�C\_MSE6SU��(��� �2 h��}��TR���\\aQ>ft��Y�7e�a^�gi�|n�h!d�.s5B�>����o�2 6��M�<3\_g�s&k9W�v���7��o��|$ҒM&7�/fVAiVA�� ���� ��W�k�5�i��a\\x����E?qj����ٱA.�� ��-5�F ��k�4lRj��;�$�K-҅ւ�I��pHS�u�^�� �G���^xn����=���H\\�. <��1:6x �'�\[��\`ڃ$8v)\\��E�,@����-�jU�|m�ti\]��~ͮ�'����E��H�R�b-��hu���^TL�韋7���"��+�?z�Z돞'ټ�T,�!+ #?�ͬ���S�:�n�%�&G��Ʀ;�r~���z�1���86f�����M2N��w$�}��r� ���DUнc+�|�fA�����\*����~�#��g�-�8��7(�7��$����b3D\\j�b�i����r�4լ���gR;� wR�5�s+�?����X���m3P�g�a@���O���.�U p�����P�X\*�G.j����ҽ� ?��h֤Ű,��Z�%Z̎c��b�{D��O��2h���5�À�� ����^�&ے���ݫn��θ��-k�.��IS^֏�1z2W��V�Ֆ�u�tS�(r;U2Hh\`��QHs���0s�oM��6L� f�� ƩF�7��{�\[�L$nš�����7�W��� ����w�6�p����g^��ރ���e�Sp�EX�ȹc¾v�� �+���s��rڪ� ��ǿ�ؓ���>}j�^���ke Fvw�R�O�g�E .Y�R@9$��)sb�B�~nQ3˦$���� qʩ&38k u7��6��s�Wm���o�1�7��\\�G����j��:�!^�ҡ3�z�EU܈��~�^t������g��++H /^���$�g��5�aS� ���\[�V����4}/��vst���\]&@�ʼn<�-7d�lY\`1 �"���F�a�\`%s處zgv��#��������wQ���=�A��W�ub���K��}��џ�)��V�0F����x���!0b\*i��>J�;q}��i��\`K�b�����l����p�C�󁄢P��C��\`�Uq:EK��ʁ2!���SA�N�#!-�}�{�qI�}�b�G4�ts�h��� ,��9�A�i"����A��Nv�7)w��'wjsz��,Gq/% �p�:��O��#�9)^5�p��R��U���q���'r�I?��Z �1��Z��棿n,���Z?�BZno���\`�xp�������� ^�/�60�|�z--�xi�o�G��qd�מW��s����^���X��ȩ��p�>)-�R)��(�}xU�~��A4G�LA�{�;��\_$Z�dN'cȨ��h�+��L���^�J\`�g&������Y��N��r�&S�� �j��A�G���-�9�t�ܮ��ҥD�Y���~Hmӑv�&��f�oƌ ������(���n�Z���pi��B����!\`���;�� vs|ޥ�@S�H��������l�<�1�<�5S���α�������EEhQ��mz}��������6��M.�2S9��$\[9G�W.�oϨ�\`|Ss��� ��aj�ӈN�?tMx��)��{�?}�4�Q���V,�O�6��\*OQ�V���?���TG7Iy��p�47�����Q�K���MZQn���Bbq�pap'�9����|�" >u�R�=ʐ�?���l��ZV�UZ>Vq�Į&��kr�>�'�O��?�ғ�+Om@W������vF���e�2��EW��v��>T�%P�Zc�ۊeҝk�h\_# ���5� ��j\]���E�����&�Q�f���>E��h���E��:��� �Yt�h�<���"�hV��b�$ջ$�7�u��� ��7~��Dڒ�����f�(��\_�L�-t��iɰ׭���X����Xy�Im�=v�/F�&͛}�~$?Km9~���|�L\*|À����#o�w����=���ި CoWs�,�f�5�9��;!Xt�D|.�n�i����U�r%�fe���\`a �� A��N U���+ � �(:�Ѯj)��,� �J��9�.-}��Ci�L��5K��)͖�JD�QT�5 ��tI:��\[Q�y�'i�{VР�k~�5 ;V�#�}��/��|-�|�v݇�\_ C:�\]̒܅���2���\_�J�M��Q� F�,���� �&��vA�A���Vs�s�=�G�����μ2�R�� �D����^h�\_��3��%������R ���&={�N�W���#��E�\[,%qj�Ӷ��\*�����Gi�����V��GE��8c�%���,�Z'������G)��?\[P�\\�r��>�9\\�oWf���1�'V=Bټc|�x���-jG�/� q�?���E9�;���m���D)�ݠ��Gѩƶ^ ���9��z�+\\���lj �j^���E��l\\�#��q��i ���JD� tBGy�@��r�OC�Ϗ�^u�?".��Q/L!�(�~QZnՔ�F���U\`\`~�ۗbی\* ��^'�/�:�\]#��"����K���.̲�J�(�I�+���\`\\���+�o�!w�\]��w���5'3m�����eb�d�"���Q�d�r՘�d���ڨ�+')l����"fݖ�� \`L"Z<��C���N/��eP��������12qD��f�B��K�޹�\`hl�u7���{��\\�G����o��39ˁ4Zo�w|�. ���-q�\\1����n�;Gz�,y�}͐��8.�+ '#�����b�hԃ̿\_5����x�Y,�t�����\\�/5�m�q���,K���G|�$�ZEa�̵���g����.%drZ�k�~wG��>T Ϫ�|�� � T\*����A~@#1� @u���Լ�o���T��n���I�:c�U�y��-0V���� ���8\[ˋ~��/w�rLGo��1ze:��8b �! VFĎ� �����4<"�m���Ok�L���7˟m��?�V��D�PEX%��;� ���d�Gԩn�9��0 <���Qd/H���+ �-�S���Do�i�mJo�xG���y1C&l;�(kf��n\[�6/�Է�>|n�L�\[��G֖XǠ����؛����B�x�n���?�oQ�����Q����:�/Wd�\`�E��m�pp�k:f6LN#9��je�OR���'���ar{�����o�\`����W3+�cA��d�Ȅ���T��k�����c׍ߞq�%h�/�l�^(���g� �\]����Rrê�}� 3k: >��չNh�%F�l�e�A�B8���� ��������;�wnL>E���>�J�SEp��Q{ʐ��i�=Y����:����Ǚ���ΐ7�+�MV�D���ӕĐ����P~Q�%'�tYeޢ���C�H\[?i����I�;�nM<��EĽ�0:4�N�\_�2��P�DѤ���9��Qb�����L�� 1GG�鲂�uG�Dx�<�I +<���1\[�W I�\_}�cH��BG�j-��<�tM�۲@���q�M\[B��fN㩟�p���\["o�����z�E��9�R��N������m�MtI��d ��J�sL�렦�$\*W@����M��ʱ�����3�§o��/�#��7'\*aVɰ����@Jyy�a�������BM|T�&W�a��r���i좸h�YȔ?� �%:�aP�# �\\��x��qi���-�,z\_m�GU���Q��Wb�����t�ꊣ'�a��>T=� !g����9�� �X��D$��p��+��{�4F��Lm8}�!'�\]�������i�:@�\*�ӹ��� ���L�1�Y�\\W�\`���8�"M6�J���>���x:�  �X�Ely'X���Z�专O�l�K���x��ޅ�7 \_��Ys|�W4\_��$���k�6JULj�6/ܗFI���usix6V��GgTQ�-�ݒ������AeF╶����}��\]�4c��������ep����ľ��4Ax��2�F\]�d:.�>��^a¨մ���\\G�f����ɓ�l%ד��L���"��(6�{�0m��$6��o �0�󫮮S\]���5��&�P4 D3T !>�}o)��C�|��G0�R���D\*)�=����3�N�X������h����q���Ǎ����ac:s���Y�S�Ŀh�fM�Tc��,c ��7�<��w�~�\_pI\*�%�Vg�Z3����F��i��4m@�>��?H�f(}��2Jhb/�J��e�����'.�Ŏ\*����M�E!��("X�O����< � :��������|�����|��T2��ۡ�F�QGɞ���siu���/��N�SywI�T����!��.D���d��Z�䕱�S�\]\\G!�����>~Z�:��3�EH,�2Y���F"�{Bc{(������'\[�j:'����Xз�����Pڋ�8�.�����Z 5�Ğ+����$,� ��U�kbp���I�\`U,�e�X�R���G�'���c{ �O�����c�oE�T�p�h:y�S�����SJ)ȍӔP����7�%n��\`G7ژ4f@݆�p$|�@�\\P�)rBa�� M� �G�uZ�h�i�X"�����F� {�Q���ؼ�e� ����=��l�e$ַQ�� ����t�c��n<+$��b�iD�z��V�Txw\]�"~^҃� �v7;�賆\]2b\\J�yʙ�Po(�}@8�לRr\*��/q�"�V\`\\��X�c����l���{�����^�JWR�@|��KB���pQ�;-T����4�F7��'{郱<6\]I{�2\`C�i���}��#���\[�Y��Q�q��'C�-����܈�� m�D�@l�(�L�t�e�2�L� ��L�M�cl��\\1-���T�/?5��7h��=Іp��GY�h���\_'�����\*�qL�S���(�g�Kg؈�Ї�#&g�x@-���q:|��bv�XhmVB>"Gk,�i�"\_ƒ)��b �&��4�O���8�UK����j�U���Qc��+�p�W��h��ข4Q@��,EӐ4~��,��N�� � ����+ұ��}�w}����D �5/��$z����v��>�Ab�4ٿ+���}W�5�NM\[�"��XF!���9L�k�J>�MQM��۟q<��u�v��Ba��o i�񚇲���\[™j��1,j<琪�%Q��#�.�9�.���r����\_�� ���®�I��73<���mC\`���;%7���ݠ�����z)��\\�NK�������VA��Q�zp?;n,��Q��KZu�B���o>��;��xޜ����y���?=� ��pZ�,����w^�m��j�4=��� �=��|~��8�Ur�\[���<)�OsNs�Q�b�3Q��\\�J���d����|�F����O⽋��4�\_O��e��m�+�����RPڬ'�>�U�\[q-t�"��g&�H�p|���M�"E����'�̮8� P^x��#\_L �c УؖQW ��N�� K��n(���\*�4�v�v(d�:u�)� ������z,�t�����nm��!w��Q�#Q�~�wVg�V�7��8�g^�-^r\_W���˧ݴ�O^@\`� �u�H�Ư,:�������+j)t���9Å���x���&��h��kx@a.\*��l��l�L��Ĝ��C�c1\*�jQ��T�E����;0� ��$�sW��8\-"Վh�-@pJ�� g���'I7�%4wX\]+ x�K���\`�)#���7�?վc�<����+��K:;�8�n�~o\`�|����b���NX�> }l|��z7�\*Ȃ�Kl��t�ĵG��1�˴��F�Z�1��Ч'V�ueV�x�:c!w1F�%�LeX��!����^��FF��z2�zUqr��87I��� ��sj�.Rմ�>�L�A�ᓽ�����5�ѝ�0��K��H��������+��'�كy6�^���9�UB,Q�M�\`��\_�N�� �N�ulā�o���~�X����io������� %��w���m>�k�̃�8������1;��\*�3ɂn�Y3�����\]����ǒ�4"���ܪ����� ���uNDv�b��e��aP�����yP�4��Ѝ�hUL�u�Y�qC�>\]�ǜ��ꍿ�9d���Uch���������LJ��� �u���.��z/�h��b������ ����\_�L�U�Ӑ�2?�����6.�|��V���ýq��D�e"��B,�@�%��\`�h��T��Njʼ�\]c��%�{, �����B�E?�;dr�����k����n�hM�F����\]I�G�t֌-�Ȍ�I\_1�yߖ������\`4�Ә�X��wo;�y ƪD}�|㥟���-Q�لm�����A:��F��rO!�u�8\\�Ûb���V���B>} ն 7��B-�W���\*�˴ ���P��m�e/��cq\`�y��$\[L�2�z����;9���,�~:��\]ܤ��H9C���.ج�GqpWgVG@�!�ǡv��߯q~�x�2n��y ��?c3(0���N��;���@���V M��>O�ҟ���!����F#�>���>�w/s��z @���?;��g…��=��I�1��^@�3�nP�DzB�S?p7N��\]����W�o���C�� ����k�O�%�&hp\]#9��' ��ߖw���?����'�RL�(3;O���.c�a{�k���9�D�nN�۟Y�~h���ɿ�V�B'���1EF��+W�g�h-~� p1Y�y?��Y5�#�S��,~ퟨ�c�2��HW{���e2'/�5b=�9����#4�B����&���ct#�O�J>���B �f�ʒf�0G�L4xV������"�!�2������F �9X�P%{��\\Z�7����q���1o>��\[�ԙ����8�\`�O�I,�����N�6���$<�mZ�����' ��^��GW�m8�Dq��!�0C���\_�bq4�KG��; ���\[){C1>�T�\]�a<+��/�-%�(��ْ^߇\*�”)0;Vߣt# �,8�v��H��)����ǯp�ݣ.�<}� �&��� ���\[CQ����V�$���� bg3����f�0:��Y����\[j9H����B��5u�t�ޟy2��w�qTi�x���dk}KѼ�S�)�����7��s04m\_�f4�L�����2L���h(��X���/d�{#�֥N���U!�9�1��g��N\]��\\m�N���d�l N��<����.T����ķij�\*Wڔ���?�G�C�0�j�S���?N����|�h�����.^"0��g�U��a��rU24=Ki�i���P�<���R�%����4ҧ��Z���C�f A����ޕ�^�i�����W{Ov�d��1�#I�y��S�IP�Y}��)�0����vY�S-7�W�y�G��-�4��T�}�GE�0���+)�kY��x�X�(�m�x�bu�U���� ��)/�nJ�Gd�~Q��{� )��S��ۙ���\_�lZ��\`F�l�oB�+��9�m�ʻ���@{' ��w�%�N��.ilG�q�@��0������UpBR�A8�a������x0�P�CO�PV�ޣ�Iw���A���?�W�r!��쬟�AJ�\_���G|� �M�z%�D� ���D��+ï"���^�x�^P+3�ĩ\]�1�wK��T}�3��8i"�� \[G,��\\S{�p����Ax�D��!s�V�j!f"�F����J��\`�w�}��Í\]0#9�ˌ>�k��b\[����r�$!�fa��YO�9$rx����j'�v8��S (�k�M����CS I�@qҗ�W �e�a���T28�}g�cJ?��o�i�֐���0ՓT O��&��.{���\*�$�ψk2�-�B��4�$mg���H�8�'��A��rXN&~�<�?a\\j\_n��0��M"�|(�\]/��w���ʆ�~�-C� |� ���� ��r�Ѥ��MR�h������\\� �B�e�� &miM�z�vܫ��G��<�c-<^=�t ���Q��F ڿ{�����T� �xr�F��I�ٻ�D���\_S� =���!�,ߧ� �L�-���k�De�$��a�ϸ�s��%��!R���$�Q�0!7P,�������ͤ��L<�. �y8�Z�� ���\[я�Z-Rt\*ǎ��\[<�C ��к���֬�mJ� 39��^)Ƙ�y"U���(����Ң��\]���-�MՓ�5�GD�D|�K8��� {�R����6��%����y��D�E�R��B�/��x\\?��$hq\*��VB0�r��Z�Q �u�ˆ����a9��$�\[����\[�i�E���$��1Ik�� �Ȫ���Wz�Jw�jd� �.�3�PŖ�Ͱ�eL CM�Ɠ�J�J�s��5�d}��\]���.s�����+��G�|�=��H��P�$��qb��j�BΙ�~�v��i�J�m����.$M'!L��u�Jщ�����?^R2��\[��}��5~Ʋ��\\w�������f��@\_����yP�p(0(�c��-��V���� $̬j��X�����~���e6�=EK�y�I ۙ6բ���-7�H�l20�V�@Хj�ޞ��� �.��F�4+8���:��O?�9�,f����� ��n9��G��5-�\\p�%�BX�n��#H����’@�y��.�L8���L⯠�Ʊ�t�u-�2��\_�@ c'P���j�v�\]"t{M���n!�\[6/�a��Ad8 �WB�Q8O��g�T��=3� O�0D�Ӹ����+�=êB�S ��7���i�w�f�,���V,�88�<��f����\*3�Z��IS8�b\`MzhZ����FfO r&"U>�ۦ\[��rjh�VJ��&��������d�'B� �^���;�o�{�Ĉ��q���d(���&���/Q��(�X�� �%ihY��O��Y�\]�.����E>�qi-�Eq�\*jA�);���\*\[��H�������R;�uF��\*H ��;µ�E�;/�xɫUqd\[t%20�튽�;�M�;\]:�\[���㠚��6�gDa-�7}��~3)� �ēT��u�A��ٕ5\[���G��r��L���O�!�(I Բ�����!魃0�8�i��  �s�ǎd�A�%+�}\[��g��.F��2�X������s �=PTw��M~�5�:�i5N�Y�Ip��\*�N n�L�u�Z� �1�pd��ZqRTQ''\*�,�'���c B���\\���7��O��Y@���>%�-:�ml\_��w�����hS�<�;��8t\[���-�S�ph�Ħ�>�m�%{%>Hl�&3�����9��-j^¥l��t'����h��,&v\`��1%���I@���o9���C�(�tY���Ԅ���0�uW���Y�G�h���D6��L@��q\_$��� �!߸�����4���ՐS��%(���{�K�Wr��F\*(�!�7&���"N6�nL��p͞x���Wz���2Z��h|PZW���w9~� ?��z)�� �� �?Z A�|����~���\[)X�y�̽�#�a��� ��6���ay�㎂�l��G�\`�5%ǷL�M!���pZe\_\]d$3S������L�B���B����8��>>5v�95�8�,3C� %2"\]R����=24I��s�OE�#�}�l����C{A �sX��f3yt���/AX���\_p�q���1VJFM���Ma�I��W���>X�\[�}d���~KNw�P��N �r��?�ELX|�T�\_�3�"X5� �< f��ሻ�:�m�I>���ٝ�5겳.��Ih��z6�ZbĶ�ĝ�v;|��4}z�B�/\[a��I�J���p��b�L���q4�"FJ�~;�V��2E\]������q���Z��H���dF7B+N���o�l+}�'� a6l����h��-�M�l���4���B��������/3l��R���m�y��K�u���p�KB�S��� ��^�U�\].�\*?�o�v ʃ��s�쨓^G� ��x\\�l� �) �ȴg�U�8Jp�V�Qt~�J<�V$���ߏ"�2��r�����&���҉��B���\_���p��-}�������{�:|B� �� K\_b��m���\_U壁&���<�C� �غ#�\\��(?�h��H��t��\*6i��#{\`Xo�%ֽ\[�ͤ���L�\_��k�G5e��0^ldb�Wdbp�l��x���5�W���M�#���P�/�\]ZxZ��Gs2� ��R���nT\_����\*3%҆��}��U S#��\]�\\���2�z�23?�JI\[�z2��~' ��mN2�Z�'a c)i��V�|6<��+DmS�����NJ�6~�VT�n#�������\[�5��-�y� u�މC�hycK���bB=Z������4כ�%�y�\]��� �w�4��(���m��a�ĽK�C5�0ק3 �lR2U=|��?�ә�5��x�e�~+E m����Y.�"�0�{(��^8�Č��-A-�pnUc��QO�� �ei��E%��'+A�@)6�Z�N{���A���\[ˌM$4��oIՊd�/�A�:s\`X��s�a�Ԛ��\_Mҏ1�����c�E��~�o �ɝ�n�J\*E�6���I8�-��G���� .KD���\`�Y��}�n�֒��j�������S�� 4��q(�=ͅ�ˡ\] v��P�v�Q.����ǻ���e�K\\7C�\[��lv��9�g���X�C0tM ���w��g� �c(dY�c�f���ww�Sf \*ٗ�y������ \[b��qq 0���t����\\���uP J���k ����aq�ji�ά#9L��Uy\[l\[V���m�����^I)=w��D��b �>�՟w�- ��9�-����\\�� UN��X�{���G���5�M�F�̫�V��EHΡh,��H�\`�i�7 ����RE�Q���%Э�2;���\\��.�U�Kpqb��Й��$Tv�|&Io�ȯW����ܱ��q�I�z+v\_n��q��Dh(\]�m���.�h�o��˦� q R�?�� >�c �Í�N\[��?�ԯd>R!}\[������<���� \[��/um��\[N")�B��+��DO�L���h���C������g�����({�=��a�;��\_�b\`����\`b;�$��� �o#� ���I\[\[�B|�t#h7�(~��!��M(AC�6�&� ������e�X�\]H�@+�u�̅�Au#\\�P�iq�I�'n��/����s���nUƶ�k��8w/ F��OL�x� CA}�SR��F� 2F�nIN��R\_ϩ�7. v����ɮ�+�1Jh�d��VP-�x-�1k��s�;Xّ�z���b\`�������{�a5��?�14�=� j��c�\`5���Ń,{�4 B�(r���iu��\_����d�(;2��d�P&P���f��7֋��,�J�������{+S~B\_&T�%�<|�� g��:U�>W�����p�����?�����ؘ���'S��{�9\[��������}v�d|S�-D#��������E ��1��>w���.�Y��ł0Cd�z�dW;Ԅ&�Elj)2�����᠜��΁����C�ѸbS���ӉY΅?5�<���&�j���+��zc8Ji�G���QJ�P��u���O9b�je;O��h�M�S��=�OC��L+|O����/ע�"��At�m��I6 \*w���:��oMX:�a�z�7J(��\_�q���\[�&�"�X����e ��M�y��)I M�&+cՀ�!��}� ����J��V�y�UoڹD6 �V�� � �Rq��V�3 �� ��J��"&s1���3�;l.i����g�(� zJ\`��Y~����{�y�\\����#��o����E�Yt���jw�B�9xޖP�S: ę���)"�D\_x �Q�\[�}r�x��˻m��\\�aI�)�(XGi@V05�wU0��H������\_ m����¾-�B8n�Д ��6�\[��nH�s�����H�A�@<���\];#�}\*���'L�$U��gԽq:䂾xϓB��\_�UD���)n��^�a�̉v2�@�5Vj9�j0{�;��H�W�w��Gl�E�S�k��I����N ����2�)�RVStAA�����B��zѱ�݈S������t�y�~y�hLƟ��.���˞8Sθ9�hz�o����rߪ�� |;��S��i�g� ��ra��w���v��#�+�fb����\*��dN�1k�5L3�� ��,���� �B!��sn\_��$aND �\_R")�&8��̲���E ������t�(��M��w3M��H�!�F5\\6e�a�0c���\[6� � �aW�O��Ht~ P鍩�P��ә���p��"b� �v���\]zHx �ėےd�ư���}�H�g��钾�Db����C�^sC�h���nhՂ ��f��x��y(�8�%K\[��Sm ��@�S�nXB�-rI�q�G�xu�k�zJ��\[��<�wM\]���C�'J����<������+�Q�ތb6Vg�A��0G���i\_� &lҤ�aQ��BS\\��W�yMܼ��0ƫ �5�-���+�T=O&H�9���q������n���G2��{ZPd�0ݔ����T�J���i>��7� � ϥDRA�9��p���l��\_\\R�c"֏Nd�\`�����(>��A�n���q~ݠzR�2�^�8�ʋf�Yk�{�vx\\8Ϣ�f����� %?'�(2��p� ��3��֭'\]k�-����#2S�g<��s���������k���Sn �'�cHl+�V����A��Xn+�GV�����'f��-/�����/�� и���ߋW�Dq���O�ރB�z.s��;:ţT� S֙^ j�+q�@�~�mW\`>s���O�a��{� �6\_�PU9X0KZ�����z$�ml.%�ԕ�6��^U���� F�i��x���c���>+,ϟ�:�@�?�O9�� ���l��P#�%�f2��d���\`i�ۧ��%�y.\_����T�/v�P�1��cNmw�v��\]r����Ұ��e �O� 甩o���� ���2V�ѻ�9q~�\\�\_�!����Ϫ�Xm\\m5I��ڂE3�P� "����\[\[\\C��R㬫M��!V\\aZ��Ӑ��q�Ș����m��\]H�"/�8A ؐ s�i,�?8}�m��\`�̋��\_�����!�>\]+���pq�<�,��0� \_}>?� ��UŨ7����"�K��X \_����=fG��Ri �P��B��p�o��>�ޭ�ux��}2�����^�t"n6���/B�H����/z EȌ�#�?p�0�&e��<���(ע�L�6g��|D��x\_e�L����T��q8�|?�ΤH����p�7 ѐ�\\u �����umYޞ�}ҙ�I���HQ8&�Cy�rA0}�\*�or�e��v�'��>&�{���e��+��q�FN�ڙ\*N^� F��It�k��YM� � ��\`��ݎħP�)�H�lDf^��kb����T�’�D��}f��tb�lv��B:+��S��kW������F�@8�t}��.�eM �)��l }���>����o&���m��V�'��l�CQg+g ���,y���F��i��Ӊ����V�J�g���N\]�w�����IpD!�m����Bi�� glY�PK�\_g\]I�NK�54��B������82���c������7�32�>զ;�!�Vٲ��O�����lh$��\_�o�f�9V���uD�\_�7�rDrzS���@���֢��Z�á��Xr�@2�K��b>Km϶Y�?8�\_� ��ng��V�V�����!ɑ�o�-��y6.M�'�#uVz" \_�\\\\�ZKU7��,Y\[��JN����@�y��j.�0ݟ�Q�l�H�ig����CRN�P2\]��rea,Z�4�) ���C���Y�����~랃t��\[\]\_vgC����2(O����6�9$�l���=R�g��%�����H'�jrF)w:CMB����\*=5,��v����H G��R�)e�\*2�h\]�\_��Grt���6���xF8&b|t�ɋK��S���6� IKc��@ܮ�l�ѕ���Π+���g|"nb�9m���������%����R��}�l;�X�ήSy�F;��ʠbU����޻|��\_�빒^�nf ^� �j�"�^� o���/�~�c��m\`�?̵�ض��軐+�F�1ޱ̎���H�0\`-r|��Zj �3�3�kb\`�?�7^�c�J^�L:~镦�����I��yb��!Z�"���ebZ�Jxwk�H\[�s0�8�fK��}#��2�ܿ�j��M�� �u�Y����&�~��Ҍy\`ف���b'w�Q��;͵��6��c��uH� 7��H�K(����}�� �����{����ٓh��.jU#�N �,,�� $�\`��\\b�v~6,U8�����\\w�G���n����ܭf�Mt��#��\\y�!�Z#qA������i�3Gk p�;�0�y& T�Z����2�\*��txo��t:����Ԭ�$����\]���LH~�r�~�չD�����V���ϵ���L}��k>��r'��s���f���&��r��^��|\*Ő��z�9ؘf�B�����%}|Ͷ����L�<��G��\_�ON�I�x\_�c�\[��1<����,ˠض�ˌ����Vh�p�a�!ǸiDH���U+|�R�ˌԼ0dX����yx𿢧nJ�!?}!Vo.�����?O4��:h/y'� 1P�^�\*���)5p 3�#��ëO��"{^�\_�t�Ivl��c@2#��=�h�R�f��l��ǂnIЉb?7�<�����/�������!���)܊����E�\\Ѹ����cs ����w�?��)�h��K�M���};�QE����f����Qè2C���e��N��LZ�fUt���(Dd��$�Ky\*��u��R�z�$�j��@@�'c4\`�����\`e���˅�7��#�'Ŝ!s��"���#h9Ri�����a�Y9��K�(e�Q#�4:��O��C} B؟��kt"�G�\[E1��������G��5�ٯxB\`��V#���\]� �Ʒ=���JK\*)99&����gJ���(5�c�q�35=/�G� G#j�������l���y\\V��l�'N���I����C>\],f|P�n�j�O�\]�y'�� �e/��4&8 ��IB=�)�|2���Pk��f��/@<�?\_i\_~U�W�^�d�gK��k�\* O21� �$qH�8��\]��\]H���81����nT%|��r/�}AR\]&�KY���i{ ����Ъ�/�c\\�̓�P�z|"�}��=���r�U��9 ?A�H\[^���}�.������Uz%!9��\]��@p+���DG�#Bʮ����݁���J-k��K���%E>�m����ٓq4C,�dz\`D^R�\`y �rN���WG�y��\`\[Nvˀ�l��Vrc����ѱ& ��k�L�����\*)��d6����9�4��m�a�q~w�F��nՂAӠKi� ��~��~��\[��M\[x�kn�;��!���1� cO��CT����r��I)�)��oL�p찮�S$�P:�\_�C^���ϫJVs���$���6\`\[�\_�w����R�$��U{��oY�Y����#�uHp@3且����$��wa �FNBm2K��i�,"�0Q� ߉Uu�=<�e����������E}�/dr����c���s3����"�~�zRM��:M�\[c�^A���CFx��I+B��#fj���hr��x�e�0\*�Ή����m\[��v��oW��Z��Om�Oq�������Pf��>4.�e���X�,� l�4�b�{#"ELr��S��׾ ^��i�+ܺ^O�&�m���\\y��Ǽ��\[+�f��|ee��;\\T��bpáOmb�I~��B�b��A\*�������?L���="�SPs8k ,�r������ ����a��q��NX�\*=��K�����^���� ��T��8Z�wLm�������q�7��Hq� U��\]�8�•�:ɍ��+�QM{�kY�i���3�� �a�GI� ,{&z��\*�e�:�p�h�F�KֈN� �+t��I���AY:\[�I�+W є���P�\]o��=�6^D��� Z ��lw\*�-V�>7>a� gc�n�m�\]4K���ےAw����?��X��f��}=��(� �w\`20��>��7��y�1���\]�\*����VH � �dŰ�Ϝ+�����L���!��CP9�>�\]�"����t���.R&Kp��׳>h䲼Ę��9>��ZF�����38h��C�$&\_n��z�F�m�5���ɚ��7�2���@r�B��&�ns$(�k�\]V} n����g�P�0�p�H�UK�����h�� #���G�ē$Gx���A�¹��H�A��v����N��/����DC���w �O������\\��\_�gRˋ�S@���'�� �����p��2Ԁ�<\`d@�bu�p� 4wE���>�\_��p��7\*Ã6W�\\��o�u ���)\]�4�k�(� G! ��s-��9����B�e�"w� tХ� R����x�HYf�E%Yk��|t��+��3�Ԟd�Sf����At�sU(��mј�jnQ�!X�m鸨��Y����i��� {˴�sz�mѲ\[B�x��tsv��=��� 2��!���ҳ�xi��p\_q�� VX76h4��c��!-k����p ��\`��\`sM�u/���kLUaq�L�N���s\*%���� ��V��b��y�~|��ɔ$\\N���'P:�@��/�� es��Ǘ���� +�Q�9I�RM�1�Gp�j���bq0˲<6uV�\]���&�%��ݼ��!z��\_}K/���^�TYh���Aܡyb��KJ�\\|�b�\]D���%���@�D\*�u��s��u���u ;|Ip���xvx�ئ�����FV��\`�hΆ6TcN~@�r\`�o���X�ʸi{��Upc+���(�F�e''KS�5�~�7buS� %:A+v��B�a<퍂�I�c/(B�+-���^�W�xI#�N�+�Mbf��\]�P�\_!?���8<���!7�"@\\A� H\`m��B\*�p����0\_���qW��{�L�'�k�r��x8���;���Pe��a2,��P�6��ꀩܹ��8��+Hp�#낥�DM��O-���{畼l����<^f�t!�u.�ۉ��Za��i=�d\`�-��� �����dD3,�|P\`������'^B�$���t�29�A&�3�����r0�o?U0�BDwX��?F��J�v�)��QΝ�胥B/c ab�?si\]?�Q\\��O�LXi���"<�w��&J�8I��IE�x�����T�.-��;KI\[g<�8��N(�nHԑ� :0-�L����C�;6�(1U�GI�u�,��'�V��rbx�\[�\]�G����= �ƶu�A�\\���wV�5��\`,��F���I�?�2)�筨< %.�\`�lj��\*\[N7w �8� ��Wz��\_��Ń\*�x|��4�0��m=Ҝx@�@JV���A �鿦 �6����I ��s��B#�/�΢�l������ޜ����<.d����aa>o���j��Z�0S(2�O���ZGe4\_y7�c��������{����z�C��%!jՂ�\[����r� 8�v����l���w��K�\`#z���Y�~�l�^�p��5�\_@"F5{�&9t�,�҈\*$�t�EF�-Tǔ�;ɻ'�\[��A��~'��R��L�Iǭ\[{��p�pUOQ���6FI"~!:���ӽ,�Y��}/ No�,��!��0�j�9߀�\]����C����"�k鵍:�\]�s@l鄩���@��E ��~�롤������ɺ�T}�c#v2,&#�K�e�.⾫�o�,b��2�JE�� ���^(Z�Ig����h��\_0��\]@C���֚ws�O��ΐ0 ��!vR��v6T�����q���4�5��P�/D9΀ݻ�j�6�s�N�����7=H�a�M\[Z�i� τV��G: i��� ���NZ��zН���I\*�\*�„Ơ��o�d�=H����7�qQ�s�� ;��|� Gع�!�h�Xl�a�u��δ�͝x��$��f\*'�5 Ģ�P�n t�ϱ���&�z��PF���뾍\\Y\]( ���Kt���Z��c�D�^�wwl2�gW�l�����l����W��V-�,%H�r�������^g�y�u��r�=ֺ�pHiH�!ļ�Q��! ���Y���#t���\]UMP�sE/8��3\_�M@�V�7r|��FQ�T\`B\\#��3�����oFf2p���Mk�����}��$��8�z��8�w�; �� Z+��,=|�\\-Ώ�Uy�� f�Uqm�j�Aa�j\[��9ɉgR�y�h��x�t�D\\;\*�2�=; ��Ƚg�� pp��x�}7�%�b����x����/+L x�H���Ү\[��L#\`�\]��{��� ��h� �NR-�y8rd'd��}8�h��|�nl����o��?$\[��!(�~�ҀCu\` $O�!t��k�1r��!�d�IJ�� �(��o��"ڻ�v�V������E��.󛴶�% c+��-�f�q�JM/���j����A��O�� |���.���H3ob����bDQ\`�'g?\*�4.2A�z�1�\])�L�gA�wh{V��R;�3���y�~�,JAqn\_����P��?�\_7�$��;DH��I�\\�ͦa&�P�M:ٕY'��A�\]o�p��j䶵z�fq\[���t����CI\]�ޕ'����8&�\*M!�X<��Rd�F-�,\\�\`,>�m���SgS�,t;f\_䨉�\`���vM�13۩�$��\`�u��eg��C/�u���9ú��ս <�����&��8\]��������i��%�v�%\]�����U 8�ۙ���&5�3���{:Xr��:dκ�P\_%��E%���p��W3� F^5M��Į\]�H.ģ����ݍ�ya2 \]9�e��kg������n�R%��;���%�J.Y\* ��� 7g�1�H��c@��>H�>岽�;��,9P���6Jc�\_���Q-�� U�٩e�6��H ���k���H��\[�l��V�b��w&&�R�9�C��rT�}�v\\��^����|5l�Km�%# ����g�y�Ҳ�|��un\] �e ���K6���4��;g972n��|@� �|>�|F��B�AJʤ�'"�s3�J�� \`/Il|�� og�io8�� s�ϊa#��澝#��@�i���h��z �����~(��(������L�쵵?�8W���2�����\`���'1L^i�|��������"�S�gl� Hg���n�E4�p#78�O� SO v32"��(0�����,r?$(����ì����|N�r��%��n'�i^�H-��d�Nـ��I���R���)w��ŏZ\`5�>&eC}�p���|��Ks�\*�u�jP6��-߹ Y����)��@�4��&�t��oA���0=Z�p��+�o����x�R: ���H�Y\_n�\`�: �е�>���72�Ij�כ�UڌvY7�h��&�z 2�\]�62�nN�2ui���ü#dԉsgUi�=��\_F ���y���zբOՐ�B\[\`q����9&�\](��nhx ’i��bG� �o�Qa� ;������.��/�F���@��;���5z.�Q�b�#T�=�G'g� \\Io���u~�P����tǤňjFSVL;'F��<4ҝ<�8~���m�d82����L��$Ba��1 ?�\_�5��!\]��:��c�)��R��EhXL���Q��N�z=�0VbX�q2‡�h�t0\[̝��a� b3�/Bz����ٮ�x\`��.饂ր� �&L\`�\_D�:}���o'�.�д�ݭq6����4�a�u\];��,�=��e�ݲ� �L�����0�8�V��\]��xYms� ׺�����g54��� �XH��n�j)U�8E���O��Ezsu\`�=��&y���zZ�&d������F2\_0�Cv���8%/��D��=c󻍎��r����3S�m�����.�=W:S� E��\[�!ɋ��ju��D�h�c�L� �=��%ȎV�y@�j�9a�wvgG�K����Z+�H��Az$+C\]��� v<��&հ������tw'5���m� RuI9 �^�SJ���w�Dc���������(C��ڲ�چ���۟�����$b酀�ǻ�\\�Z�o����X�\`��uu|� �\[�M��c�@�6�I�s�Yr�������"I������h^Na������L��9�Ӂ�h�v5�OMlVP�w�u;��҅e�{�U�l��3�׉1�Nc#���I5���:.���6C9>��q �(�\\\[���a �dz�3��k7�%I�t��/pE��݋�F���m �\*�ў�r�n�M���ʰB�>�v/X���~� ǻܺ�Vl�Lz��H���p;"�K�q"��t�s=��\[ WV�u�R��C�z�E�E6�+8��9�?MF���Î�%}n�}�˚@�R�@X��:,}��i�&ڣ�Y=V�uZ~���7�#�9���\_�p������'Ԏ3�����;&3{ƘX���&�\[�� �8:���\_��c����!X��\_��񓣓x��g���R����u|�TO���ʂ�Z�,�\\E�?x�>evv$��c,�0������� ��f�~d?�=~|��5�,aKm&���\[d{�"\[�tm\*�r�s<���/�w���oZ�I:>&ݥ������Τqb�:�A��a��yhM�x��@����Q4/���gÕ��f�"@/������ &���pt���;J�'���S����,Y��U�)�y?�~lʍ�Y��T����v��,��5���\[��R>I�x&��n��;��k��RR�~j%�׹��� ���5�ݺ�Y6Í�����Eq�F�u<{'��%�-�Ȧ�������WӴQ�LjjFg��z�\]����FN�,�=\[0vi��%��ұw���� ��,�灚�GJX'�қ���E�M��~��ah�$/$���� \`�O���ϻR?MK�܂�x�8}�e���y��D�WZ��7� �#�a �i{W�?�1GK��׾�u�&�ķ�#�1�@ ��\*"����Wn@L�7����Vl���\*�M(E^q���B��1{��~H��=��Z%R�,��7ٗ�J��&���\]�{���l d��q�?��e��!����Jh�XB< Dꨐ��Vf�M��,g(�7�\[����G����̧^��?�M/"Q��6����\[j��.�H����)�K�Fk��}$�ʠ�� ,ҫ�&~\_����^�K2ӓ��yY���⠻� H��\_��c�$@��c�JVk�캐(&Rp�DFm"Y���� $�N�maYO�m(�-�(�e��\]Ѹ�.���a�\`M��ԟ�c�sm�hvo �/�S{��P�Q�Ee3O\*&� J��lU(4��+�M���N�Y�6P�J�H�c���8=�\[�m����B���c8>�ok��h)xl�tn}� �{d w4yh��P�3k��3�v,y����!;�4u�z!�I�� l���Sb+ͥ|-��,��� �"T�e��\_\]�&�t�=�p�W$�\\:�"�:��Þ�� ����&�s�-S�~�O��2-����B��^T�q��4��-E�dF%��ޝp��k�n�,{$P�\]��}�j;�?"�U"^�A��2D�t\],��^rۀ��)۽�-�$��R�W��U��Q�B�a�j^�8���|\*o6��L�^���b����Ք<�^卡����GTpJ��H�{P��k�@�.�"� � q��� aj��$K�+�j �7\`�¿��i��.��t16�7u%%��r I���?��������z�\`q�m�E-�v����t��������c��\_�Np��qp���9�߇��h6�IM"�\`��ьR���%��}�;S��F\`>�.�.�4 �/sv H�ڠ�I(�ė=-(!�Wkr�/T}����w�Uu�Ĭb7�(����s��|�\_���ֈ��M?׺)���!�ؿ����U�o+ �V���V5�Tyj\*^���D<1\`������%�� ��{US�j�O�� ĩM{��v���η�\[���&���������ހ�1���vcߥ�>B��2��M��?mK���C�cBn�q#������>�W�p��� z��8�3 C�Ȋ��C�\\�t�֔��u\_�b��(=��b ���$9v�\*����'�����LK/:�G-Y)�t�劻���\[M��5���IK\*Ru��N��� T<n;��� r�ݍ�!���ȭ�M;\[f��E�����7}��v����e �M����Â�@�B��ʕ3����{�H�Ct�ѝߏA��Z��u��\]4��L��� ��\]zZy�^���@u���ꖾ4�W�N���J=~���m \*�%����} F��?�J+\_+K\`��=!�E�v�vo( ~�=w�x�):sT4!P�&N�Ẑn" )�zY��Q\]�a�h��ìU��@��������l�v��O�i�U�����a5�� ���Q�zA�x��+&�S\]?/AVL����Y�.�%� @n\[=�H��$����$$��������ԶPFc�b#�ʽʈ�#��u�;��Ԟ��6xNr�'�@4�Q����P�$Ɉ�~Y �\\�>�?�'\`�J믗�����-\*؏���nG�f��v�l�� �8m��K>wQi�;K7G���e�I V53'm�0��R�k�G,��8��n'���D���,�VѨE5?�bJm�\`� w�V� ���Y�� �T���2�.lvݨ�ZҮ8�e� f��bL�E��1$2�ܺ��ág�.�i8�P62a�KI\*j��i!��o���梻�Ƙ����t�I�UV���E�n)�}�t�2u��p��:캍��?���g��(b9(�d�m/��ޡ����\_�Sȴ�P�� ��W�ʘ{���Tķ�;i�$�˂�$z��(.���Jl郴\*4������JBT{�6�p̕"\_7}w���1�O�-)�mW�kၹ����$Ҍ��QQ-����-����p�4�^�v�U�@�.�M��՘K�Y��\_�n�Pu��c�d���m:W��ut�a2X 3�c���,�O9o2�z�{�m��.���IE����� 6\]?��N�hi�짶�u=� � =o���g;H��<8���G@�����s������"w�&|k�3�j�\]\_� �pN,O�Z��XcBJ�a�.�4 �/緻V�D��zN���D�!����p��2@��^��H��=�ĎG/0�F �SF����ʾ�ee#������7�4�9\*aR@ߤ��\[�\`��IȢ�=���6�D�\]�#��e�%:����0k�;h�Xu�����@u����Z����wn��҃V����gM��%��N�M�^u�I\]�,:�3�U^cA%�m�k&�xڋ�������^�Q��� 53�2}��Ƈ�Cqٶ\_����� rL�ro=0/�a2���\_8�\]�UP=^oZt��C\`�ړZ-�cT��q��I�$J�L������N/W�T�zW�NT�� ^8��O��rx\\���"X���V��^��L���\_�7�T�%�r��z��d�z���,�Q��!�8>��q+w\*M�o��C�G�4b���������P�ю� �� X��? ��\[�o �z�j^0�,a�1��~}�ZF>�)�ƿj��ϡ�� �4 z ���m8p -��\[hqԓJG��^��r�6v�D�/��L�Hӊ��6����$��A �7s8E;|ҭp�E<\\P ҟ�����B;3LtqY�@�#������sX6�$��9��1Mx�/�m�~�q�y5�$���;f��6�������a �-�e�l��I%'��UۥTGn)朒�J?���~����K��蓦X)��=�?�/������Z�P�x��w� J�I1A�S>D�B��f�s�����\`���-�gm��V(r��8;�>}g���&y����H\*�з��������/�j� v\`+\[5L�g>�0�YgӸ�VE? ����j٘m� ���sߡT���y0�V�~�~v�m��̰Oe6T�Z��a�x�rŵ�⇿ۛ�q��=��js(�����Ie�1(��>ΐ��5>��N�� ��o��˓2� �I�Xn�7ۼ�S;�vp��7t����%6cq+��\\� ޴��\`��G�5��\`.� {\[����J�F;�KP){\\TEOO�k���gi�&��S'�e!�}���@E���?��j��F,̦ƢI�����ji�3���v��� $�/�����o�n��W�n�:��P� ����N:a�W�����C�P��j����T���4�����6W ���\`G�@я��$ۺ���v��g�l�+�'�\\�ǦgI?�iՖG�z‰�M��̯?;�v��\\~\_J�����g'V�D!u� #Thb�S"W�f h���PQn�(z;f"�}H���dh�7���8&�I���z�m�����n �H��VT���pO�� (��i��.N���\`kɮ�%�C%���\`/.�Ũ��Q��y.��B�%d0^r�����;����jj���q��O������Uv�T���.=\_g�?��E�i�q'�e�+J���L$�D����vp������yGB��0yFr �L�������U��{��x�����(l��yt��Y;>L,B���{\`���X���gu�W��ʗ �J>޶��%�n}�J�N�J�<����M\*x!�\*mF�ޔ^bխO����\]�9���\[t.�'C�j�tq1�MG �O�N��U��g@@���tk9��m(���\_� %78A�i�v�s��/�����hю�׽a�����կ�!��Đ7t0���ǒ\[c a��d�����<��6�I�:@tC�΍pDƶ���(����\*�{�%q���q�ʠ�ͯ@��qP�=7�J�B�9����&���P?�|��̔1�%��H��ٲ�����ibo��"����F�x��r�|��H��X�5���?��-�,^k��f=����������?ˌXwǧ�Q\]��#�ҋ\]SOc�,�=��-�:.�R�ؘ\*� 2$ )�}ň\[�W��Y�uڴE$\]�O\*���X��\]�V=�l�1-�����e(J˙P�Z8����󳻛.\* ��5fe�G f�/i��8\_;x�\`5��7�H�w�e|哵\\��Z\]\]>��Gp���c� 9�%B˵��붙�5�5> �G�m����N&�"4\[թ<ߍh@�i�º�����k�c�=�J��޺���要}�aIm@�U� |l�@�n�UI�\["�$���1}��$t�. �z��\*�$,�a�⸵H�q�X��9�v�ґ=�BaZP�\[?��c���.����Bs��eO{�\\(~B���0o�"�b��\\��c�,l��?�K>7��URS��pr��\_@���z⒴ ���I�ُ�ތ�ԟ���;{� ��w���u,ŧ^�үP�c���I�� A-TQ���Y�\_��gkh � ֙K�o���C��r���ddב���O��ʹ���F�%t��@���\]��^ .� ﯍���K�h�+O���z�aD P��v��t���n�۶�,F��%�H\`�y���eO#b����8.VT,b���O�Q��G��-��@��F��emg6\[����h'҇A����9JL�„�ܸ��G��W���ݱⲣ~4,xt�qͺ|l����P�>蘟����t���l���f��S����m�df�){:(X=x� �|�cUz$�gLU��B�E4o����K?0������Tx�1��"�8'��1Z�7L���$E�h���9��%2#'�C�V�n���D�+�IbqJ�6��Y�J��7�W���?��b�y=�㚝� �8�d���+��6��@Jg�@R�T��\]\*B��G�N\\��|���8��{��'UyZι "�$��f=��b��\];G@?�t�$^�~�Qj ���69i��Go�\_h� ��/��J�½�x�Ik�䗹��Os5�,�L����<�����xЉ�jޚ��iN6uIQ2�:K ��j�=^�  ��Ғ.�O����y�f,C��3����� �%���A�W�c��ϙD�@s�i�2��>D��Ab3\]��F����"�Ќ�?0\*��}4V'T��T=������N�����-����;�P���F���&o�?� �C/�x�tf����ci,j{u��X�e�MU��-��-�B�w?ft���#��gK���w�tjm���PI�ᰳ�G�\*��3�JeR�!�4H��U���Tj����P�5P�6�!�K�E��w m=':���4� t���N��3W7�TS(���K�����$ ��ň�\*�oP�~Q���Ӿ��}�����@�%$��\_�Re�&�x���x�Hj�g.� J���W<��ZTmٯڡ$��gH�omܤ Z�pE���U��t|2��\\�;f�AU�V���9 m��ڙ�:�V�E�2�Z\]�3>� ��6�:�2�$���!s�����������$~Vz+�K)�xqs"9��H�pEA�:�#���zZ�}ld�gyyM꣘���k,����G����h l���q֮�� ���U�����o�Al�ƊR�c����Z� ���� �uyjօ��(��11<�p�U�:6���A���hSF�\`�y�͔ǩ���2�:���gt���U ��RT=�@,�r�hO�g6�G�ƦMB�C������D.Td��OgP�2�wOw����̩=ı���� ����o�P;����GV �i�\*7٣@8.��p�d&#�ߙ�OĔ���uk~�f��Yy�bD�Dp�N�Q~���E��f{���2���󼚻<���L �V�\_�#}�T���3v��$BI��C�� �N��E���E<�ߵf���\]�yOO'��i�Sf�Z\]�$�zT���L9�\*уO��$��C�J �����N|�0;C��y�a\\�2df��e\]���~\[�4\*�\`�MK����|@͙Ά��5o{v�%��o�LF�Bͱy����팦j��Mq����Y���s��L�2�� ��~��ѳ4\[GCi���f����-�x���m��l�\*V�u~Оbf�z�;�^���gM��),3�<�jT�;T@�\_��{�( #O��C8@�F��b�� ���yC�9�������̋|�R:e���)�3Ă��ō-X�&M�#程r��1��R��� Gg�X��\\�S���(��Au��"z/��xzU� ���Cr�9�z�C՟J��J5�@1>a���Z4��g�lY�=�絍A�zX� �Q��;��<�o���h�sW<�I$�d Ǔ�'�݈ͥCc�i� 6�(��s�/�Ii��e�G��ӡ�q��9� N��Kl�}���d��8^�C�m^C�\*&��S�Շ\]��),M����X��&�p���}^>>�IS %���״qYfa\_dX�cX�∨N�&R;�C��\]��y�)�Y�t����EO4��o6��c��i7t�6fc�r�Y'4E��BQ��߷�7��Ckr4�6��D^��k�\[}Q��v��C��ٔs��#S/���(\\����?�U檧�CJq�.�Ť\\$\*��7��%d1唩�=���;b"hQ��M����dsI�+b }n�B�e� ����N�;}�0��Z:"\* �.�:���\]��\*��феC���.ȑf�̸���h$�$z���K^���� ���:��9��V��Й�0%�Y�$غk�X�N#�M����wpe?�\[�������" ��/���}����ä|@�\*�LU��$���7��b�g���y%���-��c ���b&�v� O�(͏2xҢ%��=I��A�����{���)�J�w��jK\]�؂��14R�Ty�Rj�m�v ��wgm}̓5^Dہ�����z.p���~!�� 坔ڭ~\`&��;2���%C~޾U�˰:�����}���� K��e���q���>���a?���0 ( R�8��-ܖa Ɋ�N\]O�l�������W�5p��76VW��Z�6���Z��G}���k�H�߆��8��3SK�jˬr05��H� ��0#�' %�y|T����O�wF� ��B�i%��FE3CQ��^�É9�ʌAN���/���(W��Ԅ���M��Lշ���X��������x��\*�P�x�:?��W�\`��z\]�$o\*"�"�f�R=��=�p�+�n{�T���3�nx0Vz\*��)��(KG�/�>z�����AY�P���?I���5i\*"'�+ZP���97�Mc��+Q,f��W5�a�����:�Qc�{�n�$�fC�65d\`�cV�Q��2�� �T.�u��i�dW�v�4ojo���|O�֋ V=BK��l�M(�Yh���{����to��V�2k���R�5f<�80�} �o%\]���-V�cv\\-'����>�i������UV��U}�X�W��(�� �D�h�ŏj���o�+hg�����e��!���$��cLN�ɟz�0'�r���\_2�>�\`Rq�3{b�������^u>$�yB2�{F��T �@�t�H�fC��z8%�t\`d�����(B� �͍z����g�;�ݪ�o<FA���?;r�� �S~�n�m�G1Qt�w1�0�E ���5!����M5���-Τ�+������P!f�J%.u�\\�0�.��,��eL�,'s1~�G�O���Cݠ�Nf��I��jSp�;������e�Z��1�b\*��{5��\*9o��y�R�X��o��l#��� ���Q��3O����#��sF�ط��� ���Bm��!n�?�� -��B�;������Y�B��§��L x�\`.{�T� �دvC�Nrh��r&���3ss�TEY�'��~5q�����rZb�.Ie���v�uQI�z�\*u� �����\`��A�����6�}AC8+w����М�����%����f��G�x�������O��c���菌�Ý������s���b�?n�Y@Tϱ��)��e,!\\���Mo=�N���������A4��,�f#��~F�x��rK+E�Cu�dR L���0��f&'&A<�#�j�Ѝ�1�N���b���U�17\_�!�-��C���I��7�58d�+\_I�\_Y�1o���@{��i�3G�,�� f��ɮ��w��1�\_ ���� z���Z�71/u��G�����~�"���·#����������&�T��tp�g�6�����䧅��H��f /2�ĘJ�c�w\*R�X��I4MPL��uw��-�ron�?D�2�E� ͞; �з��F�ȼ����R���+�S�Hr���\_�r�+���"�\_ltI>�=�ļf��WK�8 &0�!����\\�7,~�m�K!�@�OE-D��K'���Տ\`�8P�\*�y��+\[{j+�#J�|����.�!\]����wd���Z�v��!@W��q��}���c��N5/��7Y�l.��s�6ȚmDB���\*���}�.��E��p��S)&��=�A�e�p�DN!�mټTv�3��Ʀ:�oCj-<˻L#S����iw=����� y?BهO��y\]L����S��I��&mSd��/��oS��Xr�Ѽ)m�G���z��y\*n1j�I���$���K�L����#C������C{����\`C,��I�s��?��\[�ٽoO q�c!:��R�ENX��ug��׳s헋f��p�~\\�d�Y&�AKoM� 6������9أʵ (� �ã\]�%��D!��AYiX�92|%��GVV1��|�}M&=E���hݿ�������~� Y�\_+���H��M�=������vB�"�~#�o�s����K�+o�"6��F\]7V��Zf��iKJ\*sQ ��!����t�I>��}�Ala��vt�\`P�Z��l,RTL�q8�gɩ��)Ҏe��j�n&417�����3������'�v,�4�:^�HoD��j�{dCȜU\[�6&��\[ǻR��3cBȋ�QE \]i�Ey�kk6�r�-<s���N.����S��#H��U�\[�m%�Я�ؑ��I�B��eW7 ����JE�3ԅ!�@6����& y�/����#1�T�u>��q8w��m:�e �k\`5�b�U�ĕ?�H���(v���\*���نMp���-�N����Gy�Ġ�=��By�}���xm�|{o���ɺ�s9���Y��t��+(���a��װ6��=xQ\\0(�M�/�WK���F�\]il�:RF�M���oؤUw����(��9�/D -<���^~\_�C�.+��?�#0I���<ҿw�D�v� �k���F�@��Q�� ��"m�CD\\81�ߡ����r���2�^�IZV�;M�.P�ғ#h���z�����9�w\`����Wqc�c�s5��m�И��{Q$9jKߚW�f�� �L\\�Mo��>���v� �&�6+ov�)d�T���>�9����K�"� ��hփM:r���@ޅ�������tz�P�6��L��X�KV��� �B�} ����0�mHt)�r�W��@l6��d�}��\[��;\`�T1&Z-\` ���u�4 \`��&�~��R�W;ꜛ��\]#���o�� �S�m�fQ$�Z-�( @�r!�J"r�\_R�xD��G��� ������\_M\*�%� ��ua|�������NѪ7ԅ4�r��:N����p}�8J S�1c��f�� ttŚ��%�/��?{�+8 �"0o�u��HM���+���(���,����"�K������e�;�����~��v���γ߫�Po��Iu�0��o8(+1���f�&X"��R�6I�OA�I?�Ml�Y} Z��)iQn� (x�˲W�Ğ�/E\`& ���,�s �AԨ��V�z>P��ۦ��>rY�I7�ڇ43iGjo��e3�EL8X�Ta ꗏծ��EÊ�T�8�G��U�a�k���$)�B�� �ȴ����=����W���f�;�$�ڿp���� Κ�e'8�tuﳵK�SU4Tx|�ʤ��v�Cـ�Iv�Pr��Qd���'�� ��J���8��F0X։0�I\[���X��\*J�d������ (<�4�}��Lh�������\\4�&۩�i)/:a� 2PW��^X��-�j gY"��P�� B\\�����c:H���a��<��,X ���El�aU�����d�vv��X�#�t��S��0�92侃\*�FXw\_f��G��2֫g��}�W�V|&�did��ė䄉<5��/��O���I�b��Z�X��l��g���d�.��R�N}G�u�oQI|#���@�Q�̳��y�Ų�>��V%���4r�hq�6���Q������xX�\*ȁ�����k�tx��C�����#|W����<U$}(J�(�BF�J\*��>���4��K��t��bc�H�C�+��&��u��2FR�v��ͪ-�� >I�\`0����o�� P5����G)P�'9�r�BL�;9K��@�cn����e��� b��B���\`{��8�,�F�LzU?8�ӌ?�\_9s�\\�Լ��-Y /8su+R�Z%��e�Ў�V�~t����±�yA$��{�L��Ԓ�& � ���6ګJ"0��!b�5O{x�\\s�E\*e�\*<�H2��w��� ��L�-�mqœ�����p)\`�G���z �lY�YS���.0�c� ���+i���\]�#y�Ȝ��n�S��i�y8��Q\_��Fz�JЯy���sqV5K� ݕS7'v>�6�jmN��=�ǪI�������=\*Y��ʛ�u��6��yS�e m'�} �T;TX7�c �A��6Y�n����%~�'�^� 6��ÿ�$L�K\`̒��wc$�h���0=�V�$ǚ��#\[������u0 �i�Y��،?����}�|�;�Sj��nhz/÷���pD��6�Q�|X��t����$7���}W�Byd$8�vx��G\`�Vý�\_�vI ml�~ &�A����Rӆ@��H��x�ݕ1م�p����Ás.��QC�˪EH�K#+���0q����U\]�١���������(�ɋ��'���Ѧ�����}��G�ɠZsq��ru��+��R�貘I�\[|tf5x�� d���Z�� L�x���PAy�K��:�տ��!�>�qCM�A�I.3K������Z^�Z�4gv�Q���'�'S��(���l��E܅�xb�c��n�P�vK"w�p|����#W�D�ӆ��c���j���z���)Ȩ�wo���H5��K���O\`C�^}�5�&S�~c���0�{q/�\_\*����,#��+\]}!�&\\ߊ��yT� s<�J�\[���, �Q���\]��a)|��:V�L�޷���}��#AZ���"��u״�Y�$e�b �}�� 2�;yF����F��+�=���۠��� ����07�\]Ҩ�F��f0�,lf�B�е��)���Α �a^���()�}'mt�b������'�mh�oܨ\\iz?I��0 \`!ʽ/?��R�!���UC%� �h���r�\_���\`&��߼-����M�=,.� y$jW���\\7�S�p�v\*�K�z���s��"��,-@/��v�� �#ۻ�\`Wtpٶ�S���ޡ�'1q���3S�����ũB���S?'��A5���є�\]^�ǐ�@����Vk���^�\]�z�,Qg�9�G|�V����II�ɾK�·����\[�x6���\[���K��L��:Y�ߙW����?��Q�u�CE��洽�g}S"�(�L���q瀡�&���,�k��u�C�Z�����r�Q�c+ J�V�F�@� �\_\*�����A�h�iC�$6+�P�Vu!��R ��OuW9y p�����ä�ƽ�f�!\]����w ���p R�B�X|� <�� W������ݹ���ԇ��!>�w�g?@U��HxQ�,(K��+{�9%�"�o�������Aƃs�A�>K\[��I~�j��{=��c�ָ9����hs-K�SMlO�}�4�yG�X<���/�^G~��x#�\[/$�K�n��~$�0�\[k� �F,�V��@Eò���q?�0��@�;�.�@�,�=��i됀���{�4�5!I��\*�N�t/ �7���+���~˻p�G����� �dú��n�8��;�?���v����n���ݴ�-V~�NL�r�<'��+�d�����ZV;�2��~����B�����8���e�0P9 :�EH�:��AP�49ʭ4��oq�TX�oD�F\[�c�:�L�ᔢe���|����O.� 2v��(�b�k�9�5?�8nBp1.�����N��\]����y�d�@a�9����q��S���h)��\[�����:vi �l��;�Uꝇ�g!�����>���u��ې�W1=8��?�J\* ;�����\]�\[Q9֤�6q��p6���mK��\*���i��o�S@dh�erv��t�ӈ �#���Lp�}�-auE~��1� ��������a���Ov�׷�����l�^�b� ,��WH��H��aیB =��I�"���,S�)�#$�L-���r�Ѧ e�Oq��O&8\_m�ۍ� ���\`ZE���1��,h^B =�>�%\*�T�;� z@�9���9{�K(�>�Qv1�7������}��e�i�=��rS�ؐj��1�\_���7Qucv�� ~a��ƞg��\\ i�\`�L���z���#��|.�ִ�)t2\`И��:�)�XH'�OV��b<\*�T�W�c�r��g�K\]|��0����)����A�9B����T�\\�����| Ԣq&3U�k��H@��E�Tbp�)IUJ�J��������k�s����,S���(� �����C���{�2��W��t�ň�� u����l������;�RG�5��N���1�LaOT��\*��;�ܣ�w����S=Р7\_�>��pyԊ�Y)�pH2%r�<ф��T$��S�8��ZIO�{��P��Rd��wre"kM�d��� �.�x,,����5F�� ���3��k;��7�a ��� �BdWD��r���uT�Ak���=���L�{�H�S�\]4�ռ��~a����x���y���=p������ <�Z��n?��i�I��;��4 #|>�M����d���Q��ì �GU�I ���oJ� �K�&���0K5r���$�J�W�u�!C���U� k>�ܝdo&����qYH����q y¬r����s�� z�y(>j�jc��6W��wCƟ�=��T1M��:�?\[���l0�I��ڃ��q���K���7ɕ��E��Q�\*�%�=TΥ)1{�ي��%�%D����Ie��VϞ��RQ(H������������b�����%���G\]8{@�13���o$�"Ƚ\*5/+01��RI t:W Ξ�: ���%��@t̋R+ �2��B��O<�Gu��C-��g-��֜�B�����O���6�t��j6(�e\]��#��ɟa�R�@�KEͻ�2�<���-k3�����K�١F���lۣ��rz|s�n��TD�=���oY�X=-J��xUN�\[��=��^�Ѣ���V�+x'���V\*���K���\[P���� sy��䱱�%W��-J�8�X���H�ئ\\a}� \[��@�ﭬ/M�\*��+����s�w�ڳ\`�h'� �U�;zu u�2�2UN��@H��Pd��bs��E�-;?�����i����3a���U\*/|�M�Z���CwSR&�#c���X�e�g��m�)W�����R=�V�G�\_E�4�DL={�xط�0\` ޚ5���N�|z��˛;6������$!��:JcS�UU\*y� J:ձ(MH�|�������m��X(x��$����V1�;�|j�Zl{�jKlQ5A(~+ud��0�)�Sn =�^���E��Ը?g�42��\\gF�vIs��7��� 7xDT�0Pj�����{+�M8�:�"�װ\[� ��������-T����M=6��GQI �QO�dG� v��A<ު�I/�� �Q�Ax{��o�E� � ��\\FvT}�Y}5��g���\[j��r��c�JI <(��Ƃ�������\\Ʒ=>=ۇL�����}���\*�ٱ���{�^~Q���RRT�1I%�c!H�<�\\�Dw�kD3@��wSڇ2��!���k ���W6���f��bX\[�1(s w4�$��f�#i�\]W�����\`�r��/�}��c=D^�N?i���&QD�! ��x@�b�#�03�s�u�Z�P��>rRE��JPlV�Tu�%�}I��lE��E}�9�c��If��'��{�V��0,c�uPV�����Jˠ" ���o�p������^��Eb��2uJL���蛡o47n�ow|8ANh�՞{�&����u�X�\`x���X@g�)��11�S?7E)�pJ��O\`�~���t�U땂%�Q}�uv $�=S80���������������=0�J�y�b;���u��l2�e7�f�������X���}I��ݒ��H�8@5�sJ�/�՘P�mH�����pU"5#�:v)V�\]� ��j�I�������+�#hRl�����H�XjcFzw��J㜢�Ф��M6�m��A�L���4��ݪh��͹u���l����BLpac�i΢{�,�U:��6G^�\[N�9CS��.���!���V�b���Qtb�n\[�����L��\*����A��Ծ鿁��6����{#�!����%~��m�{ē�PLV��!K:8�Q!X��>���J��͑5�k��Lr����UI�Ҷ�ī������6��r(,� ���?북TQS���$#I��df@������k�\[�\_Õh� �k��O��B)��0��\_h^�c �d�4��KTY~ٺNS�m�\*B�þ�m���{�s2�\_�����PEAexj\]�^h�x�Ќ5~��� �&h�\[M���.��r+e�?�\_�������A|de�y"�2�J{hX�B���O�0f�1�α\_0��#U�� ��@�3���NH�!���xaBA�6���9}7���U���5'�Ft�-�o;;8i��\]�,��������V�:��\\�گ^!���<�:5p�$\`C�d�jA\\� 9F�\\� �תT��>���ڎ�cA{"m��9�0 \]�\*��\[�t����V^o� q����j���J��p��,Gp�o���d�p�!كZ����߳.���Uk��q��c�Fn��0��-�1�\`S��w,�kW�B��߼��i��+z�&�JW�E�� 3�:�N�ʻ6����֟3G�gb~��\_ ���Tt����.��JȞ�,�L�n�j�ڨN9&��ݷ�sR��-\[n�n\_$iQ'��� �-ʋ�)=�{�w'@������HA�@ț�Oj���=2bG���myTU��'(0Ֆ�z:�!ʉI��W�}u �5�\`j:z� �UK�6�c����g�/�՜ϝ�"�F�ІE"0ƩJ����\*�㮎a��Z#��S@Ü\\���0�T�c8Qm���\\��˘�a\*�q��#���!�h�|G�@@if s�� �˪�C�\\{���yÏ�����I��\\��4�\]Ś��A��7�\`�G�c2�� ���sl���bW lN� ���aF�K�Zf����e��.Т���L����$�u4�:�m㽌�&9�=\[|7��Y���Aj����U����x(I�E�z��/M�� =ɒx� �j�ll�F�K;sk�9�$���,����<�V$�6C�3�i$� ����~�1��2G=�G.�U�Y��o�q%vZ�\`���vUI�|~���X�i�����Uov��� t������–�ɥ�l������R�#�\*If\_����K������UʌC��J����=��G����Q�7� ׼4m\\�O����|�mb���-��8�1�"�yz�p@~�myM�\\~��k(��j����ƣ9�6\\��d�#�/�Ҫx����7 r�DZB\`;���R�E�x���>�Œ��� �IpZ�w-�ߊ�z}.u�ؗ~lQ��2ì��W :�D��= ���\_^�q�ac�͟z���1�KiC� \*����Z0����⦊C����y�׭�CU�����6���Hϒ�r΁���Y��;AkRұ�+�RcZ��†g� 2��n�:\\��~R�u�f9\_O�5��"�i�C��6e���� x��1���\`�+6^���4��\\�:י�7�\_��u#mG�7Q�Je;W싏t��P�!�ډMitC@\\g��qi��(B���0#Oj��D�V�\]ddP� j����ݘhr��:��rk�eGS���H��\*G���8�mY�#Ț�\\E͝/X���d�^=��?c�A��5��f�Kz5g\\=ZQg� �c3j:��\]�� H�r����T����7��/�MK��-w���F/\]W��4��sRز��B1I� Z�#�%���i�% ���-Wa'\[�����ZW Bv�1.Ö<��ل��)NMw@5�Ch�r�mve~��T��aZ�\`��w��y�<",�ػ\[\[U�\`���<��M��+�S+������o����Z8o���)�p(��Y�i���>I8<4@��m5��ePm�G)����b �}�\]L" 2�Boզ@,�Q��X�W����\_!� ��Ko��K ,��z818DU�a��n>5/�+D;|�\[WY�\[>2�q.l�sAݴGF�N��6�.��� g�h��Z��i C|��΢���� �-K�UE.|���8��^�'��8���sKW2Z��)���@��j��E�o���3�m\*���W3"� >|%�.I�\*��~a��/X��{�GM�q�\`�ŻxA��M�x��k>�� ��" �\\�n� ���7\_�r$C���bY��tea����6��1t��:L��.�1. 4)���B2�^��\[�\]�$�cט�q�G��%v1rr���ʗlz$Y�i�E�v� E���L�'���p}�+�;�s�3�>"�|y̛�D��L�2�3�f/�k��'R�\_bR;�8uF 2��8r|�M�%-����/�y3��C��k\_��3�dŸ��0U�w�alZ8��4�����pw��� b�>�\]�}\`j� &��D�8�����ٚ���8�����,�y��x-X E�&K�� 37��C���m�������K��Z���z�M�e��z�0eG�.|�aD2s�������q�Ȝ�uk��#s����\]O>���@� lH\]6R2洮�����\\�"hN�ߑ&�-��T�UFmِ���ItM����;��vJ/?J��7'��fc��^��L|�$)��� �b�d���L���)�߰=@���\\�\*"׊����b�rl�:v=�'?��F��ӭX#Ъ�Td>l5N� M\[msK�|�YBf�Y;���G�0�\[(آs���a�p�Gw8���\_���8��W7����~��6GS2���'�O��i�5O�\_�~���P�g��X�\],<�&�V��^Ʃ���0� ̰�.sZ��$�D�\]����\*�~��Y��ӫd�j�f��}��f."�d��b��\[7��|P ���O����7;�t�Ę?��e9��V",��O�xP�/6ωq� ���<�ҽ�v�u5�r~��� �m��\_ w;���n������ Q:�;���%���\*�� 2�gP�!�8�(D���9�X�"��=c�����Z��,�{\_/2�|��S�%bKQTh�� �qfP�)�|F��Q��u���zj�!��^s����-�"Z���Ng��m@"�XG��}��/=n�вŕ��ne�����ї9�Ϸ\\� ��-"\[���b/��L�� ��2�.� �\[�xGY��QL��d"=�'�:%��~&v c�-�1��$�C�w! U� ?�f�Ϧw���n�A�iƢ�aik�“Ԡ�)2�o�����D����Ƅbe�G��M��Pk�M����������\\���iA�®���RD4�\_w1Lb�}�K��(n����h8���qg6���xL ��fF:B#,���b�k�V^}ӄp2\[��h��B�{)H�@� ��\`y�<ɜa�%��= ;� ���n9cu��+��}�-R��� �� ��7ai��. ᆽ=���y�1��'HA.���\[�s���������(��yL��Z^��IW����}�K�\*��Z��5yQ�‘2,���}�8�}��Y�y e1=�y��H���oTPm;���=�7�g?PXp�ڧ���g�}��� �l�\`�\`T�L����L���H4�<���cϕ���SĨ�2г�-��y��uf�8\]����|�k4 ��K <�e�s��|��l��j^�� ?���Z�����쎼��)N�ڈ��^N��(���z�<���N>tzc9p� �\_�m�3��� 2?��\[O<�6���\*���;\[�m�?t��Me�4%l�e 7I��.X�vc}�a�DњQ��WAya�c$�\]dn"��uZau��Q��\[v\*���9���V���H��\\dt��"��޼#Rj-q�f(�Y��e\\������YPD�X����3'��؉O���i����Vh��/L�Rh� �{�p�lчT�y��X����\*x���\\�\]���h���= jLU�dSbB=y���a���5�W� ZMN�y�EF��3��!� 渡 9��+�m���0���)�%{Ɗ��#l����}� iT��iD��-.������v� �e�����s��Z�F b��AjL�, qT?�l\[u~���^���m3��}G+P�����ci��+s��K�<�������/ s�����>؃�QC˜��j)<��p\`��REU�\`��O�\]� ���EM�L�}Y�OH��20��\*��w\_��S����-�S��'@�X�aS����ktUP�� \\� 4}���̂~��Ο�E\]���}ə!k����FS� A�R�����nP���B��:�,&��p�o���~e�����G�ЧF\]�'&� �4�2mIwXQa �\[� ��R���}�W�q�5O���Z��'wr�B�kW��a�Ȭ�=he��'2�� ��Q�q�"w��R�yx1����n~�}+)��"W,ƕ�x��� 칈�mDw���W �w��G(�qԔ��$��b�y�b�a'W�3E>�Y�oQ��Q\`W�Zg���y�pu'8%����o��5\`��2΀VA\[B�f�Q�uJ�7<�P�i��Zy��laN�5�Oc�Y��-�J ��� ��#��l�5dƈ�o�@�+���~C{�ٻ��Je$C��l6���5��Ԅ��L�S9����)�4I���o ~�m�|��F;�;��#E�3���qt�|G�=m������Z���ڬz�R���CvG2'} ��7�T����)�Έ7M�+�Ї֌��Tg^ P��,��)�^�v#!�/���P�J5�\\b}�����2FpDj%��9���r�=��v��\[V�1b�y)(u�4�cq ��\\:�� N�7Qi/�?U��tM^��fTj��J��MuN��Ɵ�č!�� ��?|�f��h�@y{'��n��s� � ���i��4;V�.)���'0�����5޳���U�P <�� �x� �i8p��3��~�^t���z�ib�ܐ��-R�…�W�1�Z���wi�1��OwZ��\[/�g��E(5p����%\[�h5��w��,� \]��l�e�r��o\[��q0����,��q�=���3�;z�ڕ(lK��\_�;��L�J���%� �R �\]Tv��(� �ǡ����n����u���R���nnq$ϸD���# �l�\_@;H��r���7{�/OC��Ҩtp������H��$����-\[�B�l&�I��CwK<�8���߶��'�b;S$�S+�,�b��֯�졔8���Z;�����1���2j�;�2eZ��J)Z��uk�y�1��J�Z4��-�x��m#DuT�W�NW��h9C ����}����YK���� 9�<�H�/�1~)��\[�'r�܃�?��/��GR�y�3��@��^q�7�����%0��{��L�/ �S�ȇ�AdY�ۖ�+���:IJ�H����\]y� ����\_��/Bu��Ь�7���x �\*���O}ZF�.í�5h�0\*�g�bk�{���}�\*J�����rDӋX������yD�дܙz;�b �y�5s�ZM��^�����H \]1�j�')ו�ϓ&���p�t�Ë�IN33%?�+�3�k��=�I�Wê�\\z\\L%eЊ~�����L���lCS�c{м�(���8Yap��Y��{H�=';��a��)�d�$�Ei�l�6��Q:7=�xEVS��7tW5�\[�ΖJ�F�9�\\M�s�.�4 4��\\$�\_-D�� u�:������;x�\`g���ɯ�O�?堃����|j�j,j��.9���Z��u ��\\�"/n�T���c����j�\_�?�L�͎ �d�ߦF�P�C�ی�%�y���x,��k׿��ޢ�\`���T��68>�~edW��g��%��H��{��go4�ڛ�@=?2�\`� !�Mi�ej4"��Z�Y������jr��A1Uj�h�Vi����ȡ0���:O��(�e�0b��u���n�����!ZHV�g���Fr��\[žd;u{�y�Δm2�~��/�A#�\[Q�Q����:j��:�6�Y�}O��O������HcNj��ۮ 3��ƣi�\[\]^��i>G��%�|X\[c�\`d+DȡU�7� ֣<�%�� ȏAF'�w0$�� R��I%�)+�1=�vG�r+zgct(��Ft��P���YU��7#;�7q桎Z��C�Qi"O/U)��\*ݚPo�B�߾O$ ֐�L̃Y�><�U��,x�a�bJ��;��Z�?��0aInA��g��\_� �K��.��N?�ϧ��JU��0��ҧ�n-~�K�V==���o�F��� �r,�0n��-�Yg^��h��� �x�䓮��=��Y�������!�4�������\]EĻ�����M��ȁ#�2��q�(��57�=&�G�A~�����88�aev>�Qp���7�~ K���eYws���G��s�u\`�wDH����\\vQ�zrb�nD+�g�BӀz���")�&�����J~��hZ�Lխ�\\,vG�NEU��i��Oc@@AH5� �r�HE���CzH��QO@\[%\`�~�Zeh���Ol �̀�cs;�ÜT���� ��Afܥ�����$���Q�\`���-��y �3��d(<3�JW���y�Հ&lui' lf��F@��.m�!���h��|�s/�nT�˫(�|��(-����(��9��R�d�' �,�"�-��i.�2j�Hm�'���VK\_!h�s�ί�@���P���)��'��:��"|��QT�E�<,h �N^\`��h��2v%������� ����k���(c��̩�~�R�� ����\`����O�����\*�m ����^��L���y��7��������791\`��7��!�(Cj����7T��,~��>�gԨ���i��V�)�&S��c��!ȝF�C�֕��L�0�>�;U�@\[ �3�A���$� �T�/\\cj�i��gQ����EG-��(�ԫ��-�?�DM��\*�9����O��l�=ɘ�z���3���Xp��MZ�M 3(�XI��˞���硚⯁��A��#����FG �?c�qNH��>\]�����z���I6b/ �\*�:��p�VH�c�x� � 2���2��\]iI���r��U�������b���/؂I \*Zi���ډ?-��f��Y{Rtu@rK^��F��������ϫE\`o���w{��iJ��҆��w��z=�λ�}.T4\*D�隠��M�w�>D���c$w8��˯,\`ݻw��O)ʣ0�GN0쒹7>����oi^N�>&�\[�m�;N0>Yg~e��GW^�D֡�4-&�E{wk#�KPƓ��04\`�b���9������qk��gV&��=q�1�ĤDA��w� Knp���oQU{�Z��w�o��q:��o8\\�#��j�b��VOM�bwÉ,�������g�//W��f���n�}#0�ɨ�������\\^�0� ���y���>m�n@�a�o�B�qy��)���Ju�kܷz1"z\]��:k�f�9�U�/�Q���ԡ{v'����F���n$�W���z�Jq !ƫm��.�gܯ���^:o�\_�^ۘ�έ�܂�zu���E+5 4��E���$x���O� �\]ő�"�G)Q�t؉�e/��7�KqtrL&���!�K��rJ+2�~V�g�� �2�L����(�\\���\*�=�n�WY���z���S�:����3&h;V��+�c�D 3j\*Ъ���D���Y� ��YX��!�"e�i�����5�f���~�<\`�|�q>�zѬv�S��y���;e�j8����&@/<\\�������~��\_O?����3�\`�)7!ꔛ��W���!�����=n����rc�.O��d�/�~Q�,�n���m l\[��{��k��KH��- ��������2�u0!M9�wP�s 1ԣ��p95�A�Z�i��(�DžoGk$h�ֺ�����8-Cu%1&:Ď�����\]�3$p\`�:���!�ۧw(�"��U�� ��^���QCh���c �U�z� FV������j���xj����X�>3d,u)�@B-;��ͯ��?��y�6G�����Jf͚j�����9����'mϝ�+���(7y��%yg���\[VAٟ��ɹXG=%�ބ4Rm�\`��cA�( 4,�D.W�)%�+����o�V ���G�"(�͕+/�Ĥ��� g���RճBE���Eʠ�\*����-pi�Cf�O����ܟ�d��<�<�2�M��4GaD���L��o�Rz+&�jM!�ѐ&�!Ͽ��W�Eу\_A�b��F6o��2��-���y%�0�Hz��#�dž�� �\*�q�|��wM���R���P� �����^��Z5���%�IT�?�dRȓB���.H��":�j����W�~���~�z��Q<����$��"K�����oU�N3�Ģco���U��}?>���c'��sd#Ҧu��!,Ǖ��0�+' J�/N#���4y��SO}���n�>�Q��\`@DY#ʴ>,�5"������k���f��K>�|�S9hv�e��@�xZx�-����\]��rFl�i�-�Bڜ��mE;=F��%Zھ��z����h�xY���)�7���WA��,ogt,�'�䢂�\`ZZ�4&����e�DRK��,!\*qR�����)B߽�{9�|�G+p9~�A\[~��}�h��6 \*�0\[H�G4�\]��s��� �jpq����\`C����A,�����\`ߗ���W�\\~�|O�2�s�^�$pH{o\_#��?�\*��3<��Fh�'��%�6|?�ҍk<�?��ǵ�M����> ����� $s;��vQl-��� ���j�� �j�$k��N�j�u� UG�v\*W�:�՗�?�{P������;��L�;OZ!u��� �N� �Ia�$���M��ڮ �ݾ��"%��+L��?x�C^��/��4�����q\_C���\]k���׎r�択N�m����)Io�M �SG�ќG!{ę}̝{��3�a��Nr�:���; �6\\��c�QP�q�OJ��J(\\şĒ傐X DOn�AGl;�V�o�J�4 r�\*5�F��D'��\`�P\[c0���"�YYk@ �h����>��"���@�:�ډ(t�o�DY�\*�A@:�0Cv��6׀�����m\]��2�e�ev���8W?F!=�ڽ���� �|.La 9g��l5|H���̏�\_�}�����p�\]/�z�92�UV���<|�{���~�G��1㷒��5�9l�e��{��'k��yUR��#R����R~G�\_{�$������\]dp�j����=�� �a9wE���lr�����N̛�ho^���#���\[A���{���jč�R�Ą�@1^�0f��唶۞7�����8�� +�\]�\[jӦ��m��n ��H��12g����6"�0�s��ֻ���&�GB��-X�o�Z��=J�8���Ӈ�I�\]8�,��uE6�ax�|М4COP(�&��9������f˞~ |sa�u�#�T8J��vy���K�� �����\*�S�a��c���:�i���U\[�Y!�#Sז\* �r�#(�s�Oϭ��n�w6j�Br貔;����!������ ���k��J7�d�P8 '+��ie�B�0\\���}�˗�^�Bq��X��z���Œ�4�� h@xaB O\[u��K�\[b�}!�c�D��ܺ�4����)}N�\\\`�2$�8�����L��>�N4�hl���ئ�FӼ��JA|�%�����kWy���s�$b� c\_͕� rkױ����3�gOr��ט�k�#�C6�ֵ�J�(�r��%�SL�\*ʅ&4�\\nw歭�"���+�B�Ϗ�}#$���\]{� ��g������܇�g�����+TQ���� s$���^dBݞ ���@����zcܫ�a�c�>�����(��&�T����F#1u��7S\*��wc��( ��ؗ��=�n(�Јɮ�e񅍙�z ��g=m�S �s+L\[F�{�q�;x�r�IDkY�2\\�{wN�sht<\_��z���I������We �2 .�ޚ6���y�&�q�TK�\*�Ñ���N�"��nA��":��e|��j����75m���,���3nEy�!(NqB���)�b�q�������m7��ƈR��Y����e�Eb����D�v6\]W�%��(9����YhBB�.d��9�ɚX6p���|�a�䷬��� j��B��i�=l�������Ѭ���}O�K3�/aأ�{v�J؟&Z�~������D�M���+r�sJ���� f�LjD�u%x = b=��nB2� ��2�򒧣- �/��6i�m �Rd�۲K<�u'�����C�u٤�Z����w8������c�s��1):Ԓ��b߉��h��~�:\\\`)�F�gu���3�Uf�ˍ�'o:�1 A��ۧ|�%Uc��4�π#���bH�>����\\�d�/\]���ˣ�3�\[�L~Ӡ��զT\_; �'�sBf>�����E��T2�����lC�uH(Ɩ����l�\`�;�J��x,�N�TI�!��%;����D�W˸���s0���Se�\[\*W΂B� DzX4���|\['o���I� 6��)�B��=��!� @B�⌁��� f����b�zWۊB�R\`��F��!�su5����9�<�3��y$�����|�~�m��q{oXX�zjA.�׷�o�|>��A��z�� 1ˢ��4a�B��n���KS��;܄�iֻ�6c~��O��� 9k1��H\\�v�J�vzX��.߾� \[���4�M�%H\\��""3G���ڀGQ|JL\[6S =�q%�9��dt%󞋝 �;����F��Qdk�^��S�J�@'���Q�56qhj��1n�Z����U�;~��y�i���7�������BjIBl� !g���)Ն��\*{=��#pF�Y��)vy�@���0�� �ͼ��S4��Y�C�^\_&� C��F#��7j�\`q��:����F�k����8�� jQ��؋�< V����������fsK�'�q����i/��d��~ ƃkK�wFp{� �s�|Ҷ6JS�Cx-���Gj4�IbZZ�|;�2��d���q��9��cg�1���냋!�T ���������wɖ!� \`S��Wj)\`r�L\*ۖ�݊�zr�|� �w�@|���ʕ�e n�kK3��g)\*/�P�b%�u�5�q�!b��{�;J�wX�J>h1��?5}\\7ϵ4n�0�� �j\_��s����@�c�\\K���^�9yo���Y������ ��\]HY����B�dԩ���5޳21:<~z�L|ܬ��h��D\[�\\�FVc/|��(d���.����,�Q�W\]C��#��g.Q h����L �G摵��4Ɨ�;LqCU��;�j�C&9�a�\]z0� ��B�7�����F���}�!Ŧ�P����� }��l��П6������ї.�r�����(�M����q�'�h{�����o\`D���H�$��A\_Y��O 2�K�����A�%�<<���c\*г7����r$hu����zg�T�\_"��U�u����\[,��K?�)�f�k�/n���<Əu���� �����7���?�^w��z7�����1΄���i\[ �m\[ \`�#s����4OE���\`��� N4�i߫y}�b8�D�lðſ^�{FX�� MwtY�D�h��2G�Q������U�J���qs�� �\_�&%����j3F�S'̾�6�)VF���64鰿��f���������N�出��8,��0��^�FK��MnZ�V�����9J�1w�=\`s"1P�4ۙO6J�Nֵ10#��7�t+��jH���o�ݥB�CӸ !\[�������ÀA�)#ߊ���1 .����������4�+�4{�O 15��b�2��#՛X�?k\\M�e�H�EH�f!���#a�V�9�|в�� ����q�t8��R��^L�� X��g�8V�\[� ):��� �" �"gp�9��5����o�,�D"��� ȭve�TC�R���\[Ρ�P:�Am8):���xe4A������M V�ukO��m�=���\]����RpR\`�;�~����)9|?> �\]3jt��+�6�&qcu�:�E����&�� |�CA��\[� �x�&�d9����P�<�X�kW���-��d��C|�k�t�3e���'�àSı�V��RV �,��Ͻ�\]�a���M��#�@�pjA��9 P(�w 6,T�=g����ls>2<�v �c�ؾV$��\_Q�yAA����>���\[�0=kx�}P�8�m�=5j����= n���9��I��R��\[�� :\*��ñ8$QaNQs\\��kj��� � ��b,%����G��l9��u�p��7��d7g�S����\\5}98/�.aV{o�����h5^kb3�D�ZQůi E'mwo�J����N#�W� V��Q1�W�ɕ��g��חg����|C>PG-JOl�7�ű�ӻ>�1���Ε�,� ��3,���rΨ�� A���u� �� >'8�$��0IC��|�6�$���&�b@Q��|֤�}�\\��J��Z}{�yA@�c���+B�"�UR�˫� ìuZ���b0Y�F���7�OT��t8���wUNkJN���6�\\�o"��٤��A�@{{cG���Y E�WO�WT�������� ,���\`� ���4�mE������wkG���M,>^��%Yޔ�Q����q�Q�ګ���YѨ��'V �?c���A���X��%"P���-(�\`<�"vD�<�ag�Z��ˮD/CԂ�\[�"� �!�>E:ю�+q�DP��MXS�-��6~�g-Xlв���<���o�K5�ͥ��pG?��j1P�j'���l��rw.�5��\[�-��i%�=����3�9�Xq\]�\[�}���u�-2.@�����I��:�,/��Y��^�I�\[/-�B�>̝Z�i(j�)���@��-D� �������\[� /ej�rI�\*Ф�!�j�ul5<���ozK@\_���Mj�Z���A��^S�>�|⥚�����ҍ4��D�t6 s�����\]+�'��(|5�E+�:��l�n��@ڦO #U��Vi;Q�(k�r�\_��%���l�LM�� |t\\�T�0Y/�\_�w��a#H~Yj.���g4B�eM^- f۞ �^S�A0�\]d����s�N��Z� ��@��ٔ�(͢.0)O ��0���Жdn�� �3O�ɛ�}��<-�|��\_�g�fY�xu�D�\_7�9�n:l��������n��o �E�I#i��7lȪ��Q�T�\\@\['��� !��eat�̀rn���� �"x�b�G.r� ��=/��/� �8 ��K955k��$��-Ħ���)T3r��u�B��|:�o� �\\�3 ��v/{��s&L- ��.�dQ�@E�SU�/ml���yIʼn��o~�=�ס�N?^#~��|�7!��KZ��HM����ٳ��U���,Z!�Q#�+��N8B��A'P\_�)p\*UÒ�גP{w��$8X�P�޶� �o<� ����BPN�C'��='n�{���:�5�hm�@iAR>�yͰ�!�����x�>��l��ES��. 2��Xݎ��V��6�Ū�Ħ\*� \[o�Q�rxʍ��bE�>�=zxs�sB�e�� �̮'R�Z�d���W,;詂+$�^\*�+����� �zu�����e���UFAww@�ZF�������g�>Yyg�hpw�� �!fg�xG��oɉ#m�ח��$��V�|b?�cw�����$l����Ƿ��6 #ɶ����\_zn����c�mu!\`m�: �x?#\_?���Þ�P �J���=﬋��>�d����\\�u&���X%��X\*�cD!t5-��l?(D%w<~�c��і��.N���K�.Uu�S�Vԣ���K���-W��\\��=������Ɔـ Cn�<�����Ÿ|��Փ ���sk�N��X��h��g ˫�F�e��,���I�\_���L����T��n#�/�I�#�6I��hq�6:����A����ة���{-���ߩ����D�X�Ngm}�GB�R���>�o��^a#�3��)��9��R����K�c/#{ ��7'иY��\*��� �b"c�H��'��=�7��Z@I���iB����V\_'���9oY���@��NOI��楷!o�͉��x ��)D��oz�bB;���H�i?��o�����/woɾr����1<���1{���W:j��s�R{ˆ%�'�X�Ԭ��,�M��A�S�\_Ήz���:ғ�K%q��H �R�ys&A�G���@1P);,\]��o�͉�\[�W\`Ѐ�V�r��{/��!�������\\Ƨ�8pL������} Q��&�{ A���\]��z��yL\[�1���6\]�/8��ӱ\]����kv�(,$�L�\`"������et�J~�E�+ {Ÿ���B�kA(0<��2�w�V\]��/��|����x��8\*����3d��M������rZ�K�hi2�ʓ\`��x̆�o�� QqL2Kx��{��g� �� ^G��n�w9=��� v��h\[��t֤G� NZ�gJ�)Mmv)�oN�\[ �Y��Mb��=�WYW �/^cV@�A{=�P�%���e>r�Ch ����y/��w+�����B;��f�'3�M3KH?8(�duכ5��Y�9�y��x��C �P���~�!)9�(uଇ���Sdz�+(˦��� �5�h�R�WsQ��V� MGw�\[��\[�!p����w�� >��K۽�>#��+�NJ��&�gQm \\r�N��5j�$>n� �=�؟S{jޘ��1??�����=>��l���\]����3�k�j\]�\_�����Ͻ����m����8�\*Q�C����,c.j��u�v��q\\5'���:u��8^���g���E��,z2I�������h �V�l�!i/�t�Q����߽�>�u˚6�T %���Ҫ����Qn vNp湍7�wI����� mcsvR��G.6�ya�����QǛq�}���i�Q�xt;SS�kz������$�!<�����rB{v@�n|�7.o����%&�l��DBT/"����C<>Ҏ5�q�ΌC�����c�ž���\_�6����V�Ҩ$%� d�r��Z����$��:@#��\[��\*�5X��Fw���xR�5\[lB ��,c@�\]�S�37y���z�Fy gx� ��\*B�Ϥ�E�)Et;}����namRT������m�y��&���\\��a�H7$��x��㖟����+�G�:g�jmJ��UL���H 8�t�ڡ� |�;�8��7��615v��V5��!ץ����XI�;#c��4B����\[��|ϭ ��q�ى�#�!����eM�� �g�������8\]�\`Ɂw�d����k�'o -����9��G���d��>��Bܬ�����0�\*{�+ �g���4ڌ�����X嚗\[��=K��\*��i�cM:��� ��H� 0fʛZۋ#��S; %>�i�)�{����ڽsΐ �.�K��\\��ؕ��ւ��>=�Y~�g=�9K-�򖨓�G�0Y�f�j�s2wUi;�э�M��f�}����0q�"U��WD��)��-�Ԃ!Iu j� ��X�Vm�'> $�t�7�K�VK��i���t/�"Ҹ���n�:��M�z�\]=�d�0y�\`��'���<�y��@��a�Oy�2��xX�P.3/3{DTS�j��2YPW�y���U\[����m|���H<�\`8���,��M� sH75�2��u+Vs��=�ާ8�\`����P��@����Km��d�1��i3�@6�%�{����3m�eZ\\����F�=�/莸���5e6�6��k��9'�#L����\_c��H��1J��ͩ��h� M�ևޯ}����CZ)W�Xz\]פ��N)ƅ���" "\[�E�Tp�t�C�qh瓬����YmV(�:�TT� #����5�ji<����W�qzS"�=��&3� =��3�Q��¥ԏX��!��ƥX>u ,S6Y�x'W����87��X�>�a�2�Ù�,�ߧ��,��\*�57���S���-��5V��X%Sn8���fL,1v��Aع�%�� �C L�0W�-c烪@��D���\*�+�;�M ��03 ��e���Mj&�� D�p���E���ww�� \*��;g&�r��s�g-Z�h��^E��jO�c� w�b��'���c7��\*f\[8��z�ׇI��r,��,v����|u:�e+�0��5��Wɠg���4��p5��ٳ�/}���tO(�n�<����f1�=� b����T0����YJ\]�H�� ����+�Ľ-�4�L��?ƾQޮ>DD�� 9ڳ\\'�(����t�٥e��w�R�˅}0���7��eN������k�h�pxt���{�|mX�q��"����h7�/��f!bT�;2W����T~Rr��,8��b2���B>V������rސ=����ql҇Ȅ�K�g W\]�?�� O0��L�פ��)��~�}Ջӣ�V<�OJ|���c(T�.����/6�>��Tv�)���S��R�(Z6�f��n�퍪�a�{�Ex���+FF�A������ ����ӺP�L�������{�O��:�'A���̿����7�L\*�8%#|�y)t�yn�MvD�pp�\]�\[I D4��\*4Ʉ��Ӗ�W��;���7���N�D�i BqH0�9� \`l��\`���(��ag� � p��R�U\]�Q:�{�\_QI��܇���@2��tƭeŐrM���i��\\+564x�aF�3n�kIr���6c;:,;D/"2Ȇ�R���Ѝ�S�쪊Ӽ�uyս��t��T�5"S���� �����2OGn�4��PKS!�KJ�\]�/)8�\]/�^Fhb�U ��y�a���dk���L��9�����=��ߛx�����f�b^��&�8���# ��xn Xp���z��?��Y�;דd� T����g� Ƣ��,�jӞ�Z��@ ��o�i�\_�#ض2'�:Xw ���g0s\\,�8Ru�RN��Ư�&dE��|�3��vԕ���\\�k��G�"���=Gq ��>�lr,-���/��s�ӿ�v�4��l�ʼnӤ��-�ZJ��˼/���@i���ˤ3���#� �-��4��<�p"L�\_�'�| x���u�Q�\]�ZD\`c�}S��9��l�K���'��a6����e�g� Rzo���!���D^U�z� j��-M�A~�,��&���F�3��mفDLj\]�����Zs6�:��$c-����N��1Y#��Ta6J �(��\[7w;�E�KMd��t��k�3<�i�5�&�2�:�l�w�P�n�(�q�� �/-�?LI(B({�b��$�H�Uϲ��KU��vP5�^��vS��xD��׈�4��JN���pr�HT!J���A��)X~� �v� �¹�9ƏO�� ,\_uz��:6��8u@P P{N�I�(����g;lS��W�}���:\]�t��� ��ϯ�9җ/�^���g�HTډ>y�e7le�z�b!Ů��m�u�OjIN7�lo�zl6ߧ;�By0�c��;��7y��LbY��R��c��>ܓ=�p��@��9�#KA85��K=� 'b \` ��p�����l��IV� �\\T\]�K oZ�|cڥ����W����hV�v��S�H���)sB~\`�2C(TW��e� �.а�O�HFԣPn�O1�UΘE - �s��P<��疢�BkI������ �9�d����� ��w����x�ax�\[:���I8rU�\]��?.�!q ��a W���}�D �\_��ƺ�����c�N66\[́� 4�E��R ű3J�|�"�8��� t/��\\�Q 6\*�����BDhm���A��U�����kp& �\]>7b�b���mun�r k �J�-��d�̽��hE�Y�,-�n�?�� s)�əƟ���K��s���jhWj:3�0�g�7iZ����\`}�|@�.�q�5�|���Q0g��M��7�{|(�A����|V���߹G����1e�C���=$X�6�X�xJr�}�#B�~x�\*��J���-��QL|�\]�jI t�9��w +�l����~�Q�ևymɌ3��}!�:\\0e}?(�4�sI/���$��C��4������IIP?��a��#�V�QO2u� J\\5�$�tvnv���\`<1�R�i1���� ���@�.��q\[""Y��L�+���� ��I���;}�O# �\\"���zj4����pև�W���O�7�r�@��N7� ay��\_I�5����C���.{w��\[�rn���uL�Fo���k\]�~{8��mmHwtuȸ��Y"����k7��o\`�&'͐�=h� ��j<�X+�m& ���Zx~�K\].%ԁ��{�.~#EE�C��&8����L,?As� W�5/�X\\��E�9P隺y,�E�Q���uV��U�uL� �� Q~��d�.���P˫����s?m�>�hT��3���AY M�٥���cl�B\*�\\�#<<" ��9o���h�mA�b�/)e�h��A�S\[��F�=? ���S�mpji���nX���bþ\_�!��Q6����6G��Ğ"�N�̱�%���v8��T+�AVĚ�L�i��;y"�\`%+��1� {��뇐�jϬ�q�\*b��0N�����B�����)RO�O��7������ȩg�x�Y���'��\]�8��4������P����� ��;-i!�u�&�9K�aƴ�����鏑\`~�\_��T4�BoLE/\[��{��|x�ӷZjڳIR@)D�|���!������G��f�K\`��}5Y\\���o��j�x��r��������&�:�=P�0DN��\*Q��hGBV7�2U�����b�O������q>7��⧂�C0J�^}�2�)�FI��e}��L�7� S�s�l~��h� nx'&��"E.��7w��V )�. (���.Y�e^"�:?E���ȷ�D�E�y1 �4�A��X\_���u�8��y��{a�~�{��qt\]��V�ظނ������� ��ߍ?���q��3ݣ��u�B��.��Kqn�"�U�iC-9��C�� 2��{���X��}�l��Gzy�L0\`�:j���1�t�82�t��V'fHV�>\]��q���4�)��9�a��zO2e�F�M�g�����i�Ӏ��82T�J �b<���msç4lS��I\\h��X46��֮5�H��:$I\_U���"h��A÷fr�:Ʒg��o6��:J��Lu�\`�R �׍BK1S��З�<#��QW�����n�s\[�&��K+�� a��{Na�l�YV.!5K�(��V�08X�pw(@�L�h��}�yeπS�X��J�7���\`��n^oJIY-���lq��\]�5�69�p>@UP,���\[dO�Aȸ�X@��-��� �\[<J���1n,YYW�8�g����&7�xZ����2G����l�B����׷.��ɝ:!h�L��7}�A�}-� ��E�k���9����/�I�J���P���Gu��B�Ec�\[>�Rl#ʽ+�P�j���\_�"��%y\[z��+�����C�&fS%nI\*e' ߫����R��R��8��\_\*SN6J� �˖Z��1�b��l6��g��t<=���xi�L���ˉ�q\[�ː�#:T���B\[�BM٫d>�����:�?��?�\\$1V��ˆ �HO��K�ѹ�U0V����1\`\*�)�O�����!� rit�%Zj����tE4�(ya��խ�-�;�"��g^w'�6)5�^��g�/��ǩ�xv;����3t���T�Pͣ���(��;�!�I&c\_���AFp�Vn!R��Cy�}��j��o���@��Q�c.�q�^.�m�| y�b,��)�W�y�%�3��u(xgr��7�NZ&�/�y��G����"��W��6��T�!��V�} \*�\\�j'<�af��a'�\*�RQ ۅ�3�q�wCxr)#'�u'�#�����K��K���ZH�N����MI����4u�@?�)\[pUƪן�����˖0Yq��dY�������� ��~�>�nޛ{��b��1D��U9����@�C\\��9�\]�;/�@�� �57�J\[�|���K�������9�D|T?jP�m�D�|�5��~} wmT��1Cs�Q�0���;τ)d(��}���>� 7����s�j�u���¦�?��87�adW��Y���̡�S� CȲ� /NY4�q��ݟ���C�����:�3��z���&�����N�H(������zq�\\���\*7P���+���qC�)��e�ɜ�}� �Z$���B��h:�O4��ö́nBQ�WX��ztF|~{ɰv�vZ��rb o�:��Q�U {Z��4=�e/7��?xv��Vt���s��4�>q)yK�ǂ��\]���6 �Aun�K{�A(~\` �6II��{q�S���>.3�7?g�����hR�%�D^�^{�7�S1kA�ٟ �9V�\`9�3}�v����� �qQ���� �&!dq��1Ԥ2�5#���^�����a��:dх�퍟�����Ol��m�2.## ���Mқ 9��ã\]������ ��%p8/�f��X��1�;>�Cù��Q3�3^��1(�c b'�0a(т� H+9��^����)���� ?�e\[�5\`۸�ڊdQV�g��\]+W��|�d,�w��$��|h�.=t���56O�+�AN����:/\_�ù9�!8c�\[,+6��J/� f����f m�,��5�.u�������v�L8zi{�D)˭�K��v�܇�Th��B�N\`U�����6X�1� �uٰ�$�&Ì�Z@�\[Ѷg&{i��7(�J�)C-Y ;����'晊:mތʅ��;��4�nkI"��l��O�6%yn��F�\`@2xBD��3+��I�#�"����({S��V����b�6"��e����G�XӸ�\]\]��{���@�y�wVzVY�'�-��O�WyNړ����Z�v���� �ŷ�\_|�"��s�Et��f��W�����l\[�{Vs�O^��l�4�h!�#�E?S(看���Ur."��y�D=�A��25D�\]���N�7�pϗ�+�;i��Q��Ek�4# ;���:��zz\]Ɯ�c���:W��n1�d����W\*�����w�ە�f\_��В@^���2��b$����o����\_�3\]ax�^Aǡ3��=�Hf�,2P|�)fY� %��eEa�l���'�6������J�z|���Q�e��� �m�P��@b���Ѵ�wb^�'%C � ���.D��34& /�9g�@̀�&��\\Q\]��쭱0P׻+������E�Ay4��B��3隮\`�� Hƣ�"�n�=r�DH.un\*^SF"�����0\[�J��\\p�Iݱ�d��B+(����\]��g6{��7\\J�Q�E� �'�T���a�-ΩNjʆt\\T�M\[v����9�<���"E�\*���Kr�AW:f�f�15���ip�>.(�wk�Ai�$��NHQ��6���\_4�jt{\_czU1}�wZ�1=��������иnr!HOa��y'��~o�8����O��1�J��I p�NYv�I�V�ш������ +\[���f�|��jۆ/}Lf��\`�%|B�� +7;�OMf,/Jܓ.$�� }��깽��C����n���&&и�<� L���d/|0��ȋ���'pe� ��4�n�|J���ez�yS���\[���D$zZ�H�\_��);ۏ&�9�����N�֒?��twV(U�<�@��^���\\Ѵ,��\*zM\`�uP+� ����ܮ8ߖ�v�X� S/�>�g�Yz��� '�\\�F͒w����4����Uf7G6�Ó\\V�N�3� D�w����M;�����ռ(I�dR�ǕӠ��\\ �%\\b�7�'��Z���772�p+�7�\*dz����%��nT3�˿ Zu�9�͸ӊ������y�ԛ1?�+��gf�|=�=\\���=�3�sHj@�'S7�\`��kʂ�����L#�}�K�V�HG1�� ��,�>-H���N���%}����D�x)$b��A��/��6���E3�L�{��3����9�˿���t\`��;i�(��u�ֆ'�\\wy�j)Z���\]����:2y�=�-xF�ڳ{;�DQK@��~ܡ݋����� �t�H1���us��>�8���K{�Ovg��.y4.^ێ����6Jɉ.-˝��b:�.q�ʈ yVM\`���cu�L�hO݄� �ֲ����D�����!�I��l}!�.c��u�P^nL�R ��ðU �9c������Ba�v�c��t�)��pt���Q� ;J��UGƝ��Ξ�O���\`��J�c�A�4'O�,�<�\]idl�w���� =��hU�\`@b���Jկ��&�Hu�c.C68������􁈮x���+\\A ���0�f�y��Z��#��hot���UYr����pM1Գ�ފ<����a\*&e�����'���n8��� ��)�X�n���K@ho����3�I���-���.��ah��cT��hQ��o\]��\_��� M& P/�WEF�鑔n\`v���Я�#�!�u\*����P:���p$��T+�"�Oa� �\[2K����~��R�~���y��|��"��K�~ ��o�y$ޞ��4E|�F�S^��^sa�����gh�X���:��Z�� ���}�M�~�k�xH�?1�M�^)ʩ�3� E��&\_��~�S4�p�e�p3� M'�"��͓�/+�(ܫ���?lR�Q�:��d������Tk�^���\[��\[V\_��44�c�<�ʧ��I>���ޔZ9���<��i��g�8���Bk\*C"p<���M?O��ro)���Dő���{\]� �-��E�;{w��K�M�L�ˇ�����% �>�VB\\��;@#�\\X\\�L���xH(gwz�Z2S|�T��nl�9y�i��#�Z���>V�E�vD�;XH�"��!�И<O\] �.F��jn89�m �(�1Rb�T�3S��B:F�� ���\`dw�Ū�;q�h� �.M�^"�|/KV�!?L�E�O �T�˓�<'\]������7G\*j�����BA��Y��h �z�L9t�rL���D!��o�;/(����z�|3f�8o�o�:��ਵ\\\[��N� ��S\[�D��EL��3m�����=�Z�2��ž�m������v�4A��=�� k.�eI�\`��by㧰 �9����� \] �+OZ��\`O��Π��1�ޓ�Ǹ�~�t��Q1�vB��)ܤ��2,ů��B �����Ǘ�JP�b�.��8�oJ�.W�����|�/7�8�ȜԜ����z\_�@�XcA����u \`꒏����P���k����Y��gE���,t��-��1�V�-�x �"����r� F˜�s|�B���>�l?# q/#���FXFM�����@��\*�v���͕��H�o:8�+4�?9ą��z�bk-1�ۓ�ˇ�q� �����x�Zu���#;>\]���p̸�g �28���L��<�⽾�絏8޻�طx��hr�ֳ���/r=�Goٗ��Pލߐ�(��) �C�Gn���Ɣ�ƒy���ER$�\\�>܋���G�R�7q�mU���P�z'��%<�\]�F,T�� ��/�@��5��珿����W�����r+�Q�����)A$o�U���L���k��f��\]�čM�r/~4����t��SV���n���AΨ)��DU5ZE+ ( K��o�� �zGn�oI��}�89�n��,��Q�QN�@&e��w�^d��ɔVqLm�ё��#�����S�{�b\[oD�Y�\`8�ߡn��f��s�A���oV%������q�s,W����^K���$l&alQsy�N�\_�����|F���ۊH�z=�mX�Do쾙%YAT0�" Ӡ��S=M9����B8m�䫚���2ʽ\];�z���É�N!\*���so?���S�+M����g�7� �VT"�bU�8�L�"�Q\[�:��\`���y��.����W���Z�OȚ%ҕƅ���0�=lW��A�ǵ�;��4�����wxLh�aQRʕz�')ĴL�sst�\*~J\[ ��G���d���bs��ߓ�����&⟶��I�h�\\��D���&�FݫN�\*51�/�v�VdO��\[x#,�HEdo1Ȣ(\*���!{�Y99�xY%��1�^�V��E�S����,1�&����(>�EB�o��j��Hu����^�y��չ���ɥ���M������{XoZ,p��,�Qg2��z?����>�7�؝h�L�H|Ͽ2��Q+�<��d�+z� j1v5�d�B�W<�#�7���!n �ֵ��X�F:��� ���Q�-�������y�ذw�@�:���κ�3���2�E��Eh�p�r��\_�I�i�Kʙ���m\[$����T>l�?oy�U�r8�2���#K)���қ����B?:́�w0�37$ז���+��3j���SY����M�6�#J���o���?���� X��:�j!��Z�/q���&i�r2O�n��n����Z�ۆD/ŭ����cG��n9Щ�a��� A�%x��%u#��:� ����0vm����Ŗ"<�=�\`���5\]4�Cա��5���#J0(=�T�;�k� j�&xU Lf\*x�yj�F�Ţk�������痧4�%oQ�e�l��� p�&16�p��͙����1aO��{��΍��Ѥ�FٟA\]� �6�\`ܪ�~��9�lI�(�k���9,��A$Y� tW�W�\]��X�P8 Q\`I���L�7\*)y�uĴ�Ά& #�\*���?)\]w��v�Kl fPM��u\_U�%q����l��(�����r��v�U�.�� s�O%���Ӆ������p�MI\*��FRH�+� ˙^H-��r�C4����%��d %����'���Nْ�������(�i� s� $�lC�T���;U��ć� o����z�>�?��0�"����w�p �Y�6���y�NS���r�ZխO�+�6��x\]����-͞0�p��q��ɻ��o�B�x,u�B��(qCD��f���Z{��=��R��h� Pu ��f��1m{&��7C�'覭Wu�W<�$��y}ѓ�}�1�n'�\\�kы�i�{��C�#�R7����0H�9 <�Ů��oҕ��/9��w�w��EH�N�� #�-�4�/��R��v���/����Cx�U��!7�)y�l����㬧���9J���������i�E�Z�� >뼮1R�Nj� \]�/�� �,����3qq#��}�&�Ġ���Nƹ�$�PP���-\]�@<���3~W\\j���PȋL��'�T��-���p\*-�^��2MG�F/����s�������W���Wd�0e���k� ������("��qB�ڝ�Zt�N��(�5�2I��,��B�Z�\*�h�MUºx��c��(�/Θ��� ��yFr(p�~��.w���A�f��nH;閵E���H)�U(���8On���ϙ!lB���\_��8��-�s@�ʧy�Q��ĝ:ڈ!���>ؘ�dQ$�$P%�~�����S%�T�q�O� ��n���\_- �K�g��Ì���3��;�y@��%d�M�R Co �fFX e�i�%nB���������ER��N��;ݤ}=�K7R�1!���0�q�����R4���S��-NoC��n�����R���()k�'k\]��h�� \[�K�\]f�#:$\\�g\[��0B�0J3�G;����4�����<���.�s�؂�� �\_�w^YO�z���(�Ѐ1w�tVm�����.w��������ek���\_�o ��{b��3z����~�L���s\*�~zuD� r&��r#)����зo�/��Ɉ� rI������@3�fȚ�j���M�������cC���+���uԃ$�K�Cn�q��^|��T(yY8 )�嬍7icu ����WXĠc$�pr�5=�q�\_vW�!�����^E��Ig��d��B)�v�m��/��'4��aA뜵H3���\[��x\*���4�P���^��� ��E�7�:/���A����}����Ä�J�k!�+'T ���s5쿓���y����4�����x C��7� .:�� uq�z�6�|5E�JWO&tb��^�F�Z�712�M\[d-Z�E5O�M��\_.O�A�jF�A�Vٙ�c��4-��-\`r0'��VѢ��Jtl�������!���n�s �����\[4�Q�<%%Y����\*,�\]&��ZR��&���G� �^�Gl�"c l�5�%L����񹸓F0�?� ��4��y\[�F��jw)Q�oڴz &�� ��.)u�s�>�VVp�KB\[�y4�\`n�(�� ���tb�T\[��j�Qȟ�(�;t�+K-��r�G�����S�-�2��f������<�#AN�4w��/��\*����/?h�B����kv�R�ǖ.y��U�!�y)�S�j�cQ�z����Y����8��X������\`C�I���{?4����tTN%��sӊ\[\_��r�?YR��hA3-���=�!x�ُm�O�^p��0�#Y�66�����֦��\_��4ˆ��? ����-���蘽q5D����M7�oi���X1D��h����gc"x� �M�jR���iH�aW ��\]�����4�@�뗀�.�1�ؿ� ^���l�~�' �d��Rߕ� y)���p�����R� ��a�STA�R�W��3�a���|+���R�q��Q?x�����Lr�^�$5�� U��o�T�A��p1qG�pП@�6%�\\���&i�a\[�4,}嫸@�Z�ɨ�+���#Ʒ��e���l��h�Me���W2X�K�n����:nf���Z(B�Ha5��t�;�#����u'�(F�A��(ic�)a9�!g�2�<��QVC滀m�=;�4 ��R���O֢x9��N�J�����0�����▪(�\`�s\_I����E�oHIM� ތ� ��#���M<���#3�B7B� �B׍b�=��Ԣ<�����Ci��2�#L7gM�/ɡ���Vz�t\*�X�H����-���fBM�G���'.���5\]y�qY��l��i�o��� ��M�fEMMP����/�W�,�%�K.^�S�fۥۑ\[>� n\]@���z��k �G�y�� �E���1�N8�8h2��V٨s��#oڃ �: C󔐫c9\]��ѰG����zu��\`i|o@j���@�"ӷ������?é�S���-,�s����j�߯s���H�J̙��z��\]�:� ��!� �>mb&p�\`�/�q��ոd��&L8�z���R� �h0��2(#��^�u�}��z�Gm �2��\[q��T�fAKTC��:��K�.������\]-~����� әQ"-U|�����U��\\\]��Ő|� .T������ZY�S�b��-�5�6�w&����H(��w�+�E������,qM>@��D����&EK��j��4# G��f�w1j,�D(���J�h%aA!$�����d��\` -�ZfJ{~���r�M��JW�4-��.Yz3q��'Ԩܻ�3�e��� �AqA�\*������5��B �Op.���������X������<�q|V��x�e}���PV�(f��G���6R�� ��Pp��1���:�<�|�Լ�'W�W��V�m\`&���~:�.q�4$��q�ֽ�� 4�9aev��� x���a� :�j�&B�H�ޫ��FT,�\*e( b�5^n#��׹8Wo����)'��E|�k f/��(�P'��¾���sI���A�-�L�\`JN}$���\`+��ߓc5�؞&�O��}��\[֊�}}�b�JWV=�Bc 0x�f�r4�K��}Ǭ�͂�w�{����/�%G����$�r9�/ͪ���6\\T|<la��^Z�%�xc��4-��#g ���3�j|�8�ǂ)�LK�y���ie���l�u�b�Yo� ���M?��.�M<\`9��.&Sj��xtz�7 (p0D�t�@��lތ<��� Yo3��ק�ޛc�UVN{i��|�c����O�D�E0��㿧� T\];���k��D�T�E� ����!ƻ�?A��F\\����W6��{8��B�?!��6O��ݢ}\[�3��;\[�t�ߢ�d����2J��vfձ�������)�A� �ܽ ��Lj,��&�\`dt���Z�O}.%\`R4�t��i���j����m�-@�q Q�����Nv��h#�pFZZ-P)e�{w�Y��� ��{�i��k�X��Bn�r��l���%olm(�p�~؃h�Qᜍ��򥋠����^0�K� �c?}7W���/�U�x��\*(��=G�4 �Hk���\[��E��eȂ79\_�r���=�'}�\_�<��v���t޿% �G�ĶGd��d� � �<�KfqEw�t���-j�"v ^�N�K�Άwb�BJ� ��c"2 ��/��Tʜ�� �mM����!��;$�FD��� ��.y�|xW�L�Ŋ��5hʛM��� �b�>\_��I �|�嚭��cj���m4�G���\\����\_��t S��.2t1S�u!��ؓ��e2\`�+�d����@���ڎD���T��\[ z�Ο�u���;��κ���Ͽ�+oƙ�t���l�i��U%��k-��=�ypte/}��߾P��:���(�v Tj�2g����.I�ʦ���W�!�uS~ ��nR�qѪ!DT���҉̃��\`ں�A(���d@\]�Y��ylOC�����q&=юp�n��pM�;��u��l^�� P���%X��B����5�#��&���j�����lv��#ø�z�y���#F����-�li%م8$�x�(�sʝy�%귔��\\�Xr�yy�SP�Qs�����������~�@/��YZ8�{8U���Z,nc���c�@9�j|I���8�<880��Sy9�|1�gU��d��l�#���u����� | z��'�\[��Hqo��\`Z�KfP�M���P�� \*\_J {ʶ �q���p�wJ���'�-J���J=���:gMI8 ���I� :�}17 � x�%��o�{2n|ЖEE ��z�7 �?�\]��� �^{�\]������h\\�QJ��SPi�Y�N4���^�S\*�� ��͓M��VJ\*�L�!�q���4��57c��"��� ���A��i��������Q��fC�K1���GG�p���1{��z�~~�1��OK�hߚ��Wʻ��!�W���2 ����8�J��v���W�4�QW5��w�5�a�7P��7� H!F:݈��\`b���kf��C aʐ��w��R|9�Ğ�F���!������nڌ8�ͽ�\`�T�H�w������µ�V�h��yXj��g�w$|�th��q�IX��+\*w�)ð��oOpNj��{f}��R8X�����/��O+G�\\��\\����g�+ �|rl}��c�S1��oo�j�'�{Y�c�67C/xH{-^\`�<�S ~u\\�x>di�G������50N�VMo9���\`c�1�Y������\[�����:0�bg��.�f�� F.N�J\]��o҅�!w�Ł��&s x��#Rm� @k��  �!r����m��ǐ�X�C �qs�q���6�T|�LOO߇U/ctT �ͺ��Ʈ���i2@Zk�m��^�'�nD �$pn�^WϬNP��w0 Ɋ�հb��D��T�b�/,�&�C@�\[�H�3#���¥�Z.���'O@��qr��$��O��H�9�?��=�Bב�<�sX��a��Q� o�@��fx�V-�rRB�U�2JPAB��h�F�3F\]�G}R�h�m�U�\`S=N��f�cf�/e�.�|��G\[\_�SXç�V�h��w&"T\*�o�YU����H�(����T�:yf����P����t��l�9Q�ݲ��Kr�k 7c��?\]�/�l3�ev��L�����Ec}}s�&������"�|쭵G ̂����kF���J�ӷ{�v�z���'��\\x�F��7��q��Z!j���:�/����I-��e\_� �i�'�V~(������ܘ�S��yΙ�����D���f��%��+k��D�� �\_�R�\_2�,�d�H �� g�B��5g��3{�r��֝���)�Z���X��=������l��\[{�=>���@��ޯQ�і������O.C�\[l��"N�J���~-z�{9{7vdTnc7�c�,d���H����bjtQ�V��uW�S�Z���OC�y�o�w�����#�ȿ\\|�� grT�����P�$t��Ii�r���� ��)�X� J\_9�d���97�O+��h�s\[�x�2��%���䵕�缐{�y��C�\[��xS:׀�����^�Z{� s�a�\`�fA��+�x+�R���tD%X\*� �7�^�ClA�u>{�Dj�o/�\]���r{��.z�&R�n���v����"nU���4�/�\_,���U�UJ��c^�.��$�#�E7�����9G.6��ú����wˊ����\`��УS�S�I�mR� g�Uѡ��P� cÉM�"�"���K�"�&�\\ﵬf��\\Pv���J�' R��-�Z<=�c������; �:X��0���h�;t�� a�������<-KEO6�snY��ra��q�Pc�}�>��Z����/�}:)��j�#�����V�Q�\*a"�Ui�� '(!N������\]��m�)Q,}��F�k0fL����ɥ V ׫ \]T�l \[����z����L�� �~%�2J�d?Z�\]�M��4o�s׉�j �W��B��\_\]�O�/�����lWޜ��|�nTl�6�-C�\]ڹ\*����P��;�f����L�F���oӪ�"�a�\_˅�2���z����� nog��ep�)To8M �x�zB\\�M�"�j�\]�Mק��TG�stR���Q�:�Ȩ썝U#ʯ��\`�� �E-���&R~#H0���$�������%� i�}L< Ȩ��atow�i�P�x�ݜ����ns\[U���jnK���PXR�����8)T�D�=�Pl%�G��ئ!\]�+��F\_��8�����n�J���;��3�J �ϫ"���Ʊg�L�H#��8��6�g���r�.��"\*��m$��%1����(�h��&�%�6��ƈ/�7rsG�P��Z4�m)��t�q�y#:J�k�OoqR�Ol�j�d�F����Z�\`�vRǷ!OQ��f4�@��7��)��+ g�6�z{⿑!�����:+\]n�$� ��%o�����A�v΅��7��3j�pWr����x�2���QD6�#9:N�E���b��M32D�Xm��F푾Mtz���J�p�;;\\�։g����F��Ŧ��h�̝�DPEA!��������\[ј�4���(�'��w��(M?�q�y�yR� �I���Y�Ш�ڜ� ;9�I'b�Yn��S/\[^���s�s�Y� (����R�!��9XyK!��d��.�e�0����i�;b�\_58�5\[E6 ?�K:R/;��+�H �NR՝���\*�,C7�,^���|rQQ�x��G=����c���3�YBs��;�������v���$����5iT��|��.�����k �� �ݗ0IDOG��{?�ו֢�lr� ��$#��/����h�s����D�g}@�Yu�o�,��Ek�8��𐊜6(�N�G�#w !�V��7�\]���!��ټT��l�9�^���K<�U�/"@ b)1����-QU��2 |kT=+�@�x�N������S��f�t��=��!���v��j�K��\[�l\\<�O�H��O�p(��\\�2���N�\`lo�(�k:�~ ��5mk��UJ\]��o���=\\�\\� ��=ʬB���+�;��� ʗ�#RVi /I2�m��>�!p����~ �b�����&'�eNs��A�2�#/��|2c���˸S�f���IO����x#�#�����(��f!�T�,��j��>����(�/P�:�����K(���= 9~��DW�<~���+k�P��\*����2\_B����|m#-40� �q�ܧ^r��ki\_M�XTPa����'��\*)�GW?����2툏֡;��:B��i�����i�z�n�v�5(�W�5B,\]���HUzH��.��?(\`1��F;��{��gx�x���M\\����>o� x�!?ke�'N��l8�5��p#0�Mjx\*x&U?�#jM��xnM��'�I���pt��탛 +����.Z��iD�1>r�3o�e�4���Uc�r7=@yE�P&�\*uE�tt0P�V��lOX 戺h\\���wݦn�\]������z�/h8n��|����|�e!-�LL �w�g��î5�䉽����pJ�7M!#ޱ�Z� ��i�La��pl��Ȝ8>��nZP3%�0�C���A�|y ��q���\]@��0#ϡ�kE�o\*\[\_ >�0-J�g8�L�@����ܣ�?���ǟd/Ș����Q1�w�+���眴\_�N|f'Uz�l9q��t&���ć�A�eG��%�pF"ێ�n��mK�!�o���"���W4�=IejF�� �����}��O��\[(�i���i@�4X\_84�,�"$?\[�H7Z<�G�}��r ���3��+��nwp�F7$8�ڭ��������� N��G��I6�E���\\\\VQh���39H�>�Wt�\]��1�٪���J�Wmx�P3��n(���|�\[c���ddg���Q�7�r�� �0���2B����f�Z���� O� �=Cv�䂂�D�N;V�LJ5���b6�G ��H�~0D�'n;Cxe�Oh�5�b^�p�܍Vt�!����=���m~d!/0 ������,�������+,�P�Frg��%��oԛ��H��y�q�i!&�&�b+�z���O��SP��\\崑ɾ\`m⥧��\[��X>�������2P8�� ɽ�~ (c6�j���-�TSP3�4 Us�ЕҤ8N\[� iW{�����P�-���e�?�(%��r��x�o>�~����A�|����<���#�����Z!Ь�: S��Z��3�����2,�<i�qv\\ ��.��Ԛ�JK""S�b�YTd�A�\]Tv��?��\[�I�.������a2~��DO>\]D �DD��kd9�����Q썘H�����1�\[�pÓ)?� 2'+ړƲU�v� Wfk��?E�nJ�����IsP�S��x�!t��0����H��b\]��䁀��'�Q��|�+����fM��� ht!'+��\]\]�����Ld·u�����EA�1��� �\`����D�B�g�'��QR'��:2 {�IBG�3�N��d6F�����gӭH\\�I�h�I���1��uv�;�G\]̅l�cJ,�\]ƥ��J�3;T�ۋ�aڜ(Q�Z� n�#O�bDKʅP9�\]��l��v�v\]�<\`¢�l�q߀ϩ�;�jNW�|���������;G.Ac�b�V� V�t�d+���f F��u�L����?���4G\_�47���pt0����l tD��������| �\\Ii\_7��8����E�Eΰ����N�\]���"쿝Jٜ�h\_�~Gݤ~="zTH,�P1�d�~����Iޥ�F��EY��Mm"�'j �\_E Ø%�-�Ə9��G��-d�C6w JP\_�i\`T� \]}�u�:S6�{ԥF� �9�6��^����ە,�"��0��e ˺��N���n���C���L��K �k���x�\`��mGݭn=D����.D6L\*|���i��l\*Lkt��!0�D�5y�\_4�w�f=��L�:����a�L�x�Î;�gؘ���?5!>�\*äE�t�1�F��7e{K�v���3�4������-30�a���"�\_��/�Dzp�WhR6���U?mU�4��GgQ�H�=��a���ޭ��{�TAl�g%&>U4�.������7�� �M}v�e3On����� ܃;h|j\\��66 6�������rA-�q�T�u�;���jJ�Λ�Q�\_DZj������¤Rl\[Ϊ"��XN�ďZ��Ivp�h����� ��~����5�ɣ��G�y�$b��s>� �۟~RezV��bv{����r��� �����#���&��wɯ��lh��>�z�lw�! �&l���R�Ԍ�Ie�k�ly�x�J��|lQ�P����0��T�\`FQ7�lH\*��0�7�Sd�;#��s�|?�P�!��T!���ߜ�/�șL��~�ٻ\]S&�d}1Ģ}\\3�pUM� ڭe�\_Jq�.̼�o����I����fD4$�:ř 'Z�9+X)^;���Mv��F� ���@���C�� =����(Kf�F��V�o��� \*� c���i˳X����o)wbگ�u��1e�1�\]�D�"�$U�/p�I}��Y����7�చ�k ����v�5�N��(\\6�����4H o!��@U�L�4���Y�Y5�X\*\\���d�K�(ST9�&?�(e�PMn~����@�)j���B>�}h�@<��d{���qg\\ �\*^��ʖ C8-��e�+���( ^д-R�bҠ�}��|��(T��Bx�y>���%7-�ρ ��\_{4?�^(�.�� �Zav�� ;��V�\[���6�w~?y@�l�%X�A�o|�X!�#i����N{i����S�le���rc�o?�)z���0g�<@�<�\\‰2uԚ����v�����77��!l�H�z�#j��NjH��B��T���+-��X��>c�k��;rK;ܡ��2j�a!��k��do?�g"o~2���׾t�l������I�۩pb���(��Ip� �U���@>����.϶>��N�\]T���@���uFb���rT l ���N'�I��#��w:L//�\`G o�:��!o�Y��DM�?rYi�6P����eO�\\�N����D�B�����"��k+!�g��FQ�v�k�I���k�����.B�ȋ/�(VJܤ �c�iU������O�U.��~�-y�^�\*����H�p|7���$���q @�%I\]�5uw�y�&ʩᅸ먠�OYhg���+�C �c�9v�Zb>�<�i�\`Az�ΰ |�v�gw��H�0���� ٧��3�\[��T;%�s�w�FU��c�W�٥�ܜdU��D��Oy9��vر����%�A�8�O�.��c�$�PM�qg���Aj�k��\_�\*@Q��\]t�\_7WͪC���-�9\]��P �~,(Hd�F\[-<{|���2\]�����,��\*���.D�k�&�����p��\`��K2Y4HI3���\]���(x�E���q��##A �Е�!� 2 A�� ��U���� "�� #�}���&IQ߀4�lT�A����}U����~'� ��Z�M�^Q��ԯ���~�հ��~�3�v��G��ea�����e��q�dz ��N��3�j)�\[���A�Q�>^��h�o.�\]bqNɤ��G�=����ź�\]��3�ߖ2Ű2R����B�}~d��W٢����)�Dp��z�=���/���������4јV$n�� �o�u���&�k\`��ZZqjϾ:��1�{Y�Q�S�=LV�rx���=\_�1�pac�6��g���I�CCE�F�tώF6r�� ��1'\*���3N.Jo �}h�7���/;�Qp��Q3+�1v.��r�|���|��a�K��2�طkeM���}0���ھ�8��A����O)�,�� ,1�u��X���\_�ƦQ{�w�w:�)\`�^��A��Smk�� ��\*U�%A���R+\\�Xb�\*gښݧ� Q@���%��K�9o� �4���B��0�f�H�l/�� b�!5w�������r��\*��/��=�Fv��9��k�5� .�c"�o���V�!�\[�2Z�nn�t���r�@�-�o��\_� ��b�Ӱ�� �2���xΨ\_MU|��Χ�\[�� E�qМ�Z$^Oy�5�N,���U�|�����&�g.���='��I�\_��4��NH�ܻ' H���9DR�%ij?�E��0�Ƃ�\*��gp��(��J�sI��M���R� ��U7��o!����'�&��^���/�!l��WQ�|pA�p�)N��b!��TM�a���o�@���-��sN�{(��m������@�D�d��Ѐ˜��EL?�sg�LF�������U����~&~��;\`�y�R9���p$ݥR��=�Y�� �-v����W٥a�{@�̦4���.8��k���)�B�,�Dw��+H�L������Q8Ҩ���0�૏�O�ؼܡ4S�w9�vjơis��F���Sp��2�n�'���tB� 5� ��{�?�:y�9�=��3\_x��G��!s��{�W�I��H��6�8 N�E�>������;x��ԍb��Ș�$���O\]��ES�#L��G�z����!E+�$lSI #��Ώ�q\_���UCA\_��K�{�}�%��O.�ʼnIM���˟J<�$��ר-�9cʗ��g�����\\��ڽ�;����yi�@nN��/P�ɲ���Ft��ś�V�^��r�O�q���Ć�� 3l����\*\*�7� HP�����v�ym� �H��������Dݐ,+� ��a¶��l��\\��ك2N���+Bq��{\]����r<6��61ޫ:���� ��^d�X9��6�u�B�9��Y���� ���M����h��a�I�U�?H�e�E/���X:�uܿ���p�{V�\*:��i��i�I ����8ˍ @�I�|���L�n��ޔ��h�н�O"�c�8��)G�D\*�\]� fu|�ٌ��C;'��6h � 5 -� ɑot �E��{�A ä��\]!�b�GW~@����n<Է�����څ}� �di�Lu�%�&'1/��\\���\[ Ѻ��jc�Ӱ�o&t��X"(y�Y�q��LŞBa��Qr���?ڶ��ZDK2���a�� �Yp�P���Et��,���H�2��J5h�Ұ���wf����� �Q���<�U�9�9����/�^D(�'� C�Ki��{��{aoG)���\_�J���">~(���Sc�\[0����������W����8�+��K!4'B� �N� t�Y�\]��M�<���&����ifˋ�順����d���񪒠+��K�|���G�l�Խ04I&v������,̞j{�s7�����<Πȧӥ܉\]�XRٌ��R�eL1�Y��P���z\` 6�{ �h��%��Q���?=� �\* ;1F�UM��2��볼��vVz{“�2�z�S�P� }��\]t�B��r��p^(Z,�)�����q�\\�}�"�q����Y-rOgj�Џ�}����I�A�9��Jv����S�i����E�&2��K\[7 „�����K\[��NT�����4���m\\�ʏ�q0Ὣ�p �a����ew� ri��|Bs�M����&g�/+�rV�.�F����Q��^�����Bl� ����P�9���w���p)�Ȇ6+�}$�Wq�^��b�HMe+�<\_S\\b8������o�J�j��ѽ��I.�Dv�j�\*�(\[�|4�\*��̬�\]E�I�5)\*� e\_�b��f���l�c�7\`�ܣV�t����ax$���\\��� {��?$�X�iT�u��&�n���jj͓1-���^W�ؐn�4���!�CB̸��PQ��ē��:�+���%v����G�\*�n�|F�H�N\_HoMA4�/�\\�9dN�bq\\�m׉�n%�w��9ɻ�S�f�ū1�ї��ƾ�i�r:��0o\_�֊�����Z�4����32Ɔn����#ֲỆA����H� ����Y�� ���\_���@.).��/� 'D~�O�����(�}�S�����c(��m 8X~n˜�G:��K���bNƞ�iJ<�qq�Z��u-2�,�H�t��Mf~��(��ݹM�p�%s� �b3���5���t b��m� pns��c�\_x�{E�zD�1��Y�u\_��Y�Ѭ�A�OK&�i3m��t�ŁL#�7�'�+#W�i���G�!�s�\`�x7���\]��k'���LjL�W��~�0ؕ�t� )�\]�� ��8��/�Y�>Tk �-���M\\\`ӿM���,K�v\[�����)��:W�w#�!l��h�̎����q�˫����֔�e�0Qz2$�7�<���ˣIMd����W%3Ғ�����3�@�J�M:9���uL�&� y��ϋ6�����QW��% ��ݏϟ�6�����fTV���}@�Х�y\_)ME cף�{���R�� sJq6�\\-�قSm:�4f�D���eJ@�#��@U�G����T��.Z�\_�'� -�� �"{ߥ� �5&A\*ͧmP�\_ �ǖ��G���5Vg�t��\[�礡�r����@z!�^��FS$��oy� ��Xv�OB�@�t��R�7;��K�� x��͓8�koኹ��� O~6�ڝ���v~���z����c�\`��� �a� 8~{�X��{�����J��w�ˌ�^��o�a��c�������7vW��xu�,�v�|�nx�)5��߄\\X ��.^�-�,/��T�s��&����e������Mϖ"~w5E�\_�ܝ�\_���8��hܟ��\[�(:v��^f�����ş毥?�\*���9iQde5�y0{I3����~��S���l�����QQs���-�t����� ɢt�����>i�\`�܀'\_�ol�,7�M��dH~��%��� S��m���w��^r�7�Tu���H6������@�|=�9�i\[Z׳��UV�"���D�)p\]e�"\[Ym�mln�M�a{��\_�y�Y�N�2�3��uG��~U �o,T镦����@� f��'�i��m1#YA��ឬ�K�@��D,���l�a۬?.d�û\_\_&F;JG� Y���W���ng��\]�>pu��}\[ͥ���r�e��|�,&I�74��ܧ��Q\\~��3tO@cdȲR���H��r���� �&�G��x��� /#��4����g�c����"�xn��۶�>?��w����A���D8�J�Mg�Xt{=�L� ���uR��^ ��|+�6P��ʋ����u��Y��$�����T����d}��J�7�Z{���f��YϏ��ɗ�C��m��\*�&�zF����br�ܳ��\`�p�$�VM\[45\[7b?���@���#�rs�Ϻ@@�Ngɹ��'y�rE�@�wm����n��&.w4��!�g���� \\�6;�L�6O�3k�iJm}�(���6يj�����x� �K������\\C$���ݐ��q�W�f(R�p�\]����ג8la�m�(7ذ�Hnla�����&a���Cږ����\[��!XH8��9�aj�y�9��t�iI��N��-nb��蝽�?��}X��?W0���7��4����/�CI��X����4H\]^y�S�����hw��&�\_3�!aݧ+^"��GT����.fEL6��M���9~�\_\`̎�JO9Q���1VoA���o��Kڂ����ޏ~yED�'<�/Qw�k/�pO�'lHVq�� �!\]��%|�s��:�ek/�,4u�b/�C,J\[2��= ܂#�¼���S�¼J�WW$|��.�,�%mV�W:u���)\`����!�E)�F�m��m�f/��O��X >{hW���zm|�^�Y�W�u�w�o�V.�K$C$tN�Z�9��-��~s@uܦ��W���2Z��T~��0����p�j�E: �����Y�2뗹����|��뫏%"@a��Ε9�v�A�r~r� d��u�fU�8�n����D6�K�}�l�N��\[F���R�������g�ߤT�������(Ƭڄ\`W�f| ��c��?���w�8����v���~���̰;1�\\�h��ߥ�#j�(���8i��$���l�G���� %��+�/�D�xF�$,y��j�T�l(Z�����e .�H|�U=�D�z{Lƻ��N�X�$sJ2=�ӎn��8x�2� T&xr}37:�)m��T�- �I����jk� +o�|��� �D�\_M���-�@y�}���CދW��R�$DS�?�G�R�UKqbC��p�l����Wԫ�R�\_�?�:��@�Ev�����|����=�:�4�%S� �tΑR���ɧFq�x\*2��.pP�H� �S5�|�?Qo<���r瑖�sgѵ8���⒈��\]|/!�J���\\�\*���38��\`�y�\]d�����Vm����=\]Q���s5$K�ls"�\[(��;���DM�;,��n"Z�7����$ 5��U��6G�����R�Ʉ�D��#��ˉko��7�1�h���L)��^1 ���{�� #v��T��$^ɀ�����L���|� �|�,$�=+B�3u~i� ƾ�b�>(��|��3Ek�S�4В6|�m��O(��� �|�L�\*Q���9W%�OIB�\`���������Lz� ��%���zU P���9�m':��Ϊy���\*"�.�:i� �m�{�d��A!���E��?Rւ�i/� ��c�+FWI�R�ӛy�'I�"m?�t��������dwK�9 ���Q -��r�R���-!L�c�WH�97Gu u�HF��>��m�~�A��D:l�(͙Π� �,iGy�(�B�pU �8��oF�#�L���i���Œ.��JY8�:6��(��a��\\̌�C�4\` Ő=�� ����B�?C$���R��\_J��DY.���\*����AI4��8.V�WR��Z� � �v�Wt��6E\*Τ��W٥;ꓝ��U��}�ƒE��,��7>�^���)241^���(�ť�\]��u�R�NM�N��t���N<�!�~�e^iS�m$\_���Px�����P)�� fS�\\H���0�4��R;�s\[\_� %�5�N�3 �0�N��Qh�eh�g�W�J�iȸ�I�8o�J«���kv,͉el3��Y�1��M�5)I;HM���qjE5�G�/�(���z\`���C��A���,����n�\`�L���� �$�?(�n.�~'F:�D��/��m~T)�z���-C�A)��>�:����A$�B��ɢ\`b� �v�,�,�D�quh8��R������P�Uw4�k�{ ������۝s|���7�n�/�>\_B����I�1�0�&0���d����F���~�dU�-��c L��T�|���5Q�c�j�Yׂl�q'�: %�/��7Z/��fYb��m��Ѷj7+LBB�3Ȝ�����u����\*�#��������;�C�.�d������5�N�@H����x��W��<3�g3��c��Kh�sr�B�тf<��Ɍ�����D��� �b������l����H�q�X^�U���\\�v�PS�>��Z2�S�D8�q� ��%;K>��w 9 ;��)ո�B+i�?~��rخ���ї �9Z��M�ݧ��ƍ�/��)Pw�^c�j&�9i�#�ۣw��aj�F��pF��ڜ������Ed�nԶ���Ҍ�; ��E� :6'c"���ØAQ���2m��J�(����u�5Hl�Z��w#��ݟ+z7���q��l\`-f�h�:�����Ҍ� ������T���jw\[�i.M��Q����x�\[3���@����,3��m���E�Q���"�o�s��>��� e\\h }Fv��va������?O���\`n�����@V���b�Kޝ���#q�� �d S���Y�#�P����9}'�JN�+�M���Br8�I�^��6���WW�\_O��1�� 0ۜ��H�.dk�$����E#75����1�l\`o8'-��\\�4�VOv^�vم��/a�W�㞴� D�>�g"܂�u�C}�?��\\5�����җ�:>���3�;T��Vr��f�%>�N��\[O�Y���6�\]��b���5�~3\_�C��/\*q\\���k�9eo�+��<��G\*\_���$"\\�u�Ѿ���#@���é�XE ��~���L���/�jވ~��,/i�ugO偱�cizm\[D����Dܛ3�0<���'L����A+j�ٞNS z�h#i�������&�h��}�w�aq��tZE����ě���(�Uԩ��?�6i���\_DDK�NZ���W ���\[L��L����VE�WD�0��i��1,�� x̴H���ِ�D �J��9c��F�n��y6HT {4�ImM����$�>=��9 ��m�G׮\\��s�j7r��o\*W����h�^��������O�{HL�w�P/.䨀ީ�V� Z�@hz/ʽ�@E?�d�؈�a�X�n,{y����umH���9��+�� k �e2ӳ�–��}/Q�Q�.� x�K4�"��2� �p\*c�Wc�Ȧ<�Oɶ�t,:�\[��x+\[�0��9nc�r?�m\_�J��k;X���7������'�A��d���NQ��;�~X�\_.G�LD\]yQ�샿��T,��\[^d�O �M�(����Gc��z���� g�MQ�sP/�|Pg���;\`C4�"%��1g%JG/� u��ದ�bt(&�ỡ�P�6&2�� Er��\]�A\[E4�GGo?��W���e����iY�U򈿍C�s��\]�� R$���6�\]\`����\*x��%^��g�3��&�#h����ş/��G.��j����O&vOm\`2��+�8�5U�,�U���� �(���Z��>�� d�J���I�\_/����f���7�A��P\*��>Y�S$+�����\*ţy�R�b.B�����$��c�=.��ѽp�uH����d�/d�)���o@��h9՚H��p&�s8�)ws� �|�np4�P������sʌ�'�V܃���BEk&d�;����䟯M�3#��B.WO�Ao("udCiG ���l1������О��O�""<��Gi��lZ�� S��Б;�c� L8���n��D���/��Ar�Le?��z��>��sֆ�i΋0u�S��F�c�G4$՚J����\]�!k�b?w+H���\*�4�P���V�^Y�9� M�����T�K�dY�'�?�P-�}֪(�6d�A����������3�6w�mRVoa.�,1?�-x�wy�VT����\`�{n��:��4��T�%�H �����Ç7�k�����@�.�Tr���߇�.�lН�B~��ϊ�9;�IǪ��9�2��J&1� u�ZR@�� ���%k��J6|�XFi\_E������'RGǫ;�i��‚%;���ن�D����R�G)|d��^Y��  �B�zӾC \[��(�;��t�PM �7��?��f���e�%�ä4ROpT:C�Ud���9h�F\\V~��MG�� �D����<�E舷�$}E>��e�x��"J�=�.��;��J��\`C�9��O9�Ȃ��o\\�Ux\*l��}��i\`���(~EG���y����ԑ4������a���� � X��u�-7�J���Q���i2�c\_��X8Ž!�s�y��O�#�e�qO����f�<�x�\_Ztj�΄�)��W:�xm'�X:Y�r�s)��\[(�z��KQ�k��Q����,G���f����������ma7\`BB�y��ha�'�$� hL�'@�Ͷ�2h�A"aWܕ�}�� ����|�f�a7��Z�zZv96�m�(�;ܧJ}^��C߆�4���x��O�lw͝�M��r������:��uy���V��� ̒r\\�� 﫚�� �S���9��W��IJ h�D���%\`�-h�b��N͈0$�ʹ�G�Z8�Ma�m ���%ses��6�����S?u��m���+���=ea'��:eЦ��F6B�.|yd��d��<������-�T>�T)��o�\]�^��\*��i\_��@c��Emq�I�u��dF�rP 1V�S��I�f��,?kFǭ��3Ԟ���Ig��y�>o!5+h�����Amgf7v�����fo�A��jx��"��>N�<��Dm��B+���x8�I @(\`���� ���>��G�ۨq��\]��f����ٿ�u���c:~��Mz2l҄{ a��wdtᕄ��H�Ԡ(.���Sh��NIE5%Ke�ĩ ��g\_�e ����6�� ��� a�ߎ�u���#S\*�Z3" Y���D�b�\*�$tJ��5�XJ�l�B��3��y� �>>�;�&8��� F��T��5��\[MA�Q8i9��� AB:�2T�n\\�1&B^��,��/Q�頒�\]�=Evt��A%@��KB��}��$f$)$� �3�覆}�5몚�߀��j�v���� %<18 ;�� ��z����z&�Vϊ�G�K�x���� ̷z�:�B\\H�x\]���5� {�>y���Ҏ1t~+ ��|WA��b�߫De�U~=0\[>��Z��� �\\�����5����\[�.��sɄF:{M�%V�γ�w�XbP���̳\` �L�4Y��= �����6�� r�^ p2K ,F�~8�\]B�9C� �3v4����!X�vN�Vf�jU\*L���J \\Z������a�/o'w�H-��+1ԧ}��:(A��%L��D0ˁ���̀h�0������ы�C�w�i�WD}>y��k\]�\*��5�Y#��A:�{�p2�P��!6�������!��r��lfH���E�a��=�QxZ\_ �yGe�D M����+��7��2WC<��)�����ט^g7��X-��B4�j��� }�/�Lu�.���B�g��F��.��Q� F� ��04L(�̫=T���7��:�4�\]������?=$Y��\_nVJ >��@#��Cf�����~���\[����f�\_�l��\\��o�,����,zrH9\`�6�g���aNy���������6��� �R7��)����+%�a�"��>O��.:�O9 \_\*�.������&8��Wr1d���#i��#ŏ�3\*�<�h $�����Z��� �':&\_�BM"�~�6<���-����@�Pr�.P�e�ʖ':���%� W�Z���{�����:�S\\�QH�e75"���cD'ۼ�\`�r\_��~G�S�� ����\]��D���������ɋl��yh���Ӣհ\*�J:ZB��Ty�����^�(:�|�6k��uqo7p�Ё#S�����lq��x������7��l�-�Zº�v�e��vb���Dz��'�i��Msˏue�2��F�d:|e\_��o��{c���&��<8���TV�Fv1�Q�|>�o0�E��4�iI)��Xn�y������U? �j�O��6�HG%Mf����9���2��yR{�{h�w���!,��%���� z�Ǧ����s�3CGV����}��yۭ֣ڦ1G�79�z����\*��'f�t�#u�m��"�u6����|>�n��-�� 9���H拥s�\* 6� K햁�,7 ��f9�L�"|�֜�"#k���;�<���HP�;�P!dC‫�Gl'�Y�&�8g|'"icCJ����m�߼iE8 �7^^u�y�2&�:�k���S��wϴ &'a��o-\\\_�u,�VI\`$����5��G�>�J�h�#����?�����~X�I6���$�90���� 7���Q)���/��\`��e���Q8UwI��\]�n�r��s�{�:\]�\_?��n�r8����vb�@^�q��;�7=�@g�w�i0;Qm�+��M���-�(�T�CF̪N�|��\[��ewE�ژ}\*� ��d3J?��\\�(�^'D\_B�0���IƠ�g1���s�U���������B���4�J����:�B+�j����NnO�dRl�+��U?�D^����5�o8 ׸�� ���b��IO, �x\_L��q5&�.Q(o� n$�'5=�P\`�BM���O�'�P�W���g\`M�˵-N/ϗ�Vf�.�AN�q��SM��f�=���,٠�a�#se!���tE���x�}�D!��z�܋�\[���\*��\\u��s�4/�� /�hb�j |��x�G�� ����s��񠧢��%�j?���Ѯ7ut��6�+g��d\]V�0�Fj=�0��L���͆�n�f�0~���\\LX��\]A�.sG�p\[��ȯ����@��6�JH�+L%���� e�:����~:χ�Ɏk��䚻Ӵ����3\_�e����\*rfϪQe6TD��k0��,�3�W���s���P�R^ej��s}�B�, H��1'lˠh�pK�r7�������S��T�d#@kAO�FQ�8�L��F �q%��qw�H˸;fJ�@��XD3�.;BY�V���-@4U�e>�d�}ɝG�mZ(����\*F�� \]��P��x���h�J|d��� �a��v���-8i��J�'kr�z�� �6�������q?(zfS'O���\[tˋ�)V��#��\*e��$��(GoB���4��>�Y �TѦ'�+�Q��3g�B����@���4�ۻ � �������B�zr �J�P�9�չ�#m�<)H���\\8��� ���?�����1��ꀕ�wfP�����ij�k�mvt��\`S�=�-�P�7�x P\\TM��/�OS%�u�OV�M�7�){5����Sx2bs�\\-�2��B�Lu�B��\_p>-x6\*q�����z~���g���vwDUA�\\���NOC�;��qd��{��D����?n��W��E���\\֝Y�8�/�ܭr\\�oqUD\*ź�<;��\[K�3�)��\\bbⴿ����A�Q���!�\`���\`�Fy+��Y�\[��\`¼�6��\*)��w�E�sՊLy��zB��IL��㔞KA}�+3�+mM�0e�%����)͖٨º�:���2������Mձ�\\=�RJySM\\e��"@�);8���4��\`�~��C�ޝT��t(a@��jQ$I7�C� V���%9�b�+b͹8��e/��N���Ԩ۰@0��W X���c"H?{B��g^A���~On��E�\[�L�jɛy�RU�ԡ�\*�\[����-Z"�=��١C/���OK�o6��r� \*KOٞ�Cr屒Y\`�������݄��5I�毲KO �dQ��?���� U˵��#��9���iȔa�GW�0� ��؂ף�� y��2 �NEU�>�͝�F���꧗{� ��:�2������pT�y)�����z1۫����ۍБ������ߔ�t�d�#,a�f��՗��B���B~�r͓Ri��nN�U��(�3i�: ���f�Z�$���X��@e�a���\`�n:N�%�}+湦��s���8E�t�CkӠjΥ��Q�@��9�0e\`�lz�Ҩ�yT�n$�v���-��#������B�d�Ġ9��:�X��=���9����-;6�; +�܋z�.��!�B�ݦ߱B��\*{'���o��k (�L��O���䁤$)H��\\�RΙ��;�0wk�xB�\`�'� ,�r�H��舓�H.�\_J��6�o�6�2x��Xf}��4w�/����}�H��x6����J �n���S�Q�=��17\_X��wq; �jՊc��+\]\]��\`�+Sx��E"����Ծ6Y,�{=bOY\`j�à���J�Q��+�e�)���\`C�c�=V�&�%0����!q�����i��4��ic���$�a|�1� �aJ������P��po��mgװ|�\`�%�H�2 K���Gva�>ŧA�i��ʄH�Z)���֚P��6"p�iJ��������.��\[��˗/;6�9��7�3\[���K�mBK��|��)F�t�������\`�ڸܧH�xY� �Uw�gLQ��z�۝��rT���dnQ���Ƙ��̞�o �~�Zo���B�\_�c�����eW��IHɋ|.��떓��q� ���~��'e�4�7��Ω���{�C� c��CN��Aq�RR�3"���E�/b�Z�&B ���k�t\*�4���j�4����=d<|oG��'95���P�1:n�H:i�(���7X���Q��p�\]\`�|o��apa�Z��f����i/d�������yx��f�X��%m �ø����T��m���K, ��WRg���g�n@��B�/��� �������K�����ՙ���y�S�\[оΌp�L��o��JOod� ^�v�|��)�q01qi,�Ӽ�^$ӑ���j| �)��l�h~���sJ5aѧ�� Z�@)4˚7�����\\�C�D.��h�9u��3\`�쁟lZN\`\[���t����F�{��Z��V�s�m3���o�F����d��-L!��1%�9��w<���H���㭉=�m谾��bD��o�����fmK�)\[G��Y�I\*@���ҵ�䁡}���%s�#����/�rXO����$q����mu�K��x��T�q��c�O���t���p�p�Q��Δ�L��\*i\`rN�4�d�h����k���\[&Jc�T���K�~��V<�T���I66�!��:p�\_M�'1ua��i5�� ��H�.�:٠�h�N�YRIjk�� ��W�Z�g%eO�Cu���Dfi������)G�"�+�<��vF#(+1b�!k ��=h��? DžTV�&�,.2����8�7���UgO�-�"ub���=p�h�e�ҍat��,�o���+сXZ��7�0֬९�5�z$N����D� �h�W���dJRvr���qжB:'����9������Z4(Zs;� �;==ߟ�,;����cͺ��3����RD(�Sjl�$�<�k��9��0Nevإĕ��f6;\[\]^��^�Bv\*��\_��%پ/��dž���,й!P}:�@����:���0������x#Eq�7�P�M�Z�Bb�LY6����u�k)�D\*�S%@� ړ����R"�@��.�S��|�rZ�Q�CiR3O{K�g��Ԕ� |0d�&1\\�L��"rω\`h���a�Q��.���K,� ��<4� ���@Z��s�:X җ����z���x$0���?���;�v�!���TA��\\��(r��>I\]��U��R�R�\] ,Wa?WG}�W 3��M��ԐA�i\]�d֡��G�4�E2��PR�5쯦/B��<��Da���\]��+H��+GaX�^�D1X�UJl7��)����}�c0̣���b~T�Ű��\*������w��,}k��{O�:o��śI�����?��hLL�ZZ?� v.�t%/Me�䲥b-�##��Y��6� ��^\`�ɾ�~�8K��^y �� |�����(�2n~�ќ�R5�r�ue�q����\_����턯�2j����Ox�k�z��;�\_x�ZC�Jz���oX&� 7 m=x��X���Հ�.��ָJ�2wu����(�r� G���Ӝ�֤�����۳C^��A�5}�@�޵���7���w3�)p��:�8�ҫ� ���,��ߗ�Yj+W����� ���o���{���F�� +�\\�V渑��}->���ld�����S���%��"lxd{\_�&X���� ڐ{+$�5�tS\`�:G�8C\`j8���\]U�Xi���v � �p�^&�87���M��3�yi�EUax����k����C�Kإ}�YP�ME���J\_�Cn�9�(��>�ss��$�q�P�b~��꟢��zi��{�\\��UԀ�����S|@�8�Z��՟��y>�V$�q�:Dėa���Nv��ӭ���I2ߥZy��d����c�/Ě���"�����l �f1G���g��C�|@͏>k��Qhr/�2)1xh�� �6�>@s�lg�\_i|�~O�K�L��M��=fr�ϾL�ܣ�� \[/e���9��5Q��<��q!)�l�,�� '���Ų�I��z�X��"������쭗� ���L2dWDf����>��d���\*��5��j��MtR����S�D��;�)4�F��d�\`(g�(���$�ܶ�<>��o�I|P|Jk�� ~� t�APG u<����S���}!'Tc��\`���\\7ӚsU����ڏ:ca:�Lkh�D�3�\[P���S�j���Q�\`�|Uj��&dU�����s%r�$�W�\\�9BY�W���n���t���;k���+�+E;N�<\\K�5X{،��Y�\\�š��$̯�L����3DfPkg-{ݠf�#�@��J�v��\\�N�%�p�� ���� �\]|�5b�rH��/��"�K�M)���/���j�$�r�/k��^N"��f�I�-�P�FUf���n��3e��u۱�L<����ɛ�zsͶl��@o�ɔ���x^^g� y�\[�D��\_�� Y��Vu��z�ȼ��1G���='b�;�g� \[>��X�B�\`q.;���f4 a�$����0O����'�8Y���d��^��B� ��\]OM@PK6Py\_ti��b���Z��q&A\\�l�����M-�u�bz�:\\�L��W�-�\]մ�\` 6캪��z{�\] 8J�G�i<��S��ޚ�d����bg�U�f��E��I׋b�v8���#M�%�}�\]1A�ڒ�>c, z4~z��g4��F�@���sYd  �vM�16wzS�%��qW��n�\[����Cv�6�79k=��|ŁH��9\`�2f� rC��Vfc%��w:��C�B�E� /# ������� }�-�'��\]�i҅M�䷣A:'���}��(��@����� �֮�Ȁ�e��7 0Y��B��S?��Fb< ����{j\*�P�\`-(3"����wr���J\\�Ȳ�l��}~9��\[Bh�&�U\\�����Nf�:�Ӵ�M�$��W!�=�(���2�X;���J}9(�A�i\]3��W3���'��S�YT)�zwp�l�0ܾ�Y{D���R��ǜ������Nڊc�k�8����k�u6���m;�ǭ�mӚx��Sg�{�Ro���~�6��U�8�Ϛ����jK93=��Hu����b�yE��d�^C���0���:�Bc����Z���qK|� ��}�g� 7��Sv��A �d�+)t��#����1n�U#͈���\]���oh����}1�D���\*�,SF�X�0 �PqDb�j�0c5a �C���ː>��ZM>u�)�?�t�A���b�R�T���=h$��8U&�V��\`�����鈳�}ԗhF�!X%i��v�vP���0�0vRK����7��U§F��d�� ����-є��\[؏��J �-b' �A�&�����P �������:�u\]��'�Swտ�����\*h�y��N�(\_��8����8���+!s�W�-\*���$?}�\\G��h� Ud���4؄�:��F�����}a�؉1}���x\*�+�\]Āp�x�fR��'��~���$ �ȫ���Ճok�U�q�fఐ:�D��s�M��q��r�)c���L̜��+mӁI�p}|�ֻ�}<5\*Tk�=�M�P������Z2N��0M��j���ɾ<�wx4vR��9��^J��OzZ�K��. 'g9�hϚ��i�"����؋tS����8؁=����@ :�����\[$��Y���\[NK��F�԰��5^������� R�K^�uh�x��?2 ɶ?\\t l:r������@Uڂ���sОʚ�\]�%�z�#Y1k��� CSFd.��ͅrO'��.���}�c�q;0F�M����0�o�/��(�(N�􁱭W���{&-V�� h����������Ou\*����Z��\`����qn��� ��ԩ5jYkb�����%�Y���\[UY��v��'C�-���1�0�6A���̃�\\�%�|�����0���w!�y� ۨ$X\`bP#���o��MA O 4Υ��� �TH\]��1V�0at5\]��.M�����qBK+�=Hpa�uX:U �|� hC��ͩO�?�8\[ ��L��t�ߺʁ���ŷq��p9#в��gZUa|Z�O�24H�x\\����<ǯg<#nrV,�z�� �鉍U�}/-�׳֗���~�"�/�!�\]\_���+�,\]��"�i1�fӿ������̤$��)�����$���M@�1�4U��II�\`�dFrh ���%F�ݟe�� �Ȭ����%>�kY�'\`�8GHɬڈ�����}�5��C���\[H.����XU�> �v�c/CJ�H3 ^��Q|ܶ�&��\]3����XK>v�h�Y��� c�Õ|>�t��ۑ�P�Q�lv �,��ɶ~��L�ۅ΂�^�ŕ�\*|B�{Y��I)O��� �Jb��ғ�¬����������\]+֝�~Vo:�Ŵ���ڰ&�~��{!o�%E3)Ũs�����șm���Ki�~��X���V׷�̓N� )IP}o�o��x<��o�QP�Z�9�7��o����B���#�\*۳�?N<�o��A����\_��(#jTL�4���l�B�҉a�ᐱ�^�We'�; !p��\_��~�f��5��3AX�d,��Z�F���G\*~����M�󘶐�����W�gkO����|�}��)�T���m��ǿ�5��"�%���rt:��8?��q�Ď�KEH��|8��M� �����)(����@�K��Qpt�H.m-=t�{\_1s�Kc)��񜿜�z7$�&���U�����6Q�՜݈n e3�o  �<�'� �\*�㏾�ٱ-��3Sd�Ϳ'B�o�� Rl��1��ڕ3��\]���n!���<�e�� 8�@��\_骷u�T/�7�0V����\*z�����,���"۲S:~W�s�<�|��>�=ҧ�.)���!9��XE<��N4�d(��?��=��o���0mq)�ʚ���&�~)�,�|��rk&������\_�F� \`�-�����)�a�duC���2H���$�\`?'� ���ڻu^R�z��7\[�>e��JFtQ"�)H{�UG��y�U�NaB�\*kJ�9��eۜ� � �������σ�G;����̕���8���m���ūնH�ά�L�<��HxGUYI���X��eM� t#��7P��7��%�/�ԩ����@+bH�n��9�B0�Aӭ H���9�����Өy~W��f����^��o�\`Tc�0V6vw��Ⱦ�W�Bc@Wي��wн�d&}d!:N19�{>”\`��Q���.�h���A����\_��g: B��A����σ�Z���燝����i��xz7�gr㯟����ްg� m�fe(�ķ jR'r\\1B���g�?2뀝�������MtB@1�nK���4� �X�n�����j�k;�@S$�L���r��Ej�ך�r�R�:R)�5� ����w��mj��Z�z��:�\\ �?r=�� .XD�+��TS"�ڂ�/��D�&�0M!�IՀ�Y�4E��XC�@�;���t�+���wի���êV�bP�\\�� ��O�d�t�\`\_� ��b�-�Q%i�uŇ�9��V� �(�����ד�rMõ�M��������|A��B��Xijjw�,R�$|k�T��f� \\�|д���.��CC�� �\_��q�o�.���%K4�5�XU�0#��~�8!�SŇo5 �j�CV��F�G�\*br�J�����H��?�k�I�|�0��9��k.i�U̵�Fڀ���h�b3�fϓӰ��~���|���a�U\[?T �\* u�nV�n�m��gA��վ�\\x�lx�n$�9�Q\\���ꕸ�Ѐ\[��ϻ c��\_\_�<�#��X����7� �Yⓚ��@M�zB�G��K2Yu%���9o\\�{f��㝻��/�?Q��u�nG��L�H�Ք�GĠ�)l��S��C.�����g�v����~�{m7Q�i>Ÿ�%�5T(�"�3'�b�,%~RqG�Mpn.�o�ܑv�:�ĵ,ih���·�u�r(k)'�$#���r������h�\\��^�>Ӗ45 p�a�l N�pN���?\\r������̅�Jh��V�\]0��V��Ug|�����n�۰�C��W���A�b�0ӥb.~��d��m��k�0��u��� DO�ޤ��tw�~�)�\]���E���IY�"���Lݏ�79���� z�95U:�h��� ��NT��{��N�<��yS���-�\*6�|��-��W;E�.D�PC� ���L��$-����K�f��iJ7��'� �彉�RGtpI�N�7:�p�:ͫ=�k&\\ڂ�tRr��JN��UQ�W��Y)�ƲC��+��$�:���.���~��AZ?;�+�V˘\]�Օ�)��@rд���� ��A}�X����\\��>�� �)|�y�� �k�YʹSZ��i�N�B�u;������r?x~/@�Al�ArS趱uŧh����P �4���#�(C��N�.Aa.֍d8Kim>X� Ѣ݀���oh1�RZ�.^5hY~�$6B�q�����"���Eb�݌M|X%p� ��:�����J��Aq(��jo�w�5�=��7�!�������N�1��z�$O�ȹ ��DJ�u�a����|yO���TS��uH�y�W���W��n'�@D\`��௚��'O�f3# �і���L�/���\`|�B()�?���̡Bf��@zmo���s�}�oY���/U�)��Y�C~��'T�4k4�9f�| hl׼#�V���L�D+v���h@��? �{Q�.�iZyI�w�XK6�ico�ˌ��v秱����uUj��1I���;�OB���uv�O�cB��b�A5�����U�<�F�hU�s�四�Ò(�ޗ �"D�f�bZ��M��Cޟ�Q ��yWkSb��w�q��\\���V��F}ZpN1��\`�au���ȟ��z�IlK >}�;:c@E����6�B ���\[�g��^�Y\*Os���'n(�Ŭ8�6�����&P�5�����YZ�:�ϰ��M�ԩ\*P\]w$f�k$ �5�;�/n����H���b��22w�1̠ i�;7z}.ц��#ha�q �§��އ\\�����Ӥ��|��h���>�\]~�F�������~A�9��K����3�(���۩q�dR�S��e�dSnl� 3��Jru��A�i�����A2�\[(��o\[���lӽ��+y��?z�&®�߷�F�=�\*���lX ��5tE�\*�\`�"f���d� ��fF7AO��k�h�E���M������"�����2�~��&$vϞ�z�1�A�X9������kсA��H�dbq��dd)�. t.�R������Xi�TẀŸ2(6Ѩy��ª�%��~�맃��{-�p�U�˒ל�4�������|ڑ���˰R1�ȡ���eɆ�\_�X�n�\[7��\\\]�!��ؿ��F�n�O�\[�+ l�Y���.#tW��#���r��&�\]��@�V\[�'A�m ��\\S�.ᷩ+�CY������ nq Y9��r-y�g���\_-��PP���7Ɯ5���J��qۘ�Nr �bC/J� ��r�/�|:��P��@���L�VgH�\\~�Tg7^�3�GC���SL5�0��C\*�h�v��T�x�$jb���'��oH>�J���|��a� �a����H�V.=�B�{��{qH\_�m A��U�dVT�������&0�XVE�Y�Rş�^���j�)i���xI�� |\[�ch�o�F;<\\q�0i���6%l�cuƒ� �Y^�b�sjKAV�%�k��+Bƭ��nA�Zr(��2C�}�|�,�g�(�B�m T�I�m l��x���yrD��}����8I?DM��.s��Z�� �u}>��պ�asi�ϫ�=��M�z��(r�t�Y� k���}��9� Nj�5�o�Ytć��-���5mJ�2讋̋O8�7š�)B�����dO$4j�������"P�������M \\���U����\\vu�P���i��a��R� r��ͦ���b0�\[����6.���D\\tr\* �f�bO�%M|�e�r���fs���\_"}��� r0�ۚ�+����qW<=ʛ�� �i� �L���#�a�������Tl� V�X��h�@��򾮹�\]{��}�I���&��G�J�h�Ac�?ʅu9RqN��~� ���mx�7"Y=���>�-�y��'�,��R\`)n@;�GN�����/3�Öo�c�-��f�,��f�;B�~�(?��7j�u�ȏ�!p�+E@�e ����-}�$~ceb���Y���T:��׮��Rg/DNY<�,(Eɡ�n=b.�=�\\��~���ٷ�� �W3�7�5jd�L���e9t^&�%�}�W�Q#D\*-�/���R�ߝ7���uǦ�ϵ�?�I+�������ML߭tj���U�\_�CY&��a��M��3�l�8����P.��9�FJSrs�����X}5y^��\`�P%� �qf\_���T���\_�g���z� ��s�5��l;^Rn�3�fŅq�I�\]wE �0�k(b,� B«G1E� �����d 67a����z� �R�<�t���D���݃jAGq��ng����rp4��bD�"��4����il6O\\���H��C�����9��D�ekva�G���3}�䪇�xc�'���n? ��\];�g#���ٸ"vOh�JC���e� �M�����N1����R˗vFEz� ��\\4� f�I�s'(������չhd^�Ps�����"� 7�5E\`4Ck��d�rg��L�?L�}����̀�!T�Z a�e:�,��Try=�m6s5Z��ET��D ׿@�Jx��'����H��%���&p�jr��o�\[��AO�C�iIL���H/|��4�of�GIn��b!w��'|�r�#w|K�n�~5a���p���>\_�ps��q�h����8mL�o-3r6'\_}3������a�b�g��XP�� a�5�/��zʛs~���rg��@��΁��pʥ���j �R&i�E:�bB�x�Q��>H��7�Є&a:��aC��P�Ȯ�����ݰ�r��}Cy�<������Nt��\`D���@=W'u��ºt��Q���������O�Pې�B�}@ q{�l��}��?wC���Ͻk�8-9����}�x�FEn)�S� O��=ʃ|���E���ؖ�oy\_Л�ģ�cFR fˇߑ��R5{��~���<�$�����j�M�\[�(�uQ��9���L��? ��9m/�Xۇ~���T��o��=o.�ɏФ��PWWz# +���I�mg������A\[Gs��m�y�-2ù9F����� �>ճ�Ġ&��u�)Ge\\� T%��u�A���!���ʎ)Q ���̝G8\_/�ڿPgJ?�{���Y'�BN)#O-�{�������-�O�5"Гfi����m�O��z��\_V�o������J�\`>��Z�7��H^�@�\`�\`> ��h�͡��B�6T��\\;<��N����\\ں �Nr,rF=���\]�萑�U�����0hW#U�����ߥ��r�Y�|��4�\`U������K�z�m�l�AK?�-f�i�CѴ�G2iOϓ(��fRw�n#�û��r��j-��D�1x�7Ӈ��J���l.~�~�\[3 �U�H�Q���t�J���O���3�F�#Պ½��ؼ\*2�\]d����"X�eERض2�9�p�|� ׳�sTd���/8^ �y )�����v�q�o�V���Z�.S��C<�^�Z�ǻ-�8O�;���4�t� p;\]\\�M\`���c��jC%��2� C)��ߞ���>�.�f���x4 2�������˺J��1��O�B~5�:0���J,�^�|� ��9��$g���S�:\_s���q5�¾W��z�$D��)��;6 ��m�~��F�uw��#�o:@/Bz� 0>P̜���-D�������;�'l�:���Q �5g����N�w�qy�q�\\r8."7��-Z��T�\]~�p�bh'�� +���n�WJ\[���L�8�#{�8���x�M�OS�5L���ń��\_\`z�-�����u��7E �4�h���@?����1��D=3ǡ�c��Тz����\`V��� ��r{��k��YXn)����9���Iֹ��o��r���?�t=/\`)��Y�ҁ�\\fgp��b���% ���8�>�p��>d�/@�}��m�P��J�.O�W\`\[���{�������Ψ뢅/����Np�x-ߦ$����v��X�x���-��ԕEs"2�L�1uu���}8����T�~����} ��׺-� Q;D ��,F���?|��+�8�{�l�ޝ0L���=T ���#�1�3���!v����"M:�=wK:Kj���Vb<5�t����~�נ�.Iغ��\`@�AW�Iܣ�kw��1�61B��&��n �ۯ�SQ��X�D�ɪ��ƫZk��)HXaψ�lG;!F;�qy�ZZ��������U Ո٥�!gb� ��=��Dʈj�CH, �f\*A:�NS��@��֕,��g�����H���@I"��u�ȋ\_>������g��B�����GB�?�z���mk�c,�i��� V3鋷��;�y��;���:�ʝ+���Ig,���%y<�b��a�!���\]�:��^}�a�=��~+���"5��|^�m�,�ݭ7=0�'��;s��Qy�Vo�0���d����kġ�q � � �Q�����4=������ �����$j8� l�1�0�٧�x����P�-1����VZ�0�@���80���pW=��1I�8M�5�S����P��ap� �^��:�g :�ivn���4<��>��ߩ��\\P\*���'�y�ۓ��+�8�C�t�\[�!�+��֧\*u�u�>⏬�%'2W���U͌��.~��u��\]ߌ��/���i���z\\�h�ڒ奏����U������Ļ�0�blyޠ\['����O��f+�)�=N���k�|R����{ �������>y9�Ha�'��zT�/O� �3;���'ne���f�-����ym�����6���Έ�ئ���\_��j�b������1B��c\_ꕢ�1=���{���ǁI�Yِ�o(���3�} \]�����:��\\���u�JA�x~:Nӻ 7��dH�3�!� �u���ꎋ��d���˂����~�Q�A� z�\_l ��y\`�ئ�ux�Wc�� hz��)��Cq��s�\_c����!�m�~����L�P��!.�^MW"�G U@����k���a\*�؇��z����wdg�\_jJ'� ?����'j�9��~�Qm����5o2���� 9Sv8��\_���@��H7CdÁ�\[ %ą?K�KC���#N�\]J��g^)���jd\`1Ҡ��kv�� v0}&g O�K P|�� �\]S�0kJw��H�\[㕿{A΍�#����Gu��oy��C�+h�ZQ��>��S3�����l���i'��(�-�aqΥ%\[ TBÁTT�2��u���ͮ9��r��� ���� �HY��.'E�O���9�.��2�W��l�t0��nK�?�0A�z�������eTظHN/F�G��@k�\`}1��O;6t��c����#5�)jz��Ұ��%đ��Ku��)I+��,Վ Tgp��oa�<:䇡��@�T{�r��y�\*G��A�I���a,�4p�������h�u۞��9�̫d1T�;����,g�����i���:���S�&���\\��.�����"�}�u�\\�.G���\[TՍ�s}�r��� F�q6^rA�F�2�!�?I�� ��:,�rb��l�}��y@+�ФgӞ���.� +��l�TڜMf\]��تs|�����n��j�^/.�FAL��7܄ ,�#�lt���)+�����u�-tԟ�)�Q��0B�I�v���/"���� �f�l��k�(�z!�qE��� 5$������sl����@� ��������y� :�gY��ϛ}�3ln��~�\\�5���l��M�2s�\\������j� �33.�W���⧜��%�6 �v^"�nM�wg�����.�A�ۇ�e����t��\]�!�#��G �l �e\`�A ��\`c{��\*QN��e�3@6��Z"jL�}�@=92{�5�����d��М����l�E���p@�����r��ރ�������AX� ����u���^eXH�(��#�" f 6��t�/jTR��V嫟�vSI�gPs��8{��Ӊ���ɰ�{�Xu�����zh��������J<�Nt̆�-��/�<����� 5��n�-��������p��� �pxMGU��C�S z�pZA�4Nd�K%A�x+� ��|��X�w�&x���d�IK�r� H�z�f$4�cܕl��\[Y�,���/S\_�W�w��4�|(�1=�Ҥ�����X�\`�9�㢧�;�����a?.q��IY�&�,��>Ğ�x��b���d�\\��ԌO�0IM��&�\`�z�/I�l�� ����<�'Ї�2D�ˏ?��N�Z�^�Ûh � ͧ��v���/\_P~��� ��.rAkg��@�d��$,�e�~�N����Aïm�ʵ�!�K ����s� f$>{� �4���E).#� ��b.��rQ�d���Ea��r$�5sD�|I���E������ ��\`\]mL���+����"�ͮ�\*��UҟZ�uh��€�\[�&���/��� ����r����峙T@IW��OW�{���va��9Z!���1a�QG�ͽ���� }�!rm\]� �\*��t�6��!�{�!��iGV�G�J�\_r��K}�iB�9��ŕ�9��L�����\\� 7�������2�$/48�Q�^ P�66�T->��'�e,G3Z���P. �t�I�������{�V��R��ab� �ϔ�j�ߛ{��s���g��R����a�\\ʈ��y,�eM�=�QTG�\_B8$�^�%>ƴh}uA�IV��K���eC\]�,���H�?����(� �N�\`�q���F8��!Gu�\*�"�ǻ�N��-�������z�D\[|���!E=4�" .�� ˊ��\[��{ڳ����@Y��d��\]��)ӄr(O��N #$�6C�D�n�f�������\[�T~wH�{�pI���H2�KK^�?J)J��:%y��J�n/�� �����~g���G(�N���5����Dl�J��?��7�\[��S;^� Y;���}�&�:؁%�꣙��ZyoB�+���I("�E&F�a+7 ̩�"�Ynh�l�����Z��kŌ����:w0�ߨ�Qkמo��뎝��C�D%���4�rZ���:@vy�@K A�ɣ��| eO���6�\\�Ħ >\\��@��x��/dlӐgq"��2�r������)P��Y��oF�4#⸤R{���C���m�zn��ޙ,bb?�8����Vw�p�Z��� ��x���� ��&�tz�х���1τ�}��!�)�@r�ң�4w�������������oAe�Z��ɓ��M��@pvz֖�T�f���X;G�8.�Yq\]7��f��I���%�GTK��g����\*�g��c��ycWb&�O:��-8tӯ4 @��{h O#$����g�6�j�gP�^�͌٨v�@�ؖ�ݕ� �R9�106�{�=j(��;�0P�G�.v8�� �y�:��X��)��x�)N�- T�Jy�җ�FLDb��ϵ���ɻ��-�k�L�˳h-L��6TW��(��; �S��Z�� ���|G�Ћ�R�1{�Z�P6vި�P�1SB.g�/ΔW K������k� "0�Xq|�g�zꘘ�&����m(��F/\[��xB6%^��\]O����-����I�#�٪Z�3��U�XZX~��>q!PLj=dN=|d8R���Hm6�'�\]�� ���LΧdE��F.l��?I%/�� �3h��0�7v�B�Ue� �Q�'ė���x;�4�a�����&��e�&bMH� ~O�QԪ����8��o6 � �/yyU|���\_kA��\_���C�=�V���^�O��PBA1mQe��&z�Ea��8v�������~��"N\\ Das��1�F�<웆�G�W>I��M��6��3Ӷ=|���g"��X�<����2$��-8�Ƶ�%G�4TU�O�����b��B9��w|Q�\]�sH�e���3����T\_L�L��:V$�?�z�gj������7j�:�~�Q����n�\_�ǣd�� �)#B(wA/U������� �τ|��e��-R�\*�g�HJ�ӵª\`H��c���~'�M %���q����uK6��:��������I��3�<9��F�"�2@:��#{�\_(���n��W�a�5��z��\\�6͏�%ԪT$"�1���?q��L�u%��k�1h/�!��v���2Dzۣ�)ap��c�8D�z; �(�H�}��§8��D;y�����r����ju�>:)��� ��d�\[��\_��U�Q���؈b��b-\`����|�TZ��?��\\R��\[ ��#�� ���vZ��\*��k\]M�i�5y��0y�� �5�ѡ! mWL�1d�F-xmG��cp�G���-���3�wT�)�a�Y��q0��@�'�/��%�jd+��=PVN�O��!(Ri��%�{�Zw�̏F-�\[Q��\\�\\�A��D���n�3�/�g���Zօ̹v����O1��E�\[O2+�@y3���z3>���n��5Y�3c�ɕ��?�u��!��~���Jl8j �-Or�}B������}�s�M�j�}�B�����82�f�E�ʺ� \\;k�lnA��jp3� x��� =ﴼ���\` {�k�Q��2YH��A���� \` T5\_�t����$6��sMS�\_��3�}���G)��̹�LR�@?,Cvn��s\]���tsa���Zؙ�m�������|���2@��d�%۸�hY��eA>$�wt|t~c�JI>���z�6Q��AL)���aA�:MK����Z��GXH���H����+w��fw�p��&�c\*�9�@�gvؘ��:z�S\`Q)�=����XG4�iF8��l��2R���0o�ߙ����{\]�����rv$ m�Y���.�Ԇi��Ҙ��@2<�Y�?��J\]�� ����+��z=�Hd������}w\]���%����C$��\*P�x��-4Q�3�� �#ݫ�C0�>^y�R%�\*> RHgv���F�Ki�I�\]v�j�%(b����� �� � 1�sf��� jm�4��Y�λ� D�П�\]\`S����⫛�n��}��Ç��/5�!2����;�XRbv~�N�\*%b��B��š$ȍ)�Fɝ3=�~Zmd~Hi\[�/���}����!d΂������=T��%�!l��ь�Y:���ew�d|��1�^L}�;#�́X�Ѫ,��%m;.� '�6�>T���SB�Ǒ\*�4�IȢ71ۍ� \\�/�H�g�'�L���X�TU7�\[�E#G�u�\]j)V��t�s�1�h�� i��lL��9��j��4�1+G�Ἴ�19��22���H:;�i�,Qn/b\[('�a5y�ުj\[�\[���wε��C����}3e)\*�8�>U?����w�X�,m�/&�$��FK��3�y��n.w��)�Z�Q� �%��=�^��v�עl? �eԄ��E@Cմ���ll����O!Wl�Ћ�^��\_�8����I��z�j�L^(�|���b�-����i�� �I���9�<���%Z���G^��j0m����,(3o ����Hf1r��\]f�JATul.��O�B����#�,!}�9�����p �|���� �\\Z�'ߚ ��N�l ��8�yp�����A��1��^Y�� ���0 S�GpǴ �$t�u����m�i�~�U�� �Fr?WGh\[aO�xb^\`J����ft��‘��,�y�b��}�tt��-�-�r��Ք���� ����-��Ir�����8�2�E��\]��i.J"��e���ߥqc�C�( ��v��G�����j�/}YU��=�bb��<�^���We��\*�! Ɔ��,�W�:\`컰ˠTgc��W�ʟ �M\]2��Cn��2�r���Ya,x�U)�C��@\`ѳ~ԍd�ʣv��~€�%�/z�?�$ZpEh�jZ/V�#YnB���b���<�3� ��Qr�X�k�P��5��ZD���x�C�y!?���;��7Ņ� jЅ�{���vBt�>�'0i#5.�t��~)V�t\]��j����E��#��W���F����Fw��|}T� \`r8W� T��\*��/t�c��V�$�6t<{��nl0^r+�q�M�}�p4I!\\������Y�Me�s̆�a�CK&�����;y��5R��3��g�J� �����J��;W��y=��9=����U�V�4.�BC�u��{NX��c ��tw)s��Ұҽߦ}�o:pgϣa��\[���1K< ��ZB귩 ~x�h��K�PS~�϶��k=\[\`����Θ�cUR�ê��g:�����Y|�L�{A+�D8��;v�\`�^X�5O�-t)�?vJ(Qi��f&j���V/�K= ��; �dZ6d��� ��y��1�a��BeA:q�U���\\1�@<Ť�\*j"Dq ���,�I�������:���\]( @��� Фm�jx|���U�Z r� -�� ��S���v&� ��{��#Z􈐄�\\<^�\`�P��� �SSr1d���&"���m�G���e(�m�r�cfy��0}4�WP��,i��K��rL2A|4w��MA�tTvk����ZiA|���/絞�u=�h��I�́i4��ys��f#ݕ�'��U:���;���5��� ^\]�v4sȱci(�g���Y���V�AhU���|CF�p�}ӚE�$ot�V��1I��(N/ �u#i^�^�@��\\gk\[�wy���K/#9�D���Xp���̑�t2 ,r}v4�\\�H�F:�ju v��ם�y�Q��H;D���3)�P���~�\\A�7Ѓw\]AsH�������lF��l�y�i Qg�.�l�k�6�Q�"�S \*$%�;n���7\[��8�r �#���v�\[@(��В\_�~�Q����m��E�suj�0����\*t9t����L\\��iՙ���Z�G�}���S��|���HM���̚�\]�\\�ܜW~�v�#�xf�����sR'q>�Lq�#4���\_\`��q�L�uVx ���6%�U��H��B����\`\[�?��,�ž܍����vR8v�����i.pc��L�+k5<�jb߶8k�T;0P�&�BLF����%4�t���~��X��U��/>�QdZ)szg�U^�̸����V� ͥ#WN�����i�W��Ub�����@\*j�� ���>�9X p �^\[)�D�9yW�1���H �mE��Zv�\*v�$1E��J� �:)�w��� t�/��Bj�KԳcV���;u""���� ��T��9����t���v3������ؑo�E���Pe!6�32�t4D�zI �\[I�M��|\]��(����k@�5z)��RV\_j�����E��KU�6�t �9\*G�Y���^�:o��KYI���ji�%��������;��%�>\*y��\_� ���R���nV�$W��ź�����wbx�.�a\*���w��=\`x�K"9^, V�.ȏ�h�;�/C�����l:�b-={����м�t&�=k%����E� �n�tYY���,�RQQ�$/��I��g�5�j�y��%��%���6z��oE�S{O��s�� �ڢ����1�+��n��P�DNd��Q!�4+$��S�����c�D��F�X���Ά�5&5o0B0�c���(��a�2 �����$}�'��l� Q׻o��(}-�+���x>/�:�֭��(2����S�Hri' e�(���D��m���^��V���p0�x"��(�#�eZ$�EY'��ο��(i�0?X<�\]}����8�I�Zh��E��c�ͬ�g �/�z���x�y��iy�2�lZ�JR��|�9�E���h��lrĽO��=�����2�g�11 �喛�s�����?R� 5�r0O)θܿ\]���Q�}#�u��I:w\_9���nmW�h�G=�9���:�R�f\_� ��BGLYM���^0U��B�Ө����4?m�y�z?��ˍ}�l�ff�� j��?�Ap��o�$���/\_�!L�>\*剩ǯܗʮ�9,�V?܅��e'\[��p�ɑNM�ҏt�^I�SL�L�����o�l� ���WI�?s�� T�Y\[��|~��2����Y�FͿ���׀�ji|��J��|�ψ��OZ���#�8�h���i����G �z�;F0��iޖ�\\ ��Y���{q��\]CWx��ԩӿ,Hi�^� V��sG�7��'=���������}� �op\\{"Q ���p�G�>���Xv� ǏhٝX��K�����'|De�-�Y�i'v��B4ʕ����E�1���p�� ��b#�%6 ���@�g6�q���tX�Dr�9%�7 ���TfA�F) ÎK�6���D���zh�f����J�z��B���1>%T4��NMd�'�|˝l~":t����jަ�-�ұ�u ϛ��z\[��G�V��I��Dl(�.�� p)뚗�+9�j}x��~�c����"lS�%'�I-qYT�s�\[.�~� ۾Z E�ڳ�\] �:m�~������\*7$ۨ��G%�����\\�^�y:��\\�W����O������\\�PA�'��C�yY\[���Ƈ��a����� �.��Vn�X�@��k&���V~�O���45�����Q�i��.k| ���f:�!�.��Ev��.nTCRk��o���1مtY w�YL�\\�Ԥŕ���hLī�.K�v� 1���+�W��33p��e4���\]�u�u?+�v�f�һ������Z�9j��w����Eg/ 3�L��H�rr�'Q\*�4����VL���b��W��Y�C�f37�(��\[�12DT9�G ���0��!u�.J�Cm���Q�/ ��\*�rgդ<���^������E�^-x �!"��v�j�J���c�&#���ݠ��5 �$%�6;��\[0�i-� �WX����lc �rwA�!������wo��r\`�}�u>��'��o��@���Ф� ��o�3n���� !�i ��Nj�|>@JE� c: l�L���D \_N{���4�/�RtS�O.ڒ%��A9F)՞��빫�L���~�Ӏ�:��h�,�cȋ��X�6��������%Bxh� ;�d��\`���77ԟ�(�Y:׊�6b+q�y�\*����Dʦ��W���������c�1�9��,�@j�E��Zq�eKXkl3��}/� �k7�c����P���\`�X0I7B�\[�0)���l �g왓˧�P/��1���~����\[��n��r� ��e��pm�C#�S�(��S@�n�����h�w�y�Y�{y���ߔ�CZ\*��\\X�b��8��2�W�B4�� �8�LQ>�anwhU:�a�{m��π���37�϶Q�HM����eN��dK���)K�C��=�j��"p�\_��}rE&�zMO�����} ��z"O�5k+I�(��z�@����+��da�E�P�N�M�T����� Y��T�{U�0S ~盪�(� ����e� ��Se)�m�,E�/���c=vt�0I}����\_V��&d�&�%�-��~K7Y\`��S��)�"C��Y~�-�7���ׄ��R�� �d�{KLs�������<. �� �r�?c���A@D��e5�7�+W|�=�ri��S�´L(�U��i��Ez��tr��r4�ޠ�|�,���������sV�:5"6�j^I8�� ;�b�.��O#�d|O\`'ռ +\[qݥa��� �W�(���;ѡ�9g#@���!I�><�t�3y;l�\`��|�J=��d������͉�<�߃�\\(��e(��c\*���g%�,,+z��#�ϵ���B9�;eU(c,��=\_�X ��d/'6�ʴ��4�5 �>R@�7�jd %�| ���+o8H�XI�������������5 ��j��T\\����m���ݰ�l2�E�mǴ�&3�W:���6�|� )�)����+G�\`��Zʰ ��&��i�A5� I�-�9�Z��)�m\_���t�oۃ�<\`�Q\`���0�;�p��M�<'\_Ƞ0�!�{��� ��HH���j7Q<�� �E��r��(�,Ɯ@W�m5�ZW\*��+���ߞE�zn�4ꨍ�GQL3�� /������m���S�-d�f��\\�2\_��L����D���md�\`\_��M���U�8\]�C��� \*P{o�������i ����P��3\`�L��\*SQ��?��a���\[r~ B���ij�6�+՗�)f��}4��\*�DG��Pk��m�KI��dv�C�Y����H�t5�"�KP��DF��T��jK�)���e���R���ۏ�^9ĕon����v1�=p�fj�5�CnW�-�N�u��h�cW��f�h���uVE :���g�+��9��3#��ݦyuA,���%BEL�Ȼ��i�iu�M�\]��WUXY��~�mq}l^FO�4�0Sn�}�e\[�e�Ap3O���AW���5���2w1t��w�n|2��/�!��� ���c%�a9��D¡�0�j�w--\`��n� ��~�A4�H�\]�S��2��Z���Ťl:��4m/��uU&��߷)�+��� �{��lo �nϦ{;B��-<��sL���RR׀nU�E� ���D(H�G>L��{��\\�|����a3LY��\_ &⛾W���)�#����1> �5�R'�4 E���qJ��� P'Z|�^M�\]�ѴQ�f>?�����‚~j n,�9�E�I��@G흎# �x���;\]�m��� #Rx>^����X�:d���,ͳҿx^��ą��~���#�ED���/�‰ �'��VO�� n��n �?�͐�3�ӕ=�9�xYf1��~:� ��q 1y|k�z(���e�g�tZmXW�A� nVT��8ޏy{\]�! ��Q}"���$N��+��5-%�����|�X��\[����#>���iE���l��>���F<���n�A�Qu 7a�������F5�NUBf���s>S��i��v T��cYٵ��Njlg�S3I��Y&z�\]Ƌ9��� uo8���R��J�ŏfN���\`olP� �!���J�M�E�S\]��&����� ��MƓ��a��~��0�k�C{�����S��44�7@��\_H�!�&H8!n$��v��>2�sۺm��e�za��C�������Ý�:�:��}�vly8WIV�N��Y�ĹH�l�J��J8�9���ie�r��EK>��%p�jEo�������٘K�Йj%T�Όf&��Ƕl� c�b)7�q\`ˆo0ާ&A�+p�� �4sg����+�}Y�u�s=�ض/Ĺ��aЦZ�u�Tߏܖ�rgI���@��g��� A\]�&�ut�Vz<  ��mp���J�-��SL�t�"��U� �Z��S��4�y퉩}dL�5��V�\[��o�y ��=�X��ǁ �LGx���y�͗t{Ĥ���$Q�\[ڥ,���� 2HAN�J���J� �"�T�Ϊ�\_K����HOc�3"����� ����\_�B����W=�R<�3�2��2>M�+�U����ؿ˶�ؑ�ī��ib��%�Y�b8�6\[&����<�Jf�he�G\\�vQ�JK����bl��),N��Cj����p�Z�܃ϴ?aǯl�%43#WpC!U����L��>�(��g�Q%X���0Yy����vL�M�2Z���~��v�5'h!{�VXD�\`4�1=+n� ��E��o�m-��� 5�i^?�T^�ɦW�\*<�d'���Fd.�M�—�\_V?�D/u' �/�ěFHark�a�% �x��){B+jL�9h=�v7���"�����ہ��kϡl˧�Nr�@������ﺱ�M�g'D2� �\_�v�"�\]� � T������d���ӯ��������\[���\_>9c���F�\`j&E�g�5(�\*�|�\`" ��.C��jY�DTQ7ڹ���ւm���;�t�"K���,��k3���#S�m�(\_��#�J�$�2�dςf���%�o�1h Ì�{�a�N�c�2�ET��Y0\\���L�D�\*c�^�z5�J.K\] %�Tc xTP���nn��\*�W p��2k)@�i�Eij�9�>����TOj'w�庿H"�ԟ�TF�)�xt�.l��h��d�U�sDC m�6w� >�}fU��D�\*�vnY�����큥^�Zɬ~y @�6(�����F�C��0����6�~n��"�z�/@��~�)\]t\_���hs�-�\*������x?��>��Wj�ٖ���K?� q�IV����Y���-��\[#H��Òe6�P����"�\[�k�(p�C�Ͷw/���엃�p�G��:-2e-\`\]AH/=Z<�uD�?���0\]���VBk��l��5�\*o �6��5�4n^�@����|b�\`c�TfNY��iDe�!��^��'𝤿�C���U?��� �\[��o�\[����N%���Q��Pݟ+��/kb�A��+ B�r����H�rZb��0i~17xVB\\Y�䷖�&Z��}A�RGLa��,Ɖϗri��2e�0�Ap�6�WXw/~���D�9��1C��c�Ff\],�ۻ�� \*A���;��5�A�zlI�-0��.�\[\_MB ���������:�H�o��A�RuE3�\]ްXq�����IzA!��1!��;����@jċ����^�V��8�sf�����gJ�w&|x�rw)tσ1q%O��o٢�\`�yu3~U$���}?01��SN�b��o�xYh� \*9�X�ڞu���l}�x���rq.ܐ�����o���7c�j�2�+r�KD> f�L��BO�no�i-��H�H�\[ ��w�+������F&���o� �ę �G�N��r����a߫�4��/��qV\[�6VS�Ρ5�\[���ް0x'�'0��f��T��4�u�f�u�8�����\`�T6����cZ����mdi�Je��� ��˜�@!��,�! �!(w��2Ve|ֹ�3�WF�i�W�5���$e�7��eE�>@ź��ȶ���}�i�Y|SE�����:Y: rQ�Jz�Fw�G�\`�R�IJ�AT���"{��q|��gKL�,P�+8�" �w;�a�W�Sv�!�9�� p��K2���S $Q� <�x��&mm!���a\\�����;����0z�>Ak\_���=��!%�,����D�͙k�2sT��?�~'�x�b!5��|\_��OPq�fe�b�|���AU�Z���h����z:��g;�\[ͭX��T�4�V�0@'j�N�� f}�Ɛ�51L��)\]��?Kh� ����)�H��fG�n���"����w��?�T���@,gG��.a��t�CR�\[������Ø��r���JWr�Ҋ.N.���ן�l<�j�B�M,��n�3k���ij�/d��&i��\`�\\�"����f6P\`�z 2�v�dHL�#mVuH'��i�wP'�D��s/!-�r��V��ݥ��bKx�;Mh��0N�-DO1X C;�go��\]� ��\`>��.�K��� t���Y�2q���5���z\]�����:Iup{'ꖀ���� �\_:� ��ݣ��җ�J�x6v��sA�n.'0�˂Id&M��~wVˍ@\]����Y)�������κf��;s.��(�E\_�oj�<�j��! ���Ô���X�eU�x�X|j�!'���;����񋿳;r\]��֤�� ���EN �����"{�� <��KTOw�O.��{���ܖk���9^���ZHV�+��Hq�PPk'�������M�KJp��L �?����� f�v}$����+���X)-Y��:�Vx-� �t��A�r<��\[jX��zP���6�\*~"�v��XF�u��� �4nËE'�F-ۆ{�9�(;�J����DVC?�OZ������h=���~�O��V�mi��m�\\J�c������7�\_� �jccXw�����Kxb��5?�y2�وƬ=��xЦ40D���w�@�(%����̠b�4�,�6^P��x��Ӎ奐���(M ����9����;�ȕ����ó����%��.\[f��SE�j0���덮b�8��qy��Ws�RgȾ@#�Z,��t�� ��c�g: v�a��2F���\]�8!�+r+�?N!�� T�11(kT.F=#����\*�W{�ZT F�w��-�/S��D�h��v�Kw��1a�M"$�J���2�\_z��U\_^�#^�C��51q߶ǰ� Lv���hB3vis�loڲ��+&�h/u\_����O������ T��S-��UPI��!Ћ�5��d��&;k|�F�\* /�� k"v�Ȯ��x��P���Kg�G��ڣ��1��ҡ҇D Z !ט�RO��3u����Ś���Yړ��Vx�y�?�)�K7l�h���?F��W��C�vư��b"n4��M�d(��X �u|��ap���=�/�-g��Ô��X�a��\\�|�Rf���&t��� �0���/{���=Vku��r�aQ��Bn�����'�#��O�U��K��r���Uxst��ĆQ1�.�Q4Ve� \`� \\\\��p��>t��>:�K>�f� ��j���M�o,6�wt ��Ȫ��Z{uKd�t�i�.{�u�(��F���7�f?�z廱~��#,ؘӡ1U\*���G�8�����gDY���g����U<\_7��\*�jm��$�7��Iw�.I��\\��\]ݔ��Rl�Jt�����,\[#{m���V�d��2%\\�\*%u����=���h���,��1�L��r�WU1�{ND��B��2��VY�l�%\`03^� \`�'������wt=�l���>�����K��\`��A��%�L/���"���9�bK:�� �ig���t�t��Đ+4���J���?U\`nV����7X�xΞ��^�\]\`��t�Yє\_�I�%��w�x�\*��ż���%e^;CQD�t\\��j=���L,����X\`�)�P����#i�,!�i^u���x#�LGvMKͪjk��b �T�l�5�pP4u���E�.�E�|u��l�Nr��� ���B�xI���voL�EPUu��jЀ\`N�� hQ��l�s��y� p�.�0�$���bf"�SI�#�',}�����NW��K.+M� ��ī�Km.j�%��#F���W��~ny�pνXf��\`���^���O��ҡM�ByױR��Y�Zm��/F�Rq|DD���c��i�):�Ig �q�s�\_t�wR�0e�2����� ���#H9��A�З|<��ڤt�\[��VՈ��=��\[Z� W�s���ͅݠ�%� 6����ְ#����3�o!y�o�O�����d�2��=áttV��\\�#���e@V��{Jk\*d�V�)���iDϪM�4�X�'�4ZŢ\`m�N��d�i^h�=�S���p��U� �'���\*���9�b�� �=�����6A!X�ٕU�\_�{��w4XI�f�����i��\_ɍ�!7�F�^Qx ��>�����q��>^��v�P� ����?����Vm\\� b��3� r��N��fi�,p=%%=����u�� ��jT9/Vx�KءJ�9,�tz�:��NK��Y�Yg�%$;!U�S�z��S5��\[��|�w z�B�PH�f� ~��~�.ѣ:�\*��67��%#9.�2�AnC�}dlڃ�$x�� �g0%k��a "�Vw���r��ą�3����&^�u����l�b���<�Ӂ��8���UUu�ˍĽ�g\_�g����R��'\_B<�f�m�N�pj<���A��wʫ��������Bz'�\_�\`�) ��\\����̏Re\`�����Ȅ \_�� �����/Y\*�/,\\�}�.���������K� �҆�� Vq��%a�PY�;�sC(7͜WJxGW8�R ��b�B|��\`����PQ� IP��Q�Sx�%040�K���Ӄ��Hi ��L~ez���%�9����S�ʕ#�ZE�=,� LN��'K(��\_F�W� ��b��W#�Ɯ�\`'e�υ��h����ՙW~NHl�>H~���\\�C��a��n�\_hg%���o�� � ��'e���� �����.!�r�"��\[ϓݪ�.#F�fPr^!pƲD9@U94�H�Qe�|"r߈��lg���s� ��ل���^&m7�=��l� "�ߡ��\`O����B���� KMk���=���w y&��a���7(r�9 \\�� �c��X?�"m�L~�\`� ���+\[�� �3�t� ��(m�M�-��%�ʣ�LMI�:�r�z��Uv^T�G����R����?�s:;VЀ6��\`bl��+~Y�4y���s>��9����A��'�h@��{\[/\\�0��”�SGS^c}9���A�b^>\*���y�^��=�"�\`@�8���E�&H ������>,��Uu/{IW�U�p>R �d}�o60i� F��.w�"��N�@�>h�j:@g.g%ӻM�x6C��ekGã:�2W6lo�ͪ����XX�Б�#XdL�(��Q!P���BK ��Ё r�~4�d�)�G(���~�F7ә B$ϵ���M�s�?�^��t&\]ҵU�J��2�\]�����J� " H���AI� (�u�43C�d7$3�� /"h�=��.��y�ٶj��;\_\*$, �wLN\]1,��O���=�x�3 )��u����@Q�<�\`�U\`���9�u��|�w��� �\`1(�R}P��<�H�pZ���θ�U�Fvz@5��h"�j7/z {�J&N�@�Q3ϲ��f)��Ro\]�#\`��z ��mڀ��c\*X�ѷ�6�.�y� %\]�A4 �K�� �q��j�\\�$n%7FD�z�PPF���o�|˟��(vWk��6�$r���M!�|�����K�l�)�m�GZN���&"�@T�p�-L�#S��"Ƒ/���n��c�8w$O#�&6�qa�D o;��4�qO���� ��i ߁�\]k�X�����3�͉U���j�D�{ݩY��3���O��Cg�|��<@��tY��2�����k���P��¨�ļ���� �-�����2.����\]E���/�5jU�͆��wbS;��5�ɴ����%��A9\\7P���R���P4tp#�߻����}�-�vJF .hFP(ga���������<ؠ̮@\*P�@���%���,�Vĩ�Ed��⚷�Q�����}�ۑ��'��t�e\_��6�l���zH�ZS\[ ��mf�E掭xcdK=�(;b;�b����ܹ��xJ!��� ; a��c�\*\]��O�0gƍ�@y&c�Z��"m�(�nم�\]W���FZ���^+l�I��v-�/��a��zC��:nk+\* 0+�b��G��¤�1Q9�� ����2�����=��Y��76B���1H4~ns1�p��τ7�o�����L��MD�d���z�9crC�ǒD1\]8y��!Ej��SYn8 o>��hva��j#?���q:� ��@��N�1�Ff��u(�9��"-�詳i�?NW�g-����\]^�?^��� :�i�S ��D�O!#�)2��uo�/!̀�8���&c���Psh-R�>�¸�g�������o{Oh��b���9�3�!��A���p��U�Nۼ�!�kַS��=͓FIឋ�$�\\�e�\\�Wc w�Y�6O�A�&7ӆ5��d�Ĺ�R�N����+��3z\[��1"�Fq�Z�o�G�P� $y��\`�S�c�V�:�J� �Nd�+5EQ�(�R7�SCs�8ݪ��1�k��H�V��,R��k�I鈹������c�k�R��E���"m-&�a�f fiR�!�y45 \\�����D3�J�sR�x�@' �4��o� ya�2&"\_�^7,\\����!ܵ�!?\[���2a^(������Y�g�\`!PI:����u7� \\0��:�b̊w�3�o;dn���ķ�s� �&�apWF�D�5��Yᅥ�6E̼���k��4RcAn�ׯLC6 i���|5��%t��mM��"��\`Z�Wk<1u&��FJ�D�����DG���f�W��0$�Ps�7��bkcp������ƹ"@S��"�@�����8p�<�tot�&I�\_OD�!j'a2\`�h��e��d��!�+i?sb�5&#��|��7�������"�Jqdـ)�鶾+C�G�8=y{�D��D��� ���:�x���2��6C� N�z� ���J�YkI��XY�/\[;� F��M\[�! zD� @����}ʼ�XD��ZߤmWź�F/�&�q���1�5�J\]�g��\* Fx>��ִ�c ��$�/������څ�h��ֲ}o��gO-�����1h��=-�o�)����ԙk�\`��얦���?i#�ŵ��j ��ʬ�\\W�ԂQ��\*yR#���=�����&�����7�}Cn���ӏj?����g�\\!t��.AUW($�e&��\`� V\]> hs���)��i��6h&(^��b�r�?|�wE�\[�0� x���6Y�����1����U�+d��Z��?XA�pVޏ'��Y9�D��wn���,u�������g�����p�b����2�τ���V9�kr\[;x��!B����)\\l���\[|�'\`�4WЀ�����C���Q��p3�����kM4�����1ie>��s3�=����dC�s1���� ��� ج��+���, ֡�<���B\_H�\[�Jjk~�@�\\�S��Z1M���o������4}�2����$?S߂��I<�<�= ����%g\]�!�%��f����Z¬��)� ���U�%�O �#���g����A4�&HuIHm�Q�Ge��2ϭ?\_ j~��տ�Z�pThZԞ ��ҟ���h�����o/�\*:�����)j\]�9m�����H�d����2���\]ҽ���Y�O�P� M(�-p� � ޞ�\*P��26^���Zn훫^��L�=ލ�#��Z�4�<�˯�K�~2D���9rG��j���w��ov��a߫��E�D�&���@���)I.m�:>����� g�;��t�i}���XY� %v��V�eS"�J\[����r�ւu+����؊�7��ŗ�hT��h�N(��J��D��\[D���چW.�C��{�R^nL���\[\_�ܢ6�Հ�jn�, ��Y�0�V&,'�H�ǣ�+\`�Tx\]�VD�A5G�y%\]��-~W�N&�}�p�P���@�� �������M�Rk�M���f�\[�՝�#:��9ԭt�(�>����\`�����,�)�S�O�>��  b�E'������ A��Z�e��u���i��ֹ�������\[�K% \\�$0B1t$�Ϋ�H�m��U����61�{4֔��m�2'�Y��"m0#��\]Ԍ�(����A��|�~ +�ܓ���a��2�\`B�W�ث�v����q�\_=�x>\`{��26�o�c Gc���yD��0\[� ��wR�t'8� C�8Z�%0h�-Kp��K�n�0��u�V���w��04eV)��c׳�u�i�#�u������x��n��p� T���6��E1���}��c#��QY��uݗo��m2U�zR��2"��-cQ��$ᯖ��n��2�m^d�(�OJ�輕WC���� ���� ����Q��A����������۩H\_����cl�����Z���G�\* �N�^ p�l�2��g���)g �ԫ �ne��\`+����ƕ�9�\[�j��j��=�7���?E�HTWJ���zJN�\_��r�1��E�H��%�|���" P7�yf��/�Q�{\`�,y�b���R�=���ҨL��솝�$ў��l�\*l#$Vz��M�Rndb���)�L��c��H$���Y<�\`��j��V��1���n�c :ĉ��x�\\�I:�'���;%q<��(Eu�^����꓂��4���v���5� >��o�"٨�^���Gw�P����9~��w�y$��$�:c\]��c�Lh�<\*��@>D 6��D0D�������!���ǂ�+� OO ��+;ϐ,I� /1��a=w�> S-�/ ��:�~e���?�8Ѝ\_�1ܦjP���ز��#~B�a��3m\*/e�� \\��K @�M�@\[ld&�d�c�m�ɗ��%z��vаt� k�����ݭ@T7Bw($� H�XiQ��ܧg\_� ��4�'�H����ű0(-"rJ�:�0rZ�ݕqA3�٣�NH! ��5"E���1⻠��Qu��@Y:�cg �g˚��T\`g\\\\ �����dP:���?���H���X��5mWj�+�ͦm�4�Y��{��N�C�7��E����u6�~ �Wڀ�"y��!�uZ���b�����������!�+�1���ͻ��8� YI�P��\*{?����� �h-�Ľd\_MU�ʨ5TK�� sg��-��ߩ�q��\[����$j�\*��;��:d�u�q�Zx�L����OeB/כ�U�&w��L��\`\*�����uGŝ��~��n�6����鹠���on��7����᳭֔��C����|f�Sk��2�O<��խ��o�i\*m���ҏ�\]���\`�vA�4�uUN�;F���I�3� P�V�\*��-�Ka&���������6���}�g�#�V�K�@U6\*=��\_�� :��Ζ��GSH��!=�T� \_�������� !����U�\[��T��6�W ��{�3������kťVi���U5�}Jn�n�й�x@:��k��v��^7�c1T�\`�.a}��d�+uѶ0'� M���+� �� r(�P��V���A��ǻh�W>�?��X9z}Bxֿ��6 �޽�l��8#Uu�g�d~J� e1�����͐K�lK\\���j�����%�$�HS���o K�\*㤁�,��$kӶM���S������u.u;�D�H�g&͌�&f����$�� ����9"��� �Bn<�&��v�� dD\]�L��R�}d���Q<�䫂DŶ���;r�Q .�{�Nt�R1��hB?�8�p\]B����m����T�B\]�sx��Kx��-YCW�ؓ�t��I�Ё�ly��| �%���2�<����P��ύ,O��h��),T�f\`�}�u��/5h���4����m����4 ���SY��PR9���)�ص���!���Z� چ��� 8� ���a��� �����?}�X���g7��S�Zu���;ώfɇ��3FjU� ��>���m��o�)J�U���I�j-��& �x�k\*Ž��^,���q����Y%�i��yh�ؘ ^��\[���$�F۬De��6焰\[�8����V�Ը�m^�\[˹ �u����A��6���b��p�\[!� �9��K�!cGʃ�����o���!Zz7ڏ<��.\`���8/y�"R\`�p�+c��,��� �ȞB��׶����{��\_ )./�SPMhq�I2 s�z�����3ƒ��5(�<(�v}�q%���Cgѕ�T�ͻ��0��97\]H�{nX�1��/�ԽN8�����U��S�<~�>�� �l�T�זd�Ғ���7J;Hg�bvSV���4@���, O���-�U���"����z� F�豥DvɅU��kF�t "%3|Ş��Z�VU� �X�T n�����6�AH��3�O��r����|��.p���l�OƢ��fN���V��R "�;=\`�;��WCc�T��\[٧̴�Cs�9���T��>����v��8�\_������6�1P���f�;���'7�j��f�"|�Y�Vʡ�x$W�����=$��� �{└g�L�������Y��=�?���Qx��t�9�,|h� r��/þs��^UfL�F��i8���d�M0�ޙ�x�m-GF�4�!11���� W� Y� Gaz��Ʒl�N�Z�\*��g��+������Rn�ǭ .�5̓��Љ\[�^�g{�aG<տ�;qxx�vU���+\[�����RY�\*-�Ȉ/�����5��K\[(�8����yf�� ����%O�0���J�m���� 3�i���/A������g���d ��4pzw��(������F�817��+r\[���\\�9�P��^~e���­�n�ؓ�H1\\��re����?k!x��ȍy7�"�\[p}��;.R-k�%���ô����s���Z��-��\_�#ݔ2�!�$e��\[�o� �w QL���EOė �+�|�@�{�65�7ݴ�>+dm������x�|�n���㩡5Aϧt�n5��i�F��{�+�hիN���  ���Ѣ�8�?n2Z\*)�ѥ�n-�a+4�̔� �vT��� \]�x{�y��K�+ ��o�D֞�3�\]�� ��ɀC�?y�ڳt�2}\_���u5���B~r��1W�~\`eA.�g�\]����I�Y�f�+t�V;� Ѯ%\*��#7߸��Mi؜���ըa �pU-tN�N�=��=��oNG�$������mm���G%�Ғ�t懷 '��P��L�¥{�U�� ��fۇf���@{'\_�i�����x��0�Ih\_�c�ץ���4N����U?L;i�j+\`�������Y����sϾe�Γɤ�;�ĕ��<4� /bSpw (�\`��pn�"� ܔ��A׌B��G�4�o\`��� �m�� ��G �cB�r�< ��:\[����s�B�{of܆�V��Eme����n ��b�r��M�2?��b\\%���\]+omQ�a��~�pԮ�'1h�N�}���e!���z j�l��\]��‹(X6B�N\\�uly}��\*�z��?�����|���l90P����\]������})��w����D����Rc@jnG�>�b\`8��ټ��+ Z�� {b�\_\]�8j�e�h�;tM\\G��,eE�29�ߡ '��w����)Mbo�Ä�2m��oߩgI9�X�Lt�d�Sr<:5ysj�R3��AT���F����5N.f�� v�2 ��ܰ��k" l-�;+{�D|G''� ڤ�=E�$�MA���˟şg��w㯑�^��R����;n��Co���\[+P��&�,O�����#=�����dI��'h<��q��Q��V�0\[V(U��"O�;� ������'����(�uA� ��7�@�$II�̮V�ǐ������,\*J ���\*�W~q���u�\*;T�o�a�2�C��p��6\*���ۅ2�ӺgX?^��̃����|���lܩ4��n"~I��^��鵐I���s���#�9�y������ �bU3fs�8��el7��I:���)&�^��Is()qP��w�(�dX��\_y���<�nZ��:F�1��Z��-9:S�B����ww��5�5m�9@1vN�2�� v;�N�z\[>t߰>���lK���� jL�%�d��Ҍ#��!q�vw�r "o���bҦԪ�|��:s߸ز7y�N���̓M;�Q�oM-���727+�Ï=�L���ح;����qdM� 1� ��� �pQa�������uԱ����<��|�fH��K�Ц;�� T����>�5���R��B\_���X��7r!�B���X��=����˜P(sҀٌY�KD��$���7��#=����KCIc��5��'%������87"f��ca���n�����·LO�#� ����U�Α;,�t��� ��������\*�>�-V�Ct��q�����ly8��r���)�� �g�����h3��3�fE�S�����b�&\]�y�5�.�<��&�~虧���,г�~X�Rf��rф}�E���¤��\*�CkQ)Y� �Zz� �B:�evȷW���T���A�8u�I�w�\]� �P��e��+��bc �p0��OB� X.bBYi:��G���cz�\[%p�ׅ-�)�'#��oj)UDg v#;��,��SW3<Κp�#6��y�m��?���s��dq���c|�����8'M�/}E\]A����\]9�;x� � 5�#�,���\\�H��j\_�M����v>)�?O�\*�.���l&q�����7��m����b�&�������JS9I �!��) �zW��$ ��\\�4�jN��\]���q�d'�Աb��ն�ɢI�C)���n4Z �d��@b�����%H���G�a� �v�$Xu(8��P�Z�7����dm�.��ReVY ��U����6J��­����4e��a��Z�.Liݱu��R�l=vVEK\[���ic0C{�C�3A�\\�2W�u:��c\_g(�d���gy rA��?���>U�i�VL&����v7p��,LV>85�� �����<��(Y��Lx ��?�'�ܷI,����}z\]�cʒӬ �ڤ����?����s���"�E�Y�o�po�D������B�H��&|n����Vb�Q�@�)��@��F��:{#�u��>��!�y��W\]�G�� � g� T}��F��Dh'2�d\`�@��t��sAX���T���3�o\`(��A�a��}�JG|�Jk͡��oW&U��(u\]�t\[�Z\_UC��霕x�\`g;+�{%�OFN��Cb����?\_ZF񛺶\_���܃��a+I@�5�T�c�JO���4���Yu��,9b��a6�56���Y<�K��W�W�P� -�� 瑓3k��1��/>m��賱�s�OJ��dEP��>������2#(�ē�Bv��$��\_8{�?7^ݩ.�mٹ��~�6H��A��S����TV��k�"���ҷ{��b�:,���\_���H6tt�}n^1>��R\`� f\[�C�n�1�������V��+�啝�=���uI������Z��b�u�\\�B��~De�W�WNͲ��b�;��� ���yU�6���|U�\*4��F:\\��>�T�\`��f�~��S����MQ��\[��$ъP����I��半��>�j̣\_ı�)������t����\[)�����#&�Td�ۗ��$��zv�L��2�x%ul+�t�U���.^��1F���8�� ���Ѵ Ԗ�\_$L�-,�4���J;$�\\j��&�\]�@�/ݭ��p�Y#H@�G��k�f�iO!�\`͓ Z'�����B�#��tA��k�ߠb�4����o �e\_+�g�PT��@��+���| �F85H.I� 풜�W5�l0碘;I�w�Q~�We6@T��d�ߺL��zPqS�<��Z�J�2���h^Ρd� k��f�öY7)�V�%m:X�\`��h�mxF�irm�;饶��f�&}Ew>���q�C����xΫ8u��Ӌ��+3M���n���BK��C�l�\_�S��'�ʕ��q1��7g�Ys�ڧ��Tg�UV�Y=�&8��iԷXR<�xN.IC�7�}��U ��W����4��f�y63a\\�|F�\\� K�><�&Y;�7\[�(jq�'{�Q�\]���;VW̢�J��|)��d�l� �!tVY"f����}���e������n�� XX��Kw�L �=>�t��#&�#\*e;\`���=gJ�.Yp����Pވ�;�G �7�j��.E����,N�AF��МZ;/6<>�Q����E��<���=;Pd�c\[soO�=�~��?�+T�����<�����mc�>������\` "OC��Zn �rT���IT�\*q ��%�/��Vf'��K����ı�ּ�v�M}�y\*3�Y�J��7�-�b���}\`�db:4��sP��b �P�%�\[���zw.�S��1hÂ�A\`u9S;��}my�g$�r�� 1|BF5@$Fj.e���.(e���Y\`�M��jJ���҃�V� Ϭ�κ�ɩAK���Y��ǔ�bg�k��J�j�� 9��o�������ל�WDXю��%+��h��r��P�ޅF��3RCR��ݣC��V$�k����9��7��<NSb� \_��<$"I�c�ִh��J\*<����?���X(�� �F�r�>��Ԭ��~�y�o�:�m�ua�5s¬�Xma�B���,t"�-��U�.��H���f�u�"rl.2&j(Y��� ��� �=c���l@++��2+����e��sʏ�.;����V��9�y�J4���S6r�q�����3����3I��H�𮏯s+�ٱ/��ɾפ���W�\_o���ruӒ�F3��aT�a��2�Ojy���b��"FѺ�%G��G�5���\*<\_żт��y��>��x���ny���'.�1)t�\]�ec���gA�}��K�==�\]z|�qy /�$���"������W&k�+!Y ��.���� ��w�@��B���#�\*qFz� ��>Cj,s����n��5���t�\[�0���@rZ��(��Z|�>e&�%Z�4�V��\]�}�\\� ����c��\[W�6d���I���q%dЀ��,ֵ'�R ��|n�J�z�j��� +�8<��~m���A9�k �/f i���j�͝ a�� ��o��in���r:��DN-9 �D;�<y�'�p��\_������&����E!V;>�}AMiM��T�QgӞ�T<���UUsl� d\_�!0������ �kJ�e$ Z�,u\`�t#�i?�s��%�Kž��"�$)��r��F~���M��PO�3�m�9u��rJG\]��t��ž�,6 &T��=\[�+�J"���fs�/Zm�,�Y����)�mN@��.�Cr(3:d�'�07�qa�q;7�3p��z����\]���7dX6���3�ڹ��X-иq{�nK�+qq� ��X�3�\*�O�=��z��5hl=ZD32"ބ�>�5�)m\]z��~2\_i��gM6����蚳 ��\\�m����u��tWIZ���w�l 3j���&��/-xRa��2K4�! �凊{�c��Jf3���01v���:�ƀR��=�Q�$�nbC��A�� 9��0�(2O5�t(v=E3K�g���˹�b �� ���� ��{3��l�� �x5)�T�\])^�΅})4�c�M�)�T��O�x�L⠡"�#�\[��&:���<��P�\]\]��wB�\\�S!���� BHM��xgW!{c��(m�v�K��\[����$��� ����~��#�^�ȳ(��\_�@��7n�?�@-\*� ��՜o��7,��K �����J�)�/.���̧����>�Pw���E��k%d��ɐ֕ar�^�6n>��e�i�ïi#g~lW |�����Nfb�k�oǭ�\* m�QWa,8�+�� �:#ʍ t��s�C���tY��o� "�{�f6�@���ͫ�S�2�@�v�����Ϊ,���TS��H)�3����yQ��a� �E�7;�s�,���?�� �\]C�/��Π�;4�C��t�L��+\`��KEc��yZ��~�?�;�����U�H1�)�|�f&��j��E���� \`����C�/�p�e$��5�2��'��l�^C�\]�$��S�� d���>��{to&�+�����"���i��?c�0�5L\]7��Vm=�^�׷�zS�\[� ��Ő��S&�,yU�LY���K$��8L�0���H� p@,�F {V�/�wy��{N�&�2�"פ\_kc~!)�#�֦�O���ڻ����UN�������T��� 5R2�P�J��;���k���N!�� �7z�Pd�S�8�O�ѿ�n�9�EO���\*��� � ��y�~�N&�,:����5Gi"�{��<Ҍ�G��My,� ��вH��+���'� ��Eе������x�J��,�Z6A\*�a�d>����(�=\](~�5�rp�v���$�i��b\_�{�\`ECs^Z�6I�b�='#ܲ�CQ��\]��v�(�����2�S�~��&�o ���ʼj�����Y�L#��I����X���^�'ā��/o��0ĺ�G�2�v+�D���E��=�^���Ο�{�ْl��D�m����q�m�MUy�{"{�����\*�v���Mp�� lgp�!Bʨ\_����m�KZn��=�����+<�����0G�;A�V�Y���D\[�D��Q��kg���P��%��/����E�O�$��\_���b ��v������\_�,�yU��T�\\+̓S�D��r�̓�c�oW��~��o��F8>��:\*OL��+��r�pj���Hn��i�y�>��&�Q<(SY,<�:o�\*z��lZn�}APP-dt�|ϪPꏊ�� ď�V��j�M\`� ��.+\*G��4g�/�n��O ,���6uY�n䉟� ��l���$�b(�M~y e�����A�3�Z�3�O�9�Kn5���\_P\]!�̮��~\*� v�N�� �������z�6��2���'�u.��U�K�\*�#��q�{O����ތ3h\\��(��-X���ċ��P%�<���8���6{��@�/#��EJ޳�.YP��f G����h���Zږi�/�B�B����Rj�^�����4@��1"��a��'�\_�j�\]��ul��y���bO�ڡ�����./���\\���Ȫ \_���=L�<��7O����v/�DV� �̇�S���MH�%9~6�q�\]\]Q3+��� 5�d�Q��M�6\*���2��%f}�V�����3�>�w-��~�����b���n�� �\_cm��l��?\]c6#=h'8�� !����,K��0�\[���\\ʪ�I��I�x�oW:��\\q�Ŵ9-S�Á��� ���\\J�Fo ��N�#sA55��\_��jׅ�LSN|\[�J����|J����w�'k�'���������5�Mm IP"��:��6� ��#o#?�$e� %xob��<��8�f/\[&0�g��s� 35mI9�f���gG��(c�X9Fť�/=�Iቻ\*C�-�+H����& m�H"���LmV��%�!��˦Z4GV���o �ٮbs�t\[�@�kp޴�\*���2�=��y���\*� ����m�C 7 5�X� �2C���w٘�4����P��2Zgv��7T\[� ����O����O:'s�Ͽ�V�<�ͨ��-�.s��r����c��0D,΃�YQ�}�,�zo�=��0l�˞��@�͹86GWu��L�I΅�)g�ss=<� \\�� ��#���/N��I� ���#ER.�P7Z��q��� ���r�Ú�a\`�y���,�o�z�� ����ʗ�)bk�ZV5��Tr�D�w��n�ډ�?� 7���B�@ ��k������N:�k��1�=m�����U�\[�.�Ò�����HP:�Y�N��p)�t��^� dW��Q�����9��s�ޞ5ExD.���}�-��9#�5���&�㈄L�E��#�pv����Cu�EVܞ�'ü���0t"i E�YP�k��85���0n!.��mU�H(����oZ5���q��O���}�hx8';�̨K��"�a^)�����Y��3�n��/��S D�@kt>�m}\`�m��Ե"�jtgi���.���F�Ct��p��nI4�h/�����Cc \]$��r��r/�����O�W�I@�����N�cF^k�GW +3qW-^ x<�2k��Š����R�lG0��9�cgAZ%��q����jp'���;�#�l� V-�@B����Ն\_%pn�����q�MU�g�ղ^v��4��ɪf%��@לS),m����.�G�+�,.�0��� �w;����"��J6������K��f�k\*�o1�7�� ���� d�θ�n)�ŽTU��E�����S@��zz�8��&�eD>��3ph����0����̏'�\\$7��c���p��+#l�����'OU��u��kJ$�"���~/���b Ł�⇼LuO(�����X@�|����@z+�~��z�f�������~c~�h/��j�l5��M{�̄�(����n��������S<\\6��#r\]�7 m�un�\]G7F � h<���J$���8�z�Zk#��x~��\_&r\[��E��9h�Y�72�&Ԫ=�om ����q���g��įCa)Ky#zMK� kpw�h�����Ս�������h�P��Bg�\* ��V;V�H��W �z"^���~EM�M�骩�/J��)�ȋ���莈K����� ryr�T�w��M� x�� �r������Cf#�q�"�A\*���&��v¸�RV5؏ҿ�H{8�l�v������g�4�h��{'�M�P��3���Q��t��<�4� 4@w���+��5 ^iF��C ,�Im\_4n�h�� -�p�2����B+���x��HA?�ԋR\_h��N׼ɀ�4�+�csB^S�"\\�(cs? V$\*��7H y����/&�G��3>�ZC�Ssm���n����(〗2����g��� ���^:1.��N����K�\*pn 5�)���{\`�\[FD��?�}����I��o>|�\]�:�=N�x ��U�zN�/2 jMf �+餙���G�fP&yד�F�O�G��rѶ��T�>\_n�B A��"N��Y�hb�X� �z.7��'a {Ua:���lI�Jx��cܢ���N��h�k�w#YJD,���Ŵ;9ʅ����Z��\\"Y�����? ��M ��sV��Nʬ<���;�r~����)�L�B���R��'�k��^����7�>�%��wg>=\* �����) ��9�(e�^�$�PK����fQk�og�3���۴\`-�4BL��y�n ?s�sx��w����v�ZS9E�v�?��\\e�WeS8 M��ۂ���#�ִ�>���ϭvy���溺���:+|�ix��e�9���rNT�~n�(l)�%����L���\[E���'X� �\*;�r�'�.����<��td�u3�Հ5\]�d�q�&����r+��"���HTU�\* B=u���(\* o�g�ɤ��:' >�� ��ŋh���N�Vg ����d.���eL��BR���="�p�Iz4/��%G�b�^ �F�v�\[Y{����|�ѣ#"��} \\v��os-��w�p�}�8���a�� =l,f� �-�g���{���ĕK��iڅ'y�P��0J� |1��㓂���ue6s�\*�y�e�$����kv�d\*z�qii7�m�@�������W�^�r B���u�s� F�l�R"g�����Qot 4݁i��/�7� �\\xg����4�\*���lP��=��(�L��ƇH%R���I�{��9GХ ́���V�����A$,V���gŨ�m�|�V1�ch�H ���+M$�͢���t��fR� ��E�\`{�UvYg��������4�M;64�P %�B�:��jM"G�-mI�!�n�S�zT��'F�iT�\[��� $�4F?����Qd$@̙���y˴���h1 ���� {�l��� q�J��,���"����1��f�m�p8��{�X�J�>1�9۔��f#���d����M�,9^�;R��YI�K�q\`>����j�25����JIi��<��9�\`~d�ՠZ�I�坮:�J^��ߌ9�h��\]yjӣ�Sȿ��y�3)�% ��$��J T~\`#�ׁ �V}5���O�;GxR�3��� �XE� \]��K�ɳ�F�Y0ծ�y�^C� Q-� � ��i� �n�@o��/�+��ў"w��Xs Eg�SZİ��)� ��g�KΥ���6E̠���m�� ���\[�������|\]���u�}�+s$�V��lu���v��$fl��\*,��hq�����p����!"�����k�����t��R�i�=YKX��:g�i�RX\[x˪ѣ����(�c�R�B���7P�����\_-\\\\�L|?�o@��r�t��� <�3�D\[e� x�}���)������}-�ٳ��; �P ����l�x����\[ �I�v��b�7g%��9�5��8�i�ؠ\`A���ʞ:f���+F\[pˏF���{I�Uirm)�U N** --- # Unknown 7z��'Q�JG\]%���w�(�\]�0�(\]�������9��\`q����jH<�l�H�6@�K����o|ZW��wm�Y\_��\`�U�\*��b)�A1�;{����>��c�M6m��G,s2 kG���'J�ȧ?8j�/~�H��Wd��������\\cR5�Ѵ}$���!�\`���9�d� �Zm��ډ&��+>m�=�V������4� \] �s��r�e���~B�ԯ���7�w��\`�(C�\*ŒW��@Ż�K��7ׅ{�P��:��@ȫB뾝���.45�bԮ�حe�o�dc��T�d��)�ob�UdU�ľ�J��5L���J� �3���ԈzTM8 6Ob�����,J��3h��Pg=E/ڒ�\[K����R���<�G����:�j$�\])�$c�|DTq��.�8FQR\`�u�-f��r��X�.�TJ�d( k VPG���-��^��2�=ƅ�%@�T{^����߆��m�K����g���� �{�/���kb���r�>�W-z� � e<�2CG3���Bus&<���a���z��p�fl�ccW���v��?��eR6�Vݱ�:\_�~��X������7f�>p1Rف\_���J>�1�?"��ùo�Y-75?�+��\_Nōؖy9/1�\]��ɤ��5���e��"+^��0;ч��R���v����X�z)��}�o蘼�D�k#nZ�X�t�����e�o�Cr�LQ�3�.�q��xU�&�H:V$|41Gm���c�1����.���F<�@#X�\`((��>�e)q{�������å���kCZy�>Ľf�.sF��gDw��\]�)��\`��M��\`\\�~�2��4�ɱ2 ��j�|ذ��F��\`Ph��uxq�g|�����$�mfvW ���m�i���m��R5K��8 �,���C�w����Ky�Pپ^Ђ�^�pYb�ΒFz�\]y͸����y nUߙEh4�#\*� �������V���K�K����$3��ڽ�=W ��(�3�Q�\`�A�|�P�g�� Sd��� f��/|Zε�,g ���Zv�}��խz�.<�L��@;k)x�+9��\*�t�E�03o���ݷmH�xw�)\[�X��Gk�\*ڒ��t#����T��>��3k�t����<�Ե���t@Q\`S�|T��A~� F,۱Dƛ<��� ���ɯT$�V�a�糶�o�$���j:^����ʏ��td��.) \]lP������;��ſP�Q!'�+X�LG����$ƦC��Ȓ�Z��?����k\*��x�Qa�Q#�������5p���2r�5�Bp\_���-�����L�\\��������ҪB83�}�z���?�Ч �R+�O9�k1�i�� |f�7���d�!�E\_P�:�F>=@rCDŽ�t� U@;�l��%�a��\*�H� �}���"�L�&{ -⩣������n#q��}��\*�w�h6i�7�\\�Ot�\[��w\[���?��y�c�f�$\\)o���F\_w���KTo~A�G��e��v8��I��^ � �|���?"�nz�:B�e�u)?�ﮦ���'���'r�A�X�,o���a���{�,���~�{�\\C����5�\\V k�$�7/Ʉ������@�&��Y��{���������\`�SiɁs� ���0z�C%k�U>����$4���G(�t\`l�(uZ��V}���TX����7"k�m�q����J-zKߣ\\�L��@kQ ���L�!U�@�.PB�?N�{A\_Yƿ���ȻV(ēf���v�(�\_M/�AI�>�6��ς��\[yJ�\]��+I\[�k������7"%��c!"�s�r�4��܏�i�MiF���ƭ��ҳ�����C���m|Qslڦ��� V��ZC���WEz�I�5\*Y\`�0�d���\`^8�wBFi �Emw��8������������}zc�.�dܠNJ�Jb�b�M����~���RgЛ�p�h�r.VT�J�즿ۄH�X��8/-�F�Y03�fN����+�/� 7U9�x�������TM|���P;�I��ȳ���' 6���\\/� �\*���:����H�h�(����#�.�ŌL�Bi(�Ҿk� �=�/Wgmt�.�%tN6rKwjKZX�\`#��=�A����ċ��s��\\�\]$�� �02B/�R���z�\*���D$�n赁������6�ؔ������t5��>?�/��(D�iI�>�$H�CõL�q��ⷸ�'�����\\$� �VS����K9XE�bl� �̃��=!�q\_� �y�w�3���a���1#�{�\*�I��%�m�P�T�wB���,(T�H� X��Mg�G���������z\\a�z�鿄�ҋ&�t�u3y�i�5���Y�!\[pM��X�M^湣�������:~6\\>.��\*ȅImƗ�d�xsz���x�� �t�?����y����2�md��W���Iy8E�—�����nj )�p���w�Մ#:X�֐�; \]Bج�-�&5�۽�� ���A� �\]9���v�8���ك�(+������� k�4P��ZD�ג��%�t��2�|��\*Ŗ�91���8 �RG0�͎i {�������Y�Y@%�I��� �H���������Ee�,�����X"â���8mg&ב���\[�RyN���V$�55�w�<63O\*4Rmv�����)�pɛ���9�W�C�z-\]Xyn�S{���:�F��\[ Z,!hJH��� � ��m�H���A ��uX�Y�M�5S@����퐗�i�\`‚7~չ��B�҈�T{�t��f�wG�^�S�Ur� iJ��d�(��5�-� 'E�F��ϛ9�D��7��p\]�ij�����jaI�=sQ����F. ���?\[l8z/l�5pS��9U�����2#ە+��aYM�c޹=��\\e����aE�d ��H��Dv��k�?��� U'�B�OČ�''�2l+���¯��8�54�\`� Hj��2\\O9�����Tx(�ԃ3D��vh"�\*���� %� +�X�#�Q�=�tc#�.� �Q�Hg����d�S=�Yz\\�m��O��K8JJa2M+��nt������o��vW}��g&hu�2R\*b��u�����ܒ��~.J\`� �p�;-�۾�%%y�K�3��\]�(Gu �,�A�Fv�|�\_�Љ8�\[�����Z/A �%�k�.��f;%bF|X����z\]Q�s��(��C2�1��1��8���V5�}��o !��aA>���6�B =���%ډJz+�6�X�)F.b��hs�8y���w��}�D��Kb\\�cp Q��w�W��d��Mt���gq�S��2h,�"T�ӗkS�˜JVk���+�B��\[��;������pl�e.B�+���Q� ��CE>��Gش��Rro���)s� 3r�r��N��v�w���r���4�-��O4N���aF��lU��� �u4�4�Mm�x�ڄ����mkpe���m�Bt\`��X�������e$t��h�TϯM��=0>����p;��� ��l����,=�QWAm�\`Pp;�&����"�|��"w�y E=�?��>Et��q%\[�SH 2�����\*�™���| �?!�M��((�&�>�H�>c�>7 7�t�����\*���O���\]br}����i<-.swvV�#�{��Z|\\͚r.Ӧe�q\`���\*x�p�\*\\n�T ���է+\`�\\�9O�����;�'�������"kZ�},���%���d^�=L�g-�"���P��S�4R2��?B�ȓ�(�/Ƙ�������-E������)��\[��#��dDK�����1uB|����6��\*a��bS� a�\*�v0G�}Hs+��ג\[I��\_h}�yZ\_��M�h�߯����C0z��Rp�j� ~4�..���\`��oNy�<�6/�|��z�+��(vm;j��\\C��� ��~v����%Wm9N�������{f ��t�q���F雮JdHp��g\_TX����+� 8�:�S�v�>���.�#����{�x�(�E�\\�%����Kee��� �Ȱ�N\_焥|�o���հ���G�o�"����\[��nP1��5�/���!���LŴ9\_r�2+������@%�% �\\�a� ���� ���p��4��q{�B;�6PB#P3�t�-��n���k���뻨�c��I�g�K���g@l#F����Ǖ�� u˗el���������\]x����s$�ˀK@���m!S�ސ���6։%L\]�r���\\��gڒ�CD�ެQ �ы�5� �Q�b�I�n/��D ʠ�G�(F���)B��珧d�-9\*4z��A�Bc��GQ\_� �d����0t�&�:�6���8��F�}F���8�;z!����#��P��֎� U�ch�����‰8�{Y\*@#��\_����9gW������Y�P��ki�O�o� �m\*��;� � ��ܸM<��J��U�U��KP�w���)ɣӠ �Q��ﻸ���/em��� �g��WT��(�!��I��-!�-A{"@�W����4����#ӥ+ \[��D΀C�f�����n&� �EVER�C픨?h5��'�E�.���Pw���rͽ�z5c�QV������NBv�l\`&B�l4Ժ�)��؟!��朼���.��\`��ŵWA�W8֣�'��t��<\`���� ���2�����B����i�o0R���������� �1�\`���kh��� x�����Z^N0t2v��?�~ѓFD���^��a��)����6� ��W�uCr\]@�����BvY I�N2�%lFZ� �m��:��:�$���璺�È�I���\`+\_R��-\\���D�,����Ap8�pzۏio�����8;��~@�⇹ĸ�Ι����=ց>��.����J|Ǐ"\]2j��(!C��'�C͐�L��r2�����<����=�1C?\_p���2iw� 㾂���=�8��ZA�Z��M�ҦX��&�Z�3�?6"h�v��y��+f���)s���4���yq��ޢ�/�^T��h�\*���E���o���xf� 侽;2S:; |��D�\\ q?t�2|�V�Hj-bW5S|v��c��Lk �s�0�,7�����&>2��N���?������UN�� OC<�6��~-B��t�r����h'4\\t�MOL\]YZa��)P�#WD4��d �>�I����E�O+%8���./>l��F�� �|�"lo��ܟ���H��\*��x�Ip��� ��Յ{>����X��KǕ',��6���q��V�%S�mQ|իb���X�O9{��%���?�V\_#6�!xك�fgΩ�S����v@�3?-�4�8����}K�����,�(�g�� őK�V9���=�}wEk�Ǫ/�2�����8���ad6{t��q�9��V�gW�{s�O�No0.m��\]��A<���5�"N6L��#���ˏF�����ȣ���\_�o�< ��a��G4y0q�ʮx���\_�#D�(��\`�z�5f'w�q\\Z����R7�)�;1:�r����'G���C��1dɮ��9G�,K�� �8J��Oi��� @���Nrǃc���w����~%l�M�� �҉�\\0\_L1�Z��S��w����������;Q|Ww��cWI6����Hї�@�06�,EL�� �r��YMW�e����=Ex��W ���K{�Ե�\]��^~��R�h���� \_�͠2 �Na�)��:��?ަ�,��6���v�!B �D�F�B�\_۔iBlٲž0���5�Pځ=­�%�Gc���S@��Qa9�Zi� �������&� \[�ѡ��#�.Y�?��&!�G%0�����cs�ٰ��b�U�\_?�>��w82��k)�����8��D\\� ��h?%�t$����ѢJ����w!aq��G�y��PwX'��Y����LK�� LM!���\`R�혜{�M b \_������0ƪ���.8T����7ߒ�=d�'j�����m��SaND�RE(Hx�J�ŕ0�tsq'vĹ5AJ���#���\_������b^��X�1��T,��Q|U�F���—�J\\ ��S򌊢1�ţ��W�)�e���f� 7 !I�X���\[��S��\\�\\�^.��5�d�L��Wd:<�s|'�)M�MV�H�s.|,Qb�\_��p@�\_���j��Y&��-�\\)�-����?\`vߚ�#Yݟɭ�d����Mi����C�� �jl���&��R��\\�n(��p�,��=���Q�w.��X̢�CC3F�D oS� �KdhԆ���l��2���k\[��i�X,�y���e�~�ni�Hr�B�2(��$�˱N9�����#n\\�q@ �|��A�������$p�jE������p��P������,Ը'��� �麙S9�3�?%\*8����}�C��V�䱥��a��\*�BH�i,}�+�Fo ��GX�E�6��vZU ��ͧ�w,x�k9��T:����?s��Rg�Zx������X�d�^�1�.�"� ��R�I�K�M�Y���6�,\_-��m��٤��� �0�� Q�p�BSd���e�\\�As�2&�̑�1�>�d��Z�5R���|#˘;wH��'��R��E6%u��|� i {���?�1�=0�G�N��Zzp�wm�uv��<� ҉���5&�o�>�#W�����NKg�l}���È>&����we����"m���v�\]Ϝk��r�g�)'�ce�\`6�LW}D�g����\]�!������\`�������^y����m ����('��&�m>�n>GV�惮r7�\*EE�g)�<�qyK�� ����%Y��2@��-\`�� '�;M��N��N7��\`a�ǔ���R�)B��\_�N�v�\[�H�5\_J�cb�L���,)�^�{ʫ��5d(dAk�p�������6eV$�@P�\[$esr#;�b���~=f�ƍ��YQ������n@yhD���N� �J\\p\]��d ��i+��t��>���m)�2���S ���R�Ptp��½���\*0��������b$܌�\[��ϧ$� ��ll�r{��Ü� �Jd+����\\p Y�)�߂t��7��\["p��y������I�6W�2�o0��p���\] F��&��ޠ������\\�K��Ȋ�!\*\*��UƎt�V�^ĸ�uia6U���.g%j� \[d�#b���$���T��4T\]����� �8#����JM^ro�%j�n���Ĩ� t�,���"�AТˑ"9b��ٹ ?6\*!��F�����nA����B��l���b&Ő��aP��z�"��&��N�оm�.,����M(l�JG(���E�n��U�<���g|�!��P��ǃ��E{%�"�+Oşw�YRKƠ�����|�N�Y27�\`Z �.��ײ� e k� �R���>�Yhs��,�@\[F�N8ܶ ���J:އv����Z+�3��o�3+A�kJ'�����N%CL�HD4���{( <����NENl $x��l�CW� �-�?97�L����C�$U��\`��Y�X�h��bƫ�%�Ϻ��f�\`�z���Mg�ä��@1�X��A��:��Eh�\*6�b$A��D&�.&KDZV�ڱ�O��@�R�6a�O3�f8ľ��1�:�����M�t�8�)cILU����t�k�?w����hB�l�2�K�̽��\[fQ0�I�P�јȃ 75�\\�!O�"ŜwI~��8dBR���m�Q�\[T��&ס�?��7���� %�eR��ϼ�\*KҁPɟ��C� ������I��G�;7#F����pTv J�� < �&h�Jh��cb�X��ƭ��Z�7���?���-'�}跂a�8��x�b:X����K"f�\[�z�̮b9��\`w�+\],\*�8�x�p��İ\*���8 ��&q��K%)�er�2��jF�a>��P̨b�����}u�A6|5dܴ�3,� �^(\] 4=H:��v�����V(����oD�\_��G�/����ɞf\\�t���.,��:��\[�fF���z.50���5���,�w '�V��|~�ԋ �����>�tJ���=����J��IW�U��6��mߔJ1�MۂL�hۤ����iX4��D0��4��7��Ǻs��x,�5jε1� q \[:��ەq����/$�~�4;�4��P���wĶY�:9���)}� t�\*�S���G�:�WT�;S�!̖���W{{L�|u������۽g"ײ�'.截�Kj�%��lX6fr�:�'�9��V�i�����ϻ���lHT����~ �j���� җ�-�ݮa<$��5�; ?u�\_f������9i�˼�?����y������C����V�P3�����|�o›H�{l�#�>B�ZT�L��vX2�hr�� sn2�T��mK�9~!���s��E����A�rg�m�Z{���|�V�K Z"i^N e�P���׈�X�4/���e�Y���~��!��d:�/��<�\*6�d��S°Š�Oݘ�J����4fq�\[X7 ��N��o}�q���r}\_5˙E�G�sGN,LmqV���B( )������AJI�O��L�Yzj��0�2z>��݁���1���H���NJY�h/G;�����a"X\[��u�gM�U�)�\_D��I�bY���,���Y�2���j�N����.���WF��0����������aL�հ��+�X�\]�)�"��È�ZB���H ��X�����&���}F��z��N.R;BE�WZիW��� h��{r����� �F�� ���'@PDV2�U�+�d�G}����P�pJ� ¹�b�;Xc~^���~3ָ{|�k������F5aN� ��^j��UB��΁��)Q�K �=;�)��\\l9�ŖL�>㗂<,;�-3�gx U3I���{�y��;|�\*g�i{�t�+��t�%�������x�Q�D�&�á�y�#���g�������C��! J��\]0F���փ�@p�If�{�� } ��rÌif��^V�4�aP�0{\]�X�Sm t,vk�c�����!�ܡ&�Qw=M�����8rzd�/�bЭ��ZZ��\_���7��y�~�DTA9\*{8A��ʾ�3sʵ�z�%�D����4�d�tg�\*8�����$���.9�Qz�\`�R����rX�"�)D�!�B\*P�)2V���̽��0�MЮ��)W�7���0�e�\`qH�S�М�\_o�����,�9�e�?��g�L���F���TWػ��LA3ԛ�VAC��? %�P����JuX,�\]V�~GNf�� WJ�!y=t ��\[Ȕ8m��#�z���s1G��$�T��1���;qv�24j| ��κ�q,1�!Xd���  ��+��M AǚQ�g&ߝF4�0�}n�\_\]r.������h�7�;�j8��\]� 7l���\]$$@ŗ�T�.����:�u괹�O�ƀ��w� {� ՘����;ŧ����֖!����� 13K��L�N��)�� W�d% �YU-(Q�ߊl�<3\[���ފ@����Aޡ�-�T�zr DL�W kw��q���<�� �,7���U�,��FO�B���˪o����}�G^�s� ��X:>��ct�Ҽ �g�. ���� =��}�\*\\��� D�o8����Dh+Y�լ�J�O h�����z�\_,����'>��\]�P�Jj��c ueˎ#�U���^ݲ�xn�� 7^��6R�Y���n�\*5\[ˠu����&����69^(q6-�Pp���@"S���� #���=��G^�\*�U�v�9���F���T{�IDr����R�� �R�8;Op{�H���qT!CĻ4a�Y�q\\��́��A�iъ�P{Lp�0�P,��P���"��3��ı�e�K#�CI�� �xOa��D}\`����k�f f�Ц\*�iNط��\\2�C�g���.M�Eaڞ���G^#!�� �22ˆǒ֓W���BD<�+��w�p��5�k.q�-���p��ä�=�YW�p"����T���fW��c�C�� ˜+��F��k�+�ӓJC j �0km�=�a|��V�nU�P�Y�T(@s@��zu�Pp�!�X�\`�ĸ�D>R�̍�K�:4�a�x\`Ò�h���^7Ǐ�0��}E�A���j��1/��'��n�md�ܥV��Ϩ�l<�,b ��o�Dd�׸&���$��D��ryW��uK�}�|V� \]!#\]�\[N�#���C\[�Bq�� ��TO킭i�:�0c^y�����e~T�;�1i�Z��+\`�6���僷 �펡 ͏\]6�t$C��@�m\[�'��?�Y�x��nh)�������),&�������Ο$�CM�M�>��.K�R�w��d�9Fa��j�탬���Ҝ�~�X�"�1Ve^�č��LaJ���,��$������= ��녗�&^"6<ݪ0�+�\*!ǽ�\]���EL�� ԬD�g-4�I�T��I� �G2��\`|�v���ߓL�ߜ��D� �x���H�i����mI�T?a;)"6Lb� @������6Yv1�n��9ߩV���@��)��Z���<�^m �� �Eޗ��h�%w�����y��!́��ڣ��"C���� T����9�����'O�'W�bQX�\]c��MnY<��Hz�WG�"��s�?OJ����G��M��"���D������U���cc�rV�'N\`����<���>\[z�+������˫���A!���͂�Y�w��Y��G�+����D�Q+6p����;ٙ��J@�V�,<�-�v����iQ��2��?��Ʒ\_ n���=+�2i�NK�o ��tI���.�j-oANٖe���j���S�m�������^Cџ������ �����K� 6����O���yK�U��Ԍ�g�d\[��T�K�1w�������>{������%�?��!L���� ���Z��X��/�3��\`)��r����\`�2�Zw������&j�?��#��.����� �E��&�B,�%&�/d�$1��\*��9��ܹ�6���d�s�sg|�|7Q-o�Jn��5���E��&�,\[Sܦj(>էا=��J� �G"�Қ�hM�D����ʐ/�ޱ�$����Sxs�\_�v�|j����I�� )�\]I����Fj��V�K��̩�9�ǹ%>U}U���&?��e%=&���L)�$\['�O�nO�D;qa���.��� '͍Е�����l#'��ր�yx2�r/"�5�� �Ӝц�A��C>&l��w�EeD��>bx��ͩ+A�f����v��a�ܷ^�P��.��\_�¥�,�gC#7����Р5Z�G�S����&;�w���tXӝݺ��jR���J���.�1w�o�$Ҿ?��5��e5�hDB\]�� ^ �&m�M�FK�J77��$}<��W���җ̨����{����jt�����y�\]����"�M���<�����t�&�T�� ���;4\]��„��\*��?TT�V���ŧ��^����9��1\]�\_�\`���یϫ���,�aA�?�1�� �M�k/����DL����Ͷ���UW@Є Aw�| r�m�vɘ ��6�{�Z��,��)\_e�l� :yVr��\_���n�5�(�,bc��t�)��GKV�U�x ������C�\*�r��\\��p���3i9��\\��r./��0�˴��,n�EB��\]|�\[s=\_vnːz�5Gbq�L�w�e;ᛂ�K�\*�+�SŁ1Z3�w c��\\WEoB�<�P8^b�^�����O�>�3�Z�o�d��!j1����\[zm%,�b�1́�TV0:��ū���V7x�k��F�V�UÙ���T9������X��"���=���T��Va5��S)j����Y�tI\[F��\]�����j��WbH<��n�k�j�K�V0�����OL|.�o�b��8\[Ū �,���kf��FM�^e��Y�Te;9}D����E��d���E�����v-���Y\`0�f���� 4)�Ř�L\>�ȳ��Y���3��XAS�2��3hR�2쳁��g3�LF�Q�廊'f:���t���<�Z�7�vZ��o� �סO���S��1�����\[4 \]�s-w=�ڻ��S��AZ��.��yNx"��\*%��+�����Ɂ}��L�;D���$��\_R��S�pQ��P��b�� �{C�'������w���� �j>V��q\_W�n���+/� � �\`y3��e�Ť�5�<�\`�WQ� �T��u2� ��\`����/<�-Y��Rb\]��m�UrȾ��.�5E���͹�����Z��iT�O,�G�=�1�򂿕i�&g�,��0P�aJ�%�槗M�C�9�J���S'�.N� ��uM�\`�N�4,�H\\�@l ���\]6����ut�';+�D {1��$L�:��V����6c6ܺX�R@윊����WITO/O�Y\*.ɚ|Ɯ��f3I������{V14>>߻��yF�ϊo����ZeTv���? ������ݑ����0jۏypf���P��9�ӑ��R�КN�OI�� �yy�(��ö���ռ�+���䇇̍ݫ+�S����?L6v������X܍d���� (ǥ��� 2��\\םT�����\]!g4ߖX�\\%�IJ�ۀg��ég�g V2O8�r�;�>��N��%�P��� QZ������3�����W�%�Ž����geq^�v��Dh����˲����y�a";���N��JB�h� r�-�5�uI���)Q�&\\�C�'���� �;�t� �b<\[�����Xь�0��;w;լ0�-),��\_�ʜ���Hakc�j��K��˜��͍=��;����\*\]��b5��9��.�2N�!\_�d�"m�/~�3�w��&U��(�u�v��$����ob,�\*{)?�e�\*q�R��5.��i{-�5n(I��S~�!@>7��}�ӟ% G��I!f����x�����%���o�i�(7K0�(��� �����I�+=hـ �rc�l��F!~�z�l� t���2 ۊ�#o����dLM�({�&R�(��ďj���\[��tm ߲�x���\_\`���d\*QD���i�(�+\]��;��8\*���Yn�����/�݉6u s�,���1U��� PP}>bu�9~;�q,�C�k�b�u�����\_+|s�\`-=��z�0�f��<����7��r�Z8o2��/y�8n\\9 ilrO;c�ͣ��:r#i%�i�E|��㞓�BY4�Y���Iȳz�)�\]z\`y �+�(mD�F���XIx��g��!�T���.Q����AX�TCƫ�iiLi޹��L��J�|����6��d���ߎ����7E\]a�V#���i��s�����i��nI���M4HK� ��\`\]�|�8I@��F<\\�p +&���贈Y�󠁧�N�?l��5xy�N�W7��5�$���dy���C#��:��^Ɇ\*�t�.���g�v|�e��g�x� �|�2� �NH/�#Ȏ��J#�n���&�3�.� 2�#�\_:GN�Q{Ϧ%��+�2��}u��R��r���s�U'b�3/�s<8ϝ󒸌J��5esdt��F��\\󏊇�\]����R �D�o�/"��<�0q\\���;r�\]����,���>W%6���@��=G�����7H� �j�{I�g�~��VySc�r�!֜M�^�\\C�Aj�Ƚ���� �?� ���9�6n�2(i�5�\]\_����~u �=C���-E�"m^��6��S�x�aif5��� ��\`��1D�I�}t\`��H��P��١�QT�F 7����-�q�"a jO�g� \_�� �d�Y���w\]��˒�;���SX��S��Z�(�X�����v?�&��� �z����^s\\ s��-R%�g�/v�WR���S+������m��\\��0�U��")�X��o湫��� \_�ĵOm�������JTm"�.I|�����^N�%�̵ q\*Ў�\_�؛�z�j~I+3�S�j�܇���JM�0GH/xmZ����~��u|�;������ ~Κ1����n<�O +�=�u�F\_Vp��1U�d��D�u1E ��+O�ޓ�iV��Y}Z��B��/�#�RX/y�ґ�7+q��@���T�H�K\]z �0re\*I�� G"���0:Y�� )$���# ��/ȫm�K�e����u\*fA�� �M�Ӏ�\[�Z�G����Zr?�afȼEsK�jܬ��qX-�@��g��.�|\`5��)��ڞ�=�2�Un��pf�V9mF�nN0P��{�O(Ź\*�đ�I8�sĩ �O֟e��\*/��������.�r���w$�s����K{7�!1�G~�Q��}�p��6���(Kq q�T����6'��3�Aʁ㣼P#�'��IJh28x7� �xb�\[掼m�W�Y��Ҁ���������� PHc�e�Jn��iM�1�ca���/}\\�-��6%��~��Wbr�-��d\]��\*��3�LS������z�kd2�H.j���ϑ�Ir�Z\_�ː��Ś�1�a��n+K�p����"��{p�����^�s"�9�7Qދ����-U��ZZ���F�Dߠ�¢��0�� t8E�w��g��#aiXZtΡ�37���k�گXFdr���g���: ���ÒU��P��@ug#���R�@��L�L���$o�X�䫫�fA+������/ �oZX� R�o������25���Kz��O��e�I��z������g;���s��%Q�|��ڊ>1� aRt��M������ˢ �\*��A4^�3\\�3�fK���N��1��0aM0e���.� ?g9�^�.��PB6�S D؂\* %��aH>��Q��^�1��ہlA�i����M�x�ߖX���}�i� ���p��9\`GԈ�ۢ������Kݖ�ٕ 8�$ ְ"���I�cPȪ���o��X��%�N��t��9<���d��'jhِL�����sODb�+L����~K�@Y�t�0��)s#s묬����X0�Z�|�l�ј�἞�,c��Z�����>�U�?�p�AS�N K��boT���s�"=�Q�&7����F t��1w@0�Mh�gDt(��2���ݵQ���E53?�+X��4�q����;W0��Q�!�MuLt�~�3;2�:6>�S�wWE�";���g���Y��a��������\*65���V=�?򾻟�\[ُ�� �4�>:B��j�W��뮱0x�n5ӿF!QƆD ,�\_�h��OW™\`��^e�x���EBa��U0Jx1g�͛bӟ� x�U��D\\�Es��J�SM�'�A�F@!~t��\*�� �8^�r�\]�'��B�5U�O��j�R��<����~��Wk�6������w���b��e����Rʲ���O��u+�52�?�����R��� ���˯\`:�A�ʪ�OC<����yk�<��e�� �/�jq�rL#�/����Q�H�:��7ַno� yM4�/�F+k0V,"��R�e8pJ�� ���E��7Ч��2\_/�4 IJ����Z���vv\[��833bۊj�V4�jgkõ6M>�Z�g�� |�T�� �%���WHM�Ga�Os�� ��#�&��p���H)�%� �n��W�����jz���I��a�GQV'��S�o@�4��P1h�V=ɗ}�g��4�β!r������i�\\��^va�c������U��3�Q�}+ ����q#��3�O�˘<{��,�G�>Ew�J\[l�ȉ��HO�F��7��~##���x3r=S�G (���k�� �A� ��B���#��p@�=,ޥv�!sP��Д�����6�����&�w���+����{��Ih##E��?�J�P��g7<���Mه���Ѻ+ 2.8�p���<�J~�l��-ZJv����Q���#��+oSk��V$����)�|���K��oHώ��퍴��Q�\\WQ�\_q4�?����9<�%s6Z�9q��D�S8�/9XH妉9 }�Z<�wU�9��d�҂����P3 ގ�2!���f+(�����z8ن5I�#���C� g)Q��@��t�k�i:بv��w�%��,�B��i�\* y�~ \\(N.k�l+\`���XE}×�|����w��I󠿏����HPo�\*�E��f3T��:l�Q�H����W5f m$��ܻS���ٓ\*�go���Eh\]|{\`9\[>N $ݩ�h ��� �\_na����NH�b�l3-Wh@l � k��J �j��8W���Zw��c!��5�fa� F�.%J/��Ų�'�b��K�P�o9E�Ia����>� y�n�C9��1̪�V���+�c/#��N�,��n&�R5������/1W���X#0�V�9Y��A�iz��\]Q�'��}�y\*b� �0e���P�f�Zt^�.�ȕҴ�^žH�o�\]��+�K?C6��uc��:�d�(�jڶE(k��r�J�w��n�>�h�9�\[����ewO��5�lE��I��,�Ղ" �ٓ������kj�!g,���҈�~�J�� �ii�MӔcq�T��f��pS>{+ �o=��e���3 ��:s������=���A��\*��Ԡ=ۦ\`$���\\%Xސ�Z �F8���E�u��\[��o(t�y�Raa��Qn?З:�7�ph���#>pWs��\*Đ�|h4H� z� \`��r��հ�B�>L%;!�mvdM�c�t��d��Z�1�=�l� �!\\��C�W�45���{DA�%�K�Ǯ½N��mi��z$�\*?%) ��4����.q��g���F�٤ �v���$\*�\*�I �\_}!��͕:���i�PM�8�ji�+�\_ �'�e���-� .�\*���Wq�r�!TQ'bܟ����^��\*� �u��?5ei�kO���O�N&7(\*/Nh�D� ��zۉM:�"v4Z�� a�����Y�7\]�{:�\\R�?=y��RG�e�k�Q�щ�N<)�g�oxQ\]�9��H\_Dݣ�2 �t�=�# \*�v\`�T.��fg&�� �,p�;��q8����DҩUEi�ډ2#�.�f��c�>�����Y"� ��B���|���Dy,tv}2�0�UHvY�teP�8���֦�A�L���Q7w�K�3��,�c��in;X�U �}�#���c�TcV��y;�\`^w��&D�u�q�6C%ϙ+�i�N��^&�ƈ�=Ө+���DiIKKl9�G�xh��s��>=O�B�\\h��?�ph����U��g��?ȳ�N\\v�%�o���3G}�iVF�ڛ���������a�9=g���c���2J #@) {�d�Y�O�ض�DE&��.Z-���oʮ \\��9�����Dm'�� d�'�L���{#���\_����3� ���H�0�6+�սib���L�)��ڕI��Am �4P�(�;��bc �t�҆\[c�����ɲW�873t(�J��p�M- eɺ\[�!(��N�bs���\\±q���4U\\̔~�2R�L�C|i��\_p�5%> E�N�s�6���P2-N�,#�\_�̷�/���q߉G� �\\K/C�G+ ���j|�s�� ('���G�jI���W+�-�^�x��w���>��FƁ�ǎk$љ���m(Nd�N�1+�2r� �� ;ߚ��ý=� O4j��E�y��p/��m�F��@�ޠ���M�̯?Hs�N�̨ ����� q���#����\\v�d�=��H��6$�z���#�oկ��Z o���=b�/%�v�љ����g���k� �O7����:jT}\\��cJ��%����R��C. ��š�Jr 3/��c��\`�� ��w��'r�Z�2�\[��%�ѣ�1��n���ڴ� �+�fX�T�|���>��YN{9óդ������vԻ�N\\�X����m��5��e%sev �A�8��?�S���}�c�J=�\_�?f�Q�C"�n�H���ә���+�M7L�H��#,�������rZ��k�=I�u�\*ݙc�PE� ����@~P��ak��u�.O�eJ��k+nU �C�z}�)��yd�(�{�� v�ڗ��m�FZ� �|ǡ���e����j���f�e�g�ˣ뤶��ތ|K���\*J�l��O�B"¦٤���@�ѩep-�\`��Itq¼��/�2a�%G5@T�f��n�e�e�� HD5��2XWL��…����&�":��\_2H��"��==�sS6�� eH� �{����d�n���Œ?��7ܴ����\]U�q�a�L�߹eT��;��B������o�R\`����^9%o��^w��#H��K��ay���)Pxo?Kɯg8����=z��^E����)�I�s� �E5Q;-�؁fp�|�7V�@C����wvI^�ŀ5���=�$��|�� N2�aZx�V!9,С�OE>m��SI��1��P���ǝ������#�<��N����4�;s�(�� �8��:�v�c�F��=� ���hP���� �A��O�J��S\*>�����D�t?�V땚��� �&D\*�/�$x!C�#�@��b����z��2�^��L�3����{C\*�͖�WT��Y�0Ǘl��Uz5��6�@سhA�;I?���N���2��\*�ͺ>""\*\\.�趻�h-�:+�ˠ�Wn��E���lZ�<ȧ,ݐmꖁ6Y�bb�Ѹ�)}nN\`U� ��\\:-��%N���٧=���<��4lc�Q�:"~!7r~�)ND�t3ǧ��w�M��2� �)bl����o�XL47\_g�ae;{�0�� U4����-�:)z|ȩ$�=.H��dM즞�^����ʩ>'a�G�N�W�?�y!E�Ekl�?y^Nwr�Ր�\[lr}p�\['��g�5��u�Y�W9�����BɎ���ގ�\]�>�6كK�Я��9������TY��۰��9Y��$�Ûޠ!�^W%G{�+Rsk(�� /�Ƹ%� H�@��\]c��L�c��\_ZB�9�~�p+!�m0��7��W�:՜��M�tOk.��i>՜d��"�a�Y@ыO:k�24L�y�Kw�u�")@�?����:FX�ݛ��@A��A?��Z�X�8��Ova��e)PSd i�-�"mO��JAk癢\*��A)yq�\[|7��P?����M�rY��L����<�r��/���<\*�v\`�g{���H���I/&��ĿaA-,�H�\[�ʵi�y�of��� ���5��7&� �9��i�����"R�~�\*�?\]����\`���|m��@�~@S9J� t���i�q��:4�P��Tmc��kB�p!�cՒ����d.�Q�2��gj��0��\*���D3����m��^Ů{��<��a{���#����7�C�O�~ ��4�aD�fwy�a-�����߸�F�����JFߌ&c{1M�P���,H|���o�Y�c����5����dV�$��\*-���e�ƃ�j�t ?+�-7f�e��<�'�ď�2� в�ܛσ�\[������%�\`�!��z��Gp�"�'!�걼���� qt2���tF���CY.)��us,�&b�^:v� �~���U+�Zҭ�+�sR�U��������D�G� ��g�Z-�̙T0��I��I ��d�5?�S9��F�t�3���gR+Ⱄ���E��uu1\[e?E��\[���(p �� ��h/�EQʘB�.�a�|�UHme��h�Ps��o��4��\*nS��퍝f(�C��^�� ��\_����#�ؒ\`dԜ�R��M��E.��i�n��D�+�O9�+P���h�"T.����t!�u�(T<�+3�\`�/P�gc\]Pqd�k��K��ݎm��~�\`B!�=�Dtwt�t�p.����1����\_��Q8\*����� |\]0f���:D&S���%שuy��e�������H�A�n��"m� �\]/�A�X���@��l.��BߗG��F� KW�ǧ^/��Q�}�d����q�A��c�h.�Ǡu�>�n��(�:�f� ���o:����o�=^������ص����$N��I�\*�������L�J���%EØfs�R?!�4t 7�ӄ��i���S2�Jx��G��2;��8����x+t��� �20 K�8�DˉzԒS�BӺ� ��> ���^1d .�&n ��t�,R.t/������o.��\*�ܦ�Pq�GZ�<(����j��= c�bkb@qTk�ꁺ� ��ɛ�-��X�'�Y� f�\`�r�\\�m�d/.Ҏ��Skl)�guY\`Ҏ�?lE��rN�1"�ch��5O�Uo#��>gmO��Pi��8�0���7�\[ �4�"ּ��l��� ���~W�������,�&;��z�����\_�D(w�L�m �}�X�4���di\[44nʖ�=��^,�Ph45g%~+$W��d�ܐk80��ܻݯ;�J ���DM,9�=\*뉱�7$���#��� �\_��$ �y{�y �;7�vqdηN�~%�����Բ�B7p.uL|� �C�a���:�Y�Ӽ޾bV7\*�������6gah�O���C��2� �(x�l�=����v��=%����8g��.wg�� ;n�WC..> ��붪���.U ����Ō���T\*E���7��Z�t�� P��o�N�>����{��z;�$�F�\_����w�Ns�}$�EݔfP�����p}�׸��\*�2�GA�����d?��҆�bHi"Z����\_{����O�Q񯳧�½V!���(���䨕��#.4������}pa.�BE#ǎ�N�#\\q��� d��\_��fH�n9�� �pt�h��HPN�w�m|�����e�/� \_ ���es0z^i��>BɣWR��\*G�a����\`�$?�l�����\`�(\*�VY��%�Kb~$Ě۱�N@����S�$�boj(��n��E\\�}��P# d\\�W�2�uh�� �a� �����z�d��g�c��Jo i��{T��<�|�v&�A'A��D\\c�\\L�{�I�>�D��t�˷"��a0O\]s���d D ��7o篩<��1��\\~V+@��6'��kh\_�^2�� I8b��j��Tm��؁#��h�3��}�~\_j����Z�^��7����c/��6$x��+\*&V^�&��K�m�#q�a�ѐ����|D)�7;f�/��l���L��̴�� p��4QG"\_q雓��G��96��/�G�N?|��y�,^��B�\*�M�y0�!�PL����c�S�� g������笤��w�\`�5{�\]��⓴Ч>� T��Oc��i�>����Zh�}�'�gY��v��� A �#�ͳ��uP�����@ \_�6V��ڟix��l�<+�:X���~��j������p��R����p��'���K4T'xap6�3�N�z�cC.Α��H�9#�(��Y�rW0^PU�N�/;�pr�=�,�J��A{J��u�b�ʡb�R0��~ރ��� �>���?������J@nAX�tJ1�ɠU������l$�b/�+&Рѓ�\_d�ʢ�4�\]�7rA܃b�ꊲ�� �)0,��w������Y���nɭ4獷 �97��v�����8S4Œ:�m֘V��hE�����@�'&�\[�<�w�$\]J��������>�Z=fg�K\`�X������G���Pu��?�l����:q��e\[}lq��(\_��K��f�G\`-h�W#�r)��IU!�1��4 ��h��aMh'ލ��&� ��X��-�4L��� CQO��9i\[���t����=��\[�G���X�3c�2�� 5!� �喁=�Kuá9����LӂU1.�y�b�eɍD"���C|�����:��24���y���\\\`������d2ۃ��K�OM��P\*��Կ|������5��p����;B���l��)�S��6�6�x��U�@�3��T� �EO&ZY���9����j�M�̤�ZI:(�h���ׁ\[t�mY7@ڃBT��M>e�"�ٷ��' J�����h�q4�����q�ew%���T����E�v�>i��3|�����h3$����S1�6r�Fo}�k�Y%tU, ���{�V>2�fWp�d�6V��������V�T��p-K�N'�\_�jP0����-�d��z��k�Q����I��8�1���p�44l�e��{\_\`Dy?�l�\]$@s�KS'��0䔾զ���y�h�}\[^�2���k?��a)����%�?�l=bW�����7�> �R3\`��� z{X;Br��2�\[0cְPj�\]Y�AP�ὤ����^����:}}ޟ�2H0x!�M�3��{O �fA����a��9qh�Oh�f��P5 ɑ����e|7 z �"�Z�/�n�οqMׇu�=��4P����?b�D�0��t�z��������p�(>aR@�#dS�T� ��z�vf(JE���\`#b�1NDDf���ԩ��G�k���R2���/I�—�\[������Q�����PNJ�d��i�=i��y���K�Q��IpvJF��\*ք���æ3���WQ��tߝu���^���x�l��)�N�� �9q����F(���,�\\|ݧ\*\*P��J�����E�2n�\_� g�A��� 7�/Td�3M�9X�$��e0\*e�?����~@�}��x�v��ģI�b���-n���\_�P�cTnN³H��!�:�^n��Hu��\[���p�9�#XF-"�o�"�â���dr �j�g�uՀ�vQ�ý��n�b @�B���\[���72�}���vF����@�����QU��s�DnҰ��DV�|��-�L:$}��Z�'�PNJ>���S�\\��xx��/��m� zf�}0EG�<�9�>�Ɇ V��U}\]Ͼ �N���Tހv� �\*£,1�b\`�d,��� (��9�Y,�Ed� ؝���u=Y�����b�y���V���m�ċ"�� �;�b!,w�\[����n�±��R���^R:\]V\\��̉���ds�i����p��L�"�E��#� �@�{�<褶�\_���K4!�$\_��V�'(\\m�V\`ze$���Ynrd��\`�A�SƊ��2dd���s�1��)��.-�UGhr�� Z�p:t�f��B�ٷ�8�����F���>�oΡ� ��G:1S}7��6�����/�c�Nػ+�����s��q�h��ˤW&R�D\[)Q���;�1�4�Q�iv��u�i)c��m�+�4�rz��ް�\_?�ĸ����7�/h� ��T�µN0'��~D��l���\_����3(i���c��ue���=�!����V壤8SoM�X�����&�^6\`�s����S���Y9�ƢQ�5�x�O�1�OH���vA��CWeO���~U�ϝ/:\[/�ݸ��;�z �.^u��!��:7t���\\\\��������x9$��|��>(�CO�OE�2�wN� 2��! ����Q\]i�F���J�l%�ar�=s@h�)4^��\*�!�.S�gb6���Ix��\`��'�O+��@��\`\['�M�w�+� 4�� �^��o��B�/� ���"(r\\(����đZT��L b���m'@;0��/t�����)����f\]w �.=y����ߣg���1�\]� o�Ҭ�X�:����6�Eh����hBZ�\`u�#�1Xgf����Y4?�4Օ�� Uݓ�ւ��i��D�Lѷ�2��ݍyV�i0�����Bds�,Ԙ\\�!V��,M�l$=AtKwB�q��2n&� ��r���tZʃ�u۾�p ź�Ĥ������yטt��F��t���m �rJ�R������\\҅= ���.n�c��Ŭ;���o;�G� �F�(�7�V�?��U"���TRhk�"h A�m�b�-��,�\] P ��N7����)���/h�R��=���0�<�������j�� &D��\\t�%��ƋS�qh��z/o��\],kc�\`�\\1^{O�I��}YM�K���z ����\`8�y�\]4��s� S���lI\`��L��@~6���c:M4|zߟ\]çn̏�v����/�l�/�#�� ���߰.�V�l\`�ǚ�O���fg� O�:N--+)֯��K�Jّ�N�h�t�� ���\\�u�|�:���i�����5��׺�"1�F!�%K���\]��{܉��T���x�|9�m1=tn�T�1!�Z���o9�$" �&�� Y�81����P���wV���mw�����T��;S�C�� ب:��oVAt@��\_��m�������\*�X�.<�C��5�7�^!��q�d�;��%����6�惇�@�R�U��8��汣FI��m�^�����>��N���� � �6����s���ܒca����1Ij�qR�ʮ…���vz ��Ot���E�7��Cy�B�˹א�k@�6�I=�T&�3 z//"�s!^v��p��;�� "�o�\[h�ڟ1����J��O��W {S\_=�s�7l�e��OC!���茮pY\\��7<;���¹Y�&�����M�s��n�T�D���u��蚢�gB6��l@�+�7sPT#6�3<̭����"�� M�½Q�5���V7x��\`���p��YO��<��g�U#��o% �Q�AHr�d�Q�O�n��tE�$V-z?���g���Z�jc�֯|2�\`@h(�\[o����Pn��P�Y��5�h?=w�&�)���3/)q���U��� �����xn�@�b��<ZO 3�L̈��A��l�ή�mt�/�OW+7�v�7Kό�4~��š���z~R�=Ĝz�1�\_�%m���D��/����<�ZL��T漣�p��!"!i�\[��\_t�^W�8x.̟Al�h�.�d�y������I�j���/6��C��͛�+R+���+,)�$'��������l��\[�朆��Dƙ)1��Wy��Y��hm���I �:4G�yR��r����3W�bi��z�����Q�WC�C�i /�=�r�A$)z��\_V���{4R� ߦpWR�F�W n�҅��@X��Z�V8>��l�i}�1L��w\`�b�=�B8+6��g�D���R%\[IU����Ey�\]��-���iuxE��/��J �k�4��c|憄f|���1��mY�� �Tim������?��BA��& �t4}$�5�O���n:���Zs� �dl׷0���^T��=߸9Am�|��� �jrW�t�P�bȄ ����1-2�N����9�yV�ӄ8�iݰ���n���+����sD�SO���� v��S��% >k7�^�'fR Y��=����S҂{ AS�;n��Mҵ|i�|�w���b/u�o �V�;�ke\]&cp�F�\]#drl�IU� dWg,�c�m���kf��eJ� {r�{�c�S\`u��s���� c�e���=2-���x������������j,�縦� ֈmѩ�D��N��� g1��\*��s{W���n"�Pl�t�R��/��6j ��Sw���ev�j�9�qx?���\\\_�h!���/'V-G�\`<�b"}(��^���\]|�>�Q7İ��� �ӭ���5)�r^. �g͏v�#�x�K�/�GD�-�#)n������I;Pƽ� fuEzO�{�j�A���A��|5f�@�1�\\#��\\p"�fպ����ٺ���)8�􁺲����|�V�Iw.������,)t����EC�8�I��X����(4 �ь|��/�ż�����\`�h�2� ��.> ���M����9����2�$۪���K\_�Z�<F^������5=Ku72�3<�݈\_t\[8u���abbE���d ��ٲ���b�KV��%@�ozoWl:Kɚ߯znTxލ� �is �¤){Fļf�?��V\\�ov,ܙ� �@ }����$�q�������/���4{�D3\]��~A,�e\*���"��:�F�U�5=�#�##��G��˰��@���=#%Nc�\]����� n!lt\*������䟺B�+�$��Up�=c\`z��3�3X�&nu3HV=O:;\[e�����9�\[m�LCW�Dэ�b�?|\_�q��Dw-�-y�ߛP\`pR���"�b��K���\`��n2��G���Obn���������EZ�D����� ��\]��.�\*M�ֈ��ITg=��>w�� ��<,B�UH�|Ə�W�62�i���8e�j܋��ߚ�-�7�O����:���F��^sUr������y����f��H�8�����6Eڗlަ����/wW�ɷ�#zF5���\\%��SH���{$�h�3BSW��� ��|}W�:n�� {a k�� ��\*>God���h>��n���'��\]��T�d7��+Y��&ߧ���܀����7E���fu\*P� �诟����vҧK:R�3�3s�7?3Fi��Nlk��\]��\[Rݠ��Dr\[ ����-"}��XU���cacU��;� �� aKY��gY62ӭ4&!�񦦏̅�.$����@�c�L�ɝ�F�nĵ?�n{,�f��lJ���A����u���( t� n�Fd�>+��2V� m(�N\]e+;�\[�=���H�(�$���6j�~�@���b���"�\[�Bn�BF� �|Y��XJ/U^?z7,�C(�<��tq�ڲ�\`,��\]�����Ai��M��K\]aҳR� N�W��8K����<�2�"�����ͧz�B��H%��bv�����N���džz�&u�� �F����T�y�9�&G� H1'\]7����@��4 \\����ڶ��iw��i \_�{�� �~���:��C25t\_����^S�&�d���J�ۓ,�^y�}T�Qs�y8 ;#r0�'��GY��\[ qP��3|�Ӓ�e�&E�戱�!�4�|Ls���Ԃ3�A-\],9>�� \*���zJo�H�9�����‹Ͻ�d����ɐs&� �Br��j��UCb�n��q��X����,#|���y�:ikG,Ko�KEz�(�\*��t�}���҂T�n��m��c+^M��e)� L�CD�1���jo�3�q���\_���\`���R�w�;��D��\]��pJd����6H��,�����hi�MX��=PN�?S�QĿ�r���ˇ�W>�f� �:��S .Ż�@C����r�� � �^נ��O��qU���\`��=JRhb����OlXĂQ�8 ;��p~U�>p����kŸ � �!CKɴ��m������'�\_R�'�)Տė:�<�8��e���ڈ�f�W�M���C\\��6���>���Gh�o��6�׵0�H��s� X��5���p���kG�d̮o���N��t�$\[�>�)��O�|\*�1�GN���O�B;�V�����?��Y�(�fO�-�� ԑ�ǜ���0&�\[4R#&BS�\_&�y��V��磒�Lr�)Wʢ� {���\[�J쌐�r�R���2�z�j~2��n���>��y��&�&������1���� �T���3�w����.;Ya�gH��N\[�!�aB 4��h��\\������3�\\��'�Sg0�"����(t�C\\�(y��gN��n0,i�5rG�u\\Y �S�^og6�ш�Q#�#ܷ��l�B6!�t�Q�g�\_����4JM\\k$�vʮ���c��n��:�L����d���\_|AԐ���L��$ǘ�F��C>�e�)��k�̗m�ÜE �\_�6���3�"nUʌ�d��2��\_�ȻR� ��k0\[�,\`5��N��i��(� �LaN�x�5�%4Lx� ��5G<��ߵVu����X�Ɉ���6����T�\[n ��A�!�&�Ǎ�b~ �sd���7x���}l� �\_RFX��3��~���uB���#DT�݌�^W/턙��x1�Aa�r41��榱���+?W��<���M������/��<ۓ���ƥ;��M���Y�|U�ө� ���0ԽQ�p� S��{"JQ#�ݮ�%� >h� �|��j���uX0R��R�e�y=��=��D��b��pa���6F��a�b��x�|a�Q@���"�� g~QnZ+�tr�e�g����c����S��źI�.��p��V@��,�!=oĴ�w n����h����3���՚�R�)��(�et�ъ>��f�(�z�@��&��,L<�ej��=^� ����B~ yf\\OhLF��� v,��m�����eD�%QUf��j�wUPC{��uژ,2�r7��7�j��gś���)6؅\\m�y7�^�ʼn6�L�9KqJ.m�J�i���v���5��{��/�u��h�y��6��F<���r���5�~�XI\_��%\])������y��,�\_u�!���\\e��#Z�Z#��(Ś艐ɂ�=�\_5�V�߈���o�~��:%w���\]�#!�>��I�����ynn��\`�T��\[� ���!��p�ʤ�=\_�o����HA����su���#�l��6\`� .�+{'h�s�-�A���\]�)B�\[�Y���H+��Κb� �� lU�-\_Ւ9w����Q�Z^k�랦D wJؽ\_\]�v�,�=o���΁z��e?�'n ��E����ۅ��Oh��'�2$Y?<|m�����/Z���2C��.z&�f\[� �uI���=�1��-��\\��t����D(�e��#.��hG�A���H)�Vg�@����۽�2� �R\\$f�C���Kg/�JG������ m$7O\[��נ7�Q� ���\`'��Z���,^�9�f^o=�s ��v��q}��%9?ab+Շ�\\��ê<v�����8g�Y�:w';#J$mo�r�hE0$TV���.?| �P��|g����)؞aL��f��dO���Y�^�7���:�LC�P+g�֢j�o��:.�"0F�$�����FM�G�U�����U� ���"�r� 16WV J��\]���.�� fF�����{�u\*���<�J���.��AW\*�17(��b�}� '�^ʣ~'�\]���E��:t��j��V���Sx sg��p"^�I �q�H5\]�vt��Hy��b{�3)�n�1��)�6�њjP�$�:e�eN"���置�hѲ�'r�}n���Fb ���)����1�%Ǔ\`C���W�@,�)嵐����� <^��|�\_܈=����;��NR�����~��k>�V�%���p������ UQ�83�S�F��V�����i��t���t<0v�\\�jQ�mŤm7.>�֪=w��)�Vt(��a�H\*>��=���}�KM�W������%�/���)Y�wR§'Avu8���2/V��R��-B=h����a\\��|���Kch���=���A�� �,�3E�z�?�g�"��%��� �ۉ����7\_��Mm̉�\*��e�t>�ĩ\\ ���q�8�\\dv~�{���E����fG�M�EM�|��1B��D��ӷ^�� �f����=S�B�INLmB�}潖~�4�F/����8�c�VI�X����Bc{=���&�����\_)��5y�|��K翏G�,���b�\`LǐiYk���6���5��y䠤�h���q�E��6�"kJ�1ZN�D�f��1�m$.|D��L0��}J���G�o���|�A}�IPwSk�I��ؔ�M�WC� �M߇Iz\`\\���I��E�\*�k�▊z%D 5�����������i��,��j�;�� B�\[f)Ԕ@W ��XàU"�Ģz��q��J�5�T<Ϛ1����5ܲ�a9���Sq��kaGLU���sɋ�Q�vS�,%.T��U� E�;?A�kz� �(�l�{^7��奥�/����ռ�c� ,��|E7�?�7��㇗�60{KHi7Y�Ly皖X��4��)4^W���Ƈ�=@��\` ^���W�A�o�3�U���6\`dPRz�g\_�(�ВíQ6��\]o|�Βc�\_4�z�4�je��H��J-d�4�z��Ѱڻp:�mY��Ka�ќP��� �YH��0�f�.��KOg�)"�+�́1t����du%@�qr\* �������\[������qW�l� ��d��a w\\4.�I��.���xħ ��J�U҄-��YA�� mS�=��~U)j���e�hH�á�̻�ˑ��z{��GO�̅�^��{Q�2%i��bo�NRV��I�9RQ�� ڦ!�\\'��Q\[}�qU%�n0O��}�3L��sR�X68@��F���N��ʸ�ً�� <�Tu�5�/����;����Lx���!��ZQ?8rAК%�aڣ���8�ƣN��)�I��^!�A3|b=Դ��p����kzm�}jU�L��Ãw �˛ ���1S}f ᪖�SR�cy%�<:O=����1T�?z|�k��D<ߑ�}>�4\]�����=��o�3��ҷ���.3���p������<�����M��B�+����tZ�~Y\`���U ƍC��?��\[5N�Zxzw/CoR��~qv�1�������b��\\�-P��"µG���9��\[ 2��S���o�l�;뽿i\[�Ӝ� U3Kt�(�p�/-! (���!�8��Eڌ�d÷�:IN�t �D�V��D0rfpzj�F �Ѣ TX�@mu�v����c�S�P<�l�g� )O��3�&�Q�9Zh8z����Ѡ�W?%��"��GC�2M�^�� ӽ��JT�k �Ko�Z��iz(#�y�\\o3���щl%k�g���WΒI>f�� @��OÚj2Y�p�:e�ΪO��=Ht�\*��-���j+�z�� ����7\*�Q��A�r�8xbP�\_B7w<,jfh|}Ts����������ۯŪ)��%�E� S��ͮ^��o7 ����JT�vp��G�~V(�@����= �xvB���%GF)47�M߻�F�h��;q�I� B �ר.���L��r��}�=�?x$h́f�J\*�Mi��&i�\_n��@dK��ЏLV�@����ڳ=�ޟ<� ���|�\`|�x�ʡ�(OU}%m�#��5Qj)�2Z�\[�k�D�/�"�#���;5���8��9'�1�5,�U�~��C Hl+Y2=9�e��Yp=��(�l�� U�ʭ�����7��� ^���<��?��f>� �?m .F�������e�d��F�~VkA�iKK��P�⣇�X!S���w��#Y��&Mߧ���>-��dK-{��\[�&��rr���m a�� ��9�4hlW\\Z3'�p����\]I��= v�GVO҅���������l�x�i7�V��.W��D���4�E�����/+Kb=�����w�z%�\*�3$�åc%lq b��'\*��4a.BU��e����cy���y�.(p��KFD�xFtµ�����F���.u9��� ���7 $ڕ�MiCG#x�g��=�Л� L�ؕ4-̸�f�a\]��nÁ�z��� ��)�Q����jq� o.��4��&I^%A�"�Z���2�8O��J�+�B��)�U������� �����b�/�\`�����aeC�a|��^�� ٿ��n�(�#4̂\\���RdW"l����&\_6�~ ��mo�8g�Ɍ�y�S�ՠS�g�Vt��:����pz���V�P��c�����Ҥ�4�!�������msh�, 5ʀ�YR�iުЂ�Nw d��0��j��ŕo�|��U�t�AW��YUXba\_�:mN@��YaZ�Oc�gl��9D�� �c�;�� <��O���z��-���,�0MzX Jϴ|E���!��= ���c�%��0k�^�Nـ�gs���V���ʦ "dQ�#W����DF�$� ���,n�\\����oz�iT�(�aՔD\\^s䷼���W�@g\[�ɖ\\��5���Ƕ4IJ+Nda����~�d5f3jJ(�S��\`��'�Gn���� �X�se #�3j/��4��E.1�}�jS���%���^ƃfD���<�~�0>�� ��8~p�{��j %�$=B�3W�����<���, <� �1����b0Zs�.,D���c� ;\*�t�䗧���a��j���\*kL�c��Y+0<0I ;K$�A\*�\\m�TM���I�����U9�����r�(8uY��0@;��I�� ņ���E��=��~������&����1u���a�cK�� \`ɢ� �.����w��i�$�t4-�~�n��b �y@8�vq����F�%о1��^�����\\�k}7\[� �~� dzh"MJ�n�u�y .�#D�3E��At.�A�s�;�.�ܛ2�7Ÿ�˂� z�E;춴���6I�mk��aKz��,ؑ���gG�&��9u��Q+��29��WM�h����X�������+�,fE�|\\Ұ �H tZryax��}������>��������H|Q �H��ƒQ\[-��:�1� =�P�v�t�=��Y��9�u��\*��7 xYv��ڣ� 6$ ,��eןSQ�x+T�\\��\_p�@���ȼ�+��|4?���dӪe�M���֞�x��95��8'qGh��Z�X�ZI��h��ޮs�B�\]H�\`i,��LI�\_���j�u���.��f�>�\*}\]����26F�ꔭ��bkr}w�����Qwx����f}tg� Ǐz���p3�6���b�Z��E�p7����q��z�.}k׭%$���v�F\`\[4D3{1�T�( -~ ���,"��1�\_h��<�r���|3G��Un3�J^6����|�bOv���><�~��\_<��� 7F,�E�^U���m�'e!�u���u����k�3E�$�\] ���3�oٷ�sK��u���.}&R�U�%;�\`�\`�U�M�#���\]t4�@��b��3���N��jvo�\*Q{�=� �\`�s�H��F�����$�7�֮�m���AGs2i$�E5΢J���M�� m�R<�:�xJ��T�&�Զ��FH�H!2����K�� ���!D�J�������)���C���ey���l�箚���r��\]&�Jr^�sЯ��(l�6��1j�(�a��u8 K��/�P���!��oUZ��r ��!M��X���Ц+"� ��CT%�&��J���4sZ\*�3wsB�<�s�!Cܒ1�o� ��:���%G�in�;���c�����t1|��1�"�{� I "�e��s��,�B 6�3���������\*�3�\*)�����O��R�\`{&�혎~c>�aD��0�g��u!O�piGZ�4�u���1�qi|��!�B71��V�d�V�TX���n (�� ��7��p�ޛ\`���2�)�9)�$�uB�� ��=;a\]X�dBɨD\_Z��\`�0ID�߹�~�hOW&�v)�X�<� �(+(�0���R��\_��Ep�Ȩ�#V?�V����ؖ�̜�,�ў\_��p!���Ou<ϫ�=l)�i��$�WJq{� h�b� \_%���{w�Y�� ��R�o�x\\S+��j c�8�O��qS��Nr���V�ȫն��\`)e!l�H���/��OƝ�r��P7쵂w�LN�����h/����؝a���M3ȝְ����v�Dݪ�t �Nr���X�V?� Xo wyn𫙴�t zMv�ˣ�/����b�^�B��iUX\[�i�ĐM����8�����4�� u��$�ac 4�F\]�I�����D\\��f��SE�p���GDne��") �B�2�5����B��j�Ê��KXWʵ:��>z�뎔iq�19�c���NB�������^~R�؋�%����;rK���/�����$ڐ� -�9�#6�=݋I�)t.sO��Ɲq�<Ƅ���vW�Z���vACϣ�� �V�L�?@��\_���F3\]\_�{�gк�ϑiN>e�'��ݱ?}�6� u:�Y�f0�,W��̺�S?ر�1Ղf$o��J�\\hvXC^���vn�'�T\_.��i�>'"��>����}�#T�~bh(��?ST��lP����;�L{id4�\]w�0���\]6���x8}ό����U���2�hqyiTV�P�tB�G~�u�T�����U��T;������ �\_���T�sx�h�d@^��x\\��&|x| ����O\`���Q�Z��� �h!�\[oU\\?������&F�-KAn�m� ��\_�Gwc��'C�M�r^��&�n�'ڍ��-�C�X�Q���A �ؖ���~�>e�E\\X�� ��?c�͠��Wy\_����xY�n�h'��\`4o'J�Q�b$��b��N}�����\`���7�׹�EAAv�H��r�Mg?��0�<����z � R~Tms�����B��Whbf��K�v��z�'O���ٗ���W�9\*} 8d$�H �.^%k�f�^d�L\_��X��{�óhlA�� ��V"-��ME�\[�̕��������g��3�d��;5���\_���夕)�}���հɴ��esNh��~� � fZ�/|��g���o��I��s&����ү뤧\\d��k(f=��G(�^�o.� �W���#�Q ؾ��mW�E�k�~6�� Ҡ�.�wu#�����H6\_� ��%i+��Z��k��Mr�Z�W4��䐸�ƺL���$��N��w1c�~�=�v�W%��\]����g�\[��\]�D�x� ~�̇f��oz��iB2g� �!�^�@���P�φ��BM鷺�Q\[�pwg�˕vf�R����v�o�;�#;����ށz�U���NM̔�e4���\_�� �g�.^l�\`n���d\_ �l��"H >n���rC�Zv �p�|%��c��(��:&$��fF�>���f&4��2C�\]\* �i,���Њ\\��� َ���HD !�L&(\]Si@�N�7�����V�S�8�}z)J�@217t��~���H\]��œ!����g��������+HF��gs����Kk1'�v�J����q&����dm��;Sf4�A �'��P�b$"���5:�³ & 〚uB}�n��둣��m ��l���6&H~f�0�$\]=��p������! �9�C�}mG�V�I9�n�%����x-r���EPr��k5r����z�����VFkuK�(�+Ah'�6u/I\*��� lP�TUHʋ�X?3D�\_I{R�8�,������97�4B7�L�5�7�zQ� u�� mP�L3�zD��h�ٓ��hZ�%���PÄј��W�gZ�P��Tܱ\],b�?�jZ�6R%��,|B��\]8��N��o I-d� -Q�'�F�S8$~ǧY���a� (}"T~�fq1H��3��}~��\*����A��������� ���f�\`g85X�2���B,��(+�"U��ɋP���A<2W7�T�M"�#"�S���p�fZ;.,M�xo�9R�ȗ���?~a�'�t��͕Ѥ�©W��w.��K�ߞ��TͿO�+| ��Qk���\`m�%\`�7=���}6����/ Vwe�¾4�jQ&�shh\`�Zb�,%ȵg��s5��A��{9�nTgN�H׼���t3�!�\*�z���%����Q@�f�aa�a�W��Nݞ��$ۍ��a�,jo�f�߅F�Y��X>�&���Oxm18�� �\_�9�t$=\\���dJh��i�i�v��,t"�� V%B�:�����mdl0~�I�!�R��!I�i�J��>2�\_�'�t�TE�ؾ"���h-j��9��S=�ȶ���j���b@I�\`�"D$�;>8�ճ��Fǜ���W\*#�N��Š��J���� ����\]썯�Dp�!��n�۟�}\\;��|��Ҧ�yf�˼ �嚎,������K���������Izv�+�,\*���H��҅��qkQi(9�OZ���.���Η��I���\]�QCu����ʑ���?�,)ƹ�ʵO ��n0U�4�\*�"�⡿JZ�볗��0��Oo?�\`����Bd9p � 3�0\]�IJQ��K�^�}�\`�\]!�ϦY�74�?Nn\_�ǭ8��ʞ���3�G�v6�4��|ūe>�i3e�t��U�fBv�T���f�j�~����&�{ ���1�.��K)���-���^�ڷ?%���Q,�V~�����V�m<�v8$V�5���-�ނ�oV|�{b|�1���n�k�Q�,;y�:��������\\���W�f�����J�t ������j�� �\`զ$I���K��'���nU��5I��Ν� �9��6�\*�m \[V�991�^��v���0\]/z��aoOD@D�36�Q��䊟a'�&N�Eo-��nl%���s�B}��\`<8�"��E�tmi FP�s3��)� 4�CI�'�1J=Nn����RA�Q?!a1�N�� \` xZ���ޙ�y�2��kg�V��\`폆��w1rj�)o�� � �� � WusXF��=�1����g�\\a�i-���}�M���ɫ(�-����G��;��Բ��یAX)ݬS%��1��ށk��Կ�\*� �&�i��"7N������$g��5k��,X\[��˟ ��ޑ�8� f�hߙ���� �я2��p����P}�Zɟ��� �%M�G�n�K�@� ����~�לH�bL1���-#0O��"�\_C\*�v �\[z.����\`��c�G���b�o�@����$s �!b�6�aL ������ �$�\_�1-XX��7�����D #:�F�8���3��L�.S}�#K9��ͤ1�i�ѻ��aPue��X$P����$\`���b��J�UxXmOd����ɑ��������~�'�W�-����c$��v(�z�\\��9��W�z� ISnC���zKwl(��P(QnN���b@?��U���~L�^���O��,�l凙��v���ex�^���6Y 5�H�52�ǂ�ru�#9ڹ���iң(xV~��������}��Z�Gj���v }�@����ťyW��&�B�;r���3 ���Pln��\]��r�������VI����?x3\*N���DW/~ �S�+�I6��"��Ӑ��c�0��F���bzʯ��#��l-���!��H� �+?� вs���{�&��+(�b��������\]�n�Aj�����:F6�V�h 娖�19����?.����L�à ���#�˴a7lqə������u|����1��F�V�?N�l.����^��D� t����-�6k��z�>g��>�r�HG�y���.������x�z�������Y�=��2��\[�Uo�\\���2�{��=|\\���$�U�v��\*4�ׂ?^�ƈe;W��SެNa?b:�G��\`Ô���H�\] w@k7�s�� f,(��>a������09�K=<��+BL�O�\]3y��iz��Q����y�\*�ʱ���7!�;{��瑃.��fB����j�{уK�φer\*�5c'�\_6NE�B��T"vQ����ſ���������)k�<�� �~����3 0dPСY.nx)�Ǣ{��e'qH?��~� �<���eJq:�)F�Lg��O=���mn��<ݽ\`m���FR�惍��/2�o��yHM��K�k?�@�̩��A^!��lx:�\[K�7w�D� �\]n�:��\*�}�U�L��<.Ķ��ˆe��J����w�����x2��t\]Gk� pd^��M����X�?%C5��\*�(���\[n�1z�� |�Q�j�2�x���o!���9�Uv��Gx�V�\_��{�A86�NN�eoN%���2��s3٬�ۉ�X5N�ؙ� V�����9��P�2>a�W���ÜW���,W!\_�u^ĤZ�Q���X�s+\_.�mw�E���+�t�� �b?-N6� ᤩO�ھAn��wʶ��M��w��Fw���y�$��t�t��)�c��5Uo�w=�VV� 6��h�-������� �ݕ0P��E?z�� ��npҖj-�KA�wm�R���0h�xᔜ:�(vA\`���w�cO�|�6����QE&�WT>��x\]˥?������s ���Μ3��vG;��-�Wq�9Г\[YK��&�G��ʟ�$�&��Ig�C�k AAa�V,w��\]�&vw4\[! GPg����f=�)�-�\*7�VK���DcK~o���V�?Ը������a�tA=��:�aű.�U� ݇�2$��fgyl��<�3���#�h����%��gW�,C��p��JdxI�>� �aJ�bj��:�\[|�n�Ȃ�� �fkk6�=�\\y�\_�}��$�h�~E45\]�\`�ɗ�!��^Ӄ�m3�'��Q�&�J��MY����W�C �S:V+0 0�l0N�⮋���b���������\[�&�u�;�\[��\[G�AQ��}�cˬ/,FR�l��?)ܼCҴ4�z�Jɇ+�d ;���wb���ذ�j)�\[NȮ�\*���#� ��ʘ���\\^����m�XOۏ ��:/��gٲ�T�}�%��McP4{":��i���s� <#qm�n!‘ ��Вu-�1uV!�4@Pd�"�Q��qv�kK�4}��+�p�G�\`K�Mo�g�%Ϗ�\* ����l���D�3囱�i��o��;15��R����4�e�\]n��g�#k\\��I(���f4�ϩ�n,J˹��X�b���?@� �����S�-�yڇ��߆�L���/�uX�A�2�f�4�sV�\*�ECq��C�E���1�3c.�o���>����1�� ҅YW�b�����,��o�ω�Ԥ�<�n�GeR�K��lO�/s5�����=��=��z�,ƒ��|N�7@A����\_ \\�(\`�(oP0p�~�wrG��4��/��/���?�wf��v|g�j�n��R@�9�P�eۢV}�B��6���gb{��wi��� �߅RV䎢j^<���B#C�c�σ�����Ql�G>��ˁ�\[�W ���H"p���Q�͏� q5�P\`R��%R������M⫙�~yz���sZ���3��7ß�dh5����r�i�s��B�7���bÂ���#-q��6���ܰ���G} �5�J���>��jS+\_;��\]�������ѻ؝��/#׶��h��o<ݐ�&��P��@�~��yx�c��p;��\]��œG $��tR�ѻ��K�r�'����.���ff��ڋ�.\_�F�9�5�-;��\`/#{}���j�m�{��Q�ug�w��|�k\\824�u\]%�j+�؈5�x�) (k��M=�o{L��h���.�B;�@����0J��h,!��֔� �����|M�0�5#ֲ��;��\\�ʷ��\`��%�||Qi:j+�PZ<|��,\]S�TD4����#=CI�h,4,��i�Q��̳EXW�vDdz<����S��~�گ��QJ�z��b���s���M�e�F�P��J�D��'��\[ ���i�%�3�vY����X��<����g�@:%��-��{�y���C\_MH|}��s�m;r>�\*�n�h�e/�l��0ɷ�!�G�:̈����%%-M�FM��?)\\����?u5���5n%�e� �+�p�����ͭ�J�g���;���ةu#�^~���G�0��,qݒ��(��)7�Ҡ�ֹF� !�\[kѢ{ۗ�:D"������R�G�qڪV �L�KJD���>��.)�������ᵮ��VuYNa���kK�}h���}\_�x�2���/�S�����JT"�@��{-���o�p٦ �m�˻#qj�Rj\[v��Ћ>�\`fΓy��\]\]� d��������w��t��n۔���z��K��p��\*�A<�D���x��񒲳�u{�qɢǨyQ�t��N���qv~�l�\\��Kv��C��"Gg6\*�X�eA(��pӓFe����fg��~��$��~�5�����ԩM��\[�������ђ#e�J�0���j)c�\]�'�|�=��F��EY0<9e�%j����(�\* �C�LQyץ٦T�e�'�‘�nyKz�2� ����dW��ҹ�"5)S��q�Tں8� UG�X�g ĈX�s}ƴ����>/}�h�c�s���/��t���̂ T�\* W\]���5�i��\[\*?�A����؂�w�c��\[\\4k��Z23�t��9Y���α$�T/�f���)G���xiwz�2���'}w?�9��5\*������ŲNns:\]O��XI5�? �ۂO�4�c�Ȼ��ļ���)�F:#�ұ|C��0Uo��<� ����~�P��^NTt����e�ڷ ��H��):�'���}yA�ְ�ᇅG��4y �����F�G�S��Z���(2�A-˨z� �ŭ�0k��łb��r� ֔��?i��@ ��:���5������eX=�\_��Q��%�n t��%B���U �p.�5��X�\[��;���i��"T�S �Jgź%8���7�0�7o��������ct���}�1��\\�OU3T���-ج��r+�����?�ޤ��x��;��\*�fY>�������.f�G� {X�j�㥕����y&�c��wD�����V���C�.P7>���(�Xqr����c��e��ݹnc����?Q O�Y�O4�\]�x��G�V�L��HK��-�?����8\`��v���Z s�� $���B���Oy%Xz��pq�g�\`�HnsD�����/>TCB�Z��g��� 1oD����r�! g4Wл)H���;'V��z����/���l��\`kl�i�\_�6�X� ��.��l ��}�!����z�h ��.f�~�L����17��I6Zɨ��f��ƒ��X�?���?�=���X�k%8H,�Ӏ�)C#���&\*�Y�=U����!���\*�\]�g������4�'��/\*�'a��w3 i�.�T���0Ѻ�� �ծ6��K� ����\_b:=�Q����;��o�a\[���\*�i4�:��xeÞ!�>����N ;�u9�jv���̈��ϽT��K#��V{+$���H�3���^Q�G�3Y�\\��n���Q�7s'�A��\*Ǖj�\*��G�?��~�I4q~�0Ƌi@y�6~9T�4��� \_a�!�(F �5���x��D\*f��5e^�r9\`Y�/8Y�� ����ڷ��;�Rc���HY���d2^�q����pY�c&}ڠ��4�\\p�&3����V�1��k��H�U�e��Q�҅\_� �x��� ���:�W�\`��˫���º�F��\`����TK�w�)�5Ǯ}�7�\]Dg����ϵ/x�sy��Z �x��$nUIqN9#&���|��E��j��}�؜M@�AN���E9ܥ�Y���;Ʀ���X��r8j\[ +x�P���y�U=�g��t��������(�'4���Y�U�JU>C��,ɂ�r���B ����/�k�v9��O�Z�l��ۃ�C�a��-+�ˈ\\��5���b�g⢼�mh;���=���\[YR�� ����}�\[Zi\_�(�6\[F��׋Ѽ������lS�l\_$� <����2��tž��KF 2R��?=�?uuڹ�+�W̐�P�3���E����r�wFY/�͜yƬ�}suߢ��+}�X�F\[A�Q�h�B����5l��gz� �e�o�-\*>zt��ů�"�8+}��e����窑�e�0��K��f ��3\]�1sX�!K��fg��W��c;u,1�M χ�D�o�R���T�X����� O��5�ݤ@�� � (�� ��z�%�C��~Z+=,S��9M�Ĩ�v$�:\\2�(�8:�� E\_L�J&���֒���$ad�@ ��\[X�a�ݕ\*4�'�nЄ�=3��%M�=7��{7? �����{ ��ٖ�D�����x��3�V�n�\\��ց��\`��2s%w�Q��kI�C$7l�,�)Q�wa;+&$��xͮ\_����apD���<�#����m�A-$z<�;��镧�:��b9��8M�7��>�>���Tv1����js��̒LV�ܹ9\*�D��m�& S���'��Sv�d�‡�+q�.� ���?����r���>\[�YY\\�R��#+ڀ��g%��-�Z�:'d�n�)��g\\�%�e)g� ��b�($!�,!�\]�%��f�'OcנMS� =�YzyZ�Jd��3��3~����/ԥo�fz� =@;.�-��oܗ��}A�2-O��h($pK2��q����j�<�q"}�\*)5O\\nl }0��C�ͽ�FW���b7.Ip~ �A/"��t넒�=j9��(P���N��e��6���� F93=�$��� ��gk � ��~h�����#VW����1��헛���2r� eЪdqUY�N�o�#�z\_ܐ�W ׀�Xĵ�8V�E�9��"$-�~���\[�� �y����\]��˭�~�|S��P�t�΀�?tc���<�-�"��2YyW��5g��@�1��AdX8,=Å���o(X�6 aP��2���+Z�Rϗ�\*j,MT�TJ��H1�+�xm.P0�� e{E!,�� ʹ�„�����F̫�۔�{hG��3��%�֢�������x��C=�,��Y>��C1�� �rW}�Ӌ\\���2�½n�:"N��h ��D~�r\_1g �����d�m^�Aǰ����� 9���\*����/<�B��\_������\_����c�U���#Օ�������ܢ�Zj����P�HgH�Z���Dy�Gm;��"'-��}�\*C��#�=��q܆>���$�gSs.3��GXc�s���(��S@�>W�LM;K��D|qA��G&�z���?U��k�e u\_��m������k/���j�AFttܼG��b!�i��qy��y�W�~H�PF�Z�0&߽�I Cu6M|P��������a̅�E��< ������~��R@��B�R Q\[U\*;;E���@س�~�d�\`�X����������S��v����\]��6w��ӭ@j�\`���@�cC�ޱ |+?5�.L�7�����E�pJ�6P�+Z3�V��I�ĭ��A�6� ��φ��ģEzؚ���Էq�p��Eo���W��z���$�M���Q�7o���&9a��-��oTf�j?!���7wt��y�fst��R����J���}�g�� D�'��l\*�� �i��8�b��aop���1�QFHPi�hfj'��-g;���Y�U�z �#\]�޻Yb�9��?׳�-ѻي�"iG���l� �����IeJ��퇜�����/�c�r\*���ű�gtrɡ��jʖ���߱����Ӎt�"�Zy� �o�¡�^�6aI胚n��:�u�k��w@�\]t�����%�m�PT;}?/����.�P������V�'��C\_W�1P��U�6qS�=��?��FS��U�̌�\\��A�P{�U�� ��y���)�� ���\_�<ߌ���pP��M���@�X(%��d#{���������(\`R�:W)d,�$\\u �e����T���� �F($S�.W2��~5Ir9�j��Db�M!�$��z��X\]���n��T�S�՜FQ4;���5��G���-V�ŚҚ>�w������B�dž�B�\`��d�#���Uߪܾ� &���>��v���\_�pU)0�"�,��\_���?Q��\`����&DI͞Re�}�k>� �w��o�Z�����G! \*��\_�Z7�-Iܻ�$z#R�-�B�����ϯ�W�:�1\`ڌ�f ;���{��LIp�y�~l|Z�;�oR�ﷴ/ĵ�����f< ��q��G?�O�Xdb���I��g)G���S}�Z��1�+�aU\[��8�;s��F����@��Sڐ�є�"�&���J;��҂�A�D$�vB��� �+��l����r09Ml�,�?ғɭ?��\*��R��hBëg�d��H�T��y��rkڷ�����z�D.8M��'l���ƒ&����4}X~N�6fia��I�e$�� \\i��ƴ�\_lts�f˯7�ÂGک�� 5'n¯����@��,���~f� ����'B � �C��K�l ��Ј�\\��b���o6 �<�\]�0h�����ԧ��k\]�9�F�\]�#�Gx\[�.��qΥ(�����5|��†<�r}��d����DZ �U�����y�f�'���M$e��Wl!!BE���С��-����H�y���x���,�|��\_v%��ֳřMa/Q�W%PV���ţ��?O @R�4\*��m?~�K �qe����T������ @3p��\]-��O�3���N�g�7 �v|Jy�Ӎ=k�K5ƽ;��Z ���/�\_�mq�|�tuc�:���\\"#ɇ�3��J/�� �Mڈ|ɺ��j���4�DV�Xz���\[�Pm~�n�@㴾V��C��\*�ѳ�z"�մ�c��(�UyE�~�s���ط w�;��Vt,qJ�L??�wn.����e���\]�\]h����O8.�\]�"LčCr��bO4��cA>�ן��@O���M�f�c� �3k�HP� h^hd��GW�x03A9+^�Bo��: ��>za�Oc�Xއ>���V\[f7+$�\*V��{���س�BbMk\_p37�%%��Dٛ� o^"���\\�����8�9� 8+q�Е��T꿥��q�!� %k��n2Y�Wߕ\[�ʂ�� vwx���ZR䀐� bc�9���d�����1��G�ܒ���ݬ\`��N�hTP&���&�����禁�f�vʏ+��\_��3 ��F}��@�� �@ĸU�tO�G��C�i�s�߈&�f՗́��7U�4�^+�@˙��G(A\]��q�ӽx��}{ҷx��tu�q$pmT���\[�^��&��\*�+�:P;"́"o$�H�x�W\*�q��cEb�lR����qS�y|c� F����ns��?��EXM��t����" ��ٲ9g-gp�T�|�m��9<�������;\`� �rT|c\[b��ǽ��+���/wI h�g ���6;���ٕc�.ēM��k>y�9�|��fY�U$���uPP!k�zA%�O��E\[�=U4/"��֘��\_�z�TDR��M�-jyv�F���8E瀜�'��\[���� ,��p�Z��DA\]���YpK"^��pCtY�nx�z�YAFy�EDEoQ�� ��>)-~.RX8�2 :��P������@btJ���|I�(6���\]x���V'�9��4zMݣط��Emj@ l����$y��8�.�8H�!�@D��싎�Ĭ���7�r�} 0\[�i��I�#�, ���g�1�=�c\]�K�^9l ��?~�y�sk҇x:��������N�-��bG||C"U4��uU�;���$�:f����g�H�j�7���(�\\g�"���j�|���}������:�A �0s��IY�G����+��)��\]�\_�@����\*\_�jβ���j���O��-t���m�R:�1\*ݼ����7��9ai����/�f� �\\Z8�O�� �o@)H�\\kWU����!�f�!���.E���C,�z 9�������s���laAB�\[m��#����ϴ� �b�:0��ȱ앷I� �\[��J ��&�����|�ԩ\\Vc�NŮ�r�5���P��� I�(�Q�od�3�x���'\]zuת2d\[�9<�9ݕ��C�E\_���� ��\]I��\\F���ޝ�Yރ<�������<na4Ӝ5A��\*���������������#9����U#�> "�n��8���� ��^�j���AB�v�5�Y�Zf��ȩȓmB���<8��f!��y�v�ba0}��?\[�;5L�q����+0�Ƀ�����i�Jb!�5"�~�#�ke�ݑ�f��mKk� ��\\�u�rm.�Q��/:.�n�v�u�� U��O4DL'�\\�􃛢����t�}M�B̓G�rѢ�h}�(���-�0ȉ� ��\]~�c6#������B��n�T��OJ���B�Ӻ����=����'����$�K��6l��}i@�����X�\[��S{�ް&��(2l;�N�m^�W��bTO=D�@Zص �d���y��%4\`��"�\_��Q���D8���� .J)�����,;��@'5�sC�P�Z�.�&���d�T7���@���Pz���z���=�G?����1������h��h6�?��u�I����\`TZo���~��s���̪,�|�\`�F��I����&5�{�?�&xz��U{�,����8�4Nzս{G{)�U�97�5��Ó;BϪ�P�9UY��Y놚������i�����؆�\[����+،BOpZ,������d�z�fe�Ʋ�龍v��:�G�ИD ۔�@��ӫ�E��0I�>\\ZO\]F�~m ����nh��r��� �a�l�\\�wΦ�H/\`\]�����9�N9TW�����@4炟S��G�>X �㊑q�� -��|PD N����MM�X�or�=�J��בR����<�K\_����)yz�ʞ'\`��&����A\]��vk#�ew��z�ƌ6��u�5��V����\\Y� Վ\[Ee��I�K㦆���Ѵ+0H�� ���WX$��2��w��'"oC ���c��u�BL��sW�M�/;3��6����Yh��������N����C�fj+;s-CP��D^��H~�J�G3B�� �E���r2���Ko?Y��P�#\*���'�i�G��wg�I������~\*�r�n�m��| �a\`��R:�f>�)����ݏ<��74�\[��xlh�K���4޽���qGݮ�UO5��/P�İ\*e�r��\*Q<��,����6�cvr���I���5{�Ʌ �f���J\[e�� "������O>�AF�+×�f\_�c\`Ů U?\]��5��\`%E\\���!Gt U^^9�Ļ�S���:�%����3��V4�'� N�w�i4�����|� p��(�Ct�wCcX��>{�}f!�9^/ް9�)b��Pd���,L^|�9�}��kn�� �������5�Y5�h4�W���A5%t����������>�&mr� �����γ�i��#H�!�g�n槆�\_� �Y ^P�0?F�ءeYB� ���������Y �/�~ұe�>���&Ld��n\`���J�y���/�\[ \]:������f#�S�sc@ڑF��@��V2�Z�XI��k9��"M=�Xqs����E ��@�rs\[M��� �m{E�Ĕ9����T�:ជb\[&�����Qf�3?���a��FJ� 9�0y�>韞N:�ط�Ln:�vzX�l���a!���D����F��d��l�ğµ��)\*�S7�Tcx���?�E<~��wb�&�r���+z�P��j~�-:�3�Y��n�����,k/�Dq(=P�@\_gF�����&��D�#�4�V����p �v=)n����|6��ށ-�S��<�(��F���̡�咎� V���"p����6/��N�'��|�Ƕ�T�N(ٱ��/�. E�����|F��4���,���M�������:�҂� )��>�H�7��^����ivF�Q&pY������K�+�ޤ}Bc~�)�\`\\b^��R�,� �ԝw=���͸D�S({�td>��^w�i���aLDǏ'"��\`�ӚuV6�z�K3s�Ў �ot�ʓ��l�� �R���KB.���W����#|�k��B0�リL�j ��e)�O=L13 (m�S�9��\*j\`P����&2U�{�T�򑛋�^ ?ۂz� 2fp�\`�r�dV�^e;�� �0�� �o\] R<�f 1�<�u� �q��0�N��W��o.���HK�Y���5�+����\_4����9K��z\_���HL�g����.�;vm�?٧Qi�-g�BK�\]�g����ڎ�3�� ��6�3� ����%���8-w��\\u���K� ��Ldϙytd{�h�\`\]X�|(� -����—Px��"�U� l���g1�i-��PZ�Ś�v��Q�z�ԵK�r�/H� {�F��\\��\]�\_���u!0�\_2�J$�����~��͸�)�\\�\\>g{ՠe�c��\[)2 �JGE�"�Qv5�����:���uI�Xi���\\TO8͙�I�J;՟�� �f�w�\\ M9^@��~먁��'��(I��HZ(Q#�ʆ�=�P�O"�Ӄ�k�q�r�;��T�PC�م��K(�rǵ.����ADV�9����ٗ���\_��5{���w)v��� r�������4 �5�\]~4\`���J�1�-��d�b����3R���ݥ~N�/�Q+,�Z�fn���2���8��+�\]�Yp����6�����I�\] ^��,G\*\\yݮA��� C4���Ge�$�b�g �����9%�<�k������r��Dul^\_�fA�T5B,kT51\[�^U��I�7ste���x�?p'��� b�vsG�ڨ�N�q?=��z�1�2IΜ^U��\_\_����M�ΗҺ)��3��g��$rR�9� �Xa��i�L�G-h@v��,�s��cD'����!�k��s�52��գ,�}@Jh^��Ԓ��aJ��qxgD�b��Z s4�;��I�@}u���zm#J�@��i��L����E���bQ�K����̧��2t�PW:���w��2;��ٺ��#! �"J�Ď�X�^H��kP@��F�N�8V�T�"���i�uE��J�<�͢cb�;)~z9uS���z��GjI���f�\\�sˆI�V��Y��� :��-�X�?�Am�C��bkY�ʥu��?�!aTy�q CJӝ���q�lL��1Q'\_�)(N�G�?by��"����@QP�$C�����j�WW@&���0�ჶ���K�� ���;�}ԯ� 1UƲ�4�-#�0c��K0\]=n ����|D O>1\[)��G\_x�.r�\_y�Z���|�$�:��E��B��u}��2{"��J �9�$�fyB��R=UY�� >��{n>?��t�ErJ����� �.�a�C�=�s�,�TU� ��m��:���$�d7fb1̈́�4/GBFʊ����T�+����a��-fw��v��6r �I��﫝��2i$Z(=�9.?����n���e)��G=��g����J�0�"x� U�- +�s�����e�k9c�#/%�Cv��g�k�p�\\'#%{!�\[5U;� (Rj�4�L�a���S�b�ow�i�\`����\]��ϒ�6N��8ܛ��2�r!C�;+��ʯE��MS�Я�I�8-��JK���8Wv N"�ɗ-�#�Q��W��� �r�n�x~��F���\`���.����U�;#~5�Rٻ�#�����i��SCE)���.݁��Ͷ�L�H\`B�jfyL\[ F�y�O�N�ŋݐ��'�����<8޹�2�f��\*N������a�� �"�֑��MhB�Z)�ٳ�D�K;RF��~1���H7l��3��)N�q�� o��9��Q�|~��� m}���yX�(��^58�S��c?�؉Ή���p�#\[ l�Z n�Q��L��'�����sq�'�0��&զ�1�vs�IGҺ�ѫ��N�,����N5�\*���Y�?=�K��@M3�g�9�A���Y(zt�~�\\�(|ȉG�;&���C�Pg;���q�o!��}�^I6�Eh#���J�>V��ј�^�� Ҡ/��7\*��hN�b��Òu�"� ��Oؿ'\\����4J��LV�ΉX�#C���m�U�'���'E�@�f�R٦^M� �&���� ���o���.�̲ u�fY��y�M\\�{��87��v�wp'�b(��z\_�u6�Y�5����)ͪi�N'\`���%�p��h\*7�\[��ݫ��.�N F6�%���c\`d�7��+B�fm��ʙNl�9���������7ѩ�D��M�)�J��x�G M��M�ܝ7���N�� �!��gky����Pr����@i���uz�jo�y���Ҵ�@6�~%��=�����3� -��\*�^Yj:�r{u%W��q���8r$�E�PP��q��0}:�|u�85J�!��јQ��Wo�}5��} }\]GB8�:4�}��a��a�"�3�N��p����п� eD�&��JM�c����&��ߥ��ʜ��, ,�†�2����B;�xˎ�D�i��V�j8�@�N���F�r�y���\[�O�6�D\\���bZ�����  {U%�-3����k!ZR�h|�}\*A�-O��І0�XY�x:�JJl�H���4�S7D>x�\`-�C�����n����ڝ���U��d$��LE����Z����<@k�u>;��ʱ�)f�z.�RzL e��ӕ�L^̼q�m)�A������C+'�Lr,�Kb��<��ü§��)�.�Lj���QR��\_i���e�˂O+�p��1\_�޸v����1d˅�邋�2rW�\\��e���q�5��Q9�)o��Y����ͥPe�ƕV/�w��#-Q"Ѱ�K�9S}g��J.����;Y��׆W�Y��>��\[�n4��Kd���e\\W �zgp�s6�n�h|UO��j.��;�64X#���ײ>�����r�9gR��I�6 VuH|���JA�p��u=^"sضo8‡g{o��D������\*��oj�ҕ����-$r��̝�{T4����dMZέ�ٍ����Mt�#eƺ��T��b,��c^�x�P�}LB��T�\[1�J~ƙ2��#j� 8��Z?c"-�Z���ӽ����4�7��2�VqL�f��&�'� �:�욖~���:��O7|�z��}� ĴHJ�T���g��2V���<x\_����{ �Al����x(�� OTX��wJ�3^V5!e��k郣�ɻ��Gx�E������l'���4�B/w�� ���b��)\\������ְ�����>��k�A�G�pLu�=�\`��\*����r� �D�ل����{���J- ���Ю"g���X�׊ܽQ"xN页kP�i���7������|hؠ�t{wxt� x�C���Ǟگ~#� �����"w2�8�����ݔx��X���&{����j�3��8^\*D%G\\�t��)=��\_��Ե@�EA��ރɽD&�3�g�)XL��Q��q36�s�|���+����|�Ht1\[��>�"�A�c��e6������}�jUBf����C�QH<\\g�;�4�=i��Қ?����z��ُҌ ��5��7��;Ï��"Y=ۨ������%�1wц�!���O�I��$i,���uZp�<Տ���%2ߎ�eLXSFK���&{ �B�aX���NUs���Äk7u8���Ă|v�uK獔�˜� |#y1Q\]iQ\\D�p���W�-�T����O\]��Y�p9���e��4T������ց���3G�o+bc%���KX�k�&�d��\_� $��8�g1?���?՜PƔ���)���6�\*��ɍn,��f�ʱ͍#�;4\_���K���ﮱu(X�=����Еs}�㓆w�'.�����\`=�9��/v(��!k��Yb��Q�6��� ���\_�A�+�����#k�e���3�%{�++�"\*��\_���R �+���Z㍬����pW���d٩�Ú�r���T��1F���w�N�~�BS��m� =��eI(��w������v���t��<:p�����e�P �\\ ���������� ����2 �LI�w��m w���} ����GC%p��H��1�������QV������� ^�q�Р�߱��g�q8�d��zFw�L�@��'�>{Ĕ�q�E�=-m@��@��hË���w��i��Kde\_3�\_�߶�RWB��\[��'����Y �eg�Yv���T@��E0���2{���hz64U�=�\\ְ˅}a��KzQ"k\*��#?¡�9J��j��͔�o�;^��A�m;|���f��y­�y���N�Y�ң�����>+G<3�bum�Vg��)�Q�:s=� Ϗ1�i�T�P���"�}��XL�h3��Y�IBLlԦ��n �� ,b.�2ߖ�^�F� {\\���ߔ��͂|�<�RlO��g�2ң|�6S��R�i�7��+G\_��1+�˘����n�T�I^0��oW@ϮOȐ����Ub�� r?���O \*6���w�rY|�7�2��):�FFr9��A �\_�agH��ÅV;���l�b���#��®�+ ��}D�~����}��rٮ4R�/��Q\[D�n#��V���e��w>����O�T���=o� ��a�74\]�nN���H}��h �wcj����������z���%D��� ʚ(��?���#�0 )aG��G�Ĉo�G�����t�\*O�K�5?�yɤuOr��7��\[�a�u�8��W,�R���v}�����WQ��!��� Ӳ��%� �$�� ���a 5���6�|�v����4�w�T���B�X��q�$K��#��l�\\8R��!�|e�U�6y /OV����)�9cX�MX� �I�/Ƚ����� ����t>�=�\]����uK�X�jn�R�'��+� �� "7�\*.�7��E�\*�?Dz�|k��x���I�����º��% �U\[0pQ 5\*�i6���ײM"��>�3W�-�@y����<�\\�n��Ts�� =g}�Lg�"(ī�� \\(���t?���ɝ�:� h�,���U��B�G��k��4 �� �B I{�K߆�C�h8άў�M�� 9~lĠ�Fg��5���T�h�7�C�S�J��GE���5��xs�|���Q� ��sM~t��+�9g�I �ӫ.؏����U�C�vy�Gum�F �V����כ���s�9ʁ ��0� �,�0�"�v��=���T)�'{ӆ���Pz2����yp�^� J���P\\�8��z3R6AbA0�kd�\*�3�<(��~�� �\_��'M~Z н���/�� �R�\`��yi�A�Wa�9��I��C��6�UE � �\_C}z��Xk� ��$�����I����� �2<�/8�K�UR��4;�E�æ�P�oC2��KXu 6CP$��s��:� �>��\[ ��3\_��\\��b�\]��-@Ϡ:}y�#d#c��ꁝ���.��w�aI�RX%�iGuY��3�n�7���<��7\_w��c$$X� {af7l�8����X �أ��0��>���u�.\[�|�C6�C��#��@I5� L,:�����ڏ1F{��� � E��> �+Aq#�\_��b:^�(�I���1�ܖ�\_�QV1 Sf�B@uӍP������B��Y����X7� \]\\�5���.�����p{\`���Wo1-B���PA��.u\\�\_��M�O�J KK�OV|G����~x�\[a ����� r%�9Q|h��N\[S�Q\]�Ž��^�^�̬攁U��~d���f���ɽ�dhi�� ��j��SFv��xL2ˌk�0\\�,X?;$C��N�P2$2<�����L�������2��S�7n��3(Lbq�N���\_���"�>,���"�� \_25Ϩ��&ҋW�t~���&��Pr\*լ�y�4c�a\*��ȶ�&�t�a���x�\]�̅S��ɟEE�\`��K��u�7� ��^�9��3r�֑�H+F��\]�/��q��\]~�X��֟�waаI�AKe��\*l^��>͈��rE��(�6+cR����������U6M�6f��% ���^ogF\*��p���:�)4��X6�^��;,���\`��^�A�J3m����A|�N\]���@��n�^& ����ȸ���ȔNK�4�~��5��j\]CؿMs\\Qy���^� K$�A�ihe�{�� �ĸQ�a���p��A�@�����gk��:����Ln+�f 7���X�.�jS�%��ۤ �{:ސq�O�����Oz uP���%��f�󻓚D/��ߦ-������te�,2lR�{�F��/���g��bf\\4�r AU ��9�A&ʹ���eia� �� �!|�7&bQMN��\]WD@DЇ�%B G �#̘@-�+�@q�0�'�J���\*l����iF�<��e���睆���8��ލ==��%Z��� �؏<��RDI�,����R�}�\[|s��P ^;>#���� �@�5��m�B��L�<1����L���:Ȅ#���t���I�G'� �QX�������0� ����%V酓R��9%4;@��\\<�J+H �����ײ<�ۖ$R2��Q�}\`�gml���St�(1���P�.�zN��q��v�pfe����Ts����P�을V%&���\_�xr�� \[��54�u� $X����~�z3�'9M/�p��}�Fq�QJ󖜴��t�N���?�� �V�\`�KT���B��ۍ�+;�>�� 'A�˧jݡ��Sa~ET��� �AxI�h#��ki୎Blf/e� ��q ҪC;\`6�j�0M��\*u��+�\*�����2"�j�:�>���������N��W���T�\[�Y��k\`gm0�oء70�W���~YR���>bBR�2{�\]j�<� }.1L� ��f� �b���X�=z\_A �v��̄=���h=���I�\\��Y�-�����BI�<�\[:k�Sd��=��dBeN���gd���'P�y�vv�N���c�P������$ dWy!g�)�ZK����8^J�XQ��Tv-�2��\*-o} ֭/\*~�up��퀀���8��qaI�QaHwV\*?�\` �3� ����n��.�"զ{\*��a 7KA�^��eN��� z�w�U��5�>�%�G�O��y���aS~�-�;�hR��Tz^�m\* g͡���� �H�0��y^NrE�okk� �a4?V}X����������H�%��g"Bod�O\_ f�N���\*�=/�p��#����j��NY�k�e�,"��HQ�o�$��̚�ʯ�J�s!&����=���U8����������p�YX��+�!�B��2F�:��3�9ŋ�i9��>��PF��|:n� ��鰍FE�\` ����x .}��(q��žVG02�x�X���v~��U�L�kSD�ߵ�I/�����?r�9��d(/=w{;�\]�eyB�Hm�dv�a}�/��(��Pz4���z���t\`2��\_�+d�ٶ}q���,���������ބ����%Dࠧ�)��'5�j��n�r\*��� +k�5��"��!��wՉ��s��T�?}�/�%�)�G�$\]ԄX����n��}�1kDV�~O6� �mV���8'�N�n#Y�F�����u����V)�~�^���\]�D��(lM��<�-z1�w���%@j �����c��l��u\]�&�h�n�� g1틵%rKvČ��5YT���f�3t;�j&�����\`�T\\�D��;�؍��\*}���� ��!�xvI�FJ"D���W��:�\]�\_��\[�ף�ݕ�Ųz��}b���>��G�^dB~7A�h�"6f��6q��|��Lg|P�:�U����U=#J�I�:�(šG�!����z�"8�X朰�W^u�� \_A#? !\*E�E}�\]x cV�\*5�^��5���'O�F�\*�D��ܠ�v!կ��y����է�'�„����!HTz�v)�ġ�~i��7\[���T��J�� /�|�<��\[R#��!E3}E�dt��~�����י�x�ik+��um� �(��8��n ��~�z���pU �$���%���> ~\]��kɝ{���#�֛�^Ő�����>��^��Ut�\\��'��|�,Ïve\_S¤�6�����|n\*ڀ��:��\*��Ixk��\*�O�\*dD��!��Z�����G��b���"p�����l������̱�����<j����Q2.�����yb�����v�L�:Uk�����:iԀ���Z矉����Ϟ�X2���ešdU��я@��Nu��\]F��^� b��͉jR�����Z�Ia�k��ܔv�%@��=6.�䄠i< �V5=�F>>N�gn��v��{)|�Ǽ��'�һ���\*f�q �����t��}�:��?&,��)��0q V���E�V�pV3TXڪ���T�|�������A����\[b� G\]��\_���n���u��^��\\W \`����!Fܷ\]����'�Nu��7��X��YI^lj�W��>���F\*.��;�T� ���m�;���'٬�V�8f�0f"��P�%\[)/��NЌ�zY�u!�'��5�dc��7�E �4�C��1ϧ��(� �<���U���.5��rŧ �@�u6��"� WW�w�{��;P|-)a< w�2(�����\[�� s\_�X��5�7�\]�'k�K\\N?��yO\`d�͸��^{��)H�e�3>'�� e/Xj ��I7��ԣ�g���v�ܦ� �E�ĤH ��\]�,���0S��,���ԟ�u�x���3�s����. � �j"Yギ��\]υH���G�w����@��P����Ei.�/�Yc�E� c��\_��KquNrf�ƍ���cl�z�l�=����KM�D���� Q?bbɓ��Q(��\*hJ��T�K���>�h���� p���~A��njb7o���������>44��^�Nd���^ �f ���VR����rf����p\`�f���n����v�-�do\_�4l��}��Oj^��-uz\*ۄR�N-�RI�����hN�N�}Ԡ���(��NN<3��vX��D؉���4P�3H��# �y���.�p�(3� �6����aF%9�q����"�A�Y� 78���\[ x�3��8��WnR���� �C�&Q����x��Yc:!诋�6K��Jh�84��8��5ϾTZ�EM3�J0eᅖ��ʥ���k�S��Š�g� ��?�Yާ�V�cl�������z�I�S����Yj��i����,����e�.���!���Hv-s\[qSߚ�+�D���4��B�v�\[S/x!(��9��ԗ�(�~�����h�7���v~�A���}V�|�2b�Ta��zF ����F��F��\[Yk��u��;/��;�&͏�\[#����.�;2�\\\*k�V�v��q{X��Tl��9�4@�Zp�p��v��ʴ�פ�5��%m �d�>L z��kK��M��\]�fd�D;�f�X�Q���,1 �S\`CqHڕI�YI��j���a��R#�· �\*�>5@�Pz�A��XEN�mI���+>��ö�h�xM����"?I��X/mX��@t�b�zEU!lN}�j��9�r�\*�Lz\*�׽\]�9�Il��|k(�ڵ�%z���%:y���L���0��N��B�c�AWN�֛�Bd �X͆�i�8�6�ְ�n���/#�VI�%ʹ����l��:�;|D L,�u7\\�VO�������>����w�V���cx�a~F�M{�\]�\]�K���\\ :�P�e��}T~F�Y!#�ٻ�\[)%� ���B6��K�����W���>^?�R�(�e��gk�A��e��%�;���R��L�����ۗ�l \*�˝u\_���\]\`�.�xCq�a�Õ��Ŭ�~C"����1\`�e�G1BϷ�� �6z��8�$�rBL��3t~'m��1\\0M��z�p��� %���1��uE���I�!�!8�<J#2��\\�A�u�W\_9\_C�V�x a��b5XI��;��Y��>������\*�"�F�����\`(Y�ʭ�w�����wi�H�A&�+���W����q��: ��m�r����׿�\*�6������U�����J��rP�\_���A~MM�ˆZ�4;��X�J��������Glt\\.4&�j��{,���KTȻ��h}L��W$U��2���0�&���w�{��A�6�>\[<�� ��+#UE��0���O$��t�mtz$��XT0�"���{���~:(i��?u�l|������U��;h��4��?W�����Z}~S���rsi}����<���h҂%Ifc5Z���\`� ™M��5S�����\]�~���)��T��b�9����^�N͉Q�Ȋ\[Ba�b�7�6?�\]��gy�9#k���r5�lc��dO7x���@pm�o��|ɑQ��x¤ܚ6��=���/����ZD���i�B�"���;�6@-qFX\[��/MD�~dd���!f�\_�����^���Hj3L�I����f� �|�-{�J#�T���,sę,y����l!�4=�� )k��A��qA��n�W�9aN:Xp��k��I��)�� ���T\]�B�/���#A��5p'I�\[#l3�j y���&QB��q�9�=pѴ�\\V-WՏֶ�Vp�"�a����TF�� :�9ar&�>D�+X�3x��BV!:�z\[�?��cr���3�Y��p�"}Gk�h&yܛ�y:w���~S�鱗\[�a��|�<�c��\\!��MGÊ�oD�ySa��;��;k{5�֚8Ԥ-����c7N�SI^���lh����c�4�\_ְ����� �?AK�n=�w>��ɫ�yS�{�n/@\\0M�+� ��XH����2�۩�%�6� � ؔ� �\]ؐ sH� �iVV@���O)H�!\[�g�ְf%úw�^y�- �qqmH���A�>�{>VDN�Y\\ M�7�)�s4�q�a�~�8!�۴^�m�x���ʚգF���k������-�T{z�'���1ׅ�6�RA����ĢA��K���8���T!�h��Oa���1�.�m�a�U�}��~ڟ�U�k �AċJ��+��)�� �' ��iݟ.�~����R���uKQOO�4��U����� K� �Ȝ/��t䯳�� �a�S?�m\\e�m^�d����h0, Y�+���Ϛtϣ�� ���pC�T#/���Q�Ƞ�HΕ1�M�?,c�Ә>r��O�bO�����K�\_ۙZU�:��:^K�π��z p��2�TVe5�ʑ� �;X���\*�)����!�1zF� U��8�?S�~��@\_�~2�S�/5"yWΝ������W�M���g��I\[���Vw�k��B�jz�kد��ނר}�s�\\�����u,���,{�˴�}('�馗 \[�|c-/�eثc��3�Vޑ:���bY�Z�^�q���d�ʯ�R���@�N5���r��Д7&�&x"ϙ^�aH��m��k&���@�0�(� Q�~�^e�De,@��R��+AZ���A�4�@�2'�c��(L��KY���A�����j������F��灞b~��{w�U�f���k'w<֫75����l�H�Sf���\*������j��J�\*~ğS�����$�A��z�D������� l?j����>����K\*K�z�a����;Ҋ��r�κ$����(9ƒD\*�B�t^� �� Xd����uU�mʝ>Riy=�Hn� Ỉb����㥑�NC�:��d���~�\_�mP�/򽛊߲���hkbx9���n�^=hK57��GR���^h�j�>��# ���o6!�ko��>cݑ��ؤԦ1�z�\*�R Y�+� �@� bq|��c����\]/�K�=W����c������#E�������4��$��~��=�\*�M�<���~O1���yٯw@�E�����ܞ�Ӣv��/3t�|�lUK\*�L�(m6�O~�\[{p�5E(�Y�н܀��g�{J7���a������"��=�D/߉�� ���{z��/�Hv:g�9 ׎|����o�p��n8�W�G������y"�����{H�q\\,=%)��A�{�jrq<��pp����i f>���V���uh���?��x��<�I@�.��ah0�U�� ?3l�t���li� I���2yt���n� &6�����p��.I�zSS�����#ޗ�����g���16�bڽ-�N|5eo�,j��� C����\*���-p��4�D(l��6��F\`~�x(�y���ujM�����!�6����T�A�Lۚ-��|\\M�r6���si \]� #��/K��\\�o��\]����EB�4R}Չ��P^ZV� ��c^y��뺛�;��ع�ڧ���'�w��ȜŲ@\\���aTБ@AV����up�3Nr�M������7.Jg��?�#����^-Fr�kE���|l��N�q���~\\b�W~}�Bu���IqyZl����+6�u/�pz�>|�ܴ��,S�#��a�:�� �B��G�������1����U�gc�~U��O��9=U���Dz iҲh��a2���n���W4��O��RQ�Z����7����KSk�b,tp1�}�{�U��^��޸��J��y�j f�zs����E·�(lH����t�^��u�AA,��>F;XW�K��ʹ�����8�M���r�h�u�:�1 �ġ�3���H���;E�"$�J�x�����ݽ�b� 1�J�U�͕zq��i�y ���XV?�U�C� Z��N��A�ہ�u����Y/��uT�A׵�Gd��UH;��+I����.�t��v�����H� ����I"a�ٚi,S�b@u8�ga�|ʦ�顓+��Sl��3 |���r\\!�� � �DTVx����\*� a��i.���8s6אbg�=,W|���.\\6��C��k���<�,ƈ��Lp�j@���\*j���ȥ�K\_�3~����Js{ ��\* X�/����t�ˋ���� �m4%��2),�(q�A�������M)�x�Az�ѐ�$�+�EzWQ��\*Y:&/�2Ī�\\�\[�YU>?»��HQ q�@�m�x��%�b� x�q�n����Z���3������6���\_}��Y2��T���/��~(�rP�{K�4�'���9Rc�~�g���@�BF����p���i���7���1ʫ��T7ʚR���gs����L� � �C�C�\`G���\]�Ζ�0F8I�7��Ƕ�訒 i���+��D��� QeY�}8���X%}Vr~� U�# �ۃ-�#-�mFN���sG'X\_k��\[��"�f���/!!�|�UlSRۙ�m�Bq��"���Ӆ,٭f\*Q�'�r�\]iR��Ŋ\]����~�8��\]Lc�,L���W\[:3͒ MP��>=�l�������x�E �#g3�n���%\\\*B�=�3�E6&�5��U\]t��Vȝ��m=�͠��< f�>ž�p�ԓA��Q��\`���\[�P��<\*�2�c�\*G��O���\`?EW�Y�����ћn���땭�#\`r���o}xU�0����Z!a�)�|Z���Ϭ��0��'b���Q�΢�(p A���������P3Jh�؁7Y܆>u ����J Ծ���{R7�S�p�Kf\_�3 � �S#�����=��� E� �E!�/���Z�w�r��{M2D�N ^m�A�\\���9o�\[� �=�aG!�G�6"���2����y�b��e m� ��wc� F��0%���P����Dh����B�H ��g N�\]G�M�����F�S8B1T-���k�:Vb@�s�@��,�Hu��Z�퀭X}@����n�>1��, N8�"�K2.�̡u����xLϖ� 9@?�$q�U��5ؔ\*1�iBV�Z��n��BH�պ8�X�}�.'���1�g�� xh�i�����v�O� �R~��Y�%�$T}G�ˆ�Z�� ��z�h͓4�%�M���&,�7@u�E�qH�� �Ӝ����-n�O�ld�.Fې��Ű�#X�= � P��kI���:f=�%�\\�M � �4kS\\�k�{�6��Z @� �GO��e���n\[(�U�P�sC��� \]BI��V��M�����b2����.!���\`����A-�������C�o l�QLR, P6�fp���\`{bF�ؓ���&�g��B4���g�+܉��is�7������b��P�eݝ 0�a9�i��\]�O�8���e�\]����ٗN�5�d�=�R^�o� ub������|"��g��Tɢk�i�⤪���>�k^���@0 N���\*�&\*�Tn�E�K+���VU�<����Ko��8M�ԞYQC��A���t�$��%d�i\`�e�3�d�̬z� �'�/%o�@�>��f�@��sQ� ��$. �s\[C�A�Zj\`"�9�Jr�\\c��$���� AsZ�rU�T���k�� B�}m���I~ KP���4�A�zH�)��kH����,�2M���9 \[!C��3Z����/��س��(�Vv�f�,��̧A�F��39A�rA~G�\\�>���x\\��9;� � ��W������f�X/8B�8�|����lP ��J�7��ݣ��;��-�n+� eUT�ֽ�e�����-YEt#U (�U@����\_>{ݎ�r�\_%��WF��i��|h��T�؏�\*M��M��W\[��2N�L��!U��L�H��o7I��0ە�T�|ח������Tq����g�|���\]��p��A�\*�����{� z�1�M��i�� �� \_��D�S�X ��s�φ�¾mS�3K>W�4)�įy;��.H���=J �\`W�\]F܇�?M� ?�E7���2���N�dp=^UHG�k�ن�!:)��#�q&y�Kg&��s3�s��k��H�u��'q�E �N�꧂�#��16vp�oN��J ����Pe�tw��\*����Ў��Dʕr@Z��ǀt\_� aC691u���,�?������I&Z���o\\%4���+���N��t v+�1@m�vڰ���>���Q��м���W��Ѻ�˴�xS<���P\`� Nx4k�:�k0m ���d� <����w�k!ɒ\*Ū�\*�"����r �K��m!�U?(��Yn;u��^\[�C�e�ǔp@}�݆O'�y�D6�nz�cR�>X�\`�qA bpe �N0Rd�}h�DW3E� ���+ }��ٖ�\_<\]�Ӳ��΅:��}��nt�2y$���\]�7g�M����S ��:\`W�����\\����ϰ���wR�H6��� \[�熍�a����n\\�q�GD�����z�9#�C^�Z��=� >V�@��!�W���#�V�e��f�|h)�b0�\[b��52�\`="�k�?�Aۇ��ɇhP��( ���WIޣ\_��Q�����������M�j���>!�>��A(��60"̈́�C��!��R�!bd�%�~�w˻rBBȣ�➏�<�7\\\`DJ�<��r㸩̈́L5o�o yh�>�ݯ0���$ Y�����3-����@��%��)\`\`Y���ýFO�:m7\[w��{�4Y�-�l��AD1��nv\]!z�d�8{�a�47�u��,f�DSC��?�� X��T�� D BN듪�K�\*���^�A��kp��L��C�;\`W��ʰyE>/�''�.����#�v�'�8|$v ���S5ލ��^� ��� ���o�r=$�g�<�a���ai���x��I���D�fƖ=l�w >���%.=E�xA95V�v\]\[��T->��eĎ�e��G��&�͠�� T"��8#Tˁ�l�m1��saxt�áwm�YX^D���QH�s+'��� g�kpn��Q���ПQ�jB#��p�%-ރ��З�4��B�x���=��� �ɢR�x��\`Pަ��}���l�5�4�"e�3�鰿������ґ ZJlF�����R�Fp\`7� +��T��DŽf z<#�=�Fje���B7M1S¿�s?�y8�\_�E������L.G�j���#|�x�%�?T����;V\[e\[t������/��"K��n�Q|+��Ɠ��kP>��C@�I���?�&�n��U�<��\*{S�����e�;w1�)���V����n��vm?d��O��~O������0 Q�-���Oe�̍������J�N��x��������x���= �G���b�/�S �LE�߸T��� �lÌ��iY��>7+gh(���߷�~L�?\[ϻ;�{�dž��:�b�ż�#���J ��2\[��>F�s��#�6{J៟�(��Y��İ�Y�9;-� ���!��W�r�B\]�=��Ֆ��'tFC��'>� (�9/�.C��l�E��ߏ��I�GD�����:2�T䙎������{W���e�\] ��-=h�W���� �YtWy��N��#�e�����c��qC���\[7f=��R���Nf5TJ8�g=��� Lh}�\\�˶їlAK���%V���f�m ��C���G����QT�3^����swgԾ��<瘇G2��k����\_�v�B�5/�^�8-蘋�a��\_�A�z�~EI<��A��g�:m�;�ϼ��H�J O%?�ש�(h����N�+�,\*����M/��(�y�r�+�����0����Vث�r�q���\`J��r�d)f���,w�9Sָ�t�u� @7/I~�p��jϧ� �����˸�f �l,DA �E�b���(��&���}��>h�V.�H�&@}SVi�/nU��vC�«{��Q�Ӧ������mxm�s����X �5�z�/�"��#�P?�;�<ᵠ��$�r5��ul�eS\]v%1I��X ����� �,��lnu����m��\`1& >$�6p�9M.\*o�m�y5D{���nT���� ��W��~|�C>����|�O������xtӟ�\\vf�L��m�����e��$Ff��Tc���v\`\*�\]���5���S\_��������.0 2%j1��B��,�.����f��"�\\•VE7^X�6��%d"�"�\*% 5� ���� U�:��:�â��C� Jt#����I�,� :���\_�J���4��Z|8o\`�쟫3,>n��/��C� �KpM�!�ń��Pu�+���0�ϭ��P.��=V�N�t@P�XGh����^�.���ܖڍP ���s0�$�ə�}��Y,�zO�h�#��qږ���x��6${)M�\`�qE)!�e����#0�u����4�Z�Tg��Ot�%2�/�z�K������z u�H7Uv��r:ʉ+�L܉�(���UI�����\]� ���l�{8�5�^/�&�+S��y��R1S��݁vS�\]�o�h�CШ/��{~�\`Yy��\\�w��Կ�b�\[��|\`x\]�/� <���sh��0G�b�LLY��)��x��1;ww�\\� li�)1@B�>X�~��>7Rj���K�x�q\_f��� Lk�n ���������գ$���7��s�8��)4wt�b��H3WH�@�U����}�d��4�4A��^���G{A��B.rv���Sclf��2���0��q&X \\!�����k�� �V��@; ��z���h��$���К^�%�6F:��k�O\`\\��;� �qC��5�\]}w�|��h$b����we��q(�Ȩ����o��óF\_l\_�s=��k��#q�ܪCW))Z�����GV�(�ꛁ����M�I�3�x���ZAP#�K,\]ũGd7L��.C��Q���e�2g�{��p���m�i��z�\]l����^���m��J�QR�/g{�U���&'�x�뗦�4\\�3�tI#�E�BL���s���2�����,��WW���Z�\`la7J�����f gpP�\*�8 i���n�\_�m/;�Ep U?��h�>}@\]�,ς-$�f�q� �;����\_@\*���nqf恈�&� v��eg�;C;6�xb�~�Dh��~�����\]�+�77�7�:���=����‚�y�r����N��>�Cn\[|ʪ�\\P?MpE� �4\\=�~�Z�vz\*)lU�!��ίg�%�f@���uT��"���<5νF����(�\*�i�q��X i�앫Ҟ-|�����q�A��,H�wMv!�F+�a���� SNl�\_ڋ�y�G�xA COŁU�M.�\`��6ED�D��~� c��8��,;MQҥ���\*� �7ө4�\]�m4g9��+�H (r���Ȩ&R|0�lg��&���߶�#L\]5 �:�|��a�v�<2&Ɣ�\]"�uA�|Wu�UW���њ��#�u&{ ��l�녯Nڳ1 m�׊��q:ԑ�뚎X׫� �,�B\]vb�Z&����i� ����e�䎠Ο��&��H���ޑj��w�g�Upe�9:ٸ�A���F��.�b�0�������� �%>o�D�}\*k���)���Jn��Z�\_\\ 6�},� �V���?dTd����<�~ہ݇>H�7�8C�0SV�PN��8��puGd���\`���p�8���!\`,�he�#H�nCw��>M^d�&���9K�ϻ�u?�E��eu�?H�g�n.��E�g�.��KjU oz|��ѳ '?�����\_� ����\[1c����VϖI���ƒ�/"�3�%�:z\\J�%!<2��Q�a\_��}��^��኎���kЍ3�'�k�,�eQ���\*��s�bh��7�T\[�C��Di\\�'m���!�lh�7��kꞌ���=0:�P�sJ��-�bݍ�9�����7�y���\*���(v�����9%R��&��/�Q���Ȃ\`p�g�$>L^؋���D�w�\\���}/^�G��xh�EQ�$�c�2��c�5\[�T\[+��ǫ��/w��˶,���o��l�!��c�!H�Q���7�� Q���w�^T-?����^?w��?�'�v:d�=�xkl�)9���7u�O�C��"���k����� S6o����������g��,r&F 5��9��#���fY�w�� �Pq����#�r�\\ �S�+L��8(�z�E��q�ɉ�p����я�̀3L�� �s�ԀqkZx�ya��JSd�z��::�(+d�0G�n��!�,i����ُ(Y�<�} 7���sћ�(��^ O��$� ���b���u�o��������P��'䃗����0�?2=5��e�Gm��5� 8>�����\\cyz��R����beLUՉg(j�9�\*�+h���������ݡ�>z�^�L��=��j2�M�~��7���2M|��\*�ӳ(h8lr�8��� � ����R�xP��ɳs�m����}�2�䯶�v� ���?���;��+ �c��T��O|�@�"ƀ�hQ)ifG��-�IIM�Gx�^����� '�=��\[E���'�0��qÚ�Ȝ�M^e��f�\_������n!���� we�����ut\*�Xo,7j��C\\"��+BT ���L�;^�?��z�l�A�r?��?�ٰ$c8��/�'���� �b.,��H �'��U7g¥\]��c�\[��2=9�)Y��k\[S�E�8�ʩ���Ed&���I��SL�q��&��R϶7���� ����\`�f�l��1F^� %��ܼM �f#u9���{쉢WO�����}��|(;s����ܵprr� ��w�� �\];��o%��� �E�$�W��ͦ=��o�Lc��� ���\*�F�|g��%rj����)I�K�����>20�|"j|c��\]�,f.�O��{D�}���4�Qr\_��}��;jG��d9\[��A�L�q���̉Hc/�ؑ S�xǡ\`����‰�O�6e���5 ��1� �R�3����J���cz�^\\"<�I~�sS �tX���+��Ws���b9���dZ5�¹���Q�R�2���\_u�к 5Pܓ���K�o��轷���>S �ٞ�YXF�w�s �Z� ��V~ >���AOV�#4T +{q�n�@9I>JC���sﱖ��Zv��´���J!��.m"� ��\_��Mf���!�q@�k$Ja:��&7����C��vT#��)�^9�������9��ܕ��\]U\*�a�l����n�sW���#��������n��?������)��Hɛ%��K���Z\[��MB���{у�����s�&c�ȚD�#I3��b5��ۧ�6 }y�7��-�x\_�S���>�3��=0Xb~y �icO�F�|�Sآc�~� u�z�������N��b�-/$��M�e�� �tG�3鈲}/���Zw���;�����)���)�qgI�I�T�Ȝ �G ��\[{��6�E�:��Z����X<\`߀t"Oq���S�K\[���(�2�zM��z�Z�O/ĭ\>�ΏMt�W���U���Tf�z�r(C�Kd��Ӄ�����J}�����wR�5us�떼�������� �%t��\\�d;����L�,�ۃyi��������\_�\`��^�^n�\*��~�n�G� {Mc�K��‚�=GCn:�C��|�#��lCu�\[औ�'T�}�z#��|� �a�-n� �\\2�d�Еp?֑���<�9v����l�E���)�XW��� +�=��n~v�Xi��w3ª{��N�o$��wVsgU�4��7��FH�:L��� M�����yt����D�\]<\\�5�#f��r�j�upjG����;6��)J. A��h�T��z�v ,�T9�~Q/MX͒����>�8��Q󛃮��zŅ��|��@�M�;\_���NP�G�e�$A,��>Uo\`?���@swD�w�A�8����L���6�?M���^���٣��P>��x���9;��:0��(���Չ���R��e���CCʐ�����i��I mG>ؖW���?��(���e)Z�eƒ c�����uE؞&r�w�������/>���=�q���$@��S�� 趽�����$0\\:XN)�\[�h�jD��{\_ݸ�B�6Z\[���l���1Lj�@�}�F�� ��/&� \\ۊ0���YK,�1���H��q�Zq�f�雕\\^�#\[�:����6�E\]�i|���U���h�{G��V��O8ȵUDh���|����ִQMw�J�8˃n�p;b��Jc� \_���ӍԋÇ�n��EӐ8.�c9��\_�2���.�6����韝†d�k+ÿy܏�Ş����g.����g�ߋ;2�q��=i�%l� �hjFf��-��2LZ�˕���޺�F�%(�)��C��T\_?� ����MG�d} ���D���ϣr9�y����{�i䯃j�y�1A��g�Y���K����E�Q§�SP��}O8��u�\`h�&�sV4GԞ����}o2n�&c���\\��� u��ᨛ;ʞ~"$KL7�4QyK˥S�r�o\`ﱟ���)�� ���c�pI���y �m�Y�֯wG)^�\]oi�˘'�Oo���׭�N�(�v��q�{����/���B~��i�d\]��TH�+ щI�\[��I���c��9��ly�X�OBoDgl�~S��B���'1�T��ǥ�m��iX:M��\*��$J�8�+Ѽ��"��h�z<��?�e�8�ҡ�M%�:ݵ0X�ɖ?da���@3Sa\`{���,-3��?��$�l/�lP��\*�"NE|����b����}�ڗ i>GJ���ܪ\`�����G#��H�8�\]��v�����7+��I{^�����������nو:SɃP#~V����L.ExJg��RU���ul��V��cY\]���9TH5\`��IR���bH��͞�+�Fz�-�\[CV��=#H� &���yZ �uT�%(83E��0՝K0&�7�VI=3��@~AW�bq\`��Q�#��x\]֑�������b��1)U��B�t5�2!�j'X�"�q->hw�Q�\[)���B�m����%Psߏ��f�g���;������݄ڭ7\*{đӼ)� fw#�ս=)�E�Εz.\_�y�EU���҃�c�)x� �}C~��0��y4ټ“"�l��y�qt�s �O߽2-Rv�s��������/�3�����asH��⫌lG��Ce�tS�d(�+F�J>�=mNR yU�2�갮F���Î� ̿�k({~����PìN�ø����cbjw���߁�܄�S����4��4ו���X!��X���4n(�/��v� ��ћ��dv�D��y�� ^N\*�|��$����������v ��Q$ÓHcUH%����4������r�V��o7���;��eE�B��"L�i�'la�έ��љ<@���-������{�Kq�a�9s�?�:����#\`� v0m�%R�"׌=p��F��l�{��rW(tu�l%�L��^nV�ዞW��!S�JL�o�ba\]r{p\[��X�s +A�x��i�����A\_T2�y�|�K�\_��UV��R���ֆ��� l����+PGR�X��q��ɸf}�ߊ�gF�1S���c\_�zzNI\*|^�r~���e�\[�n�&\\���P�➥��N�//�&K����ו�#�Z��h�3�7��|p��L��d/l��;���K������i�"%���ut��r��\*��o+M�nX�Z��f�\_�,��5#�8����'|^{g���\[�b0eh���5>�S������8���O�٩E,q g�p9,�X?��)�a� ��z� {9�W��\\U\`j���}�g)+!åK��=l���&�=f ӵ)�X�bذPo��Al ���s0ߚ�g�ۏ7�n>���;�0�6S��ՅH�L0����9�Fh��R$�垤��e����#�F\]�Ի#��zL)�/0cn \*��0>o���$����OC��z��/>��������\[��a�N4��,�>b������l7Ԝ���a�}��&��Ȭ1@r�u���!�d&(� X�<\]����\*�r���Z�Р�8��c�v%=^b�\[H�V���E����\` ��L��f&F��jȬ������� \_��Zn� ��x\_�z��翥Rϑ�m=�v�wۇ��Y���F�o{���sm��:�%��/��ąn3b�J�t��0e���8�틂���J�z"�VӀ+�W4Ka��\`Ki�=�:�� ‰\*��؈�!�j�w͂����z�\`|v�\`�|a�m��{x�\[��s�t��˰�$��v����<�U�uw� 3pnl��ۆ��&�e3�p�Bu��� ��B����:��uSӓAV��l�Yr .(C�c�\\�x�8$��:�\`5r������>�����@ iPe;75NT�(� V�C�M5A\`���o�\\G\]g��΋7%�C�3l.S(\`���(8d���UW�JrQ����V�Jw����dŭ�􊤕��hЮ�\]5��u�\_���wt��� � �/)ټm; ��- =��߿(dcQ�ʖ����\[FhϽh�ꉫ��H!e̯�S�)Ne�N+�z?��F;�1�.s���1��C�84sEF����R���q�3���fކ)����T�suD��-��ߡ�q����6llK�K�W��F2\`�-)�\\�꒒$��2�Dz��p���S��F��a�f>�Ty'�<��W(�Ŗ2�.\*������bv��L3�����a\\�˧x�p� ���WAjG���Z�w�l���D���Y���yVry������\_�Kj��dM�o#�}��\]ɫ&������TK�����eN\*5��K�V��li�Qi�MA��C&��@�␢riy��g���j�{�#O� �G\\�^�=P�t�V�˒R�w\*vBd���\]�#x▟B�(\`�ؤ���d�|��^O�df�\]V��,���r��\*Z��� �h ���\[����\]�) ��\[��q�/�}Y2��l��߿a�����Ne��bS ��� ���x��JX��kK+Ť�Qq\[�u�Lo8 HI����� ���{BK��YŇ���G���#̌u$ n@��9��f��myYRqwQsLY-�\[�Y�Q����0��<��Y� �U��-�5vʲP�o02,�0m�<�ïR�f��{g \[۩��IA=j�߼N�,�8�eʹ�v��x���bo�5͵k�B�Z��x5�� R:<� �)� ���T:X�l�a٪:��Wj�wI>U���I�f��\[��xw��c�tDc���3���qF��7ܗ����h��A�4��UĜt� ��íC�Ib^�������:c��A��A��(� ?DhTr�ϟ}��1ܕGf(d����|���mʌ;W����4��\`�u���Wg;�4��J ��s�G.��gT��=�H\_q6�"�o7��� ���F�R�� ��x����5j ��}����H�I��G�4��ʿ-ޅO\*k�ғh��!�A����i"\[��Ŗ��C�t�%5o�v���\`�5 e���^N�nh�.�fe�ALz��-�?�6�s�ev���A�31����A!k�@��/�B�I���6hʏ��X%�GΧ���%B�� ��ڗ�{Vʮ��xjM�1�J���4K���(̑���L�����.���9~\\��d�,O�Y���k\` �Pk����ElY �Ѱ/4 ��i�\`��W�f9H0�䒃�jf'�' �baoᛓ?�#�r�����qq?����lU�)� �뗄\]��-h�3D�p%�KR��d���x���!���<./�K �p��0������5��l�/���Yb��}�S��?'N������e�����ש���b�bi�b��s�| y�F�HC��=�9����\\{Fd3�@#��t�}���I��� ��O��\[x���U%������ ;4�p"����8c�X!��Dc��k�\_���r�\[up&����+���c,PJ |�� �U��R~ث{�\`�:u�q�������!�5����S���gx+�/+S��@ 9M"�E(�l�dP����Âx���vk��V�X"�@�M1�������o�M���oRH1�NB�\\h�a�V��{dq�'���Г� �L�vǝoC#�=?��挿(�cU�b��-73�}���m�K#"��Z+���2>e��z@�����f���;��=��7��V�� \_��|ͻphZ��\[2u %6N�8������{��b ���D��;j� �э ���R�|O�cY��y~x\*ޮ-o�sA�\[)����m#.��n�j�|^fF��5��\*�U�w������\_.\*��lW�&p8 �@�i�O���+�"� �Kf�0�:�{}Ԯ��1���M7V�m��z#�@W9H���JDЖ�i�{�"K{��,��j@@���\-Q����t��h0��P�ګ�+� 6�%� ��~�~�K�֗�E�/WL�XM�7 C�S�\[=w%��.���D�s��۝��������\*d�ݺ��Ŏ���xB�15觬�Xb�x���Ӛ�aI\`����쐛��s� .̹,yi��^\[�@���S���M��b�e��dp\\��i�$�䏖��F����)S$��JmK�+��ϳ��\*EG����L3o��Ӣ��B���u��H�@P�I��E�3KIB! �0 9jZ��>dz��^<���w�����a�������d�ϑ k-���m�hm�,NRe\_����+;�NT�l����v����t��h|��^�����g��l�~�;�Ǯ�b��\\�Р���Z��(�\_�zAƵ fKs�nGt��!�P� �{��6:$qhX��o����zB�� {0~�|-5͕A� �����sT�s� �!FrkD���!���aj�m��/�\]�x������vx����(,�u�u��#�fo��;�C���?�/I��%f���XEr/����-+"J6�6-�t�D�15eT�E��X���"�٤ ���mg+���?�q�U����x��} JP&# hN��~�qZ���H0p��C��\`SE5\*�=�?��l��l���@u�Vl\]�:¹7�p���U�|�-��������,FME�����j��Xά��G�7�|c+3x��m�G�T�?D��-�#�b}�F��l|��y�\_8�i,f@�(��:�B�?+��s�q�F�L��ࠆ4N5�kv��E���.� A -�\]���I�$�hm����,���W��j�#u2��3r�����\[Ut��gq������M�;c�s���f�40.JR�E���D����� 2�eM,uQxMh���Ym||8��{�\_9��6q��Z��%���3\_��,x�h|:"oWO{�ž �EO".�=������X���r=^qEBh'�Wt�Z��^�qIZA�qyb� ����k�~m܊�}�9��l��w����� xx� �sX�J�X �R6\\�@׍a�ԯb�t�q���D����X����,� ���xK�%�E�,=����,\_Uɏ��D b�~���B���G\*����Gу1����F� Ҍ@�B�S��.�+��|����ڙ4�Ǵ�"�|�+���˫� m�T��#r�҃��n{� �jg�.����� ��I���������F~d 7�M��k�N��"���y� �oB���җ�jϢG��T�T^��>� J��$8��$�:��\]| t��l� �y���p��eһbA��-�A�1Omy���\[i7J�W��-C��f)"��4ky�'b'��x��y�����ϸ��P�\]�Ig)q�=�=2�֎/��+��g�\[Yf:W�Oy�ئNm��H"/��թ�p�v��ۀ�DL���0�,�;f�o����s>:l�����pH�0�xb�5�Βs�d��� �ܼ���Zbv� �ʐ��2�����$WŸ�W�B�M�)�#� ��b���PCH��Œ���y��d9O�!�p4 ��/�ZI�\*����z\`�.&,����\]�-��\\�T���I�1O�2�zr\\��$TB���+����e��Z ���#.��3Se�6�!^�J�9"�r3nl�?�6���$ ow��ѽd��vR� �$�o.�/�\[�:lh�"���=��{�폜&���^�����x#eT�G�QA������P���'�K��Hu� �ń\* ��Z U�Æ�W"������k�K�ئ�������� ��\[4<#�P�a�ǖ\`X#��|1� ,��q(��HT�$���YʁF��J��Fyd21= \*��� {;��Ji��б����\_ϖ8ֆ.֜0x�qj �h�-BT�67''w����� ����V��ma���Ok4��>N��$p��<ڥ���V�d�L�-�'��<��LjTɯ���m����e��$EO�����cO�X��E���gk����C��e���\[� ��o��F�%�v�h�CÑ��!6�| �%m+ \`g\[Ƀ�զ�;߯�҉�J,6m����j�|4�O,��7�Ơ�a1C��D>��\*ѷ���/�V\_#X�������'bG�f�"\` �\_���Mj��ہ��<�(�Z&�ӣs�V%O~YP�v�c�0� ��MCb~{�|����?@�0�^6���'���1�.//�=6���PBKK�7r�����$͊��5 1�&�}���Ni�C���윅&�s,�9���u�N�z@ǥ�j���L� ��NApL~�������vc����E5I��d���bߑi@d��,^�K���&������{N���ͧW�V���MZ��t����K���MI �.�:r�b���q ��Y�8΍��3�)�ie&������$1���n���o���t���u� �Kk �>�ƙ��T�����%2E����ԥS|\`�����p�������2@���P|ZC��˞N먋ؤ3��C7���o��,|��Y�x|�,�������$W��'�Zn0���:����%��\\d���W�"�R��wB8ւ �Z�IƼ����:NGm6���Hkz�L�YG��C���XW�9~�0���x�V\`�O�P��UT6��:Q\_A�1/�O:AI���cH�2��s\\y�,4R9�����7�z/\\�21e��B��S@!�2Z��L~.��c���\`�5��|g8�9������f�gM� )��5�}d9��G9\*r\\Q�� �:�|ا\`�GK���&=�w��Tyk�T�F���Ig����a�<� �bn|n����j���V���-�<(��61#K|P >L��>�J�1���}<4޸���n"����j �B�-�O ��#7 &�w��@����l�#X9v���\`� ��fWr\\˧v�g�E/�30ި����9�a� ٢l ��K�8|���(�(}��E��w5m3��gS�\_z�{P�!��c-h�����{I:c�%�^3���!�&Y�\`�hns���Ȫ����b�)�\]ʘ;�&��'^��ݠn�����e�걡@~#0� O�m̨f};����,���x)B���˕�U��k^ɭz��%C�:h#����pig2>����ϒ���� ��O�M�����#�:�؃��\_��}������+��ŏn���PI����Ԗݖ���6��9��W�#�'�rUo�7��3��~Dф�S��i������1\[���4�ئ5>O'�4fo���o'�b�ux ��(�� A��L�߈y>8�Jy>��W�A�M��e�H�H�;���ȶ�;?N��|�Kd h0�\`S�&�r\\�z8-.� �y�x�JVS��yq%.��drfhN�< >�a��YC?���z8�9c��%�z:m���E�u��m�mb�-�E��^��,�v� �l?���- �RVnĎI �}��R��@�� 1E���mSn�S�e(O�-��ʡRÉG�d\_��+K��Z(�o��W����۷EO�����ҿ>��jD��r��x���m& c��tR8Zb�W�U���Q��.��Ϣ��3jи�;^�'��<{|� ϐ܀�L��u&����9����\`\`.4ATA��;��k$�\]S�M�O��FB���H=���ʵ��\`o�\\�qo,��R����� 7B�G��1���|B��^ �K��s;�G�qꏿ�<�n�Vw�p�3� ��l|��p�pQ}B�qg�:B%;�@����a�V�Cm����•�\_$M���7�Ŋ׹���x�,G�&�0U��b�� ���&��OIN����{E7��x�� ;>�UND��q� � }���6�@"���&5@&�b������,\\�v"�4�.3 �0�ԃC+�������D꽋�e����?�)�= �1v����&B�Y��)����52-�� Nm���tE�W>�i���QO�� ��$XA�tqrQ�X^��L�=0;�fĘa�T�0'������ �PD f��)P\[�����E\`�LΉcW���D�Y+�?� :�s�|�(�d�Dvi�8g{�e�����a��|��5��Qσ73�4�r�\*Z{c�B \[!����L���S���6"�D�P�QgQ�j�p�6�'��hP8��@��J\\��Re��?�����nA����u��k!^ʹv���mp� q.�����m�p�&\[ۭ����� 珊ch"ʈ�.�\*�Զ�cǪ����WD~����Q�L�� ��-�c�v�t� V�4vf��׮�co�i�:��h�i� (}@G��3EX�vw�P�P����9Xۮ���y�d��$X(E6p�>�gT��{�>��P}Q���ė�\_�\_ ڎR���m'y�9o�p&|�ߊ�q� ���)1�����!��\]\*�ǻ�=,\\W�!M d�%���J����˾����<��Bd:!S�� �,�E �pqa���� �����������bb��価<6��L:���L6T�rS�N 1�P��Բ�\*��J,��D���qK��!K$�,��F��8�����"8�̕��jW%�j�|́Ա���\_�EQ�=���!� ��P��9��OAe���\[��8�"�f��L��|RyU�2�K ��B��H�@>�m��m�,�\]t\_�PhxV��@�{����M,�r��m��2��1Z!���.����M7���\\ʁ�3/$����\]E�(P��1�"Oz�i�զ��└9�w5���n?�ɑ'�uC��zh�X�L e��T�/�-t�7��><���h�����Ek6\`6&$W�iXa��˔��^����L�yt���G=��N"�Ԝ� LX�1��;fF0�f��.)��ù�%&~�!����p�L��l@q�^H�Ҽ4����e��b��0���\`��»�i��d\`mU���,8�1�z��h��Y��ZFf�M� )�i��V@0ϧr\_�+�d $:!��.�x&ԉ\]ٽ.+�����k�1tSUU�I9"�S�H����dm#L�N�&k���M@)���� Cei�EȅB�E�|�-,9���c ��΀D���{�����R������� ����/�a� ���7�'� �e��;�&���r��Y�����a�Z���z���B��8{�D���{$�A �Y�� ���ۇ��?)mS���Y��>w��d�; �\`��@�⬪���v�ЕJ���+���ҍ.�Ay�N�Y� c'��c�YI���H>��?�q�� ��|sI �rhƖy�M��Z���Ձ�::��\\wG�/.+��h��� ،����L�I,$m@ᤂ��f˺���ף���3rM' � �%Ү�|��LL�\_�&�ٿ�M�8|����nIPB��V^y�E3�\\���ӎ1��d�1>m 98f�JvF��q48�6�f��ZzOQh���CP4�%l���$@��Y�\*�F"+z\\yA ��\*ِOgE�� @����>\*��E�|�>|1C)���&����:�W�0����޸F ����W8=H/��#L�ʧ%�O�\_�E�&�� Bc�q��6 n�U��0y�⩤Žt�T��I���g�M���)��r\_�S���'-��gWr�!y�����U�Mj���ъ�Cr�:���H�b�9g\\iP�IA�v����Cv̙i��7��\[�r)�Tp���:g�����'� <&2X���L?0u�DO��r� M��($1"���7�1g���V;�sg.���ǁ"h��EE�Cɀ�Q�^�Z i����,+ۿ �� 3� �ӵ���,>\\�\]Y1�Sq}��� в��'X��>&�3�#i��v�6.�o:��.~U \*#�k�|)����?'���5��,�{<��g�>��!WbJ\]�1�)0x�J�,ðM�Cl�)C#���\]��9����2�����yQ��\`0!,D�\\�q5�Z��'��;x����5�odg��wl�##\]�F1M�:M\[+Y\\ԿXs#�F�e��a�����.�VI���n�9��64j)'����~%uu&k ��m���s~,V���R���?+��."� Xd�4\`�c�Ⱥ�AߗFL��d������u���z�Vi�,f�n�������7��Yչ������ĥ�+�,(���x���=���K�OЯ�F�I����z�?�oy<���3=���� Z� �F�4˱Y$���b�����͘�9����\\\]y��\`���i���z���ige�� @����?�5ʢ&���>G�ln�؛tK����}�?D �z �#ȁbm�lQOFx�Z����\\��G 3� �/Z-� {/߆nWtKI;VM}�~�-.;�V@��T�ގw���k 7�mϭ�c���\` ��^�^����!n�\*72 =��u� ��E�s+,��H�=ҖN��� �m�Ur�� m��P�R��K����a�C������L�i�����q �>^�S˺���H�^=\[��&��|A���!�T� �7�U��GI�\`8X$�Zr�=O�)6��ѩi�� �Sr ξч��t���=��$��CfJ�G���J��7�@ �+�m�fϮ���I�vv�ױK�RU������d�Y�|�>��P�C�J��5���):��lQS���Q �/�����2-�E��� f�x2����~�bm:t����x��>���N��g��z�����AZ�=�7�h�\[|�&��R�Ɗ�.١I=��H'�b0�9��gK�ٱp�Z��J�4W�� ���M\`m�b �m�׺��hV��/\`�d���,�pP��J�����$T#U��:y�x����>�m��+����� F��e�U�2�DK�K.���\]1�G@a�j�(�Tsz@��?��m^�G�!I,����U. ���~Q+�\[�:�6��y$�Nj?����L���A�Mݽ��Y���,\`��j��\\՜��=�}���S�O@�Y&�{M@޿�,��K�%�\\��E?}��tXE���(~ �H�MT��%��3��J9 |O�}tS����i��w��c�yx�|RFD�b��5�!��ԑ�LQ> �˙��GN �y��א�s�咳�\[B��bU�'���t��F�����q��e/�V�LjU�iVW��bx����h\[���ϒ/������U��&�@��� 6jWH\\V�O���6��8��b߫����j�/�nz-$�ȝA -� ������g����-�>Y�P��ݻ�� �'\`ʇ�6�Z�UW�����.� .A; ���u�nܹs�{E=5�hr�L?<��@�&e@K�p��R��l�����p �s�}� a}N����&���$,�\`��=���p��e�\[�� MZ'&��#<�֥ͱ���� C�&@qm"+�)jE��y��Ιe!�龚r;\_Z��p�dc���{B��n�sD�� ���µ��,�5U���A���\]�My Su�)�H���+��Σ>����#Q��9��F'��lI�� W�\`j\*��\`G����I�Y\]h΂��eN�����$�l�Ƃ�� ��{߱����^o�n��4�0�l�҈(�����m �kk����ܶW–~�M��\[?��|}Bs�<�mWZC|:�%\*:gw�c�Iy��\]��u�m��yx�B�%��aV�-�F.�n%Ӹ@� 0�6�S������ �F��l����\[�j�����~%��Z0#�"ݸb��h�y�(��I�\`��Nĸ��S#�Rw��y��s�s�}�|�5Y����PƼϨ�6w�.��w���ӏ����9w�ұ�ܶ�1� ���~d����K?���5�\[�.V��Zd��X< L0����\*���i����T�\_'����B!tN�U{���+��6Q�چS�ꊅ�(�@��gYrٗy���^�D�Eer�|ä�����H ��.߶e�̛\*�-��B ?^���� \\�״b�\]�Y�t�Jn"54����:�(���NЬ�DH�%qߝ�'D����F�f)N ��񺙎 ���Ό�I��;#�DD�#?לeԊAsh.�Hî���0oV���,Q�����}?���9��D�8M��5Zd��Ax 深��&&؏�-�DZ-��G<�\*Yr̦�@�W�}�����+.���1���ۻ� �� �3\*��PG6�!��v�����YS@��7Xl����0�� ��f��tГ�P� ��m�P����E���o�jz?�x���^�?��\`�)K��\`a�Ŷ�m�u����˩\[�� �K\_N�yZG�kJVZ��"z���1w%&�F��N��c��Q!����\\��b�/ cZ7�Y��V����M����5|�׉b����8u�yȣ��u��3G�SUm/ \[�e�m �Ps�P�� �(�R�m��/�qS��J�ꥂ�;��e 0� B����;q��5 �:iWqNRl��d�r��ɼj�kN\\���\\��W.��U~+���m��w�>O������թ����y�e�z�h ��Ԟ c\_���K�h^x3�"}�#f�8�y���ae�tZ���DM�����лK,���p�Z���:n ����m�$�7,�-3b4$�n����v �'΃ �ϵt�W���&t���Tn~�"���BVq��}+��@�+۹�.Nz��t-�ˬ�.V?霯\_\]��w�+vl��z���Q�/jA�M�c�w���n���=.h��s�/�����o�=��$���\[\`��K��� ���$(�wt:�s LC�wrd,��<�(����i�����lǃ� �k�H�L��w.ˍ���F����,(�UV�� �N#&t����w��/�z�P�A�·n����KX�2�%�mnt�f�u,4��� �Ά�24�Rm�9$i�W@\]��N�� ��=� ����yweJ\\�Plw��M���i��\\ee�홦�b��+2�'d��BK+�\]R4{ܥ� $\`�8 @vL�Wm�%����Ȫ�����ә\\�3�xf�(��k�.��فryD��0��<��Wf��N�9�٭%��n�Z��d��m=�{�YMjJ��z �(�y˹V$CnȬi�$(K{�{�U����Bc\_^ ����/�Tq�c�y7;4�i�� z�0�ɔ®/ݣ�\`��fW��W����u\*?��x�q:�u��"n�q��\_xc>~�^ST�9#�Q\`�&�O�Q��z3s=����<���f���o�c\]�E���15N�b����@U�$�NE�r��IL�a�d�B�(�mg��g�Eu��wv���$d\`�|6~�b/� |♸B��\]wO���GR���.\`yB�;7|��t�s��ҤN�s�O���y�ϔ.(�G���-�S��\] ��𳸈�5��<�b�E�l::ꐄ�^r��t�!�BoW��eZ\`�,G�G� ����\\7a!L�hijkL:t �FQt��QOCN��Kj�\]��(F��m�(�ɒm\_i,T)��G���߈䇐#�ӹ f�}\\�^)��Oخ��yv�0K�RVo ���p�/ v�f�\\͋>D�KW����ZH$}�� %�vIU���:H�G��Ď��M !\*�Mw���\*�,"��pZ���)V����\`� ny���Q�,��u�ruUOE��zG���$�1v�T�W:e��5�"63\`��)^\[԰��z� g5r��+ �����/�Z�i���0J�'�Rcadӄћ�ѻ�M\\Z����&Z΍�^�c���M\_�K\*��ׇ����?1F����,c���&w������U�ۯb6����i��ɣ���� I�Ш�c C�ۧ�ӈ�� �g�E�j�v�)����b1)(�權e�0�bc�ЪG��� ��8�� ��%�DB���I��V gH!|G\`hY�Q?B��oc��y)3���W19��6�٭�,�Ba�؅�뗟�i� n�+�;�~�q"�UN\\�a�< \_!�1umNlP�{�-+��~�6� ��� ��Fzě�B-��.�Yo�==��+��W��9 ��� v H�n�iSԝ���o6ė���=!�u?��8��df.�8�L�KK�ֈ䝣��qV.�\*ӱM2Q) �6b�r}M�ibG�\*-�ĨŞ����lY\[�B�l��U\]���s�M��\`����κ�+�2�N^88����peai�D���&�ު �LjRk��-z�< ���y��%� ��r�b�S\\8�YAǍ^�0g��-�J�~ɵ%���X\]�Z��4��h>֫%�=�Cԉ�w=N�&��M\`���x�\*|�j�r�8ݾ�N^S�\]D���T׾�h�7�4��9p��8䊲���j�I\*Aq �p�\_���U�J��k�A?Mf���O�����D�e�\`N�\[͓�yhJU�����m\_�ioXϖM�:�=��❿�'0Tlhg���\[�݉\`d�A~<6ͳ�U@�D%�R��y|�=��O�\*��%�T�P��{�PGB� �b���I0Rq��N‚������>F�)�ԧ��Bd�I�^(vD� \]��\_c������ ��9�<%�-v��$\\j���&���=�%B�&XE%/7����৞����\`�\[���X��O��D�CyIL\[���d��G�S��e�'"��;(̣�x��ğW�5)��2�4w�������iK�5ϐ�^k�HUz�����tb|��Ly�G�隸��XZ��}t�lV����޸W��94%+��k˳��E�r�$Alf�\_=�/� r�� X���jnФ��nC��m�#� �P��=l�}U��U��ıl�3�zX�\*�)'Y ��>�E�U~V ���R�gH8҆���/��y�!L!��t!�+i�^����ͯe٠n��޴���b��+ho\\�t�V�V��A�8�z3��f�������W����Շ�}qa"��;��^��Hf�oBJ��8�6-:�� ���o��1�rJ���V&�M�#ծ4�� ���0+uO �����|���yyul�@�mm~z� Ɓ��T�3T�Zk�QQ{���3=��4w��=X�����UZ���Ln>�5g �F��L�)A��Im�{�\]�Q9�3�cZ��ikuٍ2T�'�!S݈�"7�v�o��8�VH �4e/��2�5tBn� �i���ρG芖9>dC',����/C���O7�l�.���c4ť���82}ZGҩ"B�@բ����g�S���xew�\\� ���yN����e���������;�=�6���d�/ ^4 ���1�Q�J���3��� ���������:��˯J�k��E>$E�v��F�a��Ru�\`�05��3�ZA�Q��@��G��.1�tc��{�� �i%�:S':ފ�\]Ųr�\`\\����4{�v%?���F�@�h���Y���k���aŪW������9����~e�� �8M9����U,.����\`L�CǞ����R�ݏ�8:m�� �� �bҠ s߻���e����e:��N�M�U,<^�.S���qR��5\]�fq6m��Hr�&D��}JD��Ag����@�.��� b��3�$� �<"@ ~=�m,g�c���;��U�7\`�L�c��C\`��.1�և����D�/z���p�H�k$��Swl�S}2:s���\*�f�TV���.��xb��tv�T����F,��}S&����:(��\\SE��1v� ��?U� ��f��\[Ѽ���>��q���r\`�����/걿Xi>{U ��R<OQkIG��h\*�eP:y$FT���\[ �\]�h��G�c.0s�s\_�+M�jó�Y�8f��fe��K��\*�.��u�)������!\[@0��P���b8AH3�=N�<823s�N��Oـ�ȽOg��Ȁ�m�5�� }p�(S�4P���^\[�f��їXy����"�ň���8��72.��\[x��2K��- '�$�y�Q�y�ϒ��S�˖�N�9�g����w ,>�z�m�p�N����^�����!ٳAzHP\\��r Y K~��^���4��R zg��K�A��?ZIb��0��|T�� �~mb0޹ɇ B��LW6&��h����RI+\\��f����#�X\*� ���\\wsΧ�VVșQ��D��V�����T�#�S�{Q�\_��m�Y1Ow��J�AF����w�����(���,������Y �B�I^|{5'�x�����5��4�"�Sd�\\X���N�� �ۦu�ʑ�u�/�H�y\[y���Ƴ�6�8�����H֖��@�\_P;w& ;,�x��y���I�o��;�Zs)'�Ri����@��U����\`ٻ�x��\]||��Y��\`c���SW d���,���-\*��Ȫ ���m�@��Y6��2��=����tU$:�F~��肞l��m��b�����\[I~�L�,���FGD�6�$�I��G̤���.���oq����C��fđ�>���������#�k�{\\�.�%j)��{�����A�{�~�S�c�G���n��Q�۳��C�h+FO�2�pF�v)���2���pE�!9�l�}ߝ�h�j��\`�o���p�c��<�YS���Ft6A+�kڸk ����(�М�h\_f�y��������G���AR^ɖ��o��'0��d1�/ ��c��e������\*��P7|��4��2Up�q�F�c�"�1!� �{C澀���=���3�.7 �?����舮{�%Zn� \_�g��|����b���M/;��7|�T�Q˓���jq.�"�ۍOg.�ǹT \\ꁎ�7D��1XX��h�in��5��'���m�α���k�۩���:F���bL��U6d�Ʒ�-�����c1��L���JaH�ʗ������zh�8e3�Vf/s� \*ֺDS��;�i �ܭ�"F�\]�p��::հk�S�D���k̤����2Ӳ>r\[��Z�2���ؕ.ms1K��6��oo8�.\`��NQ����.2N�A��JF)g>���S2�&~9 ?O/�Ձ~z/g���=�2\[�?�8g����U�s'ȡ�0�M����W��t��u��׸W�H��H� �Ȝ&E�%�)d��\]p��=����vO��d���w�ٜ�IÙ���4�� ���2�聦k��D�m7 �+������8"�u���m��tWAd�מ%�$���yA)����G��+c4f?l|H�-?�f;�&^����p �C��?�<66�����U���m#ۃ�J�"�����}Qӽ\`��Z���/r"�\\�m�R��)d� J��呰��HV�DV����\\n+Kſ����w�ۈZ���K�2Y�d�DF��Qc�X���ڿv>����{ML�X�Gcğ���u�F����u)�O�b���?���}���v0��\`\\^ OV���U2s�v����k�LK���q#E��y�\`�{�ڹbT��� �\[RH(p��>�{��g� �t� �,�����v��b�����܅:/��=�h^B;����^�����'�+җ�����ek�rېYF�\`N�39������T�q �W1 �W\]�0{�Z�  ڷ�F{P�)�VK�~�=�f��w���\`��G��k�H�y�bs��5���� ����b�\_b����H �a#��dS@�̝ ���+��>J��}<�%m����h�J\`x-T�jm�~׌���k6��fw�IUI�;^� �l�m��l���p@&��e�M�����@���t�OR�(���h���d�x�������N~MN�0e:!^��n��=��t�����N�;N�ǔ���/؆�̄~��!�;�I�Z�ٺ��Eۺ\_��i4��l��T��e:���ǝa��c z�K��7<��� O��%�R.~9���&�E\`��ߢQ�׉��A�Z�>{��~:H��O+�x<����Y�J|S�sm0�,9N�����@9�7��l�r�fį��S�4�̾��~���s�r���R��MU\[\[�F��d�V"�Ön�53��^�M�K��ɕ���9hb\]��\]�v��d�i�gX�� ����'yYf��+�NM�SG�Y��R7���rn�l�Zy�J�Ń'2���h��L/y��=�H|5����'�HC"…3N�陝 P��A흈�8Q?SAD�QȊx�q?����9P\]I�-ܛ�e��g����\`6~K�vQb���q��k�^����\*��u�i�<�wW�� �%wE���w�v�@l71IM�V�//��L�#���ӌL'٫�l3/�ΨB�h�uz\*ӱk�� ���9�N���;����H,,���N��\_��4{h+�9��Վ;q��gT�n!�#U�\[.G�\]e�) U�߭���:��C6���8)-p߿����𱁛�Z�X\` ���{g;���QW�f �~'��i˲� ���,Wh����dP#��p\\���%e�I�Cq{xB>�/�v�w����/)�,��qD \]6>l\*o��\*$>Z7�e?\\��O~�z���� �������A��A�e�3���LY�bh'�O�W�R$�F�A� � �\`y�3r�p�j�Zd4+���!U10����L��tC���yNe˻KW>cxfpщ�(y�\_S�&CY �7Z͆Cq�� �TKuZ�����fܙٰ)���"��FnK�$2=���p t�~�\_=��� ����#\*B�oQIW{��k\`�3�A�������m�D�qɕ�Ǯ�b��v�\_P�6C{pKM���~�=�$L�ǀLg����j|N�����>�8:w3����W4K�$�d�E� ��Vk"tgy�Vz�1!�d�\* �;�z��9!o�{�o��v^\`��q!�B� \\�\_�� m�MT��\[$�2k�KTv�5�:�Q�o�����F�p)\\�;�!Ml�����-�v��oU�䑦%���rId��������Gh ~n���͸W�\_�bs=v0j.��>� =�mƜxC�IZYF��2� ��ۧuD|b�?�Jܚ���;AG\[=m�wm\]✽�������4��� Q���A�P�e�}��:.���z2�?�hj��۔ ?���~��8L�� j\*G�nZ���Wuy~�y��P\\xa,�\`D:#��vͻ�Fc.\_Ǚj#��:\_�"����Q:�s�8��U�K橗��"6����惻�F��D���æ�Ckk�s�j��>\\��|k�v�Fe��?��9J��~���m3OHS(��v2�&Ԃ�ݠsx�U$m�"E�rv~�B���M:F�I��F��-�x'�(N���gk�}y|h�9�\\�o�,1s�?���\_�8�6�\_"��.?�Y�Fv�۫�ޒ��^}��E&�q' qq�� a���kiO���ފ�s�ƊǛ�Fj� ��� BQ��H���?�>#�Ϸ ���c+��bZ��A�>�K I�d �){�/M�ɐ�\\��@2�E�G�C��\\��h�\`��}���j���Z :o�����=���� ��9U�+��F��$�:��?b�)�֬�)����h�J�X�m�Bl�Nðng/�0��8�0�5�Y���2�0�!��$ e+�q~���\\��}V���x�7gí��e���H{צD.a/K��)�1?=��#�'��\[�Z�����-�������f�m��-��Y�N��X��Ƿ�fg�,|�I���A��L?�0W�r�2\*����p��#�G��T�M���h�������}��(A��"\[6���y�{����mO���ǚ���hvCaVR�\\J��M;��%(.�6�C�\`���J�;b��:�f��3;&F&3Q )LQ=cL����$7y������Z6��1�Kٚ;Dj�����H8����R�᮴�4�Qig}�&����<�F��y}/��+J �����9��;F�>s�kr�� �3��� y��c���:Ѿ���X�c!�++In5�Ll������Գ�c��UW��JOhI��R�T �ҫ��/Ua��ץ,�õO�� ¢�&n����P�3FB��b��/�\*S������\*�q�o�^J��n\`�f��S�� ��3�g�l|��P@�������=��(j�3���\_9s=�1� ������$� �1��� Db�{7��ݞ}yRT���f q�\_sF� Ƨ�\`sѦ>����� +�~�\_>�耗̴'�ڲ%�&9�ۼ�/4�\[ͽE:�AƉ/�ZF���\[�| ����bP�!H89q������:S�n�<;Ρt�\[�x��n�a#\['6�\]M�z@�-Ɯ�6nc������f�?���������� ���s.��TM�Lpco�����}VÔ��f����FT6#�O�Tt��g���5(o�H�e�����j�Ӡ2��w h=����V30�� ǘ��( ��K�<��\*���bg�lj!�/4���X��5���ƒ1�X��tH���Mʌ�,x���u����&�#�oO�bO ����p0/ �rY\]��|h��$6wQڲ���o߸z�tɠ$�Wi,�@�ua7�A��$j�1��VP\*cP�֊�?,�H��Q�A(�re %<l�Ȣ��pg��q4����|�ݯ�#4Sk�����O>n@:V�Gn�>�FoP�<�7�s�J�\\��mil�l~n��{�CUZ�Ͳ\]ȃ �Xs��I��Q��p�fI���:��Kة��&��=cB�x���{��8l�\_D��%�Wdt�s\\Y�^ECk�k�\]�d�0��sx)��y�b"V�(F�����f���\`I�<2!� ��|�ifB���Ŀ�}\[�S�^�=�M�\\��;E?�$5NB~�xkS���VW�����:xM���?���w�?@8mV�J� Y֡�=~�P�WX�EB ��|��9�e���.�C|D DC^���5�S�Ö�,��tY靐|�E���+v�ę��D4�$��}�.��u��5�������ՙ�i"����Pp���b�S9f�d�0��n\*���G �}N�XO3�3�-���r2�\_o����A�y��Mi\*�������tV�t ��eM"\*Q�K8p����Ӝ�z�N�z�����&���r�VR}҃�I�$�x��\*ߴ׻2T�6#A=�o��p��.���\*�4�M�Һ�s#Ua��wk�q P�zw̙U�t�d�����\]9��eLƳΰs"8;#��9���uٰj�p�F�U\*�ZMp��4�3�T��q�ͩ�ϿX/ ��"\*�c�P��&1N����%�rY�a�|A)�� j�v ��;3ǧ���u?�ˏ,����=Gh��gGfp���HX�}%ji-q\\l�y�\*8�h 4m7j�=a����F���n�̨ަ����b#Y:�� zeI�)5AN�eh���#���2򖬃�逊%&e���y}J���rT��k ��w��+щ�\_�����K�O{��I?�����A���������h�a����\]��\`���h�l�y��&2�=�sw%x U��Cl����"ktM�7�8���ѥ���%�5�@ J3 f�\]�~�d������?��R����F�Ln�.�dgmo=�^Gc��� �sWsw���t�^�'�3# ×,�9 ��� bA60�1\[A^X��Iu�\_�k�I�EW��\`\]��7I5�ڮR����z\*b��3���ݶ1g����s�ϴ#g23b��ڰ s��4ۘ�1����C�6 5\*7��p�/�s7�(�\*��B���Lf�0C\*J�n}�i��|�Y�������\_�&\]��� �T��0Ӽ�b�W鷏ci}:J� ����~���;&�6�H<|� 5��kh�EPn�L��\] �S?���V� �A��ᢃ%�0�J�#�Q�7a�-�&D�$��h@�8�\\��|-��gR�N���.K᎚9γe\`Ϥ��������v�:9%����V��z�B�\]bN������\`�I^�D�ZW���t���N��������\*<���c�d�����ƀ��e�6���?��s\[�b��0�\[����P�t�tލ���l0���\_�\]9E�Arv�O�f�t+��P!�c���v����9�捑��-i9�r��JW ��8\`>��@�p�񤺢��5Gݱ�aavs�&��)� �A��Vs\]���LU� 9�ؙ8C���e��G��5��~u�\`��f=j����\\�V���S�-�����B�%FF&���:6��tΞ��Ll��zom� �&(^�7���r3 ��i��w4Bk+�=�ECN���+����r�2J4AJ�&"��1�恕�L� @��%�����^� ��f�N�M�����e�|�R�RP��յ�(A��iy�\`�Xз�T��f�K��O���� ��ikV��Ͼl����l�Q=f\`ٺ�����-��Gq��yH\`�w}�6@r��4�+϶�4m5N��A(VC�� �'<t ;�����;IP<�ǖRIs�T�{K�� �6U�=\]b��A��V�IX�y��<�J�?,Z�AJ��$E����O�f�mfV��)�� ߪ@F��盏����ygFx� ��X��R�knh܆Zh^k�͚$�Q�y2Gs����Lfp\[�n��l �-oY���b�)\[�0�Fjc��)y�fa��\_�˟�H�j�҉��mPݾ�i�!�9���B�W�eB����~�������{�pÛ��Om@ \[�|�¬myr���Q=��5�׉��,V�i�!�8����Ot��� �Q �M8����!�����Ŧ�z��i�a����^(���۠���f\_��\\�������A�s�-&�%�5�Z6� ����>^5�k���V��1ϳ|�&8�D����,Ep,}��k$}�H�S�a^\\���U@\_?�?�j#��A\_��<�u�&��S�wn��¤���%��\_��ms!�F4"��/�8K���e�< �L�g:3h��Y% B ������?\_��\\��d�lN�.�3� �3��N:�~s;GV��;��'Ý����q 4a�,M�5�N��&��i��05�f)�}y�����J�Q�6B�a���g�\\\\��-�(��)�\`���Ŧ&ә��b)Cv�<�Յ�\\S4��\\����a�\*W��\\2y�D��=��~�?�/�j��k>���՘E�O ��K؆à�+�L�ʷ^�a��!�k�+F|��!C։�D���@�f���=��l�ZQ/r�lM@a�\*��{�uQ4�hf:h׼5۴7��Nj4�{�%��,��@)���\`;:L%�vv�f�kN>Y�TC��l�-$���/G-|!Lli8�+�����O�ği�\\p�\_��\[Qu��6�Y\[z�Z.7�� )� &��Q���X%���K�E6�#, \]egޖ���Ӏ/�ϸp�~,𘱦�y \`OOo���u���Q����PE��CJ9W�R�����J�=l�ՠ�q�jh��@P,��O�5�� ޯ�K$���Nģ �,�W����?�O��gue������M&��bߠ\`L���5�n�� V��w$��֒�TB^0uQ��f��-��\`��ٷԊ��đ~o�����>��?�#� ����?�7�g,!J���\[� 6ŕ2^���r� <�8ŭ�3��'cJ��w��k����G�����+<��Jە�?2T����|���� g������rNW�(��'ƭ�vB��m}�Le��I��&ƣ1�&;���lҩ��\]m�� �wc�J���O��z~� �0b�M�#���-n4�u�\[�)�,e9l� $���cn�V q�6����IN�'�heD�!�օ@�<�v�KI�1 ��K8ɐ�.��SZ��AK�J?Vm���@8om�Q�X��\*�x>Ue�$ņ7J�W��ڏE��\*@�L�v����~���B�ަ�ӑ�-(�w�� �0xsQWp4��|zT����\\�u�$� �6��|�&�qe�;GR�N���t���Æ������մ͌~W�ңl����i�u�k�~&�~�mO8�P� �"�Ӽ�߿�nфNx���y-G<ݑn�ԣ\[�i�&���'�)�b�'L�ы�C��w�h�o�Z���B{ɳ����V�p��D܅?�,��6�6rzr\_���m��h'(?B��N��� r|8��ùN�!�:����ǜ̇�s%o5b^s�x�͞-�:f�byt�(9�0ʄՑPj8�N����;{ ��a�{43��&��:kt.jlzg�����v��e �KEH��ހwnΧ<��Xht��z�'R��3i3����e�z��6�ԟ������gWG�x-c��� qF�^�J�v� �:\_�;-/�(��n���Ԡ�v LE�7� ����%�$��G�"���Q��>��4��Um}'X<���:��h7 X���b���B�ߗ���ge&�~�?7"\[쿼"|����\]G��x��K�H�Lj��ɏ�0���#�=��3��� ypw����d�E9�A\*J� bn^�y�ݠk��Us�%Zx������A�uMr��֬��>�A��uf�,s��>ԴB��CX��C��d\`��L��9M\]Ő\`�:P��#�1�dY$��b7��3��J.���9߶& �O�N)X�ikK� cZ\[�\`(��b~橄c�M��OB�\*L�Jn��J�~r��Q�R����Y�#�N��\[\*\[��I��DV��„E��&�������/Р��bB�zDXʘG/g왢�������ݴ��0�G07�+#�Io��'=x�����}���N"�cX���/� �"; �\`��E�@�ژ^u�x��s��8���V�5��u\*g �6bg������d��fW�{E%�fB�\]�Y�����<��R���bS?����Y�d�yu�sS ����=�r�9·�8?��;Kl��3��cm��E�������A�\[�V��\[m¼��z�b�����H�PJP����p����\[�V��>K�P�G����,^�t� )�S���P=o����\\FM�|7؍��ɴ�L�p����ڑ�;'I�  c�@Ox�fLe�������G7�7Z��~��I�=��f�G���#�)Q�I�E���P4�~�h��&�����hH��" u�'2 a�.KQ��z ��˨�����ޢ&��=;����1��0�����ؗ6�8��Q�b团vB�& �K�F���4:�q���O�f�j��P\_� ��ص�\_�3|\_GZw�1�zSц�D̖�d=(��d����i� �OH\\ �L"��n��-��v;�S�\*ݿW^��ξ�'�ǹ��^�\\K0O��ڇ�%�p�P! ���ʸ؄�~�Dڑ���=���I����S\`���,����6�YW�ۏ�g#\*"� ��C�W��1�}Sr��Պ����w��ξP��#��o��N���g.��\[��pj��7c�i""��N %L�� Q��)݆ � ñvR l^�P�T��I&Q����}�ݨ^�R��u\*��,�jآ�Z�IOe����W�!�G��x��� ���863R��:��\[���7��bV-�{���Y�!BX��ws�z���d��C�:����eV��l�c�֞�\_�����ָ^O�8!��W�2���u1$�����x������Q��"D��OR �9~Q��֤K!M�&���%e a� �������۝�Ym� 4#�a��q�>l��R��l�Gw�\`�5v5Q<��) �Ƌ�h�\`D��d;�� Hn �9�\[7�G�E=��>�������GW��$1��Ǘ����҈n���P�W�/�E�)�\*|/����vqg.��/pJ1Ts��\*��\[W��0������A�+���j;G��ub�\*��T�f�~��6�l��"EzN׀l�Z/z�\\��Z|'۾��8�����|��,e�Lj~8WU ���6ޝffOf�a���$aDy^&&Us"j1~W���u����Z��nIn��XV"�:=�YjR�\[d�������:�lx�x�U���l� �; x2l�Q��lF�zi�+|+�HlA��av��6�Lt�l��m9Ɣ+b���r�i��S�7\*�#�؝�xXb���l���޻2�72j���Yq�����͑6�|cC�Fk�N��O��/����ُ��2U�F�e��aV�$V�\\��v�1���C�QHC;��)��#G!��\\n����}X8L��ՂX�ww\`���.ʁO�����nD�̑qJ��o(� nE\_ò)k4z� �Z.K�a6Il���G����l����ju.����Q�E�e���{�� J��W��w���,���w���#�C� �m���7�{ ΄sG������%g��u�iJ���:�ݾ��۶�f���������q3��pA�����������B��'Pf��1�ϵ�T��@�@�����6�?���6 '�ŷ��׸o(�8�\]���t��A%Fmߌ��6~�Wqo�� T���=�����d\]����\\��3dP|蛤�����B�\\���e��=������T+�n$#�����߃�We#Il���W-:{:tXI�h����}�屆\[� 8�b��>T��6�>���4�ϝ���\[����ޣ�.&�$�|��|Z^k"\`����q^^���9֑�F�y\\��2'�3䡳�\*Æ9s� ��tSSv+dAҳ$��?�b��|F�%��^q��E�C��m�i a��z�(p;�0ߏ�c�"�)Ç��|m�@����=b^|��=�i)��C{/G \_�G�CF��\`r� ��4sB��f�h�� ��&J����<�X�(�\[�MPw�yJ�ff�b�4���s��$18H�$j�iA;C�O\`���i��k�VC�pz���s�r���e<�>r'�X~���}M���r�N\]�'�����) ��%�E6����.�yN\\�s��IZ�\]1 #s6�Xdx������'��\]�\\���� D+6�Xos�Ľb?�6�K��u�7��Cr���^�a��c�7���B�x�:>K\]�@p�oڡ\\�j�soO^�Z5� ���1�>/n�)Fe\\��JVt\_%0��ch�σ��١�Qa���4����R\`m|�B�Ȝlgτ���?���>��z-\*�(�ܙ̓�N#A�ݟ�� N!p@�!�R{뵉/� yK����b:Y÷ �%���:O���喻f�5/�To��Wv \\IB�ʂh!/\*�gH�ݩ}���;|��v�k�pF��U�^ �&�l����f�-/����pP&d��^���� k,\\�e?�d����\]|��e�C�&�Z��;��olhkya �.F��{Ϊ����j^;�����tQxl�v��ܾ�x�%sB��gB$��I-���{�C�a;�j�(�.ϒ�m�4��m� ��F(y��h����Z>Q�®�����/�J�����(ߊ0��:.�c\\�����ʓ�����$iuka�J�M�\`|�%��wSr.��Z�i�4GWV^W˨�\_q���B�ٮ�i��ûO�q� �"\[� Pe�jaZY7����(�:K:f��\];Ͷl �i=H��A2�Y{QvD�×� �4\`Mp�ώ�W!�ÎL�?# �!LF���>R������LHBf8�̎KA/O��b��SPh����� ����D��g��uG�w�LYz��ܑ�tj��q�V�<4/�17���ζ\`��S,��3��A#3�\*Um4���ͪ\*���v�q9���k/-��=P���\\�v1:SL��z�&���,��\`�\[�\*{q^2���d���rϑ�q�e;���ᑭ������j����c��M�������,���@qX,Ge���z� �0\*�T��0��m{ ��� L�x���K)EB)��ڹ\`U�{A@!ր ��0��ʠ��rC����B!$� �l�����m �$�(҆�/�Q��0�BX�Ѣ�2�R�k��k��(�c��o%� �?{�nj����{T�v; �\[n��z���<�i��C��q��\`��K$���vv�i����<,z}K�)Y����Rʓ���=O�\\��+\\�v.q��=�j�.!��$��.v\[�T�tK�铢���/����N\]<;\_��.@��I��hS�F�׉{ ~$��{����l�ͦSF�����7�@Ppx�{�\_�a�ALʶ}h9�;�.��H\*qW;zx)Ƈ&i�tvS�\_��yC(�s�T=kwL\*�O|�qs�-�=���R��N�&\*�������4���|wQ~�~ ��/b��r�B8�+'���H�N�0���w eK"ѲA���+��Bh/���:�?�5����k�4�\[����{�l���J ����f%(vwl�c���eRi���V�K�j/�SX#6,O�r� 3(�{'��}8�\[x�a���ҶX�R�J�M��u�5K�ܫu�l ��!bacԈ{�Ư��h ֱ������3�0u��2t�\[@U���J��"-�~7}���5&�0��n���0C�Q1s��^�R@��5^�9#ˁ�Պ���T(U����5�em���K���k�+�kf/�.ӌ���U��N� �� d��!p�lkٖ-��{.��"�a��J�0�Uu���i�T�Zu�W�);���d(�h܀���"��)\]�t��Å ��L�Y�|�Z��� /,x�O��eKw��uV�6���҃�b��T�0 �-�W� 2������=�̸ﬢ������u��xX��"�����3A��w̽-9��خ���7�������''l2)2HX|ʾ��\\���Wi>��n�kɻ�Iш%�^DC}�i;}�n�\[�g|Y����̓�FK��84$݆�B��M\*2���X\] �C�4p/+\\Pm1�簧�j�o�9-0��l�(�4��2��Kә� ��&!.uo =}�W��e���O~XT�xo�k�������+��ݰ�1+��ɩ��V�h\_7�%� �1�)߉� oA0cO6�����D�w��YW9 �4���~+���.���i�����}/|F/���\_Z�w7�þ�"r\_C\\��N1���?ل�JiW��2Z�8�5|+s���~V�\*�m3���5��&�%��X���u��>|��N�!Z�-���c,\]}�m�����������2\*LR�?F>%���x�ؽ���C�)�aAlr.��rk2�� ;���X�S��7+��U����󛾑��<~7 F��#�����x��N����>6�Ǥ9���mS�\*��dV�����ĈV�����w�X�� k'm�<6���ׄ�B7���%��>���O+3W��$�"MD �ŶO� \\Z?��Q�Jl��e�1L�#��ϩ#c��h��>�v��j�,���)���q���\\x��ؗ�I�"2%��\\�+M ���&@ �ހU����TV�����AKG�����%0� Caop'��\`�����-ː��Q��#����\]5R�ϸA-�k� /��"kL{�߾��3�Q�"�@��L��ˤf6 ���Ud��M�C��PY���lTYy��!��,�3��C����k\*b���~��&���v��~�B��O1�s\`�0 �fq�#O�@�3� &��d�l��l����s%/O�P���;ջ�!�潬E�5��#�<��Cl�n��9���BDl�$ӯ���|~�����������s��������s����v�k�b�<��iG�}������ �������hkR�G�Z�J�6�Śe�7j��H�d{��� R���A�&\]�ꗧsX&&"�V�+��L�2�8@��}�W��KGp��\\�5��M��؂���b\_�.�"7|k;T����e���K?o�ퟜ��%�dO^>��g�-P��uL�B��0�bG�$��m���t#��j�s�U� i�Q�c�ԍ�T�B�R �2.�v;+�A?E�@$��X;�� ��gZg�ϽD��� �Mu��y����K~ �v��.�u����� C�L� ��L��&�K:{��������\[��ƀb\_?\`�nl7������ytT��ak\\z�׀��٥��s>ܘ��l���L/ֹ���W^&� F�&r%�a�PlPV�����G�C4�L��g� V��Bͫd��C �,� A;\_�y�����1\\��S��)��t툈�&R���ŀGy�V��h��H�\_�\[h�a؝�5�ê�ꓻ\]�;,��PGl|� ��ػ���98=��k��6�9 qK�=�R\`�~����\*�� ���MN��GH��"K9�#5m;�#B�W�\\R� �,��/�'��ߘ�=N��,���\\\_��0���p���\*�n03�Z���z0����x��+:�pށ2�/�Q�+��K����\]��r���r�c� ���fH�����R��^iu�Y�M�W ��A  �YJ��Cc���.�Y���U��m�Y�Z봝�,ܡU� g����ٖfQ\_ۦ�)���K�n/�$�Sf�j��/� �C rq�\]�X\]�GR�d�R�y錴���V��sf�p��E���̐VZ�pJUC�V,o��!˹�X���Q̞"i\*1���9����F}�d����r.�=~�'�����\*s��� J�v�J2�)5:�� {Js�/Q.��>x����i����6���6����j�"�, y��UM4nvVŤ\_rmL����N�������H�I۷���䦬��i�>��1��+�SK�\`���k�p���Y\_��+}��R��k�s�2�,,����0�J������=��/\]�sdI����M��ľ;���nL�1�d +�~Ѧp�u\*0��X�q �,cw�iS�h���HR��kA��@��=dW��^�t������^Mj3��n鿘�16�R�uD�L:�}tc�⣇Q�Gy����5R8P�&P�6�w�Ik�;�1��FA�h�U��g#��b�k?�q��c���cʕ3#o��L�R�+��62�M�^��U\]W�i�z�'}���������"Pd��e^E�L\[fݙsy0s�Xx����Z�Q���J!,�΀��o}�(����"�I�{D�<=�xp��C�I֍f.P'(@�#X�F\*eiݩ�Q�f�:��7+�D.M\[N~��%�� �od0�,�� IҜ�f���=ΎN u��i|�k��Rb�+� y���h������P�#+���(� Z�a^ ϫ2�Q��b�!�:ۃhVɳ�7�w�R����~=�����w �G7��$h��j�pX�y�n�����RR՚�01b{��N��a���� �ڿpr��ڴ�(7���o�x����L|���?�y+;^� G3.�\[�u�ًS�Ӈj\\H��$�hm\]�@�e۟Ӻ�N)���e%���<2�!oj�{2��:\*��m{s1C��ZSf��p�W^"���v�e��۷Г�s~аGŤ�1/����:=���8Z �oɬޓu���z\*8��C�E9�}H�7&�C��ȵ9��0����㏽/��i�i� 9��\]���W�)8Ft��k�h�\*��e9p��!�S1������z>w�0MY�n�.�ga�b3��ɾ��Bgǡ��J ꈇ�A�����#&u�3�P﫪�߮��\_f�uP���܇Fw)�7wɲ�%<�i���4\]�4:���X��EՅ\\��=������g�5\\�ڊ�'%T�=��A�׌�Ncu�S�c�K ���9���h��v�Gb�@�G)}�t��j|��6�vhe��Ĕy��\\J:�T� g�D<5ѵ�Jy40���@o��N�Zݴ��t����#����|��yT��!>��SA�Cd�f�=�#5Pw��w��c�9q�@�V�������)���E��A-�m��5L�d=���8����N&��R��C˅�v-�4�W��������R�?���+BB1�ۥ��Pܴ���/��P�{y����)� �tN���\_x��|���AՕ(�l� 8�ʦ�O�tH�f��E�H-�R�>Zy�� ��Q�-K����}�s�0^Ϊ���\]r�\`��I5���F�jB���� �R�h��Jr4�ə�Wd���O�H�b�9��� ���+�u;|~���n۶�~��8B$N�,�m�0��'�.��z� O�ˀ�.Qy����ݛ��L0��-�:k��yQ�� KB2Qr�T�ҕFJ��e�EE{��(�Q?:�k��z ߖ\[�Sշ��h��=b��J3�bʛ��0҈��Y�xup-�������\]�2��\*��$�W�6?���=&c��ǩ �B�\_uR��uz��=h3��4Qzk��PN �>G�d�@��\\z�7=�݃Ao���F3��3�y\]­�$Fx�%�I�vj� ����E���ιʑ��e �����w��m�t��('ūQ�U\\��m�ï�g2��E� ����c�l�� �6j6�?�BvG�ǭ L�\_c2�D�f��� ��.�|�T��z\[�U~jϚ��͟� a(���U3�@���?V�ci#\]^�P<$lX ,��-8Y=4a�A�X�.Cq��� �kxQ��9?B��Z/ǐ|�+����W\\���1�������\[J�x�r�t=�J"��\[G��GTݛ�#��w3�}�F9�p�(�\\KG/��a�A�Ӻ�W��0��,>�ou\].�X�k�tFF=�=�@rﱕC\`0ua��:��Y�|㔧���ؠ��RbG�0iI�Nҡ���ƿ�3T+��S!Zl�ٙ��&|�b��r��s\[�е�����P1�9�+suΞ��4C9���и��uPa�K̛�T���R�\_u&�a��4�3k�b��}/�qp��}~%��;N��h4�����)Wu{�uOӺݏ�-��}�I)�4���ON x(6G� cw�\*2����p�S,�����7E�GeO}�v��|�$fO�z��8tYW�R� K :ȸ���-�%L�%�!���d�j�˹ X�GHhL��\\}z�P@a�\[�D�D����+kȢr�M|���U\\ݪ�Լ���0�U���>n��֊Y�\`�%����d�}�T�+z�\*=\_�;Q�ݳ� ��O�)�G;ם�~�^���^��^��Ğnc l�T+��\*�"V����'�ʦ.�" �|��Z���?j�ʌyj=@�� &�1>P^c�{uh��skK��f���xN�ƈ��f�Ьz@Z�av5���i�3�u�Q�Bp�;�T��G��0H7����|��e��C� �����+��S�O�m���\\�\*�=X�1! {�\\\[HT���mZ�$lu<���ʦж��G#�\*�;ʐDM3��(�V��=P�!O�Kl�:��{�.�yႎ&O��9/x��H�S����+�A^ �Fogrk����� ���.ǜ��e1��ɳ��i��M���f\\�.����\`vwE�|�:>�MUax��nW��q�A��ݥE^�㎸���e�I6\]��4j�Ҝ؁�x�r����\\w� �7�ӎ�s��� ͗Kv�5ŝ�j�v�g�h6.ƛU�����vM4�0�@��V���V\]�$l�\[�4|�� �E��\]��x�@��V�we�H�9I��\[�3�\[�Q��p�.1������>A:�m���q�P�����5Cj�3��T����� �� �֐hJ�?��H}Wkԫ������Y�����Jw����7��@�GT��^�V�s+��9 ��ߚ �L6�i��>孽Έ|:W���~m��� ʼn�=!3y�qhֽRRVym(�5�����+(���u������\_-?�\\c?\[iT�����!I� AUx����ES������$d��g�\_G�C�x��M �QP"�v�����t�������.��q�%��|�I�#.�uD9�F{��r/8�f��Lm�j�� Y丯��.�\]��r�4����:�?���|�@?�G��O�� ?h���W�r��j\`1̽���?G���À(tf;~ȡ�t�K7�\]�eY!A7�T��'���+7xy.��I���k\\D���7hU�{����I|�6��&�"K�/N���JJK��=΋�O����H��|��|�x�����Z�B���R��=�kwB��1��vǙb��պ\`�!�C�9NI��H�R^'�R����&v����y�|Q⊙�:^$D��Ii(pV :ζ��a/�5$5+�#9\[0�^Cj�X\*"4��.9^<띕�#�CΚ�bU�^~Uh,�F�т�I��K��;ݺ Jկ &O�\_�c�3���X����.��.��OKQAĘ�3��^����<�sd �+��m�^0 ����Z���{�t�Ċ�'� ,D� ��c6�1wt괥h��vcP'��yoA�+�C��l}2�oH^��&�²�\]�.��-u���6\]����! ��˚��S���z�P� �3@OEf ВL؝����z��) &�Ġ9���ți��7N4�˵�����C�\_� &�g�jh�+ �d3�(�nR9�"?/v#aθ�ka�n9�˞mA� ��v���x����5�JR!i�4����9��X�+Ƅ̬�k?���Y����-�m�q���\*�t�\\ђ~�k� ������Q�|���6�2r���\\��>a˻�pr{&k��%^���̼+��7��:'uI\_l����B�Fa��#nA����^�L�Q�e���Wݳz��ݎ.N\*9�(�÷@�� �4��!�\*�}��u��4�ɳ��P��tP� O��ś�l��E��1J(܆���G�eg�g��\]ۙV�ejz��q�>���N�)K‰L��">�����?��h�1S�}�1�P�\`��\_r���a��Px�h�M���fæQL�I >�mӛ��m/ �0����/j���߄���e1�g������2э�ۆ��%�%����� R�\[���t0�'De��#���q,�������^���m����249�$��cJ�+%���Y>��Ŏu�H��dw�ڜcZ���p�\\��f=�J���̔h��L��+�6� Z�����/�e��b�ZV^�@"S���n(q ��uL A���y���'+��:pFkV��xM��^�H>�Tز����뉓x<(� L���3{#<؂�c��H|x�E�\*dV�2�r �����\\fX� �l���!#�qE�{k<�V1k��\*��N����xN�VsF��<�QFx�\`OKwC���t����|q���^Uci�эK}��C���VW�un\*�n{#��L-ȥ�"r�(o�����w�-�ĥ�LP����l!�}���$.���몤�)ڭ�2�I� ��{R�Y%�ι��)Z�\`��:��\_���'$/�4�s�W\[#c�qR�h�F�? dp,�װ�(��~b�R0�n�w}���q\\)�$%�z�>�\*>�^�!���;J@����d&�<��mHt�S�ʖ��"D��ma�?��P��m�R�s�j��O����}�MV1�c���T\_紾 b�s=yRL�8U�\[3���ő��8UL���=��hv���������5��J��4Ө4N�A�� ^oi(��2%�̆3ZQ�w���Veu�x���\\��\]������Mj.+\*)�/|��כ�9<��!�����|�k@煡�6����N�fK�k�Cy�q�Jh��^K�X��BQ�mԑnMd��|��^�ꕂ5�����L��\]!l׊,�Ό�k�/b�)Ά�|X湄�C�����yh�vY� �� �NQ)PIқu1�P}jގ�� �������6��"�T���}�o'���=~�7�ٻ�o�՗8k��5b��R\[LK�d��q8���Q��I�@)�KQi!���s��q�G'D�\]��ĥK+��P�M:Nz�� �D�2�d�INLwf�-ր�x� �W zc��PPR�P�K� q\]��� �Ǣ�;5^�+��4S� aaMˑ1���b�Z���^�6�S\_���o������xί��&0�\]I��n$������+v��Ӡd:2si�.z{ ��|aKXM��YU��$�?���N���K ���$��u���:W�.�w.g�\_Ę��5T޼Xi��j����S��2�g �o�������ӵ�91���@��\\h��V�m�t�!T�g���U�G@�+����բ~&+i��\\o> �S�-b�UA���il�/�N�1������a@�3C�^z!F lEV-j���aA3��<\]>�Y�KhV,h��� #l�y� v�<1ɋҰ�V\`\*A@�\[��'i\]"����cS�,rܥ��S�9�ebv~��ҞT��"6��+B\[�\\ԟ<��}�=��  �.Oݰ�b�0o��7�B��%�7r�j� z����YKD�i\[��P�ɪZ��\`���‹���ݼN�\]gmac�}�&�#,���Z��6wk�����kxj�����������ڍ�\`F���F?���ꦴ%m�X�m��b�9�|��F�" u���N���0�Pf�����y���y= { ��F&�s��� �����3�U�!���c�E�Ba�. a����t��LMK�tp�w�4�o�ӎڧ�TE��d�u�6�%�+W l~�B�kЃ��a9Z0�o����)�Y�%/"�\\P���Zm�J��Hf��+��UK�J��I�����E��/u ��1��C7:z5�+Q�� ����{�Ö��h~���=�4�{��QHe��Y����.zcA�B�都���7o 9�z̾� ����ݞS�@�n�v�b͖�3�K97�\]D����En�ei�q�Yi�s�¶��m��VM�q��#p�u �5�\_�5��E��K@#zbn’���x��>o=�}�������Gc�ޥ����+zZH�N.��W!�BnoQT��<0��MÍ������}�n,�'�7)�����./�y��y�\\�Ú�p��� 49�>� K@d;A$�� c���A<���R�4F"~E^��=���}���a�f��Y��reգ'X�\_��3T�\[�������On��6�pryHwqn����9~X;�����V�R�O�:?%�F�5٧����T�!ʕz�ƹF�A�4�G �&\\��Ski�D��A#J|Mk@�S>��F�9^g������G��H���ZM&ċl61� ��\[��Q�\]�G�^G������ e>)�>}C}i��E���i���,��c��L��6}̠� \[�Z�C���҇:5j� Rx���~��K��q&TF a���P0k?�Ę6�t�x}R�ib��\`��%\*�\_���\]ꀽjB���{X7���:#3��gP��\*|$f�����!Y��3+�(yd���� p?���085��X,kp�1N��\[�=��u����d=D>��Q0�g|B�p�\*Y����~�XB�粁.A�K�j�� �:(uƤ���l��t�9\]�er��q2�C<�g9��z�}�W���w�����ֆ�7���M��h�@�ex�>q� � �uߌdL�Y��.��o�?\\ԙ�@��� 4�a�Ѡ��) ���,��Z~v��1�r��7i�A�L0���#! qӸ����\_��XWR����!%��8���M������z#�q0��z)\\�F�\* ��Z �י�f�<�h�=��<}��|� ����i�<�z-�U���iԴ ���l��� ��1�7y�2/�i���1�\*�K��#�\\\*n���A���Fo���� ��>��>;����z�G��i�E�I=�l6Վ/0�̭�;�4�Ӭ��\`t�"U�uÌ�Xa|�{y����@?%Nsl���{3rf4�qM�F��筴��7���:%�q,N�.q����\_Δ� �.V}?�{��<�#�>��|'dx��;���^ �m�6��b@��ۧ��l� ��O��;��{ ڜ�5012ȇX��gy�?����$��m�B�yZ}�^j���V\*=  ��y|��7���B���c��\[\[+�v����.�V���B�ZRc�w�hٖOV�?-v���Ʌ����o����H�1ǡ�~>���􁚹�+#�5s�6h�e�9R��:.���)��2��?��cn�qn 8��7�-� MV8:q�D������M�\*5�)��#c�T��Rx�C3�o��$tc��U���sZ\]��/=�G �V���vҽ�O� �@�ڱ���)D萸�N2D\`f������D�G}��^ߴ�3��\\��u�R���:Z��G�u �i�\`���797����l�B�hf�҄,�|֦|iA ��B�v�6���\_�� �\[�\*�̍?3��J��kT�IR�?�v�\\�5^�D��I2�2@��\_�,wy�7\\Z�����9Ꙥ�IFB���IV�.rj}�\`o�^%.;>�qn׉ ��y2Oo" �6z��5����/��D9.Ǔd���R�t:سz\`� �2�n�mQ�= ��l���6�0�������aߘoc�06Y�%�^R�zf���\\R�>ぉ J㉖chx�w�,��C�E�R@;�� ��W'+B���k��� ���)�O��� ��w�A����9�@�ſ�%��y�C\]r�ɲ���CA�u����v�j��)P�6 ���zFj��ܖL���G���^e�")�� &c{��e'z��6�أ¨�b�������|&�/'�\]�b�'�+��6 5.�z\_���{\_�����Z�.'e"Wpg��c ?4� Lɢm²<�� �&�����y!駀Ս�����YR���� �:w�-V��}3Gz½n�u��sj-i^0�Ke�����w݋����T��;9ߤ�������ǂ�a�X����������bi���B�@X��s�k��W����?b���y����Ϣ�{�7�XtA����x�F��%��Z4b�bo���%<|�����7 @uN֔�:�4bb�08;ɪhMU\*���BN���x�ħ�<<=��PyB�r���X�~���,|���nj����!UX��"q��J�+�'v́��>U@��=��#\_"�hTy��4��z���($�:�?��fo���a�ţ��t8� n���他5�\*Pe%/jd�un5�r��� �@�� ���č���|���vۻ ��nA903\]\[�����b�hޕ�.��������@�pk�4����5'(/�.��+ԪL=%�h��,��;U�2dot^8�YWG/{�:�\]�SN4H�z?Ujҳ5A<��' ���\[? Th/� +��/ �yZ3�@!޵� ��pL1>����d�kQV�S"��W\_�r0�\\ J��QNC�iX��a�0������YO�\[.df��(��E��cA=O#\]8��<41�HQ3A \_���-��2�v�\[����n������ҙ�u���9�ԇ>��imq�"��t\\Z�5����#�@r��\`��0��h��Hr�=�'�>�sN�ؕl���Iw\`o\]�r�k��?���|��'�����޿;�/u�oM�0ӈ�/\]n�����X�ˑG}�%�iQ�������d�ڥ�qm��l�mQ��D% =���b��b�l1hb���O�5��\[.\[?S�|��\`�Y�F�r������\*�t��\\�#7�u!7�$~��'� �\*�\_an��A��٪���G��"�>th�DoZ '?lݖ�o~�6f�$����j8/\]^�r�\].�� �����Q'\[Aa��=L�i<4��$Ľ��u�Gm��\\�I{g�N�l6�������"�W�O9��x>��+(N���ƪ��^�n3�N�zH�,�Q�krZ�� �I����)ߧg�ʢ�qM%OQ������G�},���Z$D\*-��@�Dzv�u?�9�O�R��@=���"ް�'yJaL+��1�yv��� ��J)E�0���x�E�iK�}f B����A%� �)�A�\]|T�15Xt$ �-������\`2BeО9��Ę��\*�3,���\]( v��\]��hU\*oW �.��5SKJ�)z���U����vT$Δ���uF�EU�����oG1 �5��if����H��j��u��P���.�"���Vy4m�':��Ft���u�ކ�9%��7ֶ�C,u%���y\[�8E�M����7�W�1k+E^��{^34>p�xu����M\[+�(u;޼�� �EQ���2�稟Vv���>Pk��\[��?�:# ;ڐϣJ��X-�>PN�� :,<�^'�V3�N�N'\[�.5��j� A6I:6����ڷ��-B8&'�"\_M�@��L�V9�﵄Iq�.�:��:�����@a���ؾa���;� ������ �q�x�>�֧8Er0���R �\[���D�\[��� ^(��U�i|�aݼ�=:��uAr1�!b��9oh OS5�#>�,aD:Y�h�މ}�����L����y���f�Ñ��#�y��pjl �2�YL���@'\*ͻް�������#De�W?��?��y�lg��" ��#7�Z��h��Z������ h��\`ԡ����j��3\[��� ������t���\`�SxR�q��.��!����z�"��o;����\[��X��OD|����'���%���\`�(�d�I��˩��=��t����z\[�\`ƺ5Ҿ�Q����'^aϳ�^��/Z��O�s���7��/��'�b���� ��� �@��q��)x� @5� �� �r���z��Ó�L�΍�h�/�2���.&�{�PApto��)GY���a��w�\\-H�8�� ǻc:��x��= ��h��axV��i�x���wG��0Bs� ��x��f+����,�j�ǻb�2<��W�� 0Ge�d�L�~��%���p�;(��1�u�r0i�v�|�\]��">�=�����w�֮;�!���P�AtĜq��'� �.��(�dƋ�X�����K��d��cT�oZ���%�?�lN�k+;!�L8� ㅽkҐ��ǓxV@=� ��Ǡ��Z| "�b�"h��E����}�P����S��w<�a='����H��a୹��#� ��\*�w�j(D4g;A"$��\\ �-��"pM�Z:���h�dD���9:�\]G��󧖥��\_"z嗈�Ȁ�'���"Ƈ����L���� �����R��,���P �A�;�\\�1q �9:��w���&�f�>M��%�B0��%�7�0Lx7�� �v��\`�Y�.WD%�=��s�Sq%�?�eW�bd���>'�(<��%?�O�h>��h� 7���Q�&��)�Ѱ�����8GH�W�&��\\F)YY���,�Xf ��}���(��0��y�n���oO��P� @�3oW�a��Ìqe�a wG�{b�?Cý��y2�e#3|s���U����ܬ����8��\]'p�s&s�����)0~c�����i���ó� ��I���\]�K�+��\[���� ��3�e�A���(U�/xj!���X��.��7v��;k\`��T3�eP�aI��޳�^�x�╫+�l�r�u,�\]���tK�zm�� �G�k��f��0��x7홺flk���v�� /t�w�;� >�o�#U5Ђ� 3hg�Yw��v$9�0X�������6\]I4/ ����Y�Q�ilO.�J �L� :�%��8(� iɵ\*Ep�P�C��(J�����Ex$�g�q��XH�{�ҍT"9K���۬s�s�����t�+>�0\_�³ �Q��tw���;&BGZ��JrBz#�!a�:ێ7Z�}�PC(� �D�O�����9:ᗍ �䷬���&��-�\_�����h�:5��a�TJ Su�v�,�}��uJ�B���x��0ό\\��}"{��N)\] }��F�0�E:X� �Y�8�b� 7����p�f9h�P��;�Y{�c����Y�ƟM� tQ�,���a� ^W-+7��y��)������v�.0����F"U9��J�?��<3H3T@�Ip Iu�� t�P&\\�����>�ͦ��gF�bA�����)��T\[��\[kzٓqʟeR�X������h��!F��x ���6G",��476����qU>��E� ��E�w�t+)�,��9����i��h„s�\_U��)�����n2{�.���QS�H�1�}m܊�vM!Cُ���Ȼ\[\_\\��ۗ��Ӿ��3�G}S&gJ����Ǔ� h<��r�����(����V8�l\*��Xp���O���\_�$�! �Dw����c�ae�<����/� q?��潠L�\*�Zf��G 3�Pv�\`�&�!\*ӧ��)m�m������\[|��n��w��b��\`\_�N���覫"��t�آ>X�AD6\[���@�^$�/�c �\\�\_�~P��QT�q����VH�䭄U��Q\[�֒�����"?к�z��p3X�ɬ5�m��Z��Y\\",ϣ��^�>�GB�p!>�\\\]W\]�5T��~��� .D\*�tM��4��,3S�^=�E��Qޜ�b������\* ��AH����֭���yWWh��#��?��G��s���-b����ρ9���S�J�� |G=�ىe'�qNٙ�U������g\[�6 �I���N4�����~ LnQ6��A�x$����>gYIs�؄�-!��H���m7��n�CG'�0ٹe�U4�F��؎�� #� �����k��4y�6agc�Cf�mM%I�S�LG��p�� r �̎'Y�O���U�MZL��\[;S�rAl�W�3-w- SJ�XU dX�8���P9'4w�R���5�{oPzϤa}-4�����焻��Z;\`QA63��cM!�´�}T�����Q�)��Klʼ�.���MHd�\\��E�l�� �����&�� M2�0����XAm�v�q��}\\��ΐχ��#,PbZ���\[=���U��W��ȗ'�!��Z^�n:��.d�.:��\`�������4\[����-G��W�� ���V�t䣥9�m��HkVҢ\_b�"3�"1���i-v����(t�r��0d��A���,%��� �$�#Rh��்�������i���x�Ӌ�LjL�u� �-j\*cH\`4h�x���x��qr6�~���P�ތ ���ʥ;�~�\`Huܲ1Rd^�;^p\\��V|����\]d��S�����9\[�rn�2�K�������#�1�p8���R��ب�Cx"��p�����T\* �e� �9���@���f:!"א�� P�Ҫ��K.FP:���B(�Q�?B"�i��\\ٰ��w� � 0'zQ�\*h��F�W\`!E�!- ���%��vuH�&���y P�?�K�g!���@��Dl�� p%=�+ f�<%:�z��R�7p ��\_;�x��c!V���wp.�z0g�P����{��|�$k���97)k�4�m�4+F�:�y��X�����\_�oKV0\`9�CkQ�.�7n?oUi �E�&�Ҫ���(��"&�m�$׾�1w�%��j�GMx÷���F��Q{~ J{L����9�@�D5!g��=K�0S�\\c�!�\\����Ή���'�0j�����B)0����\_��#,'\\L�F�ܽ�@{')� \`��Z���I p���c)�l�Q�����R� �a�1����uL�EjrDM�2\`���� �{^e�� �+��¼h�����"���ݵ)િTj�U� ���4��ze�%)PΗ�9c�����leq&��S\`x�0pJ> �K~"J6���fD��p�B ��� =�f�J�>��j��%=���@�\\=�wYs�XdD̳�\\�YN���"j3��p�����^��x���NNT��@��|�1s%�I��D$#$|ũ�Q�OЁWC�Ls���L��ܞo̪Y�1oM(<�ؘ#a���Zr'�R�v+�����Z"��/�4��iI��BZ0�D�զ֌�Uy�@�/�� �9�@��\_�}��H(��>+��� LT $+4��m����\*�aȦ��؊̲����Q�Q��q�:��HWŒ��n�S4��ۆ|p��(�O�2O���3X���|���ו\\Ͳ}l����f����ch���헂J��3����3�Zd���dU�Ҵ�XAg:߸䵧 �5A�,᧨�f�t��{~hmLmd����'(D�v���r7/�������4s�M������<��"��SV����/|c<�s���Tv��U�zD78C���7��ف�P$ qJg��!�#��!�%՛2�-�� <�シ<�C� ��~^�L3JO2 | E��Y��Y�s��'}@��d����(� ��(�y�!H6��K��,Iw�%�T5C��?�€�꼝O�W�\\S%^�U�2g��Z� r5M�bTcB��I<�=�=��)13Lh��XE�w�\*��O��Ʀ��¨�oH��u�R'������>\`l��Y��a��H?@�:�Ұ��-Wo�0�0-�^�m�R���>6wIc�xc�����#\[�7�����N<�1�Lw��V�-n N�c�6���:j�N3\]�=�M�?e��J#R�:L���( �=��w~2�RԞ�6��x�����ݝ}�����8�����\_?�$Q����Ҏ^�(8�֔��+lPQ���3߆TL���������w���i�˜��>�Z���-��!����"�R���^1r�U �g?�G�������� ܡ:�C#\\���D}?�� ��sYD���6���J��\[��� B��S(+2Ke��Ô�L!hw�7\_�F0���U�d7�#�Z�e�.���y 2�HB��\_|z�/j�����P��E+N.!�k�AK?X�~o�>�k6� S��W��;��\]����h��Vz��m�� �����:Np����}�i�������T�Ք�C���m&%��쀳���O̥�f�u�b�5N�l�h��D�c�FÓ�#4c���+.'��b��7�ke�~�)fS���YĜ��"�����&�;Ѵ��Ut;\] ��8����PP�tvX�Z���V�.�,5���߄0�=�q����@������A��'�\]�� �$#��iv^3��Z�;�B�څ�R���g�\]�<��n9�Ps�\]���ՎhN1Bg�l�QߙA���3�rZ�;q|)�uZ=�F�|���RQ���<#h��)|F�^D��b59��T�?�zcAť�q!\[/> ;^U{|);��N��Q�n�֗����+��)4���L4@�/:+d�ٳ�W�%��X�t�/$F���>�Z��籭e�P�:����$� �̜�������iV@ݲ����eeJ ⨬JJ���>������E���h��j �Ӹ������V ^�Qn�ʝ �Y�~w�k�'��|�4�m��=?����O��x�k�E\[g�f ̝�3l�3ʲ�Řs�CJ4�^�^���Q������HE$-۴E\]B�#%$0�f�fH�A7�H� +�rQA�o@2�� �� ������Y�(�Qi����!���n¼ʰ�J0o� �nq����������iUw'� �{���cP�\_�z�k���j�)\*h̢�} ��)��u6���ũq�GR�}4�O�x�)��@~�1銹�o�"hu��>b��E:j1%���x�ƾD����ҝ��Ļ�;m��:?�r|ϥfb B�^����Rj���U}�+�7��1����#�#����+\] ��9�:ӭ�~�#0�br��tƞ\[A�r}~k!�)�Q 7 R��%h�$���k�ex�\_�v�A:� v\_HS�S)ٌR������=T�C}�NLib�s� %�r�\*3�@+��R��5@����cV\_���u��vZ{D?�����눭�!�K���L�ḋ���y���B���m��oċ��l�りZEX=v�t�s+\\gј@˜N' �/N��nt�c�7hv���,I�� 5�+C/�vҶ��#;JO#d𑯔i���\`b�,c���~̝�h�s��X��jT��Z��I{�R6H<�Pq�r�hT/�+�}����� \\-ʄ܇�b7��ج��ݤ�-&��Q^�\]��w��6�xgGgW�� I{��ʕkI�$�Z���9�O�u�K��aʏ��."� t̔+%"M ��4�\[����U򦍣�UT�n��UA }?Y7��T���͌Wd�\*�ӳGg��!(إ���C2�x?����,��o�O�E���DoQZ��k���٩=®�#��@�+zA'|ψ��\*\\�Ձ����D�i��$���/i��(,��M:"T��q��"<�g�8���jl�k�5o��-�H�)�/�d@, r^���\] 3���\[3V�r�>�$Vu�|�����z6��%y2աGp���4P .\\| <�YSI��t���"\]��b���hԏ�ؚ��Z0xK�V���\_&�6K��u�D��<ۈ0P-���L��kN�fla;�4���J)$�D��3'��\\gH�.�y�,h� òYP�w�=J�\*����K��c�DŽ �М�j�{�y�=��(Z���=��썯 �N =��y �ZV@����ߓ�%��;�N�!�L�U#�E�g�\]\]�|Z�JY�7��\*�e��n1錈Q7�w��JcO�7<��+�)� �k��'��l\[>��F-�g\]3�w#���h}������W��� h�/y�,#�gr��9������c�J#���U�g��o ";�oS8E� ��\[;��\*(k@|t{� ��'�\]�.N�/7�?4��8E���U�B�BC�ek� �}M,=��<����.8F;¯�HѲҦNj�ݷo��$�%�ҳ�^�Xh��v���.�uDd���)�1�=�|�\_� 2r���Y{>��(�a� ����G)�m�F~��� � ��4\]0�D!�o^�)d�\`�-jZ�؁LH ��jX��vیS1�E��~Y��%��Q^�����\\��D�\]};��'ԫ�A� �;c}�/�8�\]� 1�� "P�����K�3�r��H�R-�d3Ӑ�dd>�p��?Ò%�1v�˔2��V+d��;b׎�uE����D��D��f^��Yި-����~�T�x���ٰ�៍�SU+���t��@X�����D/lz���rdӥ��)���rT�'@��U?�^�?�R�o�)d�:��CzQ��?;�+߇��y��=�-�����#��!��PNP�K\_���I�a�͝��3�;�6��fҤ���0�Q�Dq�ߘ�?�xWR0Љo "yħ\\A.Аu؈דޠI�b��4SG� ���CVS����\`%(��Ư��/�\[Z$6fm�RQ�.�������.�wc��%'ÇӔ\* d@��7Jt4�'2�#�+ز�C� �K$�\`�Li�Q�A�),��w�L2��t���>��?�G�G��2Df���GrR����Oك��ć.$sI�tW���l9��ʕ?jR����έA�V� �V�wWڬ��H��4P\]���ƙ�^�+� ɢ�3��|��hr��{k���}v�Dnv��{R�� ��Ͽ��鱞&Z�<���{ �~���.?���\`G3��\`� ��˽��6�\_\`�"xd�lyiw�z�U��y8\[���D�P��RN� Ր8$ �g3n�b��@hvYτ�%���6�F���P�v�=�ɽw�:�;k�;n~/�ٕ${��^�M̨^�@��WD�~�4\\�G�Q� w\]+�K�4<�� '��es& ��BgФ��J��H�01.u��ޔr����S}�d�}V�H� �� ��b��v��\\x�"q����V�9XI��z'L8jU�X��l��2砠��f�C�������G��"}j� ���\[�iV�� ���W��/���tlݜЇ9�nh�d�\`����5���ݾ�%�jd�vk�����������c�oT��ˢ���G$��h�\\��#NJ��wbdJ�z�+8Ƭ-��.�5�t�� 8dK��pl����� �� �z�z}eaޒ�刑@Σ\]Q��q�,��j�0>"��9�G�7�t~S�È��h�����5} ��YZ꿼7 �ժ�\_��X�E����I�O.%l�s� �\\5G���9�J�3R�-�d�n\[�p�J����\\�݉�,����\]�O�ܞ�Gn�����J8Z�Z� ����������p��Ha���^ʈ�4����'�>��3��P �Q������wƇ\*Ҝ\[� �rN�DU1�ž�1 ?�FH@�Y�vK���fɠ���j�Ya S���\\��<�(p|�a�$�=s,�B+ �'�3w|�-��l����QGE�r�ᢋ0��X�'���^ʭ��:�Rw�����3I����� �\*������w�\*�7H�k{T���ǻ�n(R(���-I��'�IE��V$�XNv�L���d��c3�~7����֪�L L�u�6I�ޥ�>\`U|�c �����V~Qd�pg\*t��h5a�"��� ۱�%+ �W:;��t�IM�j����,�2s������e�{=������)�����X��L������=����A.Nc�r:5a��s�6�m�r�7�k K����"�lf�G���4�V�4I8�58�\["6�|k��s�%�vK��;(�w��BL�Ӊ8�� �\\ל3l�ǁ�Yhjk�2�B��i�� ��>���'b�h�sK����:���?�$��\\7 ��XV��"%�r�\[ x�����JU�?ǀq�����YS��UW�M��hPL j48��5� ����)�5\*xc0bj~P�-%���9��z�(���AY�~��$H��+�g tc��� J Fޏ\]�}���l�Q=��,��N����Ԙ޴o9a�� A���%(��#�l�V�D\*\\Av����A� �\]�@'u�#��C�=�Q�ں6�G.!x�ӏs����C�;��QFy�3u����0d%���\*�ՈtT����>��\*ڮ�i ��t���M2�.��n��x�U��t:�T"}���P����(\`��R�Oe��h����7��!������z9m#��hN\_�U83�B�w\[#2l}a��zk�Ζ7�\_R���7��B�K�ɕ��+�B���jG�:��3��Y���\_��idW (uܴ��.?Y�:�n4mɯ���U�v)6Tw5���+��4f���S� 6G�!pZpʒ��z- �b�'z�9��@�o�d�ʶ/�������s2�M\[�h��� \*ʨ�7�n�vͿs .���2�&��\\�Ɯ��Q��|}Ǒ����d-�Wn/% ������n�V��n~��U@$)�e7;b\\��0%��N�!� \_���Ѳ�w�H�-V ٜ��Q�i}MnTC�$� ���A����~�\_f �QW��k)�s"�����! �d\_���p�Z\]xz�$�1�Vۘ��\*�\`��ğ��/�cu�w����:�w��HV�\]h�8�-�\*ǤA�7������qF� Vh��%uz,o)u��� �z>�)�UL���"�a ���ɻ�@�� �h��Nz����Ƀ�������%j;J(!%�f�r�Ϭ):�FH�|{�Z��ZmJ�(@L�0M��>j��� 9�9¤Ԙ�?�R����$�+|)�&������\\�ΗW��2S,���{����5Cw~77zT�w��\*�idko"����|��O�\[��3D�ZyJ����^-o�u��3W��>c���W������RK�؁�s!��T���-��ݾ�o㽷�Q\`��,�7;��Ƭ�L�!�;è�7%>zP��z�2�7�$ё��=k �:�ًd�Ƥ�Q�L.�wF�� ��&Nڸb��W����� �L�\*9o�R���o�}���������чJ�OsLT��E�"��ڃ��2�� aw?�zf���Fn4�³�� J�7���!o��o�gf�Wf9o&,Si�>~�Z�o�^�^,�&�oT��I9���)?}�u.�K i�u� /u��>�0t�񖱾�5ʎ�O�<ߡ҄�i� R���0n��7�� t…�q���Y>�s��S8���ho@�ٯ �7 �����Ȧa �P,����A���T�����a� �\[�)�:l�s���t|E����P�7+�����vY���QM�� �B.w�1�� (�g��\\�pU�;y�x�'��M��8���2��yf�&+�� i �E���#�?�\`��tY �3��$M(�֦�:��~Y,�K���+��Z��c9�?@�}tj"��'�l�ĜI�7 Ī���R�e:�si�(���zەr>��3v�+�Xz�e�מQ@,�w�{/�m�� �w��7<6/T��!�hʘڈ p��5�����@!S�h\`f�d�} ���n�0�mYxı�?9�m=�F��L0�Z���!�X�P�IxR�s�̊�A1��n\[c\*+��6���(GOSO�:.-�%\[E�1:3����O����n�:lBv�'�aTu��?������?u�r��j�6W�ȕ�������\\�6���\\a�^�%��qC�0����}"J�ꓧ��s׌Ȣ�&�t~� ��H7<� �9L�N��} �Fy;@?��P�� o��o�,�m�jhcP� 5+��2�Hkǹ��#�PC�S�A+?5���wH&���v�����h>.��O-H�^ ��QH������S����bbC�f������\*�/h����bGn�l��0�59� �2�;6~,y��.�b�unh �#��\[j�2\_|Jj���a�� y���gnNЕʭg5 '�}4��\_�x��%�3UjVM3N(t �Q�������T���X"�"�b�Ө�"��8�G�R�0�1 l �\] 1�8p�y����d%E�㴄�}e\`� {j��/��C0���nas�c��b��9��/:;�v\]an=�)�9�@�Dl�#>�9oqc\`G^b �V����hS÷)� \]��&e&��U�HB��&i �Az�Y�F�4SJH��������G�k��8҅�9>w�HP�" \_�� ۊ��������Bc@Ay�G>�ǛӃ$.�ֱ���K�>��ݦa�� yŐ�B'�SEwEč���w��?�v������Zj�Z� T:�Wj�఼(˾dF���=3�svC�0�\`��) �M)��4�3����¼�\* �72mYA#pU<�~� �=UȵF$~j�-�� \[;'!x�u���e^ Lf�4�{��\`��:��?�{�n�F������߰���^R�%&�#&�1X�o��� ��q�x4��V��׈�3�JƄ�?�Pދ ��8#�<�x��>�c꟨�ꋯ�T��'6�\]ܸ�j;�1�j�ck���� ���MO�l��>,�B�xh����6c��J��^QG���ʙ���Ϯ�:̮��g���������I�lv\]��{W�N�ۮ҅�����+�ʧDU �(v��X��“�Up�k�%�+��wF2���3 0�Ȑ�@ɐB�<���aU��:Q7��? ��;�h^}0���!�b�6\_l,g Ek����կ/M8\`�T���3����<ƽUr��^hH>� ?��A�e(����H?Wo������΋�ͻ=Vj4⽏�b ZB�$Ķ�8�a�ؤ��eA��7��3�a!~A�cX\]�7&% M��i1��9�l�I\]�8#���\*ۄ�B��EA�dM��d������$=�aaG����u���vZX��b��Aהd�:Ya����nw�����<��zF���\`\*�T���>�\*&�r�@N߰0�B�YZu/(�7���:Pc�G�C-���=��l��g.�۝ �(P����� z��w��Ը��c�N�\\V��VU�H1c�wiɩo�i��y5G&�N,���e��tIo#�A����?0��B��%��\_�#ޓ��e����Nqnf��5�ݶ�V��<&@V� �^��N�5�G�-��#���nk� ���}���\[N�:9��\]� �y�'�W����-x� k �>�A\\��$�O^/ &�$�\\�G�YJ�{-\]򃷦���ߓ���G+� ��N��Ӑ��s8b Z�Ĵ�YQ)��qnP�:P-R?X���\]쇯�Y�����H��T͘��U����#�y�/>R�+U}z\`\_i�b�¬���s�L�<T�;�%Wz����T�R�d3�+-��co�֠�\[3���Ϲ0n3�C�p�e,R��5n� �f!����� �5ʅ�'w,D{�c4�$�W�^��Φ �=��r7���,�1�w�K�ͫ��;-����h� >�T�ܦ� ����+�&v5�#��e��W�^���8K�(�8��H���r�7NR��(6�)ˆ>�$֏��K��\[ݞ��J�\]�Jkv":��b�s?{w� �u��b��\*C��GN�GWx��G�2���֜�0M�)�j�j�� �w�� =�|��l�19����193����bJz���hz����WV���?��8o�e��u��r��� �����#�(�Ƒk��AY�9Vto\]q霅��sT� (\`-~��H���R��.��/��"\_\*�2xV�Q��y�u�=F69FYE�H�W�k4��ò�&$(g%�ۭ�UE�k�X8A �p�K>/��D\[0�~��##\_��i������k$�2�Q� 3��ju��5@�,#�qu��>wx��l�/�&@��9��\_�vK�z�J�9��8Z��'�NS;(����I �����c�₫�e�����+ �c�8h�D���R�Н��"��nU�'1��Cu=C� ��v�n��e\`sA�����I�k���hm���6��ߙ�bϜ9&Ke�������qC��t��H�1�Y��Z��ꃒ��qQO�k:�d����͝�\_�5TRa���=��M%�sD\]��� �����EԞ�qܟێl�����/>��t�?�yEA�����6:�w�E��Z�s���B,\_q�˭\`�#�8���fw�����\\3��^r(��Ոxx��ܻ֓�A�L���@���d�!˰WX����u�(s���n��S��!�rT�#z��F c���Ket#���\*9'�CJ�>��1��"LT���|�\]Ö^Z��ۘ\_#� )�\*�%�x��S0�tyҿ�H��ld�q�D0��\`Zw�0��j���:Ru}�jaf�'�I�6���T�-�&צ�^��m��ċY��MZ�F��hc�W�{@�W�ңy��e�j�"����"�VÌO���f��;��q\*�)�ڙ�5��t�͘�A+�ٞR}��w�V#Iۄ�0/��m����\*�~Pn ����W�RM�p¹S�l���� \`ƀ��6���J�sF���Z/�B�q�W#�n͋>\_�i@o��1ӥu ����6��h�Ͼ�Y&���SJ��!�o<\_X<�FtU��M��V5q�(b�˜���nLZr5z�-8Jh��I �a��| �3����p�yF�:Ʒ��lե����-�:n����\`9�.\`�b=-f�4��TVv��(}�����}��;�\]�Z�u��l�R.��F|�8I�@�m���A�B��f�z�Bx6��-��^�9�k= n��"���{��V�wU�� �Z)P>މ��{}�A��X5����{�՘y6����ժ���D ����C���\]�4~O���s��j2��j��M�XܮL�S����5ݦ��5��\_�s\*a���qj��O9E��V�1��U�0) ��o��%�\_+��. ���M�U�� Yz�He�R�6��\[҄�\]���P�� �M\[�>�.qK2 ����HhUĦ�X���͵���� Yg'�r�࢙u�>4�v\*�F�la �5-J4Z̯s�9�JK�b�|E(+���r�\[�� ���G2����Zd��E���ϒ�}tv�򓓜�F|j�}u�, 7 ��\[���k�B��AKR}�ҽ� i�חkm.b�0@��ӆ��T�"�)�cAm��f�������s�~������-23����Jw�'+�:A�r�t�j��v�0����}���F�l�fDjO���Ex�hV��uQ��L�M�'4�˫JH{�L��'�D�E�!R�me�An0� s�4��>Dt���U���;��,��|��Ok�s��2;���re�d�G�}Z2˙P�B\]ڮ�x6i�H�U�ܹ<�Ǔ�Π?ޚ���R��>���6�����a�"Ͻ3����v��7�ję�An��Vyo����KQ���yh��D#Y����XG0ꡩ�G\]�\\r����wkJK�1���x=��m:�S��m:3��L���҄P�Z�>�0�K��fR�s�;; �RtF���xh���X��S'�W3�8ܝW�B��r��i�0�EIOs$�CрYP�i�n� �ꙶ���hF�(�0g��c+~����e,< �gg$���&��r��Q\[ꋄ9����0�39#�y�#�0�\[!9��~���.��\_.�5���y��!��q\[�ԺU��f �OqC�3 2��@'Y�����@�(;���%�懙I��D�R��r#+o��ɢ�����x3g:t!�Ήz79�큃P���%��g��Q +�o��ć�/:�)�ߊ.\*��d}p-��ًL8�Q���E0�\]�K��Rي��gbW^�V��leҥ��\[ �5�޲K�S�gi�뼡w���Y\[�Н��{����B���q�̒e�D;�PM�5��a�o�n��#Y����۲�$�i��|������ ���qP��SR���F\`���Ś�KO?{va�5\]eW�^ �2ȵ�g6��k=Y^˕"�� $3���^�Lܪ�9������O�����!���t��t�6%R�����q��𳦯�F��'��9ncQ��)�&x�T�8�\*���!0HF������)\\-x$B�\]ǣ�;XB�N�9���En�Nt�U�<�oP /t�������N�9l�> I�D(f����j��-��@QC��P^�B"�FZ���%2�E�\[f�gD��m+(��9L� A��c�Q�h#�����z�w�0x�3�,ډv��nh��\[-�j3G��o��o�,�7��-w�q�ޭ��� ����Ρsj^�����?1����}�\\a;q3��)>�5¥,��sV������ ��\[6fm)q��� ��-�"��-0��%�5�K&����M3�w�����M�3��g��̜����, ��7)�<�(2�H2~�ԧ�l�3�) �$7\\=��}\\"�����%���&��2 �" m�N�Ĺ���(��,��Y}�4�g\`>�\`"�����}���M{2#US��!Q��1�Z冣onX� ;J��v��$�DF� ���%�����\_�{�u����Q����m֟�����RW���5�3� �D�g㜙 pߤM���\*�v ���®y$.�& ��"�\]��Y��7����>� '��g��$zR�0#��d�&&����̛��FCu�w۳��~mA(�\_E���$$Y�;��2OR��)9g.��f���������A\`s2X/���{����9DZڊ� xDM�\]\*^����Y�\`�&^��7���F��Q�g�f:�eHZ<�8�A|��+!p��x� \\��4�.��YU&Zfå��,�=�w9̴a\]8 �����Bp�a���9Yn(Oשk��H>\[�7(��P�צ�eZ�w~��{8�%��~e9±��{�EarD<�޵����-�� -N�w�K\*��D&r��^�ⷕGF�TO��Ј��;9��3����/2X�b+攤܀J\[6�{��:�Y(�L��-���A�Y�G��j�8(�Gϕ���=�l:  .+ה�,|�Ęq�)���� �Ձ�wKD|���BE߰x��k��e��9��sKd�����I� ��\]U΄'D���h dZvh�MX6����{�� ~S7�zQ\`l 2'�=\_o ���K�(b��:.#\_}�f\["�{nz�F��՞@k�ʺ�{a\`|z����Ş���xn��"�E+�ٗY�NK���;\\�2.�ý�m�>��׮�U\_�QA���L$�B2~;�Џ���'��7���bF�s�8' Ա�:�>���􌅳z�Y��P�%�!�\[&v�&IjWh�p�kv��3m�<\[­�^�h��DL�r�E#��I�Ոџl�J9��\]F��i���}������kg�Z��<\\�r�6�b�:��-)o)}r?�������~a8�� �IkAS����5nc�f� !�wa��d�(�=���I����m�8$ç����3YE�s�����e�wO�U,\_P����I����t!K�p����g����6�(��� “Z�;�Y�2�@��HKvp�кM��dɗR�-t;�F0S���3\_����\*�C2�}K�ȡ�X����J��G������%�t����s�}���3��ŎVĕ�H�.jGe�ڐ;:J��䁺B��˒�{f��f�!�V/�P�n��ޫ���-@ ��٪����Ǝ�R�%���ؘ��a�N�6�P���'�j-���C\*<��u$ڟ����n�A\*�3K��h��C��Gg��b0ؑ��˾զy<�?�,�8��a����s���Y���j���%��}�U3��v��|�+Z��\`}>�˵� +zā���H�����a��|iI�G嚏�(ŝ YkG�;�i��j\*z�G�h�筏���Ԕ�y/���t��\`� ��:Ձ���(���J\*�kV&|�~�h�\*"X���G+�����ّN� ���y@��.8��B��9�����r�<A)n�>���j t>j�2r�6� lcQ��\`=�\*Y��kn�{�H�R>��֪��^e$ڐ�5�$.-���I��r,���S�7 ��.�n1�\\.l�(Y"�p�ū{�x5���7\_���2�o>����i��6� K¶+�?�blk�\\�F�}y�Iiҿ�FM rBV��R3�\]P���� �\] ��xڨ5�\\��h�f��ֆ�����ۖsAo\[� �1�g��N45�h�H�mUOt������e� ��o�j�9M��� S ۋ\*3���9�y����.%��k����)�E��m��&u���J(B��־>��^�%}Ϣ=��{M���Q��z������E�D�kS���U�W���� C��И^&�0E2��>�R3P�|f�ǝW |o/\[ʼn5F<ɀ�vzo\`�b�)ȁ⩡�\[|�M\[Ϸ�5!��v뜝�kS�\*y�Ճ'�S���f�V jwE���;�w �Ph��Œ�+��xs��A׻�ՙU�u2��ޅ��b�'�ր�ÑѢ�9�� �b�\`k�KG��^���e� V�͈��3^����������ր�R�F��Wn��a���W��\[�ip(�Tl T-:|\*���6������q���K-�ϗs���\_x,?��X�!\_�#���e\\����l�����n��>�\\-�����/Mg�� �ꢗv�� ��#�66�|�u&7{-��D�$�2�~c��a7��Twbte���v\_4��m�eV/�ۨ�Y��2�Z��8�0�\\m�3��vz�15E^)��3B��ɭ\_��"\[,�,�7b�d��є���9��I�pm$œLY� ���a�D�Y}\_�T�D� ��>��-1~t� ���J�? \_' ��$�/��E� LI��Eu�Rpy�CNw�a�� 'ǻ5=�7�V�'u�> ̛H3���q3m��bˠ$�w��,hKy L:k�����=} �v� ��\_Z�ܤ��\*�i��6i����\]����͑nߓ�1�<� "��34\*i�#�-kH�8����ͽ��oժ+y�^��>�)��u�a��"�.�;u\[4��f�m�/�\]ɇ���E��C�}8j3.ft� B������,.��%��;�ͮzjٸ�E���%Wg�$���\_1Ԭ��؄?B~�������L �c( Z�eVaOE"\]�� $hʨ��/�+\_>6U���k��V\]��������� Hfsh�T�ǀf�����0��D��)�\[��G�O���7���q����\\Y�0��}�66�@ u��D�s��w ��s��n�p��8���e\*.zs�h�4$E+v�3w�n��B߇��6X�M"�.�^�� �����Z���� ��i�<\[Ք�$���$q�lբ����Ϲ|����l�m�@�^d)��UJCgh��������e��h"�o�5��/��Qc.!��ce��߼7/��!%�{�{���������k��m �� v?V1�N}\\�c+��6\]|��%�5�< ���4^���Ƅ��v\[��g^2�#�U΀�8~\_&��%�����0���w�Q���oq�c5{�&4��Dt,��ʎaɩ�G�/B�֞I������M�z�m�qO�ވ7���f����M �!u��Y(� �8�PJ��@� ����Ҝu�W\[���>�\`#����I�������ALJ���ő�ڑ��Ȥ��{3EaG�(���m�y&����}ܓs�=��������Ҍ�m�J�Ʃ�ooP �s�k+��͋��4S\`��\`wI���vWs�LŠ�3Q�Z\[,�f�i:��JL�����7C��\`�/dAh���!�; ����nܽ�����h�Q�v���e!�;�rHvg#��+�7RC9U�K�#Z�#�z���‚��zs�x}0\`��0���\]jQ\]$�n�7��@R�qj�v��GV�X:|q��6.��n�y��������� �I\[!���4�����yV<l��H�z+6�}V�(���B\\��ݛo��~FhR,�˲ٲ �2۠���|���F�� Q�2�.v�^xH\_И��1�5���hK�IoO�p�j��Yɫ�xrd0�E�i�md7���� " �V�@b;Ip��\]��7���p���T�y���B�p+ !Ϊ��!Iw���X�TGÜ"�S\`0=Qɺ����衄u���k�g��~��/I��H������f�=A�@��Q!����~ৢ���5橧o�g����f���μ�v�4�Yu�%�}$jz�h@G� ��l,rOL�D�1���,�ܕ�\[��^���׭�b:�+B�{zkVȂ�/�D�M� ��Ρ5��u�>V4G����>�Q�9~>����I\]nj8��\]�sZ�R�ZY͕�,L�\[G�h���8� A��:�-qv�w����Е4�f�;��\\�'��~Xe>$�2&NJR�<���I������O\`a?�T\*t�-=M3� ��\]�,(o�"#��ړU�f���q��ZC\*-Iζ�>6'����ܕ�M7�/�+�1��-�mvò���f��ʌ�T���M;����2a��\\��>y��dT"��6/�\]6�� 3� � k�8���N�s^P��c4��Ӫ�\`ޫ z��==x&ڏ�2i�f�R��J�?2�dڇk�.d8��&�I�E�:D�^:��U0�zi��'���r���&-� Al��б��#9�y�c��(���Bk1��~"��D���n0�^��U+a/��eD���B���0�ނ��q\_�u73��3F@�6�kWz�v�\*~q�Q���n��{�1���z�|�Q\_�͏dY0T�6�C�E�|(aM��L$�}�n\\ע+ ��T8^�C����(�}9+X��V�6�!ڥ\*�Y$�kےR��2��d��l��+�ίl\*��7��1 �K�(|� q�LNU!���� ��>\]��m��پ<0"D tL\_(t��ޚ\_4pH!\`�\`����~+��1� �l��}�9 ���}��Ò�U��8�=X�.7� ����u/�R �p��q ��"M��?�2��W��O���L܆t\_ ae�%���<�Z羕H�?b� cl0-�0;�>u�:і� ��� ht\[�\];m�3��i��xL��z �M2�;�\[�\[����A�Pqd����CfB{��g5�Ц�� � ��O��v�\`+�xm�-����L�����}�i��B���%458��(�kٺO<��vM,P�օ"R�}1� �i���1�\\�~��0�����w��Z̔xD�s�t��m�Lr�=�C�2e� R��ř0���z�c�;����tx-���𱩽�J-�B�}"v�y 7:�z�յ�Ǵ��ʈ��g���E)��Sg��\`�\`w� ���sc|/��%��9��� � �W��TumB��y�gv{Lz���E >�Hq4��?�@z�\`�.h�p�cE�8����� �QI������� �ӄ�V+w�y�r)�Y��\[x+�Wu�4��?����4�1�X��c̀��˄�Mip��skZf�h�H'�}�r7��F9, D#n�o��n;�@+��ב�������(�>PXG�xp�F��s}uVǟ�L\_����ĺ�m�h��^��5��\*p�����;�����s<�i�IRotC�6�ZR�佩�3����4��\*�9�����X�p�� g\*OK�N�o0z@v6aˌ��S���0�\*D�%h��d2��71��� ��{�v#8����gqy�u���n\_ �~���<2w����M�~�y��mR<��{��V�� �l�UN��lw��S/�. �2�+�1n�����%�f�1�X��GN|� ����d���vѠUy���Xb���Gҧ,ṪQ����ܖE�~a�n\[U�Iz鲅���V��� T&@� B�W�\_��n�|n\_�<;�w�V���e ս�K��D;����'P춦������+tZ:���5��?&�M�H��TI1��;F�;�����;Ε?FU�pU��cFtf� ��w���?��1!JNɉ���Oj ��6-�1&b���cg\_�M��(�/iq-�!�X9�7:#��H��@��{2���ږ�.RVdqV�>|�\_���Lbg�Q$�O�/ߟ �\*�Rq;�\]�����եP�VxZ�O\`�\*�����g���@Ԑ�}>�uKy �z�ʧ�7&�3��+{̮�TFQ��הzk�7���S�L�#POY�L1|'���Q�����I��ZycꤧA^9��$Jo��B�|$�t��j�� -��f^\\c����\`�̢e���6��{ =�u�kݬ;LL���Y���f���B:u& �!��7oZ������ʴ�EB�x�d 9��v$Ѹ �r���m�~۸� ���m�q˫{�r�� �������dH�Q���Z���E\_�TƓ�^���%04� �L�4���(�=��0׋Ē����X���O�L�\\R��G� ��W�N4���3���>���Kx�1�@�-Ԍ+�䒧�9�K�\`Z�W��J�\`5��@���Lu�)ۍ�x�g5\_.݄��ӳ\*���u��S�e~�s�o?D���a��;a,m�p��\`�i�-V���/ =E�9��#��H���K$G,m�i�����#��(�I\`��\*�u��ղ,�xg��� }�M� 18v�f� ��L��t�����6ٟچ0\`IW�S� �h��L�� ��3�l��S��\[��y�x��b0��Sdv���m��,��#'x������o#Aߒ@��d$�;ig�/C�������ڣ.��$Ol!�8�����k�%%�r ա".�R�����7�x8p�a ��9J����0f�=W}�$�4�\_�KZJ.�����ԼE���n�kl�^\*�k��ϓ��K���O��8�w>�s˲�3�8����Kf� �q�I �á�<���@'.�ʭv\[7PԴ ��hC}$�8h�1 p'�?l�ґU���i����'�ԡ5�<�۫"rA���)��p�+SʟRB�'YI�w�־M��CJ���6s��8�A����\`�4���'Ll;�>|�v�=�= ���#t�����.�q�H�0��?�6���\[�\`�\[�<@���x������Kߋ̘%�C�W����;?CZ��pK��v�p��ͼ����Ż�:�Uo18G �\[K�~�e��\*�g\*�Q�sj�S���ٯ7m�Z\[�{j�o�<�Un��|$~���vEz w�� � t'��׿hZ+(ﯭ\*a�4�Nޑ�dR�±=��Q1��Rϩ�SdSun�NQZ��Z!}~�Dj�f�-vE5-k���V�W��ivX‹���,�5C�\]�����e�?ǰd:��׀�����gNˤ-�>\\�G��+{~�������Z-�~+v� �ɂN�n ��'�wuW���h���\\(�ף���J) ��f3>D��Y.��2�-��.��i�֨ښ�� }́Iy�E�5�\[�\_�=��݆kx�� �����V ���Jc� %#p�96Q�\\=hi%���E4�!�,!���?L�tE�Bg-�r�\*m�N���AO��j��HԂ\*��qt+��=\[^{����!��t��L#x}����B<��1�B\_�K�T���-I�C����� X�'(xǣw���,<���/�n�#�\\��l��.�\[�If"{�4���\*9Sp��C����Ȼzu=�̥�x \_�%4\]�=��4&�fQ�?�O�m�T�5� k�IS����{��,���Y?Gq�\`%�@��T�cE�'&�O�w\\�ս��4)�b�um�G�pr��M���o��كOd�1�;�jp� y�H�'�K��H/D�ÈU�94��Gq!�w��v6�%��XF˃?e���6Μ�6Ud?�pH���R��/wb�6- 7�P�P�g9��@0V���\]N@.������~J�g$�KÂ����I1��#��\`/֪��>�2�'@�P<Ǯ����ͨ�S�u 2�.���!4��1f�� J5B��<����C =)P�Dq��H�� Nc�z�YᨥM�K�U3����q}i�k�i� �{W�����n�J�Q�Ի���B�dI���w����y|�R <������x?�3FÄ�z���?zKU&� ���CcsE�gL�\*D��Lܹ�6�&���A�8�4��s\[��#$� ��c\\�ܢS��´��\]�b����#�3 ţ��\`OR鞏ǩ�;��a��'>��}��������+j�tc=� ���ӈ�� �l;�m�E�?�GJ��i�ȓ�I}��ȵ)���s=ʈ�\](A�b��K�����\*���z >�3R���� �o��g�8��)7Hݿ���2��O�A���K� �\\v$�9��4��c����;����A�H3��u҇���/N�B/}7pP��6�WR�,�ʨ �O�D�(�4H\["�?�C����N�x\]�TJO�\]����\*ѷ���8 �vG"%K�ض���\_N��YI/����6�t/��(j���3���"+�>+ff�0rdB���|+���\[^\*�HY��� ��92��Q�ґ�'Z����� ��H��$N����:~5\[�{��=���r�����I�8��T"�l�ϑ�9��FQ ʋ�� ,�o���R���6�\_À��J�b���X��\]g� �-�U���\`�%QjB\*��6ia�\*Y��G�̈��I������g\*�'A{�peĽ�J�@����+GO�)3���8)�1���ބ8S~TM�6@﫮�E@�tv~Qڟ�8�w7&Jמ����3�� ���;v�\*l�iI�'�\`�|��WTI�NP�>��1pGӋ�\]��kfz��)˵)?՜��Ãc�۬5\]Y�6�"�,N�/�,��}잂o�V,+|�����y�=\[���w��@���E���F����yJ�q\_�򯧭�^ھ2p�kǀW8bdР��������g��v\[\\(,��d��#��E7Ev�Mǡ3�L�?3��ާZ�����-�Ze��Ɏb^��25���|c�b�Å�{q�䙔�����׆.��i�rϹ��uP5Q�\]�����\\HY�t9��\[���2D�I1,�| ��z\[�6�H��Gn�b�����@�ZH��dC/G�\]R�/�Z�A�G��V@�ښ�T�s|NR�JF��~�C���9���Sk-����B�S֚�F��V��n���r���2�����~�޿������X�i��<� H���/dZը��e���� "��n'%��s<�w\]�DY�Q8�L�ܛ�4\*� i�,��,�ĵ~Хڪ�ص8��"sJ/�/u�Q�&o��Ӭ@ux�b:$��\\�~��?�k2p-i��g\[��P�p����໑81|CϧI���ѨW�\\�َ�U��1��J������>�.���t���!����� W)�������'H������4Jh����ˋ������+8����͖ɕw���'��#�yk�Ɍ����D����a\` ��xNU2��D+����w���o��3�șx$5 �Eqb������0�a�Ÿ��01�<����i��l7 �1�=c� �~�������t�=������^���:� oL�h�:B��ͨ�Д�9��${Z�e>gPr�-CG�ql\*�y���響i�p6��4E�O'i'.Iw.���ϴ^e��6���N����ij8L��7�Ѐ�\];M������@9 k��@���" V�B�W�\`k�B�a�2dR(\*l�âSg�jQ��<����Nԭ��\\�yT��)��Y�QHw�����������I2;EU�y����|��\[06�KB�$7��0Gܮf&��|,/S�o�Ivݶ�az��1�"5��լ9d�F�Ul��X���PfȨ?;��/�.i��<�E"��ߩu��b����=�����)�1���ߨ�X�"! �N;)VJ5O21�������"��b���l�8����"\]w7G9��H��ꐀ3�z�O�V͌����}lB�\_�Ln:=)�~a�A ��{'����ǜ�@s���R�U��6�j ܓ˄ߎ �\_§M��(�V y����QJ�v ��<��G���Z�|\\�?\*zy9v�T��#M��?� U,�9�Z���!i@h!i:���Biđ������D' )s�8zx�ƿ��!C�Ci�d@���Ф�\_=�����<��?���g\*�R�S�W$�8 �?:���j��3��.�������T��H��S��۪��0�CD��r���Zoz � ����fJ���<� �|���,�F��x�uuɂc.�%h #>Z�Q��2��|�1|��q��;\[���6N�����K,�g�ԝ�Xy�0\]�,(��ֱ)l~�� bD��,UN0w�7\_\]x���ձ"��IF�4��g�.�Z�\`�.g���8 �4����<݆y��.��u�t�LY|�����R;rY��Q#bd��o�n�{rS�T�� �xڟ�������WX�p�|����L�j�S2=F/v��5P��Qݷ@B�1k�i���p�ob���3C�'�p�T����k�+��� ���v����W�#Z�vw���MmI9�-�f�NJ� �:N�h�\*|��sQ�Q����Y$Z��6�s~�+/v\]0��^�?�mI3Ci�9�\_�ձ�����V�A�җ5�j�|�'��<��J4��۳�8�$^$�mX��F��3�E��G\_D1B��4�w\\�����\_L�\\�n�Ϳ �)g�o�g�x�a�� ��E�� 2 ��}�rb�7R�D���ij��s�A���2Y����S�������1�PG{�\_+�e�Za���1^��I�\[ڤ\_��TvC�u��Ϫb�A��u�YPdž�<�}S��Ӄ�1���M�Z5�m��mH�k��,��0�!Qv�\\��W\]����,TM:)��nX�f�C�L��+l h���I�)��V��,sn�����h�2Ή��c��ݪ6�NW��w�x��IDy�S��.6޽%��so�\*WZ+�H�J�v���v�\*���Sr (�TH��=��eKP�#������)H���N1���,w��a�N����}в�.te�3�U�c���\\%t^�Ct��-�z+����Fw"D'��&G��%llui��0�)H�.���$��\`gf�j���F�-杘�,�/8�B�˙��X�ZwP��V���"K�/u�O:�A��\*��w��#���#�Ѝ���P%�U�N傷!��y\`�m�{���T���R2�\[�&kA��J�\]����t��z^=RU�r��⇨�m�?��W�����'�J8\_>�޻����+C=;�B� �8� ʳ�\`U��&�vW�D\_\`��&���<�4��K?8t@���"Hպ� �J��qn�+f�2�ɢ�}�0����\[��F����la��.��A�ұ���~����l�@NSL�p��0d�0^�%�h�Q��ץ�(&=C�La�������3�}x��=n��5݆>�F0?�V��\]!O� U3�=� �H�!Y���ࠉ�R4�k��L�o�+W��u�.&iއ ^��D bd�z�o� X �6\]�y����\\�G��b�Cg"�-�R���p��s-$�{67�b��Q~��p���x�)���P�?D�AnX!�|koz��C��h���9��L� C�ݹ�z��:��N�Uo��ZZh��줻��iHմgڧ��w\`�6e�5)�\\���'Uay#�Y���\[��!$�'�¥?\[��O�X�W#�Q/V\[,:b܈�y �>��@��a+�\]�k��k�TBM%:��\`޴�tC�1�R��H,����KG�01�:�ϋWSʰ�L�u��&���pp��&�q���<ܬ�R��Ĺns��� �=J�����|{�"��w� ��\_�u,�($�6��5nX���\_^�����m)�P��Q�0�Z�֖� ��6�JLf��\`κ9E��Q.L�%��-A�׋�d\\�I�s5���亍�D�c�>� hlj+���<)z�d�WV��E5��d�� �\_����"{@ϖ���CM�������m�ֶ� ����j�;�h�}�K � �Q ��\_�s�d@��p2�zybͩ�%�4�Qѝ-�.�w ���f�µ~���2�A������K��I�'�t���;?Uߟ�D���Q���U�b���5�5<��B磔��$�HH�}��!��<|\_ �Z�4\*��\_�\_Fܞ��v���y�\[WrY2;h��?r��2�$Ò\_rV���z>�z4V鯮}��� ��ï؋\*�)���kf�'D��(+l�aլر�IU�O�kF�� �K,������5��\_�t�\[�-ߊ.o�͜�V��F�����8^�1�� KEw �VAZ~x���Y2\[�w~�Bo��c����+�9\\�| ?0��Gh�~�N���΀hX!���K�T�8� ����ޢ D8�uA\*��v��!8��A��0�j,�q��U{��� ���u�����\[A�x��Í�oe���Jc\_f�he�����D���f��(�-���s��f�{�������r� "p�3��mR�n߀�f���Q �\*��5�’I N/���R���l����"��'�x��n��EuX\`��b$��A΃�p�� � \`�@.�: �".!���\*��õ�%:����/�;8��U?��,'�vqK�s"�j.j)��2�pLg'�N��\*��u�W�����ԦM)e��4V�<��\`��J��Xײo53��kn�m�螾���d9��CP� ��r�l���ҙ�#��������Źj���\\ f!��~�Lb�nc�G�â������I�uߘ�JJ��݀yb�/L\[��8R?K�{+��k�r�/�0z��� ��2�\*ydnJ"�>�G\_/����{�jc:}�a���at� �)�Kq�4 >�3���Rc���$�u�TÅ:ai���u��X�c�T%�42,B\_��g��H��flSt�-p,�Mx��T����� K���3e;��ݭf�ݿ\\S�rS�鮝AuŜ;��U��x�j�$���S�V�;.����BZ�95�\[�Y�N��\\z����閫i?�V�L'-pB������, �n�k�T�l��� d�޺֜a\[,tH�J�?g�ڑ�{%�=�۱�\[��\`�V1h����{4�� (���T����8{�Q9�R��p�K�Z�:����\_�r��H�f4����#9YQtM�h�t����\[k�Q~�|Q�ߪ�Z�d.Q�a@���Y>D�����n�{�������1���϶�n9��\\"��+J�1L�/YV���rF�om�r���!�bL��x�l��\[�\_����|���G�Ш���/|�SқK��4/ݔ�1@�- \]Uc�\\уp(���%��v���\*!ۡf�Vpb\[�>������-�6�TVnC��IΠF�1wnY���u(1��y�� ?�����I��ZCU=�9� ���fls ٗ�\`�"Ś� S�A����(� ��(T��v����@&������ |��C2�7�F����ѠM���Y&\\�c��������M0�����b��U� �\*n?A���! zT�S�ϙ�\_���ˎ�F�ͭ�~ufJY�QB�����κY}ف�B\_� � r'�T�9��9,���T�\*p�a���9�ݹA�?%}1����0��f��A���# l2E��i�7���.�,S��l�#�s\*�H�~�鵫�K�����7�h>�?S����Sހ� �Ł�DY� b��ߎ�FB��(-�� }T�t Q�к��~Mн�\[�^�1����APU�aU ��J:�,8��@Pk0���6N�����a�� ���)5U?�ԧqfe��b��0��5��' fD\`� � 7�>�.���L�XiV��~��7�f%�0���C\*;�E� dBA�nk���I$H�"��&��^�$���ʜ��bh )��tF\[惝�f��n��b�枛���$#�č�&�|u��RbȘ�ӣi�,��R�Ǿ��w�\`.\_���)j��w�ח�\\ܞ�=�r�T(�1sV����Z�k6�?���:� ��t�rV3~�{qq>�H/�m�=\[yE�R���{�0�\_�0���.x���UoɑZn& �����!��-��=Ѱ�&�'4?{���S.\[�����E�t�yMSO\\¨�q�=�1�����|�Uݽ�\]�јm� ��K����?Ëla=��!7R�0t ��#m��Ij �H�T\[�O��Lt%Ʉ7"\*���6??�s� ���j�{ �b�h �Wō�{����{i�E� %Vv�?\]� g��Ɔp5v\[�h�dv�a�fU2\*�N�MA�ڜ���ޛBoe��f������x��P�LK��>a���ϭ1ԉ�BAq�!D��� �� qz�q;�\`k$N?�M�r��5�zWY��%� ^j���Y���+}�#{U����R��r�}20ܷm�Ҕ�cA�34��\`9�銤S��Q>e&�\*�/��n���r%�����'YIe�h��~���fE\_@W 3�����0 ^�3���E^�z���O�C@D݈��d�!,���ҙ���!�M�W����wl�?�S���:"�~�C�N\`J�h�����i�Q�釰u !f�{�B�>����)����d߸�?z�����Q��DI~�W@�9QEB��|���L���À�/�S��"��m�� \]�n ��\[�8P�A�}�qW�$o�����W���T<օ�y�,��RC��^��U��\*�u�<\_N�%������wk)E��/R\[/���^g5�����b.�\\���f�-\`YF�����vq�\`�x:�#9��g���ͦ�V�k�����Q�;9���A\]�"�d�\`𠙧�Y/�3���cN��01o|\]\_��@%�Ӄ\]��›�Oi"|�#�F�DY\`\_U��þ�{w�����M�l���d�����������z� OS/���2^���Y�gW�hu�) ��ׄmh$&���(̒�)�q�|�8�N���A�\*8����kL����P�;r/mI�5P�����o����d�� ��1���\* ���A�� Jk�l9\\fqk�&\\�ٍ�F��=>Ȟ��t�}&�(�o��$ 4��1����" I�D����F ��ۈ�NĶ�\_ǽ)��a���\]ћpa�hB���lbtP^�,��|\\3bE��Ǫ� Ħ�a��\[�bQT��?i�Ux��E�N���\*�C�gT�)����V2Է�)�.��J2��3�;��V�%��\]�� �n��j�S���<�������h���=�sDL��I��o2�(� �����\\<��f0�%�i� ef+�E���1���Z����eq� �x�۱ϟH�����a{a��)��L���օha�w������|S�a(n��^��v8ऌ� �"D\_������-�n�d�����˒7��k�Q)��E�2�?y��;���Ry��g��%�yS(��+ m����� ��C���i�1q����Q7-����^}\]��Vl.2�bb����g@ �:-ļ9��G@��4�,dVA�;��!ݫ<d�}�7����Cm�D\]�i�<>0� �v�{B瓚���;(7֠����B5I�L���2w'�t:�g��/Ŕ�!���!#�8k��z h�}?J�\\���1=������pK�Я������,)�VI�4�0 ��QQH�C/�F�4�qS�|�nvy�lXo��G9v�\[5��Gyǭ0���đ�b0b�в��s/��M��B\[��2���, vr���!�Xtp�!�m�@L~�sqa��1�P3i��X�\*d�� 2��Ʃ<#�̺���n E�\*�� �F;��~q��B����|6W�^F"@f?�/��YS .�ئ�<��S�g�Z�%B� ;�f�����+|wJ�?�!=��X�8�Q2� ��@Ԛʑ y�|����0�M�u5\`�Ő���J���\_מXkl�:|�t�"��z���?b" nTVw��aSL����'Z�g�x��� U$36 C�C�GYЃa� �-��U(!H$�\*��\\ݤ�C�E74q/7��qV��ӫ�a��x/ L� �\*k/��/^����$f���k G��|�8c��&��:�&F�f�y��t5S��=3�� AߖX#N J,v�?�}�����D����^�g��q�����DTX�Y e���S�1����c�\_L$��/�ƈ�� k�Ż�08! �V�<�\],�������M(DZ���V�648���&S�FO� ���|��N�\*)�p�ŏI�ϵjc1��n'�1=�����9j�R����t�=���1I�"-}B"P���0!����:I��3\]/����l\`פ�VM�t�YiĬ>��{\_t�ѳx��G��WFO�7ћF#���d�bwcZtF�$�ۊ������ �>v�l��\[����}Nє��L6�.s�:N�'�)�Y&p�� pϸ���h�,3Η�W6�\[�B�+��\[�����%\[5��%��iWd^��kˤC�lH�� &(�:����iP E1�'�� 2�M��i�.��� �Y��&�����\]�\\G���5 Q=��vA� iR-uqoC~P\\���D�Ng�9����\`~�g8��{z&0)I��� �RN��VTM �>��e3~�%N�E��8�%g��������� \_E�d��k��a��$�?�4CUHP���8���wU �P:�{8�y��Z��j0@+\]�\_8��-�DA��WF�P:\\q9��'��Z���%�0^�!�W��� ��a�x!�tA�c�-Qg��9�� �L|�=�I�e��\*���O�|Ki��ؓ�������/>$� �Y�2s,yOzs �x$�<��J��3����E�87� '�u������V��j7��'�pE�4�LW�c:�Ӧ��}���uޔ�i1���mҜȚ���c\_O�PS�6k���sl'������'�g�/�cφ�%4f�j�����x�B��B:�<�ZBxXh�p���<�{�u�-�A���7\`�{��&��{���� ԕ��ꇭVFLX�v�/�� �3����} ��>S�BjM\]N��A�#��� �\*@\*���RWH�����RQ��XM"z�~훃���жf��ƳL�5��K���S���f�-�+3�{3biCw�r"��� 47�Oإ k�}���v��b�;������?1��M|\\ɛ\*�\\��Z��w�\_\*��ˁ�,��'A�q'�N�v�cN|9\[�bA�\`m�d�&��s�ƱXk �|�0j�����"�������I��7�}(��,��"vf�Y��By�-K��|�����.\]�"�N�/v����hVZ�a�{�X��+�$~Za����=��u��4?c�'�d���s�+����irAzd��� &ޝ��"���W���O%z�f�\\,}�۰C��wb�Uԓon� ��\`��9>�w�Ҕ7�� ���#SѠm� �Bx&���l堺w��) �<4A�dFAI��u�k#{2?~��l���{�<�Y�…L>��U�=�چ{�I���1��wl�#�kS�U�d�x6�� ���!@��\`�YM{lg5�;q�� �K���pe�1\`QTM\]�0�����ب1=ߌ�\]�CN\[|��\\K���O)�'A�jv�&����ǰf�1���� "���r s\_�{Z�\\�o'h��!"�v��X�:�N��r�l� Uh焱d/�\`/n\`�a�k ��n�7Fo�\_fSکd0 Ǵ7��X��v�ч�l m\_�����\*4�OEN�|���XC�Ô���/�' �"�D�!X��#��b5���kn��߆l�pO�<"$b�7��Q�1�3�W�-��A�>cG(����C{���$�����V5�d'� �Z�k�������n(\[тk�7�e�n�����{܎��f��9� �r��k�����I�����r8$m���k�A�ϧ��ڡ��a�S��K&�t������(G�uqx��V .����)���kۄnFtK ��\*C���#֊�S5&�e�mf�\_�8���i � �����L�M�L�e�3j⌆َ���-�.��"�1虃�HFbGjO\[���\_&g\_������D%A@��\\�IYy�b̞L�� �@�X����4�9bz�:Dg���DF��3�3�9��5n"9��E�H\\�l �⏕cq �4�Uho^>���$8��\`V���oX\`V&�\*L��#uG5��Ե��\[��@��ށ/x�NF�!c���9/� xO��wN��L,{S�W.��.�PRc��V�C������^�U{�����!Շ/&�x��Z��ib1kj>ɮ��=�����"�a������eg�q�ݜ= �Vc�����RSY/��N3Y��ȨC��ke�d5=� 8x��P(�Q��MayL�K�~��n�\*��<��V���?+�"8˾�G�'�j,��� �M�!�'�ɘ����ɡ�օ�3�a�ʱ��cK�\`��>y<�f�1ّ�|��o?�K�H2Qo,LmXg'� 4O!\[ t��(jy�n!M;���, xM��uݪދ�J�u���K\[HY�����4QǨ�;�,/\[F&����� 6@\]u�u��t˗!؍�F� |�"�H2/�q �� ��5˪�t?L�sȌV��YN���4-\`���.���2�:�N��Mą ��Xq��xők�\]�Xi����\\i���೚ߛtn�-��~~�YH��: ��a�����W �\]6� ����Ee(�Pf>���,�Mqae�Ȓ ���@�#&�͗��X�䵚�7���о��/��P��'��9���,�S�=�|�Ĕ,�p�����,��S�\\�(���rȤ���@>~��sIu�Oe��X�cX�@��Í̃��N70U�G�Q/\*s�@ѱB��J�m/|�31K��/-�wp��E ��sD\\F ��� �@���:�G:�a�d^�d�K<�h��^+ �Nx̜n1�0����eOkC�\[<ԝ���\_BP��cjnK�na>�e\*�~�S;�zxŨ�m�sZٲ�XDn�m�� ϊ���h�=@���V�\*Ay���\\X w����H��Hţ��\_��E�e��bŦQ��=)�/���l�%��i ^�\[��>�M�~�Mǜ�C��Q@E �@��+0n߷$�L�\]�Q���B����ሷ|�b�e��.��>�����w�h@z�/�DK>��e��9Af g\_��큣.f��k�M��s \`���ۭ넊�a��t�����qzrǯρe9���5�������b��y>C��G�h� ����aځ�jF<��PG��P�«�\`\\��"�����$9p2���o&v�����?�kw��ac�}Y-�@�� C�\[� ��cS�L����gP�1J�T�iNb�1��ƹ}V�CE�����L�B��5�b8��n\_�Ȱqj,��)1\*�:��2��r-�f'���\]���;��G����+>V���' �T\[?�ً��c�5���c�lb�\[1^\]k����z)��<�M�k�o�䗄���&P��3\` ��:��}>-�>y(�e�Ksu7�Rv4k-s�o.$�S\]�'��^�\*|�YyB̓d���X��3\[rd�$����O�������zYȇ�f�v��8;����������q�/�Bρt�1�l�Y"�1�}Pl���Ei�� {��"ݰƺ{7�;�vB���+���o������O�76��G���&�'G�b\]1 � mx+d��y�܃�X�\`V�W4�bs�m��!UʋQ���9,�u��M�i�Z�G�߽�ilLM�T��� {�KI%Y��%��m�q���U$��,��� �{�mQ$�+�\]G���\`�6��. B�T\`�{�Iܭv�������K�S^� \_�b\_����P���פڍ��Qk1 F^zY�o(��R� S��}t��n/���QpK �{}\*��\]d��ۏ&����#�7&c�ɴ��|~��=񇛳��^p#Fm��j�2�s �X?}��͎/ �P�!���1{�3o�x\[3�7j�K�6I�iy1�y3��tڿ��� ��i���t�i�37�y��S�Pw����il��������8�T�J��b\\�2k��.�F�݆��Rӓ�6��ʎ�L ��M���Hʙ\_��)�qz>�;��N:J��Ea?�i��o;b:G\`^������� �<�%a�zpS�3a��2$��ڬ�0��"��H�����K�Ѭ�� J�h�̟�� ��o��)�i\]~B�;WNf FP��m�v�����^��> �����$���\`5�e�d�v�0Sa�E<�Ͳ�Wm��,�{�T\[+|l��({t�\_��r�E����s$�>��f<1��w��sx�D�\\j�s�\]�2\]�Ո�� h��\\�\]-W�)��3����p�1�~3��U��s�|���c.N<��^�m�:�(#Щ9�Ҕ�} 9��O���(4U?�!U�'RY{%��oy G"������u�%TN�\\&|2��'TI�0k�W�Xʶ����Fb�O�|mW�sh9͋ê�� �~�n�ø!��^~�ߧJz�=��"�Ğ��7\]A��ܴ�M^J�v AG(V�����Й� S�aǬAD:t�D��-���B"���o r�3j�ػJ�B����B�}�W�!��3��\_!���׫� ;�5�\\ ���i۪:�K�ú��7��DI\*<������r\_ �X�PH��-S.� ~U�'�/�>,�Q'7V���~�)Zf�g����B�i�jY�a�Q�j���\_����s2����hv�V��T�5� �Bѱ!n��&Wf´��%iOӲ5�6m�X�4������ �J�M��V�B@t!7�.�ے�ۺƏ�1&��o�W-�LP�Q����o'�P �.FY��;M�3�������h���\]~J�A8v�t�Ch� ,S�\\������)"ܖ�ʚ�i4�Fw���l��Z�(�'�OF��\_'+F��g�To���e^�����Sv��0�#�)p��YAV�����sЅ�����q�7pr@gb{�����d ��)���{���z ����+?���K��������#�o�s;�B�%�v@ ��}s�ːg5�i����ep5Q6�6�\]Cɚ�m!씠WdQ�V��E�q���V�:�N��6{�v�zz�#�!�\\��ճI�S�Br���z����'V��e�I �$�D����p�����:ti����|��� 3W&�"�7a�o�<ܢ}V#yL�Π� ��\*>nD�\*�D� �^n�#.��2\]�Ni�s�p��h��;9�F�� �1��V�@�E+��+���ˇ�m��Q���e�k� \\h3�}���C���n�d"=��N���l8��-\_��l�(���4�(�uȓx���ҟ|���2k� �����F�ȓ�S�2��\[q��W��rW g�3x�\\&��(�P6Wfuh21Y\`�����Q� � �D���Π��mU����R����v�x\]9Q;��:.$Xy��� �j,׽pr=w&�0N � �"���Α��D�Zۗ��cjir�4g��ߔ�~��S���?>���̫2m��zU��q��̿0�Q{ }�cM���cP���yv}ix�v���\[�L/��z�P;�vt�0���R�!&��l��!5�'(�"?y�P�%Id���8��֋E\]�l'���s�&+��cd!D��'(�ۜ�w}��}�l~U����a�� ��}�UVܭ��Ќ?G�@ qƣ������v�{Oe؁4+\*� Y�k�7�\*�ZZ�:M�\*�U\[\*�#&،���kNA�� lGy��Y�K� N�6�Y�A5lZ��i��^�g�e��QeCW����ʠ���K�u���%���ULu$o��E����Q͵J,���)8>��w�j�C�No��'�<9i!��-k�2G��a�(�,J�� ��c1�ײL�hi���W�!��&1\*�! j��r"��Z��gAۧe��,#���&i��7\]T�O�sxbn�e�q���>�U�;wr��r^0:N=6ۋ�ʣ(3��}L�g��,�e���no�Kr&\*4��#������R9�׉v>^�'b�q�^\_@� 1vM�\`L�ޥ7\]#=%I|��pI�Ua����Q��ډ;p�-��)��;�^��ao���h�/Q�Ƽ@"�\`�D��6�\_�r���\]�n<����P����x��uwfn�|�g�y@�32H��-��� Li\*�'>"�UX��<��R��9��,L�UNm��A�\[���H�Z~��G��jIwñ6����v���\_m��z�\[&���;�S���Aaһ���L�\`���N�DJZ�w’OGl�Ӂ�su؟����3\`ֺ��\`��ו����C��׽���m:j?2�ty�\*Y}Bb:�$^% ���Z���4֬�j�'�K\`�N{�հ"��}0 mjܤt�(�3���HQ�$:2��4��f�>5���Ze�1�EBk<\]�����%�6�UWp�ځ�� ��P�����dm�K��5�� ���-��Ҥ��ь�~>��K��h�g��(�?�+�&����\\�"�؜�x6 r�g��?Lp�:��D��8�1 ���WG#8<8�!�4�Jƹ��v�(�s�vW���k �k�6\*\\?�<p4J��;l����!�G�� ��$d\`v�� ş��ޕ���/��i8���o�v�\*\`�F#!�ax~�\]#Usw�&A�ik=(:�|m�^7�wQ��?���-��Q>1EwY��#�ḙC���Z�EAZ(�u�E���0���/�M��Q�+��&��w ��ȕ�m ��|T�m� ��psqB?��&�Ҩ1l���F���8�@\[f�����:J�I�;�����܊�)34sب ��X���s�ݬu^0�-�45�y� ���u�u��tx��SU\]\*�Z�qpu���������mV;�U��3��W�ts�J|wC6贝.!Y;�!A1���,�\`��=,1��\`��Cٿh���wzR�ԉ�\_�B?z���v�^�g��n��&| )�!0eB���/��o^ed�L�A u�N���Kq�y��I܂�3�Ǟ��F�.�Kή��n��/����W���o��n��S�F$�p��t�Hx��g��f�8��7��1�5�7�Z���K��bO{�"�a�O�F}0�V!78��4#,�Մ?�!M���N�E�A��p��q���8�UW��o��/� m��� Ţ���Y��-��/64M���N�5\*��>�gU\[\_��'���diE�������zo���KY�t\\6�c��4���‿=�ct�� \_��>�\\���ZV֮ݥz���={<�t��Ӿ��!��U�(a�3�BR�Q����1vs”�1��h�k�"��=�w��2�R����%K��:� �R\[ܰ��Z��q��a�W�Gu�� �G"2�ӻ�k��U{ {GN�q���ʙ�X�V�����j�0�x��0rLC�B���P�7h��z��o������:��B�È8��� D�#ցD#��f�4��)-�S.Q�͒��W�P��qY�-�)��8�\\��x�{��\[�u0���M���^�y��7�68�yΉ��S?Չ����M�n�D�DŽX��F$g�v��X\*u��)��/�Z�9���.�S.2XZ)o��7�c���\]H�(�dt\\z\\6~��9>>Y�M�9\_�,�/\_���y,�3�P˴ţc��b/���@�f�0�Ԏ^�\*�RZ�K�>^�a�W���OpΩ��I�'��I�PFc3�=ܞ;=Z\[�������F�rl���SU��Y4�Em���&��U?��}(�)xJ)w�+mQ�1Y�������������۝?���v���b��3>eP�F� ��05�R� �3;2 �1��K�U�\*LjSe���� �"%�k9�ߕ>�������O �i+��u5���am�z��&�=��=rq�zA���S���j �H)����;�kd��4\`��r��h���'|�ʳFoಉ�-��� �׉1���!u�+�ڨ7��gw�?8 �(���ze��g4��xb;ѵ)�+�\`�Ik�E�ZE/� E���l�ڽ�aId��p��� G�Iؗ߮9�{����w'lS���2\`F���fOd��\]KHZ-���Ʒ-���g��C@f|JzQ��mˡ���?)�w9�w0���gB��\\ f������إK")�H�ؗ�'���! H���K5�S� UN߆s�磤 ���\]��'�������\*�T\]oP�{N�Z�%�\[�\]��N�I���#�� �\]�3��c�T�Q\_\\��u���)�&�e����􎪪L�E iR���H1����W�%�Z�W��O7g�~}����}�Q3�L?�%�Z��E@�y ��k��\[����c��dI�\_��5�\\�! p�\[�)��d��s���ַ^�g�;/cI"z&zZ��9f�V0�ɶ.@@U�ҭC�u��L�e;}0�I�v�Pi E�\\C-����g�X�ZQQ��>�˴i"�Bá(�M-=&��XI��W����d�2Wk܂i�! }tp��Fo�YÄerWT��AV�W�F���5&iq�{�$,��A 5�H6$ӱ�G��\`���E\_���i�tP�F�vTl|� �~�?\[3sa��Jd���>�)l�+ŦJ�~b��l�Il\*��y\`�v�y>D#�\`a��k��h�Jv'�9\\�O��h��9��ן�%��Z �T��f�����-�}��B�\*1��A����g\`�x3�a���Gi�lh P��C� "��9Qv��Bs�t.p�u7���+�Y@(ou͏�u����I\*�B�@��A̱O�B�x OW�s�0&i�Й\*�+v��فʊ#̔��+c���$�H��Z�nN�҅͑,܃И���� Bz�H��(���nz8��B"�f\\YQJ�� �SG\*e��@�J ��t�5�C�!@ �\`XZy�sX�r�����A�a���5V9ggZ�Ymy>��n3���A�U�灺�@��-m�x��Ԑ�������⧜���D@��4��r�յ")@l�������� �?\[��W�\*��=���q�q�G��J|G-�}���sE2��%�V�I�|��iI���c:FO����r�O����U�!�}��T~}I���T8�/9��1��UJ�#�4N�+�����ϴ�h���W��� 3}���|�l����:�g^��A�z1Śd��O�j�N�Cyg�����v����M\`HH�s��-X�����J�1$gd�\`.��8(�l��ht��^��>Wi����0��@���\]�L⭗�Сv������"E|�b\*��W�NS��r�i��>��\*��6��F9fC@��h0�gQ�$��.#�+�hO��N����=���l��%��m^��UTW#�KF݅b�9!� PЀ�XP���8�4���w�Y��Y�r <��y��\_��Q��v%�J��^��&�bv��1iQ� ��3s��mF%˜�ݐ�~���rץ�\[K��E�Q�D����Z �rB@�u�������'#D���יB�tQ�Ny��ȿ�s�<���� ���0�����$qnu}���":�� {-�\*2B&i�҆Y�lw�>D���~+�1���dB���A���a��>D�rFA>S\`�ݎ��܌�fc�H�ڮ������-�>�d�i�D���>���/���3^��j���:�qg�+6��%�~�DpB�F�?���w�6:��./�+�Bd�q�c3� ��R\]oO9�Q2�>��kp��q�?�~�8� mt�vW��i+')��� ��J�~����KF Z�T�@s���k�����A< ���p'��<��d)9m�L��/բ=������5Jv����ϭ��ݣЎ���ԡ��j�8��\_�)��\[�X����5oQv{�$r�,#b�����e� ��˛s! s� ���?��Of@a�@�/����>����%-<��+�k�(V��rEq�ThkI�)�5eP�a��t�|xgl(Bvg-e�g/oL�H4I4K �I +��y��M��pa�Kz�n�����eU��6�C�an�jz�G��(̼n��bU�4��2��\*�DЀW�y���� +ݍ���O���Ǐ>v�����sB�h�^?WN�!4R?�E ߝ�\[.��h�xDz�v���h����O�)R24�l�TDJY;�T��@?��Z(U�M,o���Cܶ\_cP���N�P�<�rx�e���;�����W3�k6��'��|\_2�;��j��Ɖ � hOG� ��9w�v��Fo���f?� �fB���%Iu�r��M�"'{��:��M�v�0��,b^,��,��zm �G!?K1�#s���ơ\`��Qf����z�,��ab>���s���b�� e� �g9�3������Ț�\]1k��\]�r)����#<�Ur��lfJ��%\*�3����\\��<�^\*�j6��>p�� ��h�-B{����,��T �\\9ڔ eT����6WC�h�����ݭEV�>���sy�Z SqD��\*G�Vz\[p�Ov��@�6�2�,����,f�b�b���L���\*YP�bef,48�F\_�/�=C�%��0�zҴ�f�̾���Ya��� { ���,�����ǻ�7�\\�}�~�6\]:PRG������f�\\E�\]�N�C������rg�u�9�M�^��T�}�0x0}E嵊�Nl\`���f� -���^�QF����1�4sy�v+rdCC��5w} %#P�0��}��\]�����F�%y�f'S�\[��/UK�V�ע����6�4���'!iLd>?�PU��ޯ��Ll�E�FLt���J�\\�����s�z �!�zA��\_���L�A�GX�|�4�R ��QSL��� )$֠k�i)�Cf~�EO �cnE(�.P�\`Ĺ���|��{qFOt�S�M�U��S�q�kTy(�8����\[��ߵS�@�梒����c?�� �X�a��R<��3�T��Xn���6%��Ұ'�wB�������Y4���x/\]�����.��#�9Ļpד�J!l�����/L���P�&�~��RG�JB�t��$��# &�eZ����y�q1ؤ����Q�wK�U�R�,�����ce({��4�%��B�ےF��d�M��/%�z�:���5-�P�W���6�ݶq�\_�3$#����� ���$2��\\��G�X�{nfs��Ĩ����"%1 �S�4Z��B�^��C�%<5:6n��d���y��9 ��8\[O�c���\[��TKC�O���fS�$����m�ҫ|�D�\_�MG���#�\[��Ow%���9e�����E�"��\`B�p��(�Ñ�s�!D�G��We�����$��&Q�=s\]�)�q�H�� �dI �͊��~�����ږ\_�=�@^(�b�����x�J��9���Aa���9M"�M�H!i��B�F�m��?�� 5:��L�ָ� C� �c�&\[��=��h)s�dži������Ʈ�(v���.��̛ʰ���f���+��2�"���&)���6�ePq�.�c�ޟ�#¢῔0���!�qiD˹���� ULj�Ph�|�'4�ȗ�EN񪥆Ac>�- N�שG�VW�tB:��� �6��H$�/���T\]��4��8tk�?<��ۨ3��,2�j�+M��0f��ū��L�\*3� �s\*����f�\*etz�JQ�/��x�Ȗ� t�� ���jsk)��q��B���h���'-�ũ�,����\*��,O��U��K!셛;���f�\\ãp�eux�̑� p<�!>y͐���s����m�/�v� ��72QC��'�4����M='�%��˥��p���-~���8�R�ӡ���i�d�g#s��������3�|�5A��s9u�t�+�F@ĆRL��9�wV����p?I��G�H�ዩstq�E��P���)��ݤTLR����á޶\\F��՗{��s�P��ą\]�U�����sAV�a�\\4���2ņS�Ȕ�8;�gM�Ѵ���/{mn\]Y��1�$���DI�+��p�p�}�۳����vx�C�<�6��;�dt���� �3�� -�J�lL����Y�R3ح��6�Cش�0Q�Aq�N-��/2��J�\]P����YiQ�Չ\`�S���M�c�'��t���6s���bn\*��LI�}�n\\�4l2�:�����D�=���F8��)�V��0������z������!�T�ܗ9���¿� �9�-����+���H�}7o�lc�M�D Ӝ�� x-�T��ej��i���5 G��E�������)�3�#���R0}��u^��\`O�j"��L9΅�;�AםvhQG�N��LO������c�I�U-¥Ʒ���IT�N3�IF����2�-��h!F,L�&�V+"��א.�dkO8x�M f6ԣj�2���S�fW��!������%v� ˤI\_�<��Y�^�0f⪚�\]i�Ve�OA���n�P <�ȝ�����؁E>�} ��HҶ�\\\]Z�۽uv2֬��V\\DJ~?�9�>���2�<;a|�+�K bG��g�?+�\[)��������P���-���U5������|��i�nl��B����싹$����(n��v#��9��:%@���̤U�\*��ǃ�1�4�X�-\\��g�G���K�z� �T�Q9�\_,U�-�D6s��p|j-�Z�܆�x��8J��ٱ+���ۋ�mS}���,M�DG���wFs�\\W�\]�FʁX͸�G��3���W � ��V�F�F��X�Ҡ�て\]�/�w���DE bh������糠<�i��8�A\[�ʩ��L�jNw��"�:x�Z{C��2��Z��j6P���vN��� �B���NF��ze�rx��:��}� ����O�=yd��I��qu>F.Ѻ?�o��{�k9A��ZG�j'~eL�6�YCppz��=�ζv�7�ZK�Mz\[I��D�k���bN#���͚�����3;��x�\_��9.KzO�-J�xK�W� ��\[4�)�1|S� �^k�����E ����"�Z��{,h��9>�H���J��%��{ֽ����r��۸���������ca����#Ә�'��,���H��\_B��UyMq��d�f��2�g1E�46oy�"�ۡ�\*(����:�∞6��w��z-\*7O7����o�d��)�%HR�{c��P4+���^4L\\c/���/T!T�33� t����Zn����7�w�%��4�wdw��( ���Qm���G�70�J%�=�\]}�7=H�d l�Sn��� ���{�EdT��l� +���Y,ח#C\[�c����ӷ�6���d�(d�D�P�Qza�seC��8�>b�k�������-�<���s��A3�v84W���Z�Vs K�I�X�~����\]��>-B:�� z�k���. \_�"�vS��s�G��4\[�ݼX����2٢"��{Ь3�\\Њ)��\\t ,>8xB�5�"�UT�Z9"��/f �E#u��Q�<р��QkyJo�G 5r�)��$G�\\Ҽ�T�ב�������x�XF������?8�c#f�L9HglXHD�Y�UFР ���>B��$oJ�ߕO��9�lT�s��2����F���hOj������^kJ9�2\[o\*�7-���Ȟ�|����u�))�\\�eQ�KJ/�K4iG ��C쿾�-� ��Pѹ��\]���x��D�f�x�˱2��ފdL����pF�=�w�qR�C�.���ӹ�C��sQ�!8�f���EAGX�E-��͞�9�����xT5�U.g�O4o5,��N�y:����1�:�q֌���T=�.��Hٺm��w�!�����W<��x'µm3�o��Z�Q�V�r? � 4̑��#9����#1\[��Z��T#e��I�\\��u����ڥ�K���:���gUo��t�^5�6.� �N�z�Wy<��|��5��U����x�x!4:1^@�uC��R�(��ER@.����S;���Y�aւ!Bc�Tñ~��I�K��a�����\*&3�\_ɉk �.\]�BM\*�,����oXe�6�=�� ��ߥ�e8�.�� F�x�\[C�E�|>��59�P���ƶ��&�/�O5\_m�{|�/1в����2��/t�>X����e�>�J����}� �{��N4��\_�4|c(�+k�;� ( G���Z9�F:����<���l,A�V�}^�+���F���,1��ӂnB�Ǭ���t�=���bz�u/���g���-�%7a ���=Gpߐ�T3F�.���A28�|�⺛ ���s�+S�-����kPV�Dޮ$o���GZ��WS�\\��K� ����p�R���:)sb�v\]�H�e\[�q�/��F�e���8�2��~@���h4� \`���Y�����T��Ch-G���nbL��Q���B0#A+3IFK��~옌�ДuB"�,i5V l\_�N���hO��3O�AEa�{�i�IB�I�?��FA�Ai��^P�kbfz��K��4�������ϝu,�����O�xCdF���D'{���������OQqo�ɝW��e��h���rM���\\&T��}��^M�\[/������U\*�;l.�<��)�|��Ns�2�� ���\]m� 7��\_xI�j�w�G��٣ �S'g��W0=�9~�$���}�t�vO��%0�ֲ���C�:�ܑ(�"��eƁ\\$��i�.$�$�(%��v̐���6��3�@̚v�+L-a�4�!��6�:c��"=j��;=?�ᓟ6Rz{�#g�ӈ�u����F���N�+�.�+ֹz��|�.C,�X�GH��V�X-�Iq�������.� Ʋ�W\*U�j��pVe�$ s\_Y�-�6��J��k��hf�����W��=�\_�~5�FEgX�����Q���p�\_��Ժ�"�,�g�u�}�2Dvx8��MU�F o��S�����F\`p���Y-�2�=����<�=u /�n�cL��j�q�L�~l��v" ��.�5��? �#���? ���=�:�<�e3A�N�FG"3�QI9��~� 5��1�\`>�Ô�K擰�m��t\[�\*�\*Ԅ�>��\]���n��z�ʅ�.\[eF�蚥�X�� ?AP=���cb�5��.Ì�=��6���h���Az��c��J��,�#�8��ykW1i��l�:�����6���r ԇU�Q������B\`Ö�S\\^4�\\ G���X�+��e��3�� �Vk�5��L��}�a�Ӻ��w �X�27Hwp�������\_�i��C�1w�<�Ȯ�� \[ִTN~�^�P��.&��+�)W #�ޛ�+�S%Rb���/����<��w��9�@�vL/��iˇ�0�e}�T�'���r�Y#�?�G}W����\*�k�^eڌ�����e���;&0:<���Ι���W�4�-tU�G�j�D�dz����L�5;uq���� C~�L����Q\]��\`�{�A|��a�(t\`d�;O���;�\*Q�qDiF�1vYK~���"m� \[;Xv!����-�qh�4�AW�P� ?�z)|���9�!\*k3��l �b���֜�oD�-��#o�ɫ��ߙ��䡳�im�J�C���3�8����>��ן0����������Ȯ� 7� �u,�����d��aT��c����B ��%��RK@�ѓ��/�lD��C�yu�Kb�����4gVF#\`�;��Pd�R<�$��}���D�D/�>|�a4����p!�c-큀a�R��}8���!�Ο�c-�ܠd0�5������4$u��u�6��<�ѭT�~2��$�q5��$x %�#AmIj2�� �w�D���O@b�2$��� h�we������o��?\[ΚOҷ�<��4QW\\���G͌OR&�oQ̙M7$�Ek��YLe�\]h�W.�ح�P��<������qu������i�|�� ���.���� ��8���..ޓ��e�~���1R�ǔ<'������4;���ˆ.ʮ�|�O��dЎέ�����N~�}j�#��p�:�S�+�0�ZA!f��̨��\_t�/��Q�b���u���;��V�V�\*X��\`���.Q��D��3�= �#���vZ�����4x�2��AK�P�2����@����m��g���f�w�.vlR���=�É����v������h��…ΣND���\*s=�QZ/ 2-� ��HD%y��9iw��m�%�|�}���t�q�F'-�}<�a�$<�x����B�b��ۺ�Ɩ�%K� ���5�����^�)옍xx��^��T�2�7���0$ԇ�x�Dې ���|�ԧ� 볳\`��#;=���� �x��Z.5��Vâe��JI���f�,�K�e�3����!3�ULO�6��ď\_�Yw�P��KT$�9���桑�tL \*f x�����J�$\[�K�\]��\\}@�\[Uh�Z\[�'�1+����|��O�e��>���L�������،��� Z�i\`�"V �4h$~�cx(x|+��p��w�L�P��� T06���g�Z���&\[��Ъ��ȣ�h����1�2��ټp�G������ń#���^S��$�E�2h�(��#$��l�#b�L�5�t���� 2��z����TV.�G戝R�Ža�"u^9�=c��R 2���J�|�X�K�����eZ�pt���ۮR�\\S�MtO^���c��~�\[��:m݄4\*s'Q�J��6� iU���icl�+5�^���\\�8�� ��u�ď�:6�<����)����\[KK�b���t�i���Npxm���S���� �b��譼��\]� ��C�. \[:b��Ϋ����}�����='�8�R��T��-�H�� �Tj�� ��O�����\]�\]B�����l��uz\_yO̍�l>�\`޻T�b���q�'h�$)Є����:�a��i�\`?�\_�H��f��W�:�λ�6 |�$�9�\*G������ �"��������뻍�l#K��"��l /i(���a����$޶!\*�P:49�t����T\]�M \*3!���z�� ��z���ͮ��s���\*nX�\*؁(�-�����)"�435�\_���@\_׷�,�;����{�Qd���n����T�.�)��Mj���2���C�pe���l�C�y}��������.��Ґ�$'�g|���� ��O� ���v� ���y)\[�).S� Y��a5�% k���j�3���+�$55�u�t��=̋x4 M���"��kvDd�dAR\_f�#\_w�Z@sUa��� �\*�@&)#+axp��Qo�n��B�N�+�ْj�� W �o��f�3 w?i1K"F�'��TS��'��������Ό}�#3$��Yw��)��w�7,��@C��U�B����P�ݳZ��tv��O��|���HtI����H���7g��F�TO�.O�n�����+.�Eǂw@,4���\_�¿���4<�Q�X"� K�jY�Y\*p�ִ~7$\\>Y���:��� ���3 �'�~�W�b�)#ȗ�B\_�8J'NЩz����&���Q@C��&�\]�0�J�lf�k��Q�X��B3Cz��Ř\[�B4~�\]خ-Ž�e�oN��u��̵�kc#����i�\]��q0L��m�F\[�#W8���%oWSnU˧-Y��2yǚu��p���NjW=UZ �ͼ��U"'��~t�}y"��� /�ì��\\n3��1X�}�U?��\*m�x&�a��U���õ�u39^�8,�7@��}r��:m����ݿ��Zd%9�S�{�wv�L�ب۸��k�q��)�GJ��e�vv>�%O�L�Z:G|dCq�T^ƕ� �f�lQ;l�\`�\[J�� '7W#&�L9�7�����lJ�����5aw�\];n �\_�f���FQ�^�.P�l���!�/�Ƌ���� ���)p �\` \[M���g89�(��8�g��g� y}�:f�y�5�~ �զl�y8 ������;��ۡ1�(�jT!֩29/J��E<���I�f�e�qtP+�\\�\]IJ�� ���Ų�� \_��\] �Ʈ��� ܿg9�=�ŝp�^���|���w�����\*P%4��D ���׹�O�\_>��w��I0m ͨSMGe�� G�Z�����f�����+w,�>���G��B��)�½Z�嫫(ޠٮ�.N�6\*u�x�)C� �xe�ҹh����O�.Q�r�P�^ 3\*�+x���V���<7/'Սl��Rܪu���%���J\*��&%&���JV�a m���Iئ)�p���q�R.�f-��Ӥ� ��\` Z�u���f�޴�\*}=�� B���s� "'��:䠽�k�C���B vu�!��oQ���SPҫ|�mR~D(N)�f��7�.7��?�J�����WX���j��ٖ�NѰ�4��8༞Ϭ��0� ��$ �=3e���?Իi�RF��g9Ih���|ܽx�X<��5���>���P�\*�y��~, ԏ�Q!ہ�@����{���?7ў���\\�)����Rr� �"���G�L��O�?κnՙ� ’^<Ѓ,���w���ںN�Y�JR~١���ԍ�(����q�by��ɵ�q����w�Y�ye�}�|֧�;\[�-��L��к�����i\[^��j�u�))�M��� y�\\��I��&\[5f=T��Z"�\`!�t�1���y����A?��Xs�����~�oWY�E�������m@w���w���� �|L��!�� cq���j����X�� a�n-���J��/Q\]�{��-\`� ��w��c؝� �N�(L�6�b �-��,����>���L�0péE�\_~n5�� ӞȓO(>��? �g��t�M}��$w<��W��'ޢ^�� ��4�\\3;��T"�0�u�\`j�����D�q=��U��w°6T\](i6rX�)X�7���MJ��o���A��������~m����K���g���������^�V��y��b���X�M<>x���P��#�ȷ0?��a�j�ؕz#� �n�ݼ|8Y"��a�����Q-\[��ړ��?��qi����}���c�ipU,��MѦ��}\[w\`���(35E�zS� ���X��\*\[� 3�w�̺F�&����㮤�5>���ˇ�����F�Q�Xr��k� G���^���J��\\���K��;HE�9���c��9\_s���ֿ�p�E��^���QP2�g�ʪ�P�@���=<��WQ���̄�7uEkh ��.��P�D�������o2�ʠ �Y������k�����|��>�8W{o�s�m A�s���d9g�N&��|�c�9�5-sM�%6��膵��$��K0����i�L�-�~\_�x�h�z��׀���gB���^��{{u���Ƨ U)$V�:�f�2�H ����\[d���hz�p\]�B\_��D���Ճ���\*�R�/Jb��Iʧ�B����?��H�421+i�W��Jt�S\\ �pT��ɬ����ey�a��Z��Y�<e�DD�:��jdk�\_�ꁉWDSæ��n�\]f�H���5�.eR8�x?�t�����@ �d�UT��n.3�0ڼl�"�\_G�9B�Xɠ�$b�StM���oS�ov��R��ky&��-h\`��5�(��.z����f���ކ8z�;� �\`��PW)Q����u�O{�U�í�w����\`���Lu�L�m�ܵ���F� d�T|E�;>j�m,�. ��3�@��y0W4���,F�&əW�h�L����t&Q��R��|n\\��9� n��?dDŽA����v��K\_�䖠�wO��$W<��sfo�{��|��Ӑ��~��U��ص��ѣ��E)B�R�N��Q8|��&��|�1|�Z0�VP:��q� ���覬+P���/�d���j+3j s��<\_P�)M�k��d��i��$Q���, B\_n%����ʆ�^r��!\\L���!��>�n���Y����9fl���W�+C=&--��\_T!��y����Q(��������� fAyWOګy���g��U�ә�G����7��Z&,@\`c�N�0s�d#�d3��tt���8�,&����PC1� ��E�� ��?����CʹC?L�elD{^������3q��N��"��@���WY��8�F�;����8�>�}\]���\]�Ev૊rV^���\_�G��P��S��ა�$a���O����˞<�� ���R0��C�\`"����Nav j��0�cdwKvZ\\���qGh4�H }���3�ȟ�T�\*n���"Q���}ӽA�V���V�á��F��i��(�Κ�؁�%4s9����o�v���B�k\[�C�tX%��P�ژ7����U(B��YQ��i��8�v?/s�����4\\�0NP��=�K�ļ�$g���5��H'���U���>�M��xd�17DBP+l�R��|�ױ/�hE�%bX)���A��F\]KQf��h��"�إ����t��Y��Vd)������4O9UW�Kg��J�����@c~�%\[�/��1���Bw�5R烱8>�Ф}��\*��B������:W���V��;�k�Ů��cc\\E�Ջ���2\\y�R5�?ؾ���Ƅ(��(��uw�6xnj&3���� �SJ$̊ g�B���� "\[#&ē)��@�� m~������Φ" o�1�D'8R��Sr\[4�3.>\`(cd :��:Yؽ���z�Z����m�5o|�e��� 3���'�C���}�0^F �c޸��-�8�}���)� f U\[M�,�\_g(F�0-����� � �Л�Dڶ)�  �.9U!wae�밉�5ٛ���l&7yu���xMb�����n����j>�:���B~\* �HN�b �߂�� ��e) ��&�B��DZ��C�>�������Xз\_m|ZF��i� \_{��� k��^�v����\*�w|� �fy��>����ݘF�m�W.!��=�q�\[��d�J��\`gg��E�h�6��e����� �gJ=R����VXЕ���v=\]'V~@�1�qHр��va%��%a\]"y������:���%�Z�y\`� �ګ���2���U�j|�# &Fe���c��w@��. ���)��n�4?\\��&�s\]Bɍ1�H��&f�Y�ܼ��G�=�~��#~�잔�Brax��SR�9\`�}Y�=��ތ(h��@�\*h��ל�\]sTAS��~%��� �0��^����F��X" \_��!��ƖT��8 +8K/��� u݁����# X�蜌p����fx���?1��k�Yqe��t�\[�50 �r��H�օ&)�"x���gS~"�f\\A��.�{t0�zC����ֈ0i$�q�s�SK�;te�|R�tK��J:e҅��=�Q �"7�}���SI��a�s�a�j�g��m;�}#-���V=d��� F�\[3��8K�q�ޭb-Ss��vkj\*f/�m2�W��j�v����ɔT��=�� 4����=� C@$���j��A/��͉�b��X�:�'M|Dt\\��-�m� �/G�����O��+qi�{���M�e�4\* ��ax\[���n4ټ:ؚ:\\h����DžP��R�Y��"�b!�����Λ}�{������������2���T.�h+��\\F��K?� ��o�,.�UB ����J>���E��W2�@��"k���5�us � ��A��7@�#��y��Iz�$�A��LJ�\`O������l��'������L���MC@�PQ%���w(C����M���~V/�IGmA��:�ۿ⼘��������Mဍ70��Ǚʺ�8���c<@������W��W�f�(|����� ��V��������s�h�Q�g�GDžkv�W 8�sF�IH�i>�E�B��۱-���͟ �魬N�F�pv�o���\[��F)�wn.\`�(Co��̆��53pxw�V�kF�n,b>;��ס�H4�ۛ�� �")�2�k����B�1�i\`ק$��W��,�OΐYTa(\\,�gQ �F'i��� ���s8�>6��ԁ�R��Ѭ6�?�<�\_�2 ��GJ#^��� cew�!;C-����C�I�\`ae1�m\`���2��?v��լ�G#���/�2Js��)cYc\]TA��)��Q���E�إ �Z��L��F��X�����8�/8�iV��@��\*R�S8�eQo�Һ�҄V �҅�x��0 ��4V� }��\*��Fxu���U�)X�l�����r!C�<�. �� �G3!v&�mhz�|P�ijj Arz̸ʢ���z�X���raXJ9�T�@RA���U�Y)����)���Ҧ+��"��� B�\_MS�˧\*aK?�\`�;�~y�2��+B���a\*a�aɄH�ݣ2�0L�;v ۺ�V���˟ ~�fXL}l����a�Т �M����b����5�2�>RW����N�"�ر�;,;���n-\[ÉzW\[% ��Z���b��4s(�K,��\\۔G�Qlb��עT�4:#k�|����"���h�3f6��S�}�X� Q� \*��3�}�\[���e�f#Mg����\*��w�~H�W�s�Z�"�b�\\ncE����F�$s�;{ �$pb�(>M�����^TG�T�w�̓�3"�� F�Pɬ֧f\_JG'�3�ڼs����S�x�M�<��4�FYd �� ��� �P�3=��U��l���� �/�jD�Q1���r���:�y,1��SJ>�.qaʤX���L�E���WD�a�8�Q��+�t���D�,��� o}���י�"GśÉn��7ͮ�2���\[�>Ȫ��ƌR�C��|�#ҥ���)�O9c4�πK"\\rzWG~Ȍ �Y�Y�J�d���E嶤��4~Zw\_#��v����9�%��4A��g���\\G��Xb7��n���m�+H;x1�� �}���n#�����F��q����B�23g�0^�!���%�9�&�Ͻ����\\� ���|'�z�9mʼC�m�I�$ ��6��L��Qg���g��tfc���i��j��$9SB)�"�'i�l9�v���� �41�B!vA���1Z�3�LQ�>�g����as�/.Y7Zg�:4�= eMZax����g<���= ��=s�v��؄B������ΰ�ӵ�>����xF �q�dY”E���W�/��\[��0��b���$�IH���P����I\]���T�.�w� m����ዃ��X�t@t�����SsEU (g��ɂ��eL�"U�?�����(F�J<�T��!�����h�R\`�ď0܊6����YM!�c1��vw����L�0���$2���^aB蘔SZ@g�.Ѩʖ� eE��!�M#:ϛC �O�L0�ɬ"щb���}�!�Đ�T����q3=�0h����ߛ鑤|f�0���4�%�����@^��@x��"�̻��� ���+�^�DړAJΞ��T�1���%7��th zB���PM�ؒo��d.sϊ}�q�<�O���R��|���\[� m�6\[X����k^DZu# \`�^:\]�:i ���+���^�o~��+ V@z�v�B����p��eQc�� �:?OvYF�6��\*�����!��RQT����Ԥ K�>ŀ���,E<�\`�����a�T�ѝ�k���k�l��V�(����c͕q:2��F\_;��'�fN\`�X��3��!E� m������ެ�拓�P��\\k��8س����3i���Y�q�ʮ \*����U�On�M��+�r}�X�59(��P� ��;O0B�u�ۼ�����hy��=�X&}�ߚ�KB��ӏ �=�2�b���-$mc���.@��~���8�T��bd�e�\]?K� z\[l��2�-X��?��.;s'��ơt�F�� ����p��?�d�;���I2H�<��y�;����!�a�lg�$eB���7�7�?\[����\_Y�{ �jw��\_�)>@�\]-DB��oW%q�u�J �cXg�"����n�#ٷ� �\*Iޑݽ����x\]�̇zѠ���vQ���&~wʦ�s5�ٹe�rWA\`��Z��>�ٕĂ��nj�9��\_rxa�\*4@�tv�ǁJ��;|I�bNB�g�}�� 7�뉦��Jd H���3,�1{ZR'�B?\[)Y������xO�q���n��> h�7n�o�d�(dC���\`�K�)�����T�ko�W��ԏ�}¥(�g�|�Bz֠����i;=�d� ���Z����4^� ���� �UW P�3m���5oT z�3h"�����Y��-�#����ɜ:A��=z4���� :��L�o�9C)�G��� -�\*཈sݏ.\`���\[��񲎜��p�=˭��d��\\iok ����q�&��&��M��ߩ�#̱�B�솾�{ȮN��xKr� oQYZ�S���w{����Y�����\\�a���TC%l;}D�953 ݚ������tV���� ����d9e3�˱D����r�UW��\_�o��4�TӶ�'���-!X) ����C\*��L;���㺫��� \_��� Xw�bd{ �D ��4/ԡ~\*�֗���;Up⹮gR���\]Ռ����\*D?�.�YVt�;\*��{.���))���O�$�\*�\]�==�œ�׻� �����MrHt��@5�Ѡ��29�����X V�"�b4�����?q���Mׅr�+\\�gJv���ꭊ����Lt�k9?|��e��Y����BE��}��R�����/󙯁�-��DkL"�� �1�6���q�-���'C�Bˇ�p�x͵�q"w?��������&%Z��gQ���}ng ���b��":+ @ڃ��x�������\\�UW� \]&�8�4����g�\[�4bz� OqB�񳔖�#�^�DZ��—Ş����f��AA�˴�Fؖ����C�=6,��@�˫��#����}%�S/��B^�� �kUy\*�?i�/F%^hb �lH�xa\]���7��̌-uX����|=�\*0\*̊\_��(ɵ��!���9+�\_�n����W�����z���w!��dU�m��x�x0n$�i�KtP�^���q��\]$�9�KrKI��b�ɭ��Vf�}F��\[�2��g!׉o|��z)"S}�Q/�rn��d�6�ZS�� ߷Q �n��?|z�i ��� �ջ\]l�G'l�+F������ b$��S�\]�$��j���Gr�T�̂.\[C��K�ѱ�Wމps>1Ibm��\]R� )�� F�0�%ڀF��h�䂵J��l��\`����e���P�L\[\]�ж �o3�˰��Ğ���\]�t�\[\`��V�X�)��{��3���z�i\[5+�( ɂC;��}�o��A\\G('9呌\_�w���z��-��Y��7��M\[\[��6���wX@�-�\]�Xz ��}\`�\\���ߴ���YY���m�.�B��� iø�K~ �1,�ob5D�I�����#��=�'i�^~�W������ԏ�I{+g��9(��E����ݏm�2�c6QN�;$�M�������w<(���� ��B�D9��a�pv��Z=D�Yɺ�)-j��X��(83��O��Fa��j��\]�� �: �.�e����^ -8x��<(��\\F+l��mG�o��'\]1����:��LnJ����U:�d�NT%�|2��r�b���g�$�����2�y���t $��7oY\[�!ʐ���&)I%I@�i��e�2H;�11�$�ꆱ����2�J7�"�̮��l��B����8bf��M#R�&α��6 �L�WEi����6��\*�F��3ţE��:9��\\Đ����~��������1ꥻ&0Z�V��Z�d��\`L\*��\[/�\]���W�q����ᤕ^oD�q2����0bx����~��Rs����)\_����Cᛯ}(�I"V�t�0��@O�7(ҟ��O�͎��k盂L�k$śO�l���f��rB�e��gOYDaDXt�ݳ�����Ӯ�$��޴K ư��m�F#��ʣ~\\&�}����Гa�vA%���O�|A�p��|XVڈh���u�XWO��p��D%�2��@�q�c��3�����"X��q�p �Y��5g�\_�Dx���Ծ#�C��wG#I�ip&��i9��@ ���G��I��\_\`cZ#�\_H�a�\`��-q\`�������(C��:e��j��?W� �e��ӨM�I�a�^�B�9:-N�'4����Q�V ���#Vcd�4�@X�XP�E�Nu�i����K�t�1Y��І�\*�\[��ױI�K���DK{��\_����?c�uIM��R?\*.=�18�p|���j�E�%�6 �@�5+I%Q�Jj�� ��;� �����%{:�-\*\]��� �\_l�����%�,��S6&�S��jHY ���9�\\S�\[$2�Y��K��'nӃ����ES\*73���׳��"��8eY�R٭� ka<��n �+piȁI�\[�� �W���Jo���@� �"P�H���/gJl"\\�Hr����T��܇7�Y��T�"��;�+8 5��%��/��\`�x ����h�O�@�rD�ɊDo�Uc�I���\]G9q�Hh?�p���Z\]Md|��/�-J�숝�8��g&�j��$>�%a$ �N�(/Ŋg֟�\_������@����͸s)�h��ϙe�f �� �̌�~!k��x��A}� � S����!��%%\[�\[����I\\�7��ӵi �� ˳\]ޛIDI�?�(\[�I�K�A� $��N�췑0�^h�6��2�����+�����И�\[ڿ�0,S�t����s+ΘSLq��!��a Ti�JLa� ^�L�"��%}@:7��祁�%k�v��N�Q<� J�(�\]X��&/>��A5�oRE�\]������A �L&�Ғ��ʬ8\\���Y��|6�fip���d�ylS¶�<�a�e�/��Aߔ�Z����m��:<�U�A��m��yB0q�͵�Z(;�fN ^�ܕ�J����;�eb.# i���Ro@n�bƳ^x"����j����\[Y(i�\\��\`�%���uzk���:#>���HSտ-�5A$�X\]��O��\]-\_i��|��,�!���z�P��j�$���V�U����= ' �5�~��H���p��4�Sn��X����OE�i�^�Nj\_(�� |P�?����ǖ;)���v��7��FH����U�A\[���lG����h(��)Y� /�����Ƃl����=�Nxf��qH������S~�Ө�$�$ #ݑs��)�~�AP֒��u{�d뺲txW|��\*�8Ns��U��� 2\\I@��%��P���-�@1Pk��C9őE�V(Q9����R?�|م\`���I��\\!�qm�b9���t��4 ,�p��<��ҕ�{�+���0 %��I�V�IƟ�e�Ĝ%D\_�K^�m���M��ԗx�R�zT}���\]?󎠞����\[?��zuh:w��Su2 �2JM���� �C59���Գ�4�.䋎h���MC�o'Մ�L&�-UR���m9.\_��\`��'��Ĥ�\`\[E/�7�w{A��$�=��O�; T�^\`L�d&� X����P3�GͅNd,= �0���O�ɠ��&l��\\ �\*Y��$�ܙ��S�olg��.�Ճ��G,^���דq�R�hX� �� ;���\_��~����G��W��N�u\_��M������\[��Č\\M�����q�A���.h��\*H�\_ڊ�d\`�{�T��C�ps���k8z��R�����$��m'xls1/�u2��Q�\\~xR'�2^\_���!����|���U� zHf�\_u�S? �F���8��T�^,E����N��c�y����E~�s��#���\[�5�q�H"-�J�=��/�J���\*��J\[/m������>|��GQV�\_?ŹW�M3\] �U�<�j��.\`��q�ߟʄϧL�&.?EnqoR'u������&���gdYĊ�W+# r3�W�<�ޘ�:U~���;1e�O{��Dl�ܼ�V(��i71k ���Q���:X��e\]I9�\_X�r���åS��"����#i 7G���5�S�/��8i�Q�{�}ʃg>�M�R���)(hbyj����ۻ��V��� ����c"+Jvʅ�y:�\`�頙V:�\]j W w�l�h>��\*���^o���O����g�K ���������3���'�?�N\[��o���jCy�e�&a���&#Y#���c/�J$�\`Z�\[^�\]ˠ�7ڮ+��n����\`��a?O1��t@"LFIf�\*|�o q"=�z��c4}(��^F��W�f~��w�ǥ�zI�eF����|������}ݙ HL�ʶ&k��/���XUX��\`��&��s.W��C�m㑣�S��N�e��Ki�2�"\\k��Bi�Xkg���,D���\[\[:jh-y�A����G��I9�!-f�:���\\�a{�E/�}�輦��7��Q|bhk{�r~�Y\*���L� ٜWb�U)�f1TF�&Ty��,ͬC�,<���V�Y�S���6(�^�.�"tE ����f�������ޤp�N��2��ydN!?$����M8���f�h'��=!DgP��\_ۜ���mU6:�ž�쒲��9���E\[N��K�����}u�l�&�~�n�&g6MT�Yv>����ir���;������R=V>�������)��2\_��w���� t��^�H9����UA8\`!�����몜\\��M1M̡�v��^w#ΫƬ ��TGq�U�M͠\_6�3��!ȕ� s{�0�I���DearbuԞ��������<&�9t{\]w�T\\B}E�ng���B�� '�ž꿬�h���-O�g�9��Q\\�^�&���댜�|�Q����a�т�6j��ϊ�qq� N�#G�ѯ�Ɩ�4x8Ƃ���F� �����9�O��Y��&UW?�����󟊃��ɞk�h3�E���fkE|y��|l�D�ɲH=�h���ӈ\\��r\*�����9 2�Q�c�Nb+�c JPeb0o(=���B�znCc �Q3� ��@��{�N^�XI3����O��,���a>e �����R�씞����6SNY\[��\_c 2�2�)3��C=��2;�{�Uw�%����~r7����>��+�vw�i}� (K�������� =>��C��|6�+�S̀A�����8M; f�v��$ \[��ޫ3h~t����R �X����+7�f޸@��MY��񬨛3)\[�5�I\]���{<ח5��\_P�2�׀���,�R�e���"G��lL0x�s:�3�!Ȟ�K�?¥�JDE}�ʬ�#���uf�^���1��>�L�, ���)1���+�P��k�J�1{�� �a�ݘ�'�͗�����a!�\[�\`����\]zgw���d��Z5O -����E��!�5������ L�������4C���.����H�#��7����\*�������-��J\`�?�}�~�DLл> ,&"����fCv�?������5wl%\`5m�(��lS�C�E-Mя��� �MGvEe��\]"����>�"�$(�D��̜�״-������+�' E�ɧ�ԡoy�e$�~�)�G+؂C}f''e��SkSKk+�r\]"�d�a}\[:#>́��� f�ni��Ί\`$ݰ��O�Q\\�w�2=�a�7���h��1�4Y�$�W����l2&���I"���Ο�+�����6"\\��� m�����\[N����1�� Z=��JMK� �=�l������t������\]�..8�w�f�1�Ԁ�>�����T�����s�lr�U�T� 1kE YH\\fv�o��u7/C��o���T\*��<��8�N��\`������\*-��N�6h&�ޢ��P�����(!� (�P֨ٷф�y!���̮����4�(Ň�Kۨ<� c�GB��ߋ��<@�I76�X=�o��B$�����#R�+sF� �r���s:̷�k�X��Y�b�H�t�=� �G�!���7j�Q\`{j�d� ��S��� \\+�K�/u\`�p,��q~��Et�����>DC)��� %���Î�y��cY9��\\dp�6UF��������9�EiծB�&��V\_�/\`�6��ܫ؉f��5F}�T�;8���lVd��PyY ��^�t���g�\[ �U��"n���0�+oH�Ԗ��(��pC�9MQ$D�Kt��1���ET�m5���ۇk6���>�Ǘ�Wʾ�^j=������gZy��hxGL� �\*�p�|��2V�.� c6�T��L}\`�Te�� .�L'D�ݿO��Z8�"m�cY����D>�K�@��9S˻���5�����x��� &�}J� tw������m$fտ�Ӆ�j7fꉓpK�n3. ���SQl2���@x1\\χ��!O\\� 3���#�=�<$��Nd���#.\\����M����3�\\��ϐib���x�K ���M��}��@iJ�@��q� K �\_ �m#��� �%|���SY�����\\9�&\*O0o!���u�s���i��;2��.<��:!F ��kN�Ūk�� ���;�^�S�2b�V�UC�xl|�5c\\�7��<Î"bli�� lWz�;�D4�J�� ٗwײEȚ�"%�!��lf�z�4���2�~9#�}�l���#���H!Hz��?��͏ �\]"���󧁁�,��;A��E��w���ڀ���8pL�=�\]pgY�ŇI�� G�i�:M28�CTw���@�ԗ"},���yI���{|\\ �n����!��f�b�+|�=7Z.�kn���<��rcϵ�����g,�1w��p�M�x�;�=����\]�s� ���Y�F��SB�qa�{�3���a���j� �'�%�Sj@mx�>�pG����� �;�� �6R��{3�Hy�"\`5��?�S�a5�Vھ<��X�ɴ?s�{��d3��GB�H�S��=v�Z��t0Ȅk�F���S��1P 7�pe���ā��I�|���ɦ�G���|1�\[���5:,< Sy����c7+ݏ- ���7����u %� x��9�,U��Ȉ�KT��v�9�7Yw�a� ��ޮ�l �;��h!����~\*������QTzV@\[� zO�O �W!AJ�\]%�ىA\*Ln�j� ����WC��,W�Q�b\`��Kb�h�Y��\[I���\`z?Ş� s�3��9N\*��ku�L� ��r A4�'��\*w�>�b�a�L�W�BT�'���b��c�f��%�pWqp�BU��hì +���-�Y�U�pj����I����|ok� ���R��1��a4�Td�p��t!S�e�5��F�C���#�zX@u��.�Y��Xp1:}�{ ,�@�J.c�U}̇� H��N&�5h"<�S���<̍PHF��e�:����\*ZNY��\\��N��-��4<\\�20�#,�y1d���(�c\_��DE������lQ���9�GADۄ0Y���}dlF�Y-Tzz(�4�59Г�,L��pH��O��khuMzn�?���ڹ��/�9�s�M7�B3:�� @G�����n���k�o�������g6�R1.�>�/�k��Q\`z������㑱Ie�ŏx:���̲ �!�ٹ����I���=\_���=���d�#���B����2��0�t�Ks��Ø��,��G�-�ix&�A�)+<��B� �:;�2sʬ�N��\[c��+�Ή|B���I �j�I��5�I�ES����)v(m���H�h9�Q����P/\]�\_���>rr#�C5�\\'���H�t��\\b��֥Ar� �\[���(�zZ� 7c��x�����f(����zX|I��;���٦'���U$E��I��Ca���bܓ�l����p:� x 22��jo�߯BҊ����~-; �~C�6�@�lL�p"·���X���f%4}��i�}a� Ya�2��"똀�\]c����eKL�I���G�ɶ ���{��C�&�S�j n>���6� #�a�c4ϑ ��))yN��ᗲ(ʭ�UY�@�\*NЬ<�U����uu��W�{N��/�R.�չ�\\Y�z)̴=���׎��߱�iiu���)���썅�J@gJ���p�5 %Q>�"�� �f��FCF�\`��eV�\]�JʀwoK��}�w��1��?��5�c��ה� ���GvΧ����O~�� j��ȝ�\\ԉ��+~��c�yڤ�@�AiW,�7��q�V:�W\]\]�y-�n^5�t<�Oo���ș�2T;��Dt\[��H������h����i��V�'{?�U�-р��πc9�CZE�R�Uϙ�B�)��ޞ��:�7 a�e�O�a䋚�|��b��Rdo�\*ȣP� �Ox���)Z=E�@��C�m�&��7jfS�Fg�S6�� j\\ ���y� q�9��qԻYs��&�|u3EVd�H��q3׷�^8P���#�p$���c��8��Wy8-Ū)�y�y�����ޫ7��fa��>'�$�Y�7��o9�d mJm0.LI��z ��w1�I�Y�F 8���%�T�\[J(��E�u�����vH�m+� �?t������ C����P�uAur�(��f� xz�bl���n=Ѥ\*�A���nT\]���|�c����zf7W��\`��gz�.):\[�"���!����9�+-��Mj�����,�@����5�h\_��?��9%�0��+���8OxX�����LTш/�����g#�gR�!4��B�y�kX�$p��^OZ�!�P�uQh\_�Df���Yg������� �vo�t�� �i�"t z~y�rU�jIIߔ��8 \[�ǜWL�2���>�ʲ �����~l�\]l��AmL=E�ʍ��7Ibg�c�Ʌ�4�� M\`��Xˌ{.b��>��ۍwI�3J�?�L� $�t8�A�z<�����j��5���K����:8�s�f�ڞio3&m��g>^�$sG�X�A�%c{��i��6>^���3Md ~Q�@k��(& ڮmK\_�X���\\r6�o�{�e?��A�8�O�Ve� ԡ1z��p �ä����� ��vm3z-���X�E�۩,�l�n��� �ԯ�q5�!��\_��ȇD\_@�>�O�6SS,kHC�ҽ ��������l+4��ToJ�B�w�O��&��/��Hb6�͂�52�Sx������9H� ���� (�wֺ� �@�\\J4V�~�&������jF7�R-��d�v$\`KwK��6T� �P���H����1@� �kew,�)\]G��U�l���g�lLh�Z���(�Z��?d䇣DH���m�>���������S?� F�Y�����:=w� ܭ.��U��U)\[ ��p���m������L�7\`�� I�@�=�s�Q>�&��g+Y��A��5F�����^G/���\*\]D����;x�#���Φ�g�SEBI�-�K�|f�8k��ә���9�zE;�g���½�i����r9����^��B�Y�4T$a�C��֤@�<�z\\}��690��qS��ӧå�m&����¯��O^Xu��^2������@���Pz��������$,��� s�/8\[ݣ�k�M�8�/� h����%og0�K��9-�X���Д���ծ̹C��D4fX����2wmy���?��� EUR�y7?A~����N�6�6��ud���Ձ|�����7q&��CJ�D����u�\_�TpY��Z��h ���Ȧ���1�Nh(<�Ţ5nj���\`(�d�=E|�J��5�w ���;���&M��0���RXpC�!H�0��%f�o���9\[�"�\\O���\]A�K��X��?�wt�X-���6���5++mX�<�˜���&�8���@{�T�\_8�-��& �G{g|��p�@�zX��e�2��f�E\_1Z,�ZDk��eB8�d��� |}Q+�ɱ��":���dES�v�{-o���kU9�H偷�,����<�f5��r�d�c�Z��~���|��S���I���/�)�Y%8��o�(��{��S�Nꅃ�b����b�l�����lbY��~���A>������ާ{\*�C��2+$>��յ=���DW8^�JY�iJ:Js�$�#1ޫ�C�䦍�y�0��R��׈ ^ǎ\]JT���fDm��.���� �fa�-����FPUע \] �5 � "��wF���۰BD��: .p�J�Z�X,\*����·���i NGʕ!c\_Czd��1�S#���O�ߗ 8�����^��Y���х��x\]D�\`.5�Io/����$�ˈ�;��h �\`\[Yb:�����N+{�\_�b�{w@UR,��'���F�ǰ�� 9!�������9;{� ��5��%w�S�YAf�7 ��q�p&��"&�~�C��!���$�V�a�z�(�;���+�v�Z\]�&��nFx��/�^���t��u����=�i0�5,�yn�\*�h.%�ԮH�)%�����/Z��|sd�>�;����=!T� �̯����V��c\[$��B��EB��S\_h�I��%��S;J@aMq w�w�����Rsi��|<0��8 D��{�������.(et�\_ +ĩ %��;�Ӆ\[��U�o)�S�v��� I��#�����.��\[\]- ���|�C� )��\`|�T�9�G���T�Xۼ��.�����)�& �5��#�e S{�8^Jg!kL�'%������ ��<�.�O��/7c\*���!��۟\*����Mb�s�t� j� ���韚�d�ۥo}J\]��q�c��ׅ5����E��g� \[h��e>�τhl� W�XU���.Hy��E�+pE��F�\`\`��n��"ɟ˙Ҕ/��wZ�$ �I7HG��:sn$��U�ĬsGW�6�2\_���X��%��D/֗\\ ��n��q�Vn��ٽX\] �~�h�)CSe�s��=��G����G�y95c�{�L/�f�m�~x�h6@��#���ʕ��J~���l��@v��@t�P0~5�t&yx��?���Q�w� ��!:��/�ej2����B�>\`Jg�\[m���H���%���q��8szaEʿl�$���f�\*�V4/Eڛ�Kq #Ѷ�tQ5A�1���V\*�@�i�X�Cv.c�B:�ѐB�����C%����\[�|��Ӓ ljE~7D0Q�Y��#����B����"(�b����$���&v�OD~���g��5ʍ� �-��9�B�� ��>�A���rb���;i�W�MY1�\`���oq��.>sb�@ʺ�{H�ux�}������ �A�}j9� \]�\]�87�Ou� V��^qA��\`�\*x�@�N ���(4p�?�I<�Nx��!龏�� e���.��?��v�1�\*a����,"��-��=����fq\[1�/�+�7Q��������c�Hv�i���Ugِ��B�qHJpvuH\*M-�ׇ������@����; ����v���{�H���i@;l�w�95�3�eOY7XD�ܽ�w �������ͦ�sc�� ����W,���D\_�L�꓁P�S���\[�����Ik �́���2����4VO͈���Q�����0~W��I�kW�y\`L}�Ð�ŔL3�����/Hql�������\`ԫ‚���|�|��W���9�3lN"+���2�E�%� �r�j�KZ��&����W��ͺ�ׇ�����EG\`m7����e�1!�;���f( ��\*L�v�����\_�z��Ȇ��l})�,�w�̽��N�/\`�J�/ۆ�X3\]�)���q��l�Ϙ/V\`���7���V�~�\`脩7��Ł�EfX�+������A%�u|��x��w\]H͓�5uC��݉���e0�4Yo\]\*�?;Q�T�P�O�zLg�fW�B�3\`�x�E���Ѥ�R- B�s��?����r�� �����&����o ��\_���k -w6� �E��Ӳ��� ��w�ns�U(��؟{8|��\*�C$'^����@$;~�Nj����å�5%�W�|��J��u��՛��gp�BԠs@�A�nj@��j�}�ë �=�j�n�E���8;e�\`��'o�u��q�Uh��10�/�t�2�E��c��GM�����GDU�I\\�8֪֒}s��-�>�J���cA�cr�z���Ioy��=-Ǣ���h��I\*�t\`D��y8%�;8�dg�cۼf�/6s��H,^�@�U�gχ� ~� &Y��'�����Ҕu��j�!���|g��&�O� 7tuo >�\]gx"k�EY�x|Oԓ�0:�G�r� R\[�����~�u�\*��V2\`(Rã�(/d������~��S� \[Nu�e������O �yD 7�݃���:<��M,� m1C� ��>ޗ��X7\[�(L�b�t�x��|E�H�l�I戙� �@ɜ3c�z�����Sb�a�M�\\p�r�Hk�Q�S��� eH ��� ���{�GZ�e�e�v�u�����o�vݲ�t�zn�d�ב��v%h|EA�8O�(ZD�YB���%u�����\]�"v�3~�r�ȚaZ�\\�=��\`H ��ג���{�\`�z\`��ȅ�'�u�8��w��Br���RN�a�B� �޳W捇 �}'u�&���A�XJ�3 �J��}�x0̑�b�b̳\_�^�",=�qo�I)�8Ǡ�a�u i����!��e\_x0�R������b$Fb��@�3���(�u�� �SW'd�Nq��cy��ͻQ�|TGG���Vp6é\[7�������>3D0�VG�7��z�2M�R�D�Ϲ������$Rَ'��Z��j���B��'%��oi�Fb�1�׿A�{n{!R}�L�,��!iG�t/�i V��r/�S�v#$R��\[��8nem/2lڌ��.&5ϯx ƽ�k8V���Z�i�r3U�E�����/9�f���M�VC��f��FQ�^A��k�� ��b��BS�㟲Bۅ.�)��Ѕ�"a�㉨E��EcA9|-�l1�7 ��=��\_p>k�+����U�ʅ���F�v �/�#�ݟG��ĉ�������� �v��taI��!0{�W��~e���@l�?1z�V�i�qX�X5�L3++mƶ;v1� <@��(e|+�f�?� ��q���  w���Jڍ������ ʘt��'ү ����Ҵ /�/�����$䮸�@&�j�c�<�%��7�Ii�>��b)GCur�q����C�~j���0��q36f(e�r\\�tc�سз�V$7�{�'��o�9���'���2�#����a&gI��Ia �OT#�jF������pg�ΰճvٵ�k�>�� �����J:������u�=���\*��bΙ���Hh�mDR�!�r�)�b�E��)�3�aQ\\ s�P��?/�֘wL6���ΦG7H�ƈs�z��JM.$G��=i<͗�gT�ݳ���8L����=02�Jf�L}��LD�d�O��7�6���R.��b��)ʽ������G�� ��,,W�٪ (���k�������^�\`����p"��~)��U�� "� �K�����)�U1�ԛ\_CC<��0�!nI<�Ym����.;h��d�� ��% ���J���@W�z�����:�~���}��f��o��ť�=�k|&.�$��v��%�Y#=��So�S7�IⰼR��l�7 ��F�^}Pmf��-\*����� ��\_%m������\\&���)�1���w=d���ATP\]�sW\[4< !s�i��F��7���"��2� %��Y~&�� ww'#{:�H����\]�Ϡ�|�iw8�Yn�|J���CQ/�t\`��,�X>V� �5��38޾O'��0X�j�X5Ҏ�F�C� ���s�E}C�������-�淏���|����;!0���i�#L����N�:�N�qXs���R(�ڰm����D �\]�d��SYM� TT' �-��4F�1r �=6s��;��\_{1��E�,EY����p�������$6%�����? ��x�p5��Bƥ����y ����4�r}�c5�63���mur�l�IJy�S�67�\[�#/ eS�j����\]���6��B��py������zw�m���Jj֨ˢ�ѽ��'1J�ׁx��wgy���B�oÒ����!�/�V�=���T��Vw��%1� �\_H��#-��X1�(��u7 ����ܗ�g���/ߖ������c�qy'�T@���)���\*\_g �.� oVx���v������GH��!�\`�)��EXu���Y?,^\]\[�z�f�ܯ��iD��b�Ԡ��'g�K�Q�,��lo��9�J!xN�:pvy+߬:�-l��k�1t�@ ihFZ�?Ȑm�\[\*��Ķ��?6Y����=ˍJt�=�|;�{�R�L\_�����r�,S�̧Rp���~�ƠQ�U��V��pr�qLFu��F�4� �̶<�l6X| q�++��'� ;v�Ѥ&o��g���e�AV���pyG�v��)�w'A�6N��!L�g��� ���g��+�c�E�Q,#�5e�|6�a(������ �U�\_����B�=�1��r,�q/�^|����NGj$ �㼔���MM��o��h���v¡��i( {dS K۟pn��B�$�S����u�߷"���XH#�:�YH�� �"�k�.0���b^���:� ZY��\* R$P!��Zm��� ���� 2w:���\]�1�s�����yt:��ᨀLq��,2Q�L0T���\`잨���V��E��׍�Paڙ�y��\]n���v����\*��\_F���ض�o��=�X�؁�h�������D(f(��i�G;��\[������X�t!ޡ��\]��j� �i����-W={�5�\_���Z�DO���=���U|2lv��c�V;ݕ!���K �\*��m��e���Ôi�+�����"��|�9Tǔ7쪼������ZE��JL�{p�� ?X��<��5Y��=W���3��R�\[��V�ز������\[e�����P��ŝ���\_\_\`t��xBx�O�����M�c�o��v L/��"�h8'ٿrA\*1@2��u������wL�2��퍬�������3������{�3sR�5-�O��ҘSz������9O�W�-��ܘS�$�H��Ls�Ƣk�<.�"5G�8J�6�I�:���,�M����(PN>��R�N��S��� �:�\*�<ҒX�x��>��Ř��W��b\*>��±;��\*�T��&=���de�CS%�M�Y^is�\_0ȥ|�4.�����n��V��� >t�%2���8\*� �Qn�֕T������׍Z���t�\`dii�?k����������Db��jN Ԃ¦(qx�\\�~���"HbT�ʖ1\]f>0� y��\*��ښ�|Gn4,�Os��O�Ε�C�͡����RU-��d2��u��ɒ���<1�M!<��b/Cm̓�q�%\*�wk���z;�#$�� 4�A��\[Eaݮ!�Ӱ�SԻ��S�ʏv��/�&MD@W���iRWz9��Cۿ�PB�H�/r�FPE�����yoP�Yݟ�@OD��(�M��il~�G����h�gF�4D ����M��3�<���v�>�=�5�.N��d��C�q5����5��oU� � š�C��h�We\_p�|\[hH'�\_σ ,�c�Fƽ ���lƈ�~h4�z��6��$|�D��Yw�Z�:���4��L�5��)��o:F�3��� n\_�“��&ã,�P���B<ӿ��4�ׯ�f�T\\���� ��Ԭ�o�K���Xᯮ&N��.\_�˺� ����,\`0g;�������\_}��Ǒ�?<�Z�85�+��bX��y����Dt�M6Bx��,���;�O�0瓄�|�dd�=�K��cL��Oԍ�| l�.e����\]���"��Ӧ�V\\O�ɯA��8���L݁@6�@�a����%�z��wT�O�9k��0���Q+k# �5\`�b���y��)����hN��덩�{�C2F���0�y�5��+g��\*�D��'p���>������?6$\\�F� D/V�>�\_�qq�l��g(�5\]qK(�����/��NP\`�-������U�)����W�r����!�|S�v���g�݆0�'i�H�F�9���"�p�8o�� \[ R|PCJj�U��WIg�gk���=� �wP�� M��B��� � �Z�\*�W����Ȗ<��ϕ��)��m.�<~���E�g�c�j��)I�i C5��k����3%�@�,$2�CTo�^s�}dD�I�لw:�wR�ݯ�mZ ����Rg���h�\\Oɤ�с����m�e\\��Po���������՞B�y��V�gH���� �VC��j�0(Rw,�7kc�qM���9lRiz\]e^\`9�uX㲴F�Jv/i�\_(L��=q�U����8�8\_' �oS���JC�XMV\`Z՟��Ĝ\[��{B�Pf-doV��Gb= ���^ b��E�'-׭�����H�v��p�2�o\*1�����͹\[���Y���bd�Fa��zrZ@�p\*�%D�4��nIcID ؑ� R\[ύPOeC�1x����5�p�= � ���j�'�z\`@��㝗�� ���y#��h&���\\iG��X�Џ��8�y�B���m�D3P�B,\*�m:\_�h��g�(y��n��=ų%Dx2J��%FE�\_���țO�"�‡v~���z�S �<�j9�\]d\_� �x��\`g��m���v\\��'��GIu�\[N2ؿ�X�H � ��\[;�z c�<�Î�W���?\\��at\]\`m�-�ڀՋ7F|�(s�T}9M4zQ�O����(h02T ���H�q����B��0�wd/�"�Wf\[&. ���M���j�����j>�@Pm� ��T�(��)��)7�w�N�\*�V��t�.?��𧌘����/�xVZ\]c�<�������Xz���U\\��\\���>��U�E���ƀq��V4�(�2�c�#n��ø��{�} % m���":i,d�UY�Gq\]��^N����\*��5?��k�|���yΑT��IL����X��#�rΪJ�N �ʪ��C��b��X��)���dV?� Vpe?Y����A17e/> �g;BE�7nZ��2�߯h��i^��4� ����V������ �"�t��R���AlSa"X\\=N!��K�n�����c�S\]b��\*Ԙ��!�@I�Q��<8��\]vb���H��ݢP������iO�S�4����\_\[%c�fNI���1�pWx �M�ތX� S��|���, �غ��~�n��'�zMm�֚�#4�����&o{X� ��f�-&!�3L��H�4�b �����yy1:\_����;d���+�T��b|P'�h�D&��4����\`�ͷ�<"+(���z����dB� ��O\_5A���@��؏�Tm2��I�C �|\]��KYj&D�Z�'�o���u���ǭ���M�b�!nC����J��P�1rX��@��M� sxz�>TI�䚆��s�0����l�z�h��S!����$�\]��!�҉;�E@��܈���$�H��\\ �Q&��\\o3����(I����������p���ҳ�=�j�KF@���(p>S־L½���|���/�j�^� {gv���'~��a7h��I�c�u�.��3��Y�:�$�)\`}.�E����;�\\��w�u�����~5 �&'2������\]&�g�6��N��%�R�hv\[���❤K)��χ�nŚ�� ���'�m^��#��@�߅ �!\`����6B�~"�k�Uqq~���g��C��������d-DŽH�:Ϊ ��\]c���8X�!����Z�"��G��^�C����k����K������!����\`���~7ڻ�.�"LR\[WD���u2�/�t��/��"�������VIe� �$�o{�Y�L� �k�"�ۜ���\_.P4����O=�\]I�\*���4A�c���7 V9Z>��\*5�B���\_0�\`�H�MT����k�͗ǣ��W�\_l�r���+Z.H4�A.��;��5�l�Ւ����1p�~�my#W.2I׃��!�.���éb��1$��C� %V�v,l���8'�ա��ѥÙV�)��yG����{\\�Ȟ�96�#o|e�2���Z�Q�ҧ\\�R���}��5�C0���J���K���7�(�ȓt�E׊�. �J� ����A��R�ֺD�z$‘�o,��sBA�2��u��5���\]Ӱ\_�-:r/N���dǥ��W ;IuC��͌5��ka'v$�S9&A���F�ˌѻE5�ѣ� �"k�,>S��\_�7�L�+)T�2��y��#�w�P��ޏg�fC��ˡ~�k��D.)�����5.9F��/M����3#'x�/Dzm�T��!��9���L���Si�I� ��b#M��=���\*ڑ9�aN�M�I�1CѠ���뢷�)ޓ9m/,'�\`ȱ��ԢU�Д��T҆Y��O��!����U8�V�7�b:�����{?q��e�3N�|�����O/Z�N<����/�3sGG�ݣ�s�\\���Z����z;�M9�J5�V��5�9�Uf4����W����5��F����W@�M�E�!�U��p:C���, ��s4��R3+x���m�NP:�LV %�����j�YE'�;��a{����X���D��;�M�MA�����hm�8�5�:!�{�c&dm����n��� .|Z��u�w\_�����dڜ��}�oI� �q�O��}^T$џ�1�z)\]��y���������TQ� �\].�XP\_^ɇj2�F:{g3J><\*�;����ۄ�! �� ��(L��I���-l��/��5�D�D��w���f��!�s��>L\[��8�13}��D� �O��W�cn���� E����G�4ߜp3>L��R��Ǽ��x�;�\[t��X�6����W����H�%�\]Ԅ�R�fzY�H�S��h�>b�J�C\`�>6�F,\[�O -+k Ҟ���ruVgj��� )%�iU�LD/�l��T����x�s�Q\*ء9����(�ti�W�C '�\\��F�o�;��{ ���+�����Z�hFxHD�|��a��/A��R�$�q�u"� J��(�a�<�%\[t�j~�2�t\]\]��K��晿�Ϳ"{I�S�}�T�a$�i�甄�ʼn�a\_������g����v2M�� �������K>Pf�aj��m޿\]��B�uN�D澜�\[�衿6��Fb ��3�(����v�n�I�)�����H93��X���8�j �������P�Ez����~�Uqz�۲�f�Zx�4�X�GHZ� 4�9E��� ��l�\*0m�ס�; �PExFL��b� C�����LQYnO�F.���|i��,|�W��x�=)�\*B �ݹj� ��?���70֓�V&�f��\]?�n���楫�z��:����z9T��bƴ ��������C��\`�k��?c��y��;oq�4��W�!nl.�4�2�0���̈́&�a\_ѥ�����y�����Fj(�bI��,��f�ҵ�E¨T��㨑\`!^�ͼ\_�s/�����Pԉ�N���z�7�f�B����l��t�������q��;2�#��NH�'%A͇���JS(<$/��t��\_����k��܂�����S�������Rt+�lx��N\*���2F\\"e�\*���VC�~���-f�\*�ٹB|������A��a½����A��\] �/nC(��}s�o0z�\]�dH|�pOh���Qw6�X�໣x���.�)�\[��\\����N�� N��ya�F>CJ\[p}1=ć�F�m:#Ġ!3A�:3�=�I��w�6��3 ��±�V�q���3�Z:�k�����4������~R�\\��~�d����{�ތ�D�̈́ �:����pd}�\`9�G���GA���{��\]L$��yP�W��ᡴ��Q:��TbK����{eS�s�?��Q��K�:��x7����K�h<'b&�s�ƣ�z�{��fW\*Fє>ުr�w�X��5�Q����2k�0C���vp z� a6�m�x7���7��wL��@0 nH�>�QU�0:!V�8I�49�E�^����>�J��\_ѹq�P�W����$z�#��2�jw3,W����e� cN�8amf��Xb�������}�p�����Q����~�y�~� \*%M�6��)YŻ��B$A@��ޚ\\;\\�@ Ӫl����oHw^��cױ��h�eͦH5� �9A�V+u���S��x������2�^:��4:0x��!)Ӑ��o�M���KSo��}=�|�\*sh)�r\[�W{�"W-;�xN�s�������y\` ��������n�euA�ϐ�Bb��\*e^(����ߏ�O e��y�)\\Ri��KLy��� 9�\_�LX�@��{� �N�X��&��c��B�82^M=�k���Q�����4\[\[��\[� �߽�L��чb�C�����u����Ļ-�A\]���A�w��\_(V 鶴������܁rn�Q���I6 �����<�Y&������{/���>�&0ٖ± f��ݝA��=�'?G� ���T1�Ľ|~��mT��l� ���t�'�TK�&�(�~\`�\[��8x�h7Eɗ$F�H�eo֙ %\*-�\_o͝E?:�Ey���j�OGjC3PVW�hV\`,����O��k�N��4ԷϬ����UĚ5�i�$Gn�؋��� 25\_������1̩\]E�y�$��YQE���6�yԲ\`�7��R��+Jj���i��Y+ͣ��V�n8R�;��T��:=�;\]Ct ��� ����^E���q��6�7ӯ'�u�(���)Q%�C���4�\\xψGy\\ ��AMC,�WDA�A�ā��ʧ}�4S$��A�6�X���=b�E���.6ʶIU��������o ��b���y8����;��\[��G5핾���s#�;�}��5���Rܵ�}�/L��x�Mu�1����x��k��xD�Y���4h���w���N���n����qBjƋ��r�X��ۦ�o˅��� � R�ĢLX,� TʻA�cř������κO?�b���-NR���Mtr�,�Hj�ƒ&6��\_�Qw\_ �A��s����7T�����!�s�� 8%|������ �:�̋5��������h l����U�����s��©2� ��.���d���=c3\\Q�En�$i+{x�X�b8�\*�P�\]8�<%C��H��O��ꚣM֓�-&ۍK5g։ G�\`o�6���v�A<�|�����yB6(���M�����;� ���\`���M������.�ޡ��O{��0�Eg��C�KP\*��F��7��R��XUx�Y�p4n�au�\\.g�U�(��is����,ʌTA��lN�\_P� �������)2�G����o���g������M���' �0�\`0�\] ���zִ9�K����pc\`Q�mBՋE�w4�@��'�éaD�l ���?��|��o�N\_ Q�&6�dO����,���K��X���"\\�gTgus�}G�zhX˻F�sK ��\_n�\]au��q� �f�F a�I L\*o �FCuR��C C��B޸���d�'{\]� �|�8��C5��ZNY��m^�ϑ� n��o��d�� �A{�صR"ÿڿ�X�\*  \\C���P����k�����5��v���\`��μ������)��̎��k\\ ����e�^���΅�g��l"�6:Ӯ) �^q|�k�jB�B��hr�U �m{G����=�Vx�( ��=��v\_г���50 4%���\\\[�c�\\8��%?o%ud'w�?��>=),(\*��Ց��hf�� I�5��uW�>���p�����ɽ'j1���duN�4���fg�F���'�R,QM�2�O/Z�Հ�����N�z���'0yNŦL����Uރ��� c,��i���aK�}�~b)��-����߳�\]2Rol�N�Oj�h�1H89� 0�.� 1\_7a(k�U^dKp�E�����u��g�3z;��SNL��lKB��'��g.�� �G$���1�͜6�q�������)8R �n�tp>�=��^;� �I�\[�܂�%�\_����w��Vآ0��QE� ��Bj���+�7�B\[���n�;x� �m��ߐ�+�I.��4 #��^X�\\�d>�&����ñi�᫢�Ε�z����q� ��t#|����k1����� K{Π�?�� �+: �wv\*?��� ;3����AI~U���l�\_:�\_�w��D�n&@P�^�����%׮ 9�z\`����х��e Z�V\\�xfƁ���4�qe!��\\��9D�po����y\`i�U���LHۍKX����5%KH� ڌ|�Z� O�c!��޾&������,d��Oƥ�3W�dp�)T�\*�?��{�YЎ}\\��� ������� �2� ���!}y� u-�KB�K4H�\`���"6a�����gK��� E}lP�I�������Č � �yp�RS)��ON��v�@,�u �Ct�B:� t�(�ʭ����%;�w$g~�b{Y\`ˇ|L���g=����#��\`�r�%r�\`bt�¡L-C \*Hs�t �#v�D���z� �K��t~��\_HC��~\]���}95�ч������O��a?kL+�l�ȷ/�� Blg��r/�i�I=!238���\[K}��j� �����A�1W!x!��v%ȃ��4y��J�/�QB��n�a�.�bE��~�!�g�n�R\_2�GOBvk?��߲Y�ܛ��f�z:�ʾ؏g�V����A�po�t��Mޢ��I�/�ۙ����DUm�Q+��i�vDo1\[�6\_��n,h\]g�u �{�R�Fc��n�c� ���B���:�wY�5z��)\]�����F�l\*Τ���~�4���ͷR�5�o� �!�����'��B#�H�J�n�&�ݑzD�M���3%�%�����s��-�!���Ff�^������#K �=��0�D���'zm\_�&~�@e#k�< ɟ�m�¨9���ܛ� ��$�����ɡ7�ɞn1��&ٜ)k(����D�Ϟ�L�ϩ���1OTP�ו��\`D��7v�&�\]����;)A�To,@�\["�V�p�bJ���������7�5���ֻFR�\*��+w�8�S�A�L�mX �0�\_��ML�A���p2\[B�vz��-=ٸ X��1 ����?κ�G���K�!����"�S����bU���I�|��u%8���=�l2����Pv�mкj��Ҳ�l��L�#��r�t�QK���Q0�5YJ쎃�ߖ~���\]� ���H�G��v�I���u����t��Q@���Y�7���Y�m���rt�.�X�y^�y�ī����j�ƾ�$Cܜ�}p@fD�&�'��Be�j��,��1ɢ���UqR �0ɏu?2W�W�e,ϓ\[�2bA�JA�h�~^�&/<7hkg<���֘�� �\[�\*�\`F�q��ֹ���7�r�\*U�N\[�x���/Ǎ�� �jT\_Yx\`v��a~��V 8E>Rמ��y�jkukuua7��)Y��\\�9�fD�e�ťM1@{o�'Z� n���V��LN Ns��(�"���5�s�E�)�T������y���-ُ� ߽����n�!9�#s���^���B�P�qY�F��F��2<2~���Z���%\*� ���T��MZhs�2 ��w\*ԡ�cT���z��a�OIQ�䭉��B��a�w 5�)�t�>��k���S�n���ٽ��V��Ǥ�Ȫ՝���A�i����-�KB#؉��k�:-��RF����V-��r�Al���.��2 ���5���qa�9g�Pj{!�,�ѥ)�q�X��!$\*W2�+�\*v���n ��B\*B��,ΰ�k���\\B�(���H�/�EI�\]��b���f���q�l���W�d�pST:��c~�g��'H^?��%! .����ݟ�\`mc}|s2���Yw����=�I!.��޿������~vB�NY�8���)�r�P�{ Ԫ�u�,��D��?h� ����=�>��g��B\`��ZCL��1��Մ�5&�������\]��c�������-�=��,�jD� ���C 4�M�E�-!�P�d0�C1{�->=���b"�ض�uA�g"g�oG�� �2�P@�c�gL ����JT��7(�6l{�������p�iF�S��:S��;P�榊V��eD���v{9�)κ�6���� � �Q���#�$�3����E�C�L�3�W�WG���}�{M��+gLe2ݩ�eE���>��i:3�4&�$� JT��̲�ߪ囟� c�9���K6i���G�i�ρ��s�^}�Sc�����n2"Dv�L����ɛ�9M�f�\[c�W�pk�8�R;G�l���\[�8|�0����^T��%�d=����V�Ȟ�L����>6'�.�K��q-� 3�G���� ������9�N���RZU����2�f�V�)�=�l��9�0b��D���I������.XCPp������,e��i��g|�X�o{2��80���L\\pwt>F��yT�~t�{5��������M�v��1$�kb��x��C�/�l(’iU�:�\_+�1��}�i�&wQu�\\f!�H�/��8��4B�nE ��W|\\V.=��rOk�����s �?Z�/={coUp�K����q<D ��q�Ft0�.�Tۯt��P=�b���5�6} \[����>�J��gi�LP�Z�3��<ŶR�dB��R�.L˩ �|T2ŴN���ݱť}L6${��K��?75�Ũg��c��|�2�\`뫄%�,^�f��U�MaḮ\*'�Fb�Ji6�9��RL��Mr=E��V�����@�B���f��d驊GH����V��h\]�{��YU����o��r�{u����g����ꏴP�-^� �� �R����U�jl���VM���,�2'橤�zh(D��S�k))�" <kZ�9�PU6D���2��ޮ��{��<�'v��͎�J���3HB��x��&~�z\_w�13�\\��:�+��E���h(ۀY3T��eK����~��Q N�-�+\[�M۾�\]aҠ&��Z���Gb�\]�W�2�sɛݸ��� %)��w��-�x��CB�;���U�Ԃi���'�H���{��\_�n�EI6�Ē����~��u˚8\*,�Nwq\]C ��u��aimo�E����\\���tj���5�O\\�r�y�P�Z��\\�p\_s~�m��o�@�Y\[ңe��^�� ��w�8�c|V��1������N\`\`iL���xdD��Nh��Rk{�e������/r�! \_�u���Y�>��\`y�%p��\\���\`��:6�hj7��|B}�d\[}��V�2��2M}'���e��K�� �e���b'Z��F6���y�u}��v%)�r������S��6J|jf;T@=Sl��;��B�\`�O^�ZQ\[� �F�k�Tя|۷S�i+���o���t �<�#9����bF��F�,%6E��\`�0�k���#X��o���qށ� .�I?� �Z�A��' qϼi��'{F'�C�#���=�����/���?���Ь�ct����O���A��SzId�/O<�\*|��~��4r��%X9�M�#��Q���z�\`�>���֜�Bb^RTp2kD�9Q�9�� c:��^��.Jxn<�g�c�z�:�F���cb�3��k�pR솿z$yn�.@͏ۥ��O�?)k���߯�V���-8�m6)�o�����EQ����IwG�L���J\]�pĿ�P�\\�+��?�A8h���H{�1��sJ�ɇ�~b�ɤ���+=b�6�A�Ǖ6Lΰ�W��&���o�f W)\_�/O�M�塽!�n�Ե�<Þc��Ŕ~ q4�8>d�>n9�����e�/��2=��-�6��,��f�~�ߨ�����#�G����ѫ��v�$�1��^�{�\*�y�§Z�mG�wm&qi�w�^��̥>h���Ī��g���ջ#��\*A����4�05�yr"j� 6�������:�S�\]o�m���oKf���Jz\_�4u�DD��@�p4V��P+�i�6�#S�\\ܾ��\\�x+1�WSR K0kE 8�7R��feQ0��̠r�+�<��s-��\]b��2��o@d���M��TeXJ�x\]���·|+x^��; j�0�Mt�^\_xN2�N���w���Y'����:l�͍h�)<�G�P���d4A�(Q��t �c��v s\*N���Cj\[6��D�M����h�D\_���$�?z�tzuU�#�" A���栾�j �)�\\�{��i�2(.��ivry��Ofn��5�b����JP�.�����Dv� �^WF�T�Z�E��\`���8�Σ�ԪH�Ӊ-��;%6B�(�o�a b�%\[� @qק�\_ߗ%��,t�q�fp��d=�\*d묮��ZA�\[5I�m�6����HU�������c���4�I��$9�}"\*m�T�\]����D1s'B�>)��x �C݁I����?��JI�2S&�ⷾ�JȋB%�TՐ�?�|�$��\*��\]����}��o�;�з1DfH��{A.=#��B��%jFS�O�,�\*o��r�а��?��P�9úЉ��yOO�T�6.�m�8e�)q;�4Ӂ�' X��7R�$���5u���s�����(n� 2z���x�L�b��0 &�M�s��!xxB����K�Rj������%�i�?���������>m��\`��E�Yң����b��?( ��t �Y��܃ݒ���\_Y�P�U���\["$�m/��A| F�%E�m�w�^�?���>����� Al�hcJ�R����p،�^�� d,;�\`o�֫�L\`�0>/q�,/���w�5�G����\]Ww�sv��?���$��&)U�B��A�?�=��s5�F���=Z 0���>4Iq��1�4�j\]5bN���'6wn��HF�H�����H�<�pC�֖{>ܞ��pJè�!cj���F�Uo��A�3�O �-q�po;�0��1��0)��Y�$���$ ��p��Z.�3�����0�\_���Y�2��ϦZ���s�����S3�\_?�p�=i b�GG��\\�X�qr PYR��&f��llZ��紁S�O�X�J�gLs�X«��b��Ҭ��H�#a�12��\*���8���=�|�\[�x�e�鿐��I@�EC������M��!�����Q��<2�iҟ ��N�@���EE�〳 �LA1|��"4��/6!�D���;NΑ-��$��%�҃P~��A�҆�!�j��Ȇ.��iWd��� PK/�~���$p��㥝�p��ߚlFɣq�a�ʡ�}8X����:1� bau~C 5.�� �\[��~O��2����~��'�c"D�mTUu0�+�ai�O7�f��ÌG�ܙls�@e�&�gϱ�^�������}�.�SK��Q �v��ę��@1��&�n�sT0D���Q| /ʒE��yCل ����Y�{�����&�o-Z�r�Υ5|�ݎ��u��u����hn?Fp�Y�fy,%���ܩ�e�ͽWd� �"�R�.�9�1LN@5�9��Mg92��6G���,R k<����G���NG�3�\[,�0�l�ň��WZ����I�l��rIZ\_�)5��\` ����:��� (����t���"Dz?���P�3|�A|؝�:�#�sk�ٺ4�K'܉�ո�BѪ�%���o��g�-Z��\]�����9y���\]��Y�#^�8� ��S4ylh=���ks݃��vI��f)�Mxt��n'd��B���y�$�^�J\] �\_�&d�  ������D�~A�M���z �O �t�|�ؾ�i�Bn�x�A\`|4bB8G\]k�"�@8ʝ�\]c���m6�x�뢃�g���^+�����^��6r��NjІ���h�u��J�n@��J�\`��' R@����> Kw��֫AS��Qr���=�� �:�����I+넛�b�������!��$ABM�X\]I��X�r sN�hN1�}{6y���3���R�4�� �I��F�8G<,��=�5�A�\\��n/��r���T^?��MgR,6.��v�F��4�����J����� uu���UC�Q�Ų��������ԧX>��� "��I���mRf�"��W�ʨ¥F/ �W��k�Sx}�ӈawh9g�3��{i�t�a~����HLoԐ�p��������� 46�=C�i���P^iqit�+�'������|�րZ�x���R� }+�˘�%E^�s�bb-�o4F��d��r�!66�Am��4����#W�e� S��Xz x!\_=4�h׬٦Jx~�ϲîK k,~5Je˲��0�0�pn���0��:�O�}�\[����-���K��+���n�������I�\\�f�#���}���/�叠�Bf&��As��ik6��.�p���x�D�6�k�@ /R\]���$\]��$�����!s�#�.�\[Z�e�p+�F0{��:{�+��a����h8ư%�����—\_v��ce��N{�#�j\]�zLz�w�X���E^!�= o�5�5�I4��3cIP̓�. R�ro$���\`��廱�٣��݆�{8S��'��� ����~�j������oYJ�X�4�O��f������J��N�G!.6>�Ƞ1�g�no�\]cЃn$^N�{D�N������Hg"��h���\\l4�ݲ�E�7D��g�(+���I/�|�wb4�&{��X��Ӽ�>b�� �c~W�R��Z�T�>�̛%��Pf���m��V��L���dͦ��Ce���o��;+�� �2�\\��ٙ����I'%@��SN����\`��eqޔ���\]�GS ~\`՛rU�z��q���9�ͭ��5@?�����٬o�qb �\`�/R��U"(Q-S�xD'ۊJ�sCb�v�48�����tM��� SL��Hyl���&���T�-t�&\[��\[��H�0���Ѽ�\[���!/~D��#��F\_S���3�7��7qR1����Rj���G�LDS���UH�H4�kQGLH��ūM}���BS �8��&J��žhN"��=��C��x��S 1\\6��Og�,Z0�v�c���W !kx,��n�\`�ٝ���u�9Y��:�zdS�� 1A�&����%�����\`�ګ@̧�D4��K�֌�|�Ձ ���UB�b !�K�� a0KՐ�\_�(ՠ�V5�KA�&� ������Br�o���4򆢟�ڹ4�\*���V�(? �%j�کؑ�Б!�Q�߾�� P����Q0:�)��N�D��T�>6-�"�f �#�����ػ�<͑\]X����D���&���y:ԢO)�2���+KV@��Uy:����p�^��������K�!b�L��b}(�(����GE!v�φ��(�̷�E(����S���x#��/,�+�&��SX�ԙ��m1���a\_��b$��LS�^P�x����ɞ Y������͜W\[�!!9EP��6c��e����7F�m�S���&�4�ŝ�-� F�޲w���E�ϸ��Ugf�Cj�3���O�ڞ�g}p��b{�z�F�?�qG)�N\_��h�e�?1����1�wvs�0�׉�^o���9C��A.��@�|fa�� Ȥ� �"�I1m��HF��0�h�|��� ��v�EE��#��q肄O@�wc$� �H�B�@"� J�'-�N���䴻�GMa3���������uԯ)��Y�T���!��l��Z��ໃ��Rlw�EŔ!�������|�?p��{ "7��Ƃ������v/kXg��29� ZY�J�f�>��ߛ�N+e���c�c'u�'2 ����X���,����Pd�c�Rx��W�z�&�PU�,��9�W~���@��c-R�V�Q���VS6�E���a�2~�+0.EyG��iƣ��F�������p-� �W�����f���f�v)��$��e�����P�.kg:{Pڬ(��߀1U?�'�ϼp^z�!�ClL"�f�x�j�Y��'�)2��h�^�m(X�=๼��W�������td���䮊o\*;���O2HE� p����c�R/�F���2\*d���.1 �'�Ўr�\[��! ��LA�ߙD\[�� Q�<�WL�qH�� �,����\_�D!�=����g�P�2�j���\*�\`╬�XD��ҙc��)Ri5�����'�\[��������|~V��p/�ywW�nԢ���k0\]Q}\*'�҉�o���fK?n�pl�t�o Z,0O�$����l�~(��RD�+�P#�2"!����� ���G�� )�� ���,�l�L5zE�ә!�$�\[�.m@����2�,i��f+�9:8�l;��c5�.(1���1�� �0����\_��l�ݳ�bš�\[ ������̾����ѥ�ܴ�)��\[; �P�)A|Mɜ�V�;ٯQE���3Lݱ�La �q<2��\[\`�Э��}7w�0U �Kv,�~��?,'\`M������^�en r�.�P�;�l���1��&�O6��%�q�\`N��H���OX(���z�f��;0l��� �}J� ��s��?b#1�~�|NED�DJC��%���y:^�|���2�ɲ���f���a����\`�&X.O�)�\*ɉ�rȘ��?/��;� 0�\[�V�( �5����Bw��"ae $��i7�N�J�>�c� ��zL1D��ȟ�·F�{R 3W)��J�-OB8r}����6F�v�Ct�������6��J,X5���Eq��~� ���ˇ"��96��D�S��Ё0��'-87g�N�)���m(ےH�E���L�@^�n& /\*��Mwd� �^�-.^�I���"r��7s1O�7�=0�Z��0���6bbX�� �hp�n���HL�Cs��o\[��������;�p N�ߟ�R����ǔض{&� �����jA{����ͺr��{�-��b/R��K�>Q  ��!i\*������ 4�s��c�H� ɳ���&���/��VCAP ����:y�P��\] �����m�"�8mE�B���پ1��e�#�0׶� �ȣ����&�~������U������)��^i�tj!�Y$̷|��j���+?3 �LY\*�%����-w��xЖ���;}�C1)�/PĹ�<�Ô�7���{��P�U�4�d����+��� a ̚~p SMXe�<�:��敉�\\+�}8 ߱\_i��-'Ký��3ڐy-k��+-�x���Zծ��L=� 7<�/ M�za�X��{�-�����MwB���(m�T��T�\*�=0V��Y�F��xr���s��9JdYTt>�S볘g�i����� 3^�{LÁ�4I '���&�08�:3\\��%l�Gc%6�9�ju/$�}ī���3ȹ��~�9ğI�p���i���Ja}���;�N?X����֒!tO�FL�lyI�P#$O0q�Q�W�>ߌ�VS �GWZ((2��B�\_��,�%I |Vc���qu�1\[\_���ʭ(W��ց��2(\`�^�zlOd��-N&�>dF�I,{���u�ry��^��ښR��E�W�ё� ?y�m���2�GA��4O\]<��-R��>�߯ b��h�1�D�h�\_z΄GO��Ҕ��LB���y��gkU�H��Λ�P��W�5�K򭀜��tI� ��� kȒ�yX<�\[f2�f�Vȷ���MҰ�I�\`�GFj�W��9� $���}58 ��� �w� gs��P�nm��˼��y�ա��^z�(� ��&���vJ���b+��ĥƣ�G�x� 7� �'T+U�>�\*����i,�\*�U�|��ugNY�,b��#x����4Es�ͭ���a@� =���~���4���ʛikQW��U����z�nj &�'�������XE��ƭ���{Ӣ͏�2��yJm�k� !;�+�V���;�F���$�k'�n,G�t�4�Y�řYq<Ρ���V��ٔ$��$:��"����h���q\`�\`�t���Y>;���(���P/ ����9o��; 䠞�'�)s��R�y,M�����\*�y�6#Ԕ�Hb�E���i~��t�s�&�>��Yb?��\[J��K>��1C%��ӌ�%�O��ș�G���DP����==�ٜ#cn\\D�A�� Lb �mӸ�\`�A�^X6|L׈�4̋q�|�H ��b��>����u\*��dG��ݼ�7�����뢮���1���5�G؛�!i�Ǒ�ή�� {a2�\\8��1�T23HV؆j���\`RwH2{�l�� !\],|#^��P�ŧm�|�a-�rf�������}��Pȯ��lz��fQ�'Aթq�ʝ���f�\`���:�R�B��|-g yhOR=A����ZY��ڛIW����/I����oi7���� Du\*��H����P�e8�JH�J�(\`N��M�V%!{t�Vέ��s��M�&��^�bl�%Qu���}3\\��Z)��7�oM �޵�R1N7�w���+��q�?�U�#�/ Rd'�\*�M�%'F��>���^w}k�0S\`?U�BA�\].�{�\]�2�Jyٔ'���������|cVE/S�M�im�@bs�k�E�o).w���x��'�܂A�KDe d{Q~V�P,R���D�ݾ�\*���f��)�\[\*�����E1��GQj��h�Wcw�Vxt ��qO� 0^���q���MkW�Qnj�h\\�T��j�q ��8\[��v�,�y���H@1��y���1�A��H�g���e� ~gUz�<���5����NN���ڻ�7\]f/O�V���hr�%���o�,i}�(Б�+�xM\_�\_{oAZ�A�gDA�/�rz��Q�{(��d���p<Vl���j�Q@�;�r�(i��NSyz���Qm�\[�)�\` M�K�,\]厓tˎi���fထ �f��\`Jf��u\[)��\[tR��x�o��; UԢ�l����w�A��A-���eݬ�d�c����\\����ł�����B��ci�7q������8�w� �T��h#�g� �܎�U/�~ �K\]��C�8�g��w5ָ�XKs�j�׎���<7Zlxi"z�} oz� ����?��^���%����YT�Ä�(�XA8P�M��^F�B�N6R�=����e�E�P e���Z1�������|�7(#�E�l&���YV1��<�l��:7@Ou{��р�;��EW�(�����l-�����H-ى��i�풦D\`}�,Kg��m������?�1ˍ�WA:k �M�埽�6!���p~� 5���r��o�Җ�FU��%�lآ\]��\[&�J1U�w�Q\\S�UQ�U�3-\`;�E���)���ش���V�FFн�v����ԡ ��\]�y%�e�9�q��2d�ӿ>d(6���3B�R�����m��컟\]C���\[�W ,��,V Ռșzf��9�E�p-1׵�%�ң�f�\\F���D�L���\`�Ƚ̀(�\]v\\a��Pʸ7l��wnLr>��TqAI�ny�͉�G�vu\`\\"�گ=L�L�)!+j^�Y�䬹�����\*.�D�M��\_�c�>��=���p���4�i~p�H�O�'���l#ӓ�u��ߢ�o%��<ӡ\\�6+&Oy^��w�����f�K?���Y�#<|�j&�%�,":�D�'�&Gu�E\[�x:}�����R�R�kNm�t�#�(�gy�}O1i�w�����1�"\\ B� ǧBG�������4x�d�Y�;\_�10�$@\`VU\`#y�ܟ�Xl�h�{���9�nI�(���+�<� @�� 5ɩ��Њ��@HCs�ҳ�s��/�+�{ݲ�������M��:l@ ����A���K�K0t�\`��ߣQe�b�}��?� ���b>��)�?�\_a�u�o'�,J��A�����3'�!�Շ�U�7�o��� ���dfL��\`����S�ɘn1���D�i������:�԰R�V��Y���k���I�T1pa��i����������SG�����hn���n.>���o�=��b�c#^O�c�th�����ih i�����r$ĝ�f���Z� �{������o���� ��(�A:T���HF�^�n��u�ctp�4l�5qA���.�&"��'��Q��.nznC �Bo|9�Fq���H��D�������7�����8�w :�,l '�2Ix?c@4����:x?���&�܆{��֟�l��>'���� Q0D�Zg����@��k�ߋ \]虄w2��-��)���L W���}s4��Fk�0� \\�\\5� �������Ql��S�z:��G���hv���k��q;�.g% �i�C�u�5P�a1��\[�������u:�\`���J�Nq�o��hl�ζ5�&���w� �K�(T����\`��ԿL�A�IhFl�%y\['�e�uV�'^,����hS\[�� �����{!�m���Y-(哈���|͈�P&13�~��A�'��$^�����'������M����6){�6Ϳ3� �6\*Ї��o���c\\w٦@1B�\]�6t2.�ㇳ�)����⼗ͺ�U�-�V cݗNL@2}e.!�l�B��㾘N����1{.�0��M��I =K��%U��H�5�a��z�PC.%�ǮXS����o0w-f~)K�7���2H�?��Tb�u}�d^�-�?)3��\`>v�>Ѫ�3mM�b��m�S6F\](�%坭�{s�1-s��P�x�\`�J�V"ra\`����Q���x��0� 4�(�礙A��CC�?3?�u߰\_݌�΢� �\[4�0|$LW�h������c��0��E���ILN�ܭ���"��f:���zB�����֔r�O���"���!x��'��;~㈡�J�a��\[:��d����\`G��/�,��$=���>w��BF�����W1�E�ȋ�Lm��h�\*�^q��������o��(���r�j��0���z���x\`^/����o��·5�Z#�@�z�|1�(�U�ʩ��ٍ�;�9) ��q��3���w�nu!eL��� ���K����>��ޙ}"f���i���f�m�Fcd1�YH�J�h���䰖e,�y�2ZcM d�mC���W��+{W�����ښ�����%�ѡ�CD0?���y�C�BL1ވ<�Z��D������6��ij�� ��������BET�AX�l��~��Ub Ҡz\*��-­ni�m��?k�ؕ�Ғ����"q�4ʍ�p�a�� ��w��)�jA��y�!C�C�lt&G�i�����A��P��{l�:���Y���VJ�: M ���\`�\[F.��L��v��Yh�/c�E|�W|�+�q6?#k��9��Xd�@��%��\[;�p���Ȑt�:�c��;����Uss��������ä7J��$eO�5��E��8��)� @��c���C7Ǭ�����+����4!{:��ɦ�� E�'���v�������4��>o/a'p߈\[<�'P=i�vg$�{ڜ-4�S���c��+�TZ)�dV���6ۄ~�ɝJ�h��?�\]��2��&,J�Pm )����+{TF��U���5�٦�v^q�)o�u� ���1��<��&z�~�S р���D:򺲅� "\]��$��ԫ��ִ�L���c����� \_r7bp�=�}��-�E�vb���ޫ��|{�C1ws��H0��uj>����}ȿ}����{���E21��,^��/��%W�ړ��K��9��R�(Wrٯ�H�ݐ,�#���©��L:^t������\*�m�ѭ,�m�1\`Jс�)��+��(����'�����4v&5��:�\`4o W��r�f����}�>&S�\_6UY�>c�)�������Y�2��c{�\`���}�8\]e���B�3;�ϰx���(�u4z ܴ��q�K�钝7�:O.� Ӻy6����ھ��+6��'��֭�WZ�h�~�Z-���|�Yka�k�8��\*T\[�J���c���jLv{�!�PMf�N�𩞈�\[�%�퓞ט�<�闿4��Ho��:9ln�����B ��0���� ����V=�-n-�M��R���$�.�%K�/��XEW�i�\`ՙ����J,a�}z ��6d� �� ȭwQ��j,�3��q� hpi� К4�#-�p�^X'�X|˽�C��h����� ��Lƫ�םP�h��h��{�z9>�.��\_�+��+�� ܨRӬ-\\�F�\[dc�)bG���D���k���I�GȆ�����v��p�����F�j@ �#5n���V�2����\[O����D\[9������Ϊ�ﲙ5�>��)���$U�$@/$��w<��7�6����Dmx��{IUEs��hC6�u>���N�F��r������~�3x;f딳�<�� ���T����z��O�ӒW�H��o�U�|!� �ɓ�֡��;O�l�T���\\!��������1t�9�'�gF8 |0�g 񢪥���Q �p~���k��Wg��B\*�l>�E���#���.Lb��cZR� �����z����p�8�i�Ko\[j��w�}�S�Ԥ&�q+\]��>�����P��Fr�1v\_1��ֆ\`�'��gR��e=�M�\[�R�$r�:w�݉� ��Cb�\[�Ӵi�  6�u2�\[��hpSe����T����{�z m�;���ގ����隣~�H��2��"�1ω��'�1�c��Gi��%x�k 徶Y+�>�-\]�f6�a�Rf=r�ӝ�Z����2��n��N�%m�)�j�P�%�O�iH~���ڑ�G����0�hH�������x�̩��q�&����;�� ���la6�N��4ɢ�U��Rp�f���~��&�ui�Z�Ϫ���K������J�4�6�4����78D� 6,�:b��?�Ԏ�Q0�\[��E;� ���5zy>�\`9:�ռJ��Iff���mƽ��\_���Y�5�؛�\_� �u��Fp��~}��DH�� N�\_���-$~�ӗc�ڵ�Oe� ��#F>>.\\����դ�V{F�j�����~ �� 9r9u,\[�S��/����!|�n�F\_Z��r�Y���pwi\[ȝ~�K�-�� (F�d����\_QL���T���������o\_>0"���0��$T���7�I�Gҙ�҄�|�\_�\\�Ec����)�Syl.(pS�B��& ��f�i�1Al����\]޼�����o>���������ϊ@������\]-�5 �\[���JB�vT��wcz�%:�w���۰E��0VXc{e|c�Ӷ'^v)�\\u�˞��B�Ә�ɴG��'���y�� 0���V �u@x��/:<�B�w���:�����n�\`��� U��A\`IE-\]�#J����+n�� 0-��:LM@����-��O�>���'�p#g�5�FC� k�a�W��\_�ۇג4\*B���j��0��t@ �RUh-��7=^� ?%���ԡ9f����OB��v(����\[� ��q�w"} ͩ�Fȶ��Ճ��/�k�s��:Pm���4u9ٰ�����0���O�K��9����0���� dq \*���\]&&bxE��.�)��tgG�X\_�<:l�o��Z��/v���11U ��-U�8�� J���lf�ʈ��r9H��}�y����uΗ�ߨ��ly(�t�-�:z���SE���x����e�Zr\*'�aچ��l�fװ @Dh���?�yhE :��O�m��QH��yk\*\\8j9�UP��wT��������2w�l �쥣G�9���˗\[���~��gos�lV�\_�U��e�H�5X��<ѐ'�i�m�-w�� ��s�@��t��}"��.������������/.�#;���m��)d f��ǀ����}��\`�8yK�i���v����zhݻ)�L�a�m\*���xi�������u8�� nB�׌�?BL�͵���x�et�ޭ�/ u��wK���΂�=\*�e��~�����G��4�o�#��L�Uw� N��ﶰ�fR��,E�Hes�66^M���̽�Z��?0LzJ�P'A�����Z$��;e;z\`@+���\_� X�~;t��+�ӂ��?�=ԓTc��h�֏į�(�$h ��aU��g�5���\_���<\\�o�˦��N�ҷ=��l\]��u�~�����6���K��qyE�F���Z E�5��є�O����=5T~���Joͥӝ��-�X�3��zh rC�-��!!�ל�)(j����-�L8а��0"g�'�t�f�i �I�m�b�n.��S��GFթg\_r��s�-X!i@�eaS��:v�J~�����K�nD��rEB�-�=� �I�F\_��\[B.�9= ��5�T湇S��T��8m6�K�R���)��A ���cd?��I�mT�<Q7y�L�kӷ��#�Qg���L>�d���FI�V�Z��s�%O�Z�c��c\]v��\\�\`s��乱l���%�;R�PK����:�0�ͥ��X�C��U�$л\]���z��c�ͷ��q���xh7^C�t��Hv��l�U���I"\*�&��>�,�כL2B#\[C���.C.������cϴDt���QƮH}�C!�kp�m���G-����0\*!q��5L���+�����vO����-mϟ��Ϻ���� ���(��m�����.�c{��EFn���;؍�9�WUʕ(�\]����,��,RT�~��\\��IN���S�s1��c��)�EA�9�a?#�R�ý�"���@�/� ���K���&P>\_�&L�z�h�wj#�P���">�G��X'�Y�p�-�f��eX��;d.� A�Dh�b�d���b\[�hɝ#�W�9ɤ �e��鹪�5"�/�ez�xn\]�aX� \`���P�L+ tc�Dž\]�5��KO�7���#\_/88�sBP����5�W� �:�G�Pgn�!⫺W�ԡ�3��� R䓬"��VH� �p��o��S�<�uhߕ������UeL��nZb�wu�>��O,��E#ήOD��5z�X+���!������U��i�z-L��XSf̱\`���q�&�7L��#�1�e8-�O�|���\*�\\� B9;��0;+��:�����M ����� �1�:����Q>����;��$ɉ��o������ё�\_\_�e�;9"\\)WZ�E��#ۍ?�'�yAz�(\_(|-y||�i�h\`�?�6h�Njɭ� �ē��l�F�5�?ʇ7�N��/��s�B�1\`h2�T���w�($�>�,��Y2X�l��.H�;ƴ��#\`,�\[����!ǠV3r�L�uHy�n0��(~9a������wt�-xnJ��g���c��.l���U�T��ъ�����L���9{4�W oy&�:oMW\_%�J�f�?~�q0&P��L�1،�?���8�p�~��x R���cγ�"�f>�v��zY΀�\_x��-�q�i~��#�zM�����>� h��CFֵ�L���.�\_����� ��\_���΁��J4B��:�O��K>\[��@T�d/���aI�~�f\]6g> � �{�� �3��U��t�c��Yt�B�Ѱ�ס/�V;a���͗�Tδv��k��� ���7�\*�Ü�"{7��>�/�K7�k��?=��C��w��g������s�B���t r\\��(�A���tC�s����d�{�v���>H��~�O���PV+P\\ph�4k+�rE\`ۤ���\]�n�Q�l^��%Y�\\ �"f4���7�l��źP��u�we�\`(6��d|�{n���os�3�!�cr�/�2N��ラ�w��G���BM�RY�-�K���6��y �e�-B���\\8}6tʿn������s\\��,4oe ��.�\[�M\\-�P��Z���e�D5� 8���~Ս�{�wA�h) �9���:�ڢ1I�Mrٽl�'j���t���M�U�k��>�� ����� g��g\`�-��Ь����e�\]羝�T�dq �WE�>#�˭/ZN��c�=sQ��>��g��0���zg�y��MB��(�ex�ܙ���ɉ�2�\[p=ff�~����00��Év��ު3|\]+=��|sS '���3;����� �����-�>���r������Q��2���w�e���݈��dq�{mu����\`��sp���\[H ���>�Zh ��>+.���k��kPKP+�\[|Ϥ��f4V;�5��a2�ߋJ�Ai�aW.'ك\*�k�wn�zֈ$���p�����M�������7�ѳ��?+j�Dgai�3�V k��(\`�CBD���K.����)3)<��o�d\]>�G�x�ϩ�� \\ϡ �"Z��y�G%'$ ^VO�x�?\]Tfi䥔,�u���9�+Uy\_�e3�.��1��$ ^n���3MX/�@�!�YO�0Q�ڐ�r��D��|��9/��Cr��%P�rK����������Sd(r��p���Z9����3Ewy{@�M������.��#�M�/Ŷ�\_!�\]�>��Ku���Z!1��ɑ�c=�������� @�K�|�u R����� DŽ�� ��-^r��'���j\]ȝ2�B7���R�1�3��d2�ַ�wo�J�'\\1/��� ֦���ipt�Yd����k�2�Kc�J x�h �����7?ZFx���Z��!D��j��&}�M���M=v��2y�� }��ܸ\*�~��ȉh����7�Ey�i��tMy�Z���}K��xF�Y��s(����>�\]���Sj+-?+n�S���"��\*��� �"Q�6�p��L��P;1,I�.�����?�̂I����R�Pm�����:�-��\\ԍv��4Jzy�9����K|h8�P�7 �l? �3��;t�C��x ���N&\[& @�:�D0��\`�j�LRZ�:��~��a���)��� ���6�1\*�8 ��8�D1�e�ņ{�Q�>@�ϕÕ�����\\�l��s�V��VDZ �� �z$�&�Y&��z�6^q�%/\`�I�e���5�B/P#�~����5�g^6�S�~{w�;'Xq�|n'��<'}�chKҏ�N���qUoR��D�� \[�!�&���+�B� �1�A�1�W{�Ո@IM՞4�'��\\ue&I\`‚R0�k�:�{n��/j#�\[�� M�%ǝ�6�B��K�&̶�$|^��3�� !�?�\*c�\]"��۠�"�#%��K�սb�����s� SNJ�MIH}o���D��Z6&q�I=O\*҅�!2gEX�W ��%� �l�L�Q?�m4��#���~dg6��.��@D��(�c����尛�(�@U\]�B�M�������:�km��n��� �\[ �f��L��K�7�# ��H���r�� ����� �u����@O׵\[B���o�P\]�0 �����JB:K ���jWQ�3�("bG��e ���g�x\*�\*�=�xB!eb\\�al٠�G�)�ݲ�b�U������Yb�8��$��?�x���l�E{��/swP"�v�TQ�\`�+c(� V˴/kv��<���^��� �b�M�im@3/��}l,�9� zM��֝K�Є����J�7J �+� \*$�CW�#:h�\*q:6���\\�#±tT�~8�v"�/}���X����\[ �IZ54���찭3QE�:/��n���©��~b�����q���P��(��W�/)�\]a�3qC�nF�Jb����fR���K%���t�fTl-�Es2�#A�j�F}YB�xrx�L���sڎ=@�mP�=�wT�Dڡh�H�#+W�p�j�����L���- W��������/\\-r����O��M4��:�����x����K�;��߾N�;����N5��zp�()@��9RU�� U6=Y����S�/�U6�\`�����5RK�K\*s8}\`���Wʹ����)!+@\`W�y�)��o�p��҆4.����9��e�#bS�x"�R)ǔܳb�\\8��Z�R�lߖmY�,��7��1���f{��P�RhZZ>��;�7Yi '$�������j?��\_�/ٕP~!�����h�� Yw�� \`��CV fx�q����^�L#�b-�d�DI%N (~l����W�͠�kn^�סf> h�Ps7"�8\]8b9�O�\]�?�4OGIPW ���NN���eA���Qm,�F+�4o��L�X�2~@$1��q�D�+f#J�!�%��"\\��q�U�i0^\`���F���}GI�� � �}\]��9�eR������2�V�u5j�7b�H3�6+J���e�SCp�cz}��\_q򻶵3�|6�� �P W������X��K�?-b�����R|\_3�5y��Y��̄��jM6#l\*��{T�h1OMZxgX����\*�?�&\_m�v/����y<�zH��� ��Nk�k:��H�S�6�u��\\B^��L�1W�1�I��>���sC Z��9����h�$�\[+8�� ����0�J����+GR��\]��G%ニgB�yY�w��=V�'�c�\`1����qa�^K\*��k2��������z�� �AfW�ݰ��da��&L�n���U��\`>���x��b�üg�d�G"Nl���h�A�3\* ���n�S�4���bY�$�7�6��#�<�N���=�O�7���) �$��U�V��\]���3���-�!I�OV�.0:�u&tu���\`B����u���q,@Y�Rg����t蛀 &�ks,p~�P|N���t�O�&"Z�! �~��c �x&q��)����Dé�r��i\]侁�4q\]��Y�-�c�y+!kG��|�AG�H�Ƨ����})��w���$Oʤ���"��Bc��"���-���3sa�&8�Ć�>m��zrKD1�% �fqRY�����T&��1H���m�3W����w衒벺�z ��7#k�P2v��jT�kb͑u��m�3��.B�� ?�q8r?w1�\]k\]� �� �<�4 ����Oɹu\*�Y����P6b����s�P���ڲ�3�U�Y��!�|��R��㌇E+��$��y���w&u���y0� 'C�C3�1\\\_�hʶx����(���h�h;&'��UM}5\[��R��� C���X��v "Kw�%��;17����-�s��,O�UA��sŻd���)(��8�j{HN�����t���ed�nI�R\[�<��䮲�<��{?�W�a��&6���c��Qh�nr@׾&�m&��b��Ui�m�}�����e�T��04|p�R���&ƌ\[1lƆ��@�Uڑ�����) P�i��h%�%��p��c��7.�^oL���h\`}'�F�� i#��yù�\]oRE�c���' �1��l�+�� ��'ёe�~U�,�\_��pʹjM��x�&�B� �����JgQe�6�|\\@ ����١�àG��EzLB���P&�3��N�o��&e$���(?�"=��>�I�L;F#w�^�v�qiB�>m�5��:�\[.��VLDf0��2��vR�Ι�G����;\] �M��J�&Y\\H�z���r8U�O?:� ��s|���.�JՔ��S�����w���޾3ԺOj��;�ܯ�0L��c�ŗ��\[rz��U���p:.!�O�>��ż+(�k����4��+l��l?��󁥷��a���CC��" ��#Y��6�������4B5J#�\\k�8X���gvw�����&����餞����".����/tI��8z���8o�7�6�����=�$d��>\*F���~E����)t�3(�3�\[:�\\�y+�LT{��Q�(�/�,�ru��T����� D���6�~X�\]�zI�7��F\_�����aan�� ��K�i�d�e��T��(ێ�m��hFd��� ��R�O�\_�P�n��� aa����9e�Y��=P;�֨ķ'�DzpS��;o�����yN�34��upe:��k��M\\�S1� �{ �\[����gq5-B���,D�ؐ�Ϟs��ʞ ��~,��2��ĩq��VJkܨo���<�B��s\\�,)W��#�>���E wծ ���em��7lY��Dz"'�G�-{&pF(t�R� ��+N4Z����jX\_���4�@i�(���<�i h�@�YO \\8�SC'���yg;M��U� ���45!��u?,E&C�7��jh�;�%�r�(pט@��-���\*����p&���Y"����EG�T�3��II��6\]��� �uOG���℃�2���C�X�BJ���'\]��#����ݯg��y!I'@���4�AS9r�Н.7\_+�vi FѨ�\`��jG ��m<�=�sG-���\_��^D1\`z:��Y�=z���Fۄ��DIЏ�s��B�n����A|l�/�G| �z�+צd��̃��v,\]i̷au����l�!�\`}AV�K�؃ �v�\[�ޟ ��Q�t����@�m��H{X��Ԡ�C��: ���4E������"��c9�Q�G��K���;�䞪�����'7��Vf�d#���@�NA8�uR&t���էФ�LC���\\��qڏ22�V\`��Q \`05�d�E�4�?Ki�W4ߟ�\*���EP����2�B��pmU�\[ht��c�W���n�������ĵ-�V�Gȳ�p�?VԎzH��2���r�;��S�|�E��(�%����Sȹ�(ӵ&F ��z�����q��XG�Um��!�6 ���&�)�œ�$H��:���녏���f/�e�����k,�#��Ap����px��0;qO4}!��;+�F�:{7���m�M��y�Z���V�>��WCj�x���މ(F� ��J�Xx�^r-�بE,";�p�H׆z���\\ �g���\\�DR�e�m�" �Q�i��������xI�������c�31�=���q0����l+mV�dfp���z�v��/���UVt��\_�������/� ��K\\��ͤ��D1ʚz�Ѩ L��9���en�sH�7��TCRP�v|������vj��bL���\]�qn�҈��P�c.�I�����˘��H%޼���ܢk,��B�ㆷ��z�9�~��R;N>?��L~��� ����ᅁ�-�\_r��l����}Y��T-��� �����-=9�����Y���fC����V�59�N�6 �X<�:\`1,v ���\_�h��Q"�\]�����1+��f{\*S#�������s�2<�H ¯p�l=��S\[/���r�F#��\_�-��2tx+\_������7��wx�!���AtV� �x���ĔI�L�,���Յ�� 3��q)�{�cD�\\\*J9���J8���n��-bĆ�^ 1�j� �7�@z�hΏa� ޳\\#&\_������s/���"�""�P��ɋ���q�sXxm����� �'C�\_�(�Fey�Y7��p��2J�sJW��V�Q��P�쎋�o��ݖ�0�s�DG�A�D4a'��Z�z\\| ��@�f0�;��1�dсl�1}HQ=���J���KzF'���͝u��G>�O\\�o�����y���b�X�������dm����� W� V��o X����l��o�rO�P7��UGW�\\��b�%Ф��׉�H�\*ʆ����Ez\`jAf��b���y������� ��N�1��қ��r��DAd�ʌﵩrFI� 4��x>�r�6�s^��@S��DE�0H���O� �-��S\*�т�З2-u�TC�:5T.o^� ��#�Ǔޠ�a�@\`�;���l�Vǭ.��g�\]gH/?�n��,jr�gU���2�����84-Z$����H�\\Cx�~�\\rҬS�Ed'��Ѫ�F�m(\*CN�\]�|>jm�Ϫ��."��Ŗ��\\I7����������9Rh�i��Uvb�OՑ�}��.'cU���v��k�}�Ȼ�W�%ǩó�k����Nj�{��!^��6�R�݌��cz�\\�;��K �ƅ�{�p���T��Y#1/H� H\[9�朜~�F��V�ɵ�j�T4�0An�� )�qJܣ��Z�/ �|m~ W�>Ԯ�m�t��5ks����\_��L��^�4�R<�.H�#Fr���\_��:��ߴ �~�U�f���R���+�#lA\*�x��4C�T��M ܐ2u� �+�ڊy�SC�F¢\`�:6U ƛ$�����d+�����S� p5���;\\�q?�����#S�����mѽ��ϙ�F=�Gav�>�%��0�����\\�^�daȒ�e�;��������G���u��O�yR �\]��:��ŗ� ����dmǞ#2� ��9:V�wj�!��K-r���\\q\_��\*�������ZjcgSqŭ���I\]Bq�\]&���J/q��O\*%�}�a��/�{���ys�\`�׍��8N9y�O��㻚v�3����740j��+\_S����n�v�@r|�GV-����4���w#�@��+���Z�G�m�UgR��\\9 E��p^�>uB䊸ѸL�C"Ҿ\_�26N�0�{h� \`fR� �(�:Y��,j�GxN�j{�i��6^� � #dJ�j�-L/�WE.W��P�Ƴi �����ޞ)��q�;��s�1�����~lADmX�9v�V� ���k8��,K���p����T0ĭV�=���D�B�S~1nX�g��O�r�„��'�� �ȇ��/��ԑ���~ ����u���1�k�2xx��O��F��<C�nk��ֶl��w�����!� �폜�����.OjCj?�Y�v����t0�!�—ZFlz��|��5R8��̫IBX2����aI��k>L|���T��J�ɚn��Ѡ��������xQ;+4k\`!֯����<��|�v?s��ST�j�B�p��R����R��nl��aQS.Pd�\_T0ks��:�t��V��(��Q��Lzݥ�:���Se��@=����S�y&\*ցhM��Xnt����,v��ĥ/�s$�z�n���� s%�=��.A�� m��\*��;��p&@@".P��E33��F;�A���7�i���\]�Co{���.�M4-΢�\\\\\]��r�p5��\_�N�#s��V$Ӵ5T��<��d����@�$��5��n#�xA�ǀ(�)K ���Q��,qd�"�������� ���A.�u>OA�Oڶ�ͱ���/��\]��J6Bܻ����C!z��\_szH�yc����,e ��'��Rڟ��u�bA���R����iC&!��5�Vţ��οNn�-��,�9ܝhD�\*j�iW��%Q}�+^�bեN�s��:��Q6��hat�� =����Q+'��� �����Y>\]��N�jH��&Ru|����Y�iz����E\]8\\X}��A�w\]�+�r}���m��.{ŜNu�A4�W%��#��gE�P}b�\[��5c�R�d�l�R��2'�}X~��H� ��–d�������3�c«���p9w���� ��=�:�����ǃ��P/(H�<&�Z�� B3��j�ӎHyh��V���"Ƥ�>J�!3��2�\]�㌋�Y�i�NJ�M�py������&p��\]�����B��S8�I��\*^-������~�T'2�: ��E�qL"�s5��uq�4���8��J�3���G��pN��}V�\]��ڝfq�k�X(���K��| y���7��ad� ���7h'�\[=�ֲ ��w�� ��W ���Κ�N�^?z�v���Sw����јףl�|Wh���\*,\\��/��3�� s�Րm7���|��ո�E�\[�IH�C�Y�^�d�h�35"�ac�������j�����=n��!��)e�D|�b�r��\_r���Å�͙� �+\[gR����!ʉ�Z+�K�� @�9�F�,=$�!���ڹ�EƈMW�r���9 '9o�Tu������.��Ih�ԓ��v�Yls�߉(��b����� U�Pj-���:ȼ֠�Kg9�����vr�ʂ���l�v��$��\`{ ��1���2JE��  �ƅ'��SP���������$I���ez�4�v��&���\\���Cc�Ѐ�Y�xw�S �.;c^�m�>�g��1sl�=�B�}\*�Wa�>AD>�B��m\_� V�r�C�V�Q�$�0���쌅�\`ek''����mYZ'Nep\\��ɺ^�%����+ɒ)��9����~����p܇CT�iBF��&�z1����#�@�L��(�E���{��r�B���Cp6��\*\_�9���"R3b�#GY�=��Z�c�J��2T� -�\\\[��F};7��E�\]�,�oI���I/��B @rZ�U�D~!�t2\`�q�x�gO3��@���������=��ϗ ����%��:ܙ'ڃ%���R|��P�29�1w�4���u,P\[oV�9�|N4Cp� eFpD�=��#��<����e��٤�R7D��>7\[7ѓ7��\\7l����1��;��%S��v�iދ���:�Fy�� ��ϕ��P��p� �7,��5����#�Ir��,���Sͧ"e#�����r�z�d؋���a��÷0F��b���� �}k�����ֳ��=�;;�Ph�5��P�K� '���y��w8�ԩ�ky���M���׻��א�,� ��Pd���ۊ���O� W�� bW��ܙ��@}ڨx4�8���O���v�.��A��?�wn��?���Ȁ���T8�frv��8'X���R�5��Qzq����n��� ���\_U��F�=��~F�~��Yy��5\])�yؿu��N�d��|ٜ^6)?�,�LF۳�(�8\\�b�b�� � �B�\]�S��s~� �Ð��O�@���J\\.p&8~�U�@R��l��\]��y���e�;$�%����\[�306�62a��ԑ�=�/(2\*�J&r�4/Q��v��M��X�C���cn�����4�Cv0T����͠����w@i�)A�W�c�Y~�S�� �e��-)�\`�&��YTJ\`�:���B E9�L�R�jT��.�s�I�&�����cr���C�{a�X>�d� /a\[���\\�� �\[��p ��&JF���"-C\\�O��#Ǟ%�N� ��$��)�U����n2�e��@��|gOǺ�j���^�����Z�.�N�vf��np\]ؙSdy� �;\\v�n��բ lfB��6S��Yw����5� �C��٠�A����o�P��<�+�"���\\�\`�9���i�b�X�>,e���%\[��QNz�� ��$ ���s/g���\[�+��\[ok��!�&a\\�W:�D ؤ��J��4 ��9g�� P~z�^�U�$���34+�\_����Y2@�}<���x� s?ӯ�)W�;@(�+TY��ޠ�N�K��9O'�M�>L�}�c��U;QhZ� �Dd���nʠ�Rd� �Z���t����u� MWXa\_��\`�Ok|�Z �\\�z�ҽ�ƥH3�^ \*������6�<ҿ3 �\_��D\]� ��W{�&j�$U��\*�v����D3;�}���X���M�4-ݪ�@��I �V�NJ�F�=SZyr�\[g;h E����%/U�� fDϪv��B�-��Xk�7r�C�K�gG�H�-��4��j3PQM��j3���|S��u��y��^Z ,����t���֫��ړ� ؁�|7�w�yǭ=":ۗ���F\[0}�ݖ�ҥ��DX ���g�� �\`��l5��!O���씡n�j�D�,�7��&��f���W�I�\_�V�g^z��;qD4B�A芜��bN�(s�fx���K���m�� �U��4����˷C?~�ql��. �>� s��|�!t;Sc�$D�X�@�\[�bJ�DwY$S8���v�u \*�μgA'D��H%���P�@�\_Z>�'���(NvV��\[��p.�\[��kiP9 �b/�^��gOh�c�����-\*ӘLަ ���6�x�x�hg��/���P� ��E(�+޳8~�BnAӻ�5��\]��\_��\[��T\\��^�=-��V�20��������LdS)�����0� fC�i��� ������t��h��A�eWj��葦��B|F�q0l�C\\a+�5X+�b���J�Ǽ�I� \]�xo8����HqWH")��l�J@�b�B¢�#����4�Õ�i�;�|��'~4k��������f��m��g�\*�?\\E�M��������F:�-����ƛ����Z�����Ya�u�������:�j&�a�LnS/���\]��\`6�1πj��\_KW�Q��P+�5e���Z�W&�\_��Gū��\_��ٛ'bT��iLF�z�7��%hHy�\[�(3G-V�H��l ��q0���e������s �3����cV�GM�>-����z�i�\*��r����$i����/=���\[�����Ȧ�Jn�g-��C�Xm�g��1��F�x#���e�G�AA����Db8ʇѺͩ̐���Y��}��J� �jC�6���x@'����L���+�)2ȳ�kJ+�a䪛�|�f�D�������<-H����D��>I�:�F��/� �+\[�R��z��W)�Ez�>٩ђ���97��a7m�ժ�Cu���a�-��\_v��W�n��5լ5N�<����\*��>T�>�s|}�@$�O�>\_��3!���"фm\`��;B��+i$,EԲ$�F7�:/��n,�k��=�a��j�蔛�y�ɊՓ"�RT�Bn��a� ���7�?�(��'׊F��"363egY����� �1L�����b��qޔ�̻�BQr{����^�������e ����۔�= tM.��� e�.��x��>v��>�Ѽ�Y-�b(�=Ҳ���\[)���ɔը� �>;�R8ܯ�.��3�ZR��V�Όlbu�S>P�ܦ;�|��t��Wn���߿��0�� �&�i �6�Õni;�6"�%\\�n�0P(Q0r= l��1����R�?����\`�u�\`/�i����5j2n&�2����%����~ �dVNM��VN^s?w� Yם0�=������?+���K�A�Yټ���������&M �m2���(����R���M<�<ٍ���zH��c�>�����4��B���?�\\� �O@�I�o6/x��!\]I��X��� ����"�aJ\_���v��sMS+�V���4�)H=�w�\-4�ͷ2Q�(k���T9C��VHډd.����9T�� �~6+��Mǚ�v��y�֊������|Bka�Sջu3}3$�������ZZ~���Zz�����8y�GP�O5c�)5tE�h!�����א�� 2�b���C�G�2��&ù� ��wX��o�ghA�B�\]y��D�$��D�� �kܿ���7!�M�� M�䒽�sg� %\`�oR�4�V�^x��Sù#�ذ���{���٤2ݙ�2�2�o\` (�kV���A���ߑf�����O���hHt�F}ZL�4��<\[̹�>&R�^���uY��!#��w��� �#ctV�b����C�k�U�G�a�x����=����$��>�ͷF�\[FJ����e�Y w�QReyN�#M6.X� z���ɕeYց��+��M���Π��T@? �2k�D}�� �- �{N�.�8ƒ0+�O��T�.x���qv8X\`�$��-@�\`!�sA#��S���}�\]<�z�+raQX�/�\_l�����X �Aߵ��\`1�����$Ն�����\\ߐܧ�Q���upp��G�'�8�T��nn\]�����<����#45~����.�ӝ�#ӻj���y��i��iU�\`>��x�����(XA���V��%�Y�RD����T�N�UۂL�x���ig�{����^\_\\\[�qZ��Ӓ&v��^|���Btf��� $\\���9�RwcmNG�-�'�p�E�t$�<w���s 7���R��� ಽ@y�� ~ ˉH�gf�9�K�:�+�i��%����j%+�����3%�|�YAv�Z��)�E���u�Lb����,(�ԡi�5&��(LײN����(�����p� KdI��UH;St ���e4� ��p��Q�\*��d�T�Z�b��Vm}Ō9b��;�����~JA Wn�^&����F�1��K~zb��o>.��W�2ĩOϯ�QUG�4�3HMn����(/�%�7�^ԝ{G�-���'\_�Τ�L�m�d�,��2��&p%� ���nC|�\_EA99��1l�r(+!��=�VA\`kwR��>ؿ���N�1�M�5D��J�\\1�� �j��{����ohY�T�Ρ�x��h9���Y� ���6���@�\[�'Kܞ��p= ���D\]8�a�,}R\_}��p��O�S�^L/-Ť���N����F���lS9���q���d0�o�z/�����h1N���Y�F��K�J-�P����-m��v�R�|#D+�<��v�gРL>gU������M����^\]�-�\`c+��k�פև���/U/mߺp�'D�v��l�����:K�J�8�d�)�3�9~���4 �V�=�$�JDg���٨If:�\`�V����<�\]w��Ek>E.�@��Ú 0�!�w6\_է=;��-�r�4VFo��(��y����Y� ��\_ahoX����O�L�݀Dgch���9�y~��n�@���Ą�~U����l4 .�\\�!.� O�T�TA�� �oM0����j����:d��!ی��$��E�/��&�;�SK3�Tҭ"�I˕=P9B�\\د֤T��D���Ox�<�N�!\]��ͼ )�&{��%�}�ԣv7�����;�'�Ks�y�M�uT�a��o�2�\`�\[h���K��2��A�'��a��!������)��|K�\_M�F��� ��{aZ�\\�h����6\`� �e���d�Oo�+�4�&?mN�.c{�\]�����Eۭ|:���,��D3�e1�>�f�����n�5T�9�݅�#�nz�〣�:���IՏ�Oƚn��G��R�ȗ���LD�}�#acԊ�\\ ����J���mX�cv@�&����q!-�l��uT�"������c�7��TY>D����I���J��z�ş���"4�"r=��\`����8.+\]��wh�F�h0+���E�j�8{~�D����K��Wԁ�r�ړ1G���\_����˓�PNJ���� �{�+��0����\_�����Y���� ��n�%����)�GT!���J��NV��(��ʖ>.�'��ܒ{�� Y�� @aZ�Ҹ�v��C\*�S3��-�Ǽ�3���<� Mt���u��{�& �aWs�Tz\\H9R~Q3�h� \*���.��������+��O�:���थ�&=fLi�۷��Q%Q��N�f�/�(�Zn�����G>@ R����� \*v� ��<\*���x>.CDMk���C�\*�u !Ð\[z�����lb�k&\]�2���Erkĭb��\*�KM�i��@�S����j�Hl�F���pmc���2�\\����ۅ^�-kr��.���>8���%�a 0�Z%\_B��b�L�� tD� � ��f/}���L��\\�l�'�2�9������W�y�\`��2��8��hYcQ~�6�a>��7x��}A��ʞ\`�~>f�/�J����DF����؋p�r�����3x��F�?�aN�9�;����N�y��@㾚�5/(�����+��T�E/4,���OĜd�i� �� � ���M tnx^B���+�p��ҝ��/�/+�y�Z�E�g��U�Fڌ��� e�Lɾl~��")�~T��S+TL�\_k̖�׶���;$ͬ�gC��Pʡ���\`�\_���Y����e���H����¾\`8(�-~+����ꖡw�l~V�ЊN��0e����n�� �: �hJ��5���骠�9��MP�@ z�'q�gI,�i�N�A��>֘��3#�Gԙ=O2%V�"�m�r&)����:e�� S�����ݷ�;!�;���z����5�� e 4�F�AF��%�J-x����W�em�f�L �X}�s�/�v�=f�<�� Y�6LTI� v�@�? >�e;���v�y V6hU݅\*�)�0��K��Em:�2��A��)!�Ǹ�b����Q s <�L�3�\_��P���f�(v�g�#�#��#��V����{� ��XE�鲘2��a��WX��1���"ϥ�� �hZ9I=�������%�o�/'@�����c8�9F���o���m^xP.�lB��%������Q w�l����͞�}�o��m�Z��y{����ml��ڗ$��Hqu\*�d>\*%Hz�m/�.U==���N��4V UF��\\�<T�<�i��?���=�t�~��k�V�#LF(����C��� f��=���SS��������Pu�sKopGD��d�  � ���j���W5�L�Η��\]f}�l����|v�X��Pt����՗�h�-�S,�T��- ���P��i�ZF�=Q$��+���r�Gd$(��S�8�b hw���͸ǖ�hD���mQ�Y�T�/�1�.yp�<��ُ<l�-�\_�^"!U�#����X��x:TW���1>D��3�����{)���Z��Q���� T�t����E@5���J��A�X1hҿ+��G$kB�!ytp'�k�"� ~�2�$<�$\]\*QU�==� �\[���\`���i2��=����jw��H"Mi{��VT�,!����������6�\\�\[��\_ބ=��~(�r��m<�a^�p�p��Ͻ9�7/��+@}<�z������1L\\�td�T�ͼ�/d3�x�>x��Y���a� �h����ܼ&H�9ӛ���z�E�U|R�ZKH~i�����X�I�&\[�^�64ʠ� ^�|� ,�G�����9J��d����"-u ۤ�~� \*7(���c#�cO���mWB�0ń��\\f��e�G$ \*R�aN�׷>�2�� � �L�sA�� ��m�1 �P��ȇ����P��x�{��"\]V��L�\`@Mz��ƇRCB㐹ah,w��l��@ c��xp��\_S�gL�b˔���h-''\]��Mؔ���8 �4��\\��G��&.O�3�y�U��pM\`�7�Q�{�y��0��\`��i��\[g�vվ/�E„{�������� ��Oos��Xz��n�܌n'ȸ����Qt��⎺���W'���y&׃�L���G 0Ay����b$p�3>�@�:�������\\/hvP�\]����4�x���.�>��u�ލ��DH�E%P�V� M��De���'wtN����iq4r�%k����=�9�Mr$V���Ǎk��P�2Ԑ��$luI�\]�;�s�e�#�L��+U�!R��퉿��L9� 8���̷�f�a/���y�ދ���tD "�h��%2Gq\*h�\]�� ��b\[,{ <(��D���^w�0�d����6�� \_�֜�l���\]@���7�j:$EV����>zD�6���s���{�6��d�q�g넱�{��\]���B^�oo&s����Ϣ\\��У��b�������~\*����m\]�\`�b�H���?b��S؆�=��q�y��e����Ep6+NɃ��-�� �…f �I�|Q� �>�G\[X�d�i�\*�7�/�P�4���� �i���^��j��׍�,�t��-�� �;�G-�s ^$�W�o������\[� �9�=�\_��$��9�wdġ�iK�e=�\_X+��ړ��hT�\_�o�9�>�u�\`/�������s��\]�7���Cc�懖?�:"0\*�Q�B�Wof���o�430� �������2g��g����&�4���kˌ��Xc#+��;'a� �p��mX�6Ҵs�?�b�o���s�@E%��'b�H^;�k�P�\\���K}�J@f�������؛Ҿ�dvyV�������՗ J���r"I Rt\\ �HC\*�g{�F�"��1�ѦUu��n77�-z����Pg<��B���a����}� S����s��<&�I��d57��z•ї��\\�|eS\_��'�b�\`S!�nQ���I�~#CnE�%�>�9FA�>Få�&̑� zF'��yc��F��"�>�uW����9�q�s޾x�z.k�5rM6밵�T����\\��F�^!�UG� �\_+Sgq\\�@�-��=�Z�I�+�ٛ����+X��t���P�|uL\*�',�E�>A�Q �߽;�T'ЄE���\_�Jg�GL�\[�6�o:��.A��KC���>���t�uw? ��H���L�!�Y���Xs,�I\`�z�\\�&�e �9 Xn�\*\\&Wv7TN�\\�$�-qM��x}>�Cc|��5V��F 2s�\]{�8�!��e6\\��Vb5?�h'P��q�Gw�x��G�������-˵��I�4O�\`lo�Ӽ�n^$�y{���� ,�;P� ��,�7k�y�xz�|O"�w%Ѣ�=~��"w��d����/�cd5����S{�Dqg��\*�^\`�},3��&%�� �ad̡�9F{{���\_ŷe�Z�Mzx���D�E�� =v�gn�G�vc͒�K�;��\\f��)^K\[���A:s䩱^���&���4cV�G)� ��Lz@ρ(���d����8ݲ������� ���J�����9@�?���{�v.�|&�u�$ �)3���˂��R� L����L��s~�"�o�W�OJ���Y��$����i}�u�����h�����\\͈Nȳn������"�dD=S�+�$� �ܣ�&5y�S;�Q�\]B4Ȓ\`�˿��J�\_����%�1�a�@Q�%�� g��\_��A� x��cR.�B1�C����\]�����$�r\`k �@� Ź5@�1ΰ�H�yd��i�Z� �;#6�W���FlS���a(�~{/��=$����O�1ڠ�W��Qi��� pI̺ʮ� }U9�y�X=��H�+��Ciܧ 8�+��gVtf��4�6,�(��N��Pз��Y4��?�/g|6�ާ\`� �K��g��͊���p%��t��G�D(���7�.&����LFHY�f�.�h^���K�͕�v��Q�}d& s� 7����қ�6O��SS.aT�����zɄ�\*'�q�<�ˀ��삛��E^C��- \[�z�\*�1��VHߣF1S-ejΆ�8\\��Fu������d���B�#���J�Sq �4}�Z���\\a�k�����W��ݠ�� �I�OC�Tm�T.��M����� <��N ;��Q�8j:�cB�8�\`���e�.�ǣ��2�1�Q��0ff EX&U�\\فL\]:׿�FѿXފ3H6rm���)l��hR��p�|���<��)�d�Z5����^ej����$�ay���X�1�t�I�Ղd�IL4�u�Y�t%D�!E������i���U�'��0yR�C�sڲ�����v�>���<���^FL�|�2��D�K�T�����+-��ި�����w>̝B��u��V�3 O�� �Փ�r+$�E�m�5���x������' \[>���CPpd��|����݂м�4�$ρ���;s�g�� M�&r� ��\`k�̭���Z��P�C �<��� E��|�~y�T�1��a.D�(����!�C�!b�I�|�tp��\]xe����Y�w��@i�,�ʧ�:��|�3Z1�6��S\*T��-S���c�u�\\Z�K\]K�I-5�)�~���<�L\\+L�T�EPfj"L?�y�%=�Ylm�z�P�P��Zw �V~���Q.Q\*U�,��������^�!�/����U��������I^ h݆H6"��R��H܋լ�8�2�>�܊�vS�;�7D�1�u|Xq���� 2���-\`����\[��n�����e76�T� ��\\���jk�I}��e�Nv\]��yT�9Q�b�'%�n�.�^�\_��W�O�jz�5y�i�L=r�xX�I����%�xo5b� f|���9�g޻���z1�Zr�\\��6m��p=?\_SQ�"���\\R#�^p��<���X~��%���Ƕ� ��Ex�V��u�����\`{?�y�d ZS��=}Z%������g���ǢQ�ϑH?1��\_���!(�����M'}����� ��neѭi?32넑�,�\[�B��Q�kw�ս�I�/�|���1�� O�|}�6m=η�´�P�ĒyI�䩑Ĕè�'���M�O ���{����UY��5�I��� �>��b�E��#��e ZU�ȱ�gJ9�� z��?�徦��nmBWR���J����蟉�7��4 �F���#X�v������r�A ��%c0��l,D�p�:t�'�&��l�OL�{v�:�D�\`�P����q�^qg���O��Zr\_��ݯ?���f�O�1�/o!?�/��'�~�޿~�9�L/�w�� %3+y�xymB�ǔ9�}�� ?�}�e�Q@�49\]����{��3�� N�@ L=3g\[������=l��OQ8�s���v���&���q�D�c%a%�0�5Jd�³):{�q���H��O;�� �pڭ4�l���z�Q.������Kʛf<��(��%?� Q�K��;o䎗��h���d�wX�c$\_J�&E~� �?�1�|3��­�����!\\U��"#�61��O�u ��� 2���P ��nr Zn���;�L:#6 �yY�E���C�K������2���&�\\��pZ��#d���0w>l3���D���S��!�,�;w}Pp�C���U�u�G��!����CR�XN9�f:��� &�75�x�Oȗu�IE���Ջ!��?R�|����rq�I��p�+I$��g9�I��UR+=AC{���ģ�����Z~�Ϲ�2��?WG��P\`Φ�Sdc��\[!�ud/G�6��S�qT���͢��\]��q=���i0<�in��Xe>AM~5����9�"����{UT�5�)���U,�B��ll<3f{����x����Z"\_�=��1���\*׵����Z�(z�W���ݳYV��޵���|%9��;�� �"TZ@�7���O�n��Sn>;�U50BB^�m\`�z�"8PԴ\*���W�.p5Z8�=\_��@����w��� �U�TO�=�B����'l�a���;ZeZԄő�p1)?� 6���My�����r>� p��� �G4E�J���\[�� ��-�f\]S��vے>j Ϫ}>� i�\[44!�\\GS�P.,8ۥ^��xB�� �����s ϕ.1'X�W%� �@��K�^�W�\[ë��)U���ܙ�^����6�O��+�t2�'\`2�L�E҆�G�/�i˿�я%ŧ� \*��ɧ��=�ܖX� 1��l�� �a�0�a�������0'J�Eۺ��Q�I�f�B֝/�m���Bl�"L�t�OC���nO�O�>Xؽ�cB ����k�\*<�8\*�?�!m�'���P��O�Kg� =��r\]$1v!�\\�Њ:bG�(s&���=5M)���j�k��( 2��\[!�$�j��;�;f���9�W����N7Iw�������'H"�{u.��>T���rJ�6���\*�N���}�D�X�Y{�FXv�v�-�PCZL�7��U�\]� % �M�r?E���{��\\�\`x�b\_��A^|�����?��jd�-8�� :̉),Xz��V!j�c���5 zaYԵ�<�Cf������4n'�PB<\\��\_��7�9Z/��\`K�Ru����5�J����k�����MN?&ku�����kU��6<�����X) �y���gM���\_k��:2���\[�=)���aj�\*�6�~׈WJ�SL nA�+Bw�q���9���\*�s=1�s\`AX���m�˄�q2�.�rֳ�3�r�!��J�2����� z\`�Ϟ^V�yv��.� )M6eY xf����Co-�/���=��۬k����Y,c+�1T-���MS \_�y}d!��y�I���ֿ�� H��0�\]x{I�c��\\���V����:�D �K.m.\]��Y�� ��!���YL@��J}B�EOs p��Gw>t���ms��Ľ���^��aF.�\]�mJvӳ B9���:R@��a"$�I^q�A�T �����qO��ǀ}a�a�sᰦ󋉠W+����\[U(�\]^M}0\`O��>�6��W���MK�njhj���y��ib\_�\]�'��v�VW�����,��H��1{��?)��U����ߨ ���F�$ZGk�z�򝳟�(pz>Z�BxЛQ߾ �If��V�>N� SF��!�z�H\_�z-G��-�}6�mw1Y\_�n)��L�v���ޝb�Z���OEmI����V���J�kجHA��WW ������%�\_�:-�g�@��p�컭�$����\*�~\*g�&���\*��'t}6Ils� @-���띑����ņ������7��6?� U!��ݸ�T%��z,�>!���(�X^%lE ��7!Q��f���sLg�����bPL����2�#p��A�o�7��Ҙ6����pܜ\*ᲠQ�t�rpwxA;�#��j�Q搮�,�}�B�\]gʍ\_��\\CEX#^F�p�?0�ݕ�ς�I��'t��X�� ֿ�sk��8���Y6jo���-�io���XNۊ�(���8bԾY8K4�P��3�L�j�� �>�)���A��a��K"8c����= |�=�.��Mii��j�j�&�9�1b��5�����Ŭ��q,"�L\\yԟ�{N���ߛ�-G��Q,'�"�쪒n&�e�Q�x\\K U�-��� -\\�����i���ՙ��Gs� Y���79yyg��w�Z�ySz������|��5�{\_� ,hT��u{���d����j�� �\[���ـ�u���03(1SA���Ws��)"���Ӆ#��k�Ql�}�ɤhʉ��O�;AX�/ �dU������D��D�ǻ\_����#��06y�� K�� R3���#}������͇��Q�b�̱�r�۽x�z�q�>=�G5 ������\]Ɨw�\[x3���I� ���ƈt�\*C�׉���S&�I��NK��%\`t�6��XGOM�ri\\�&u3gI0 D�D b�U�\\>����S��\\�0sV\]re�j�7Q�w�:C9��+٠�vf�a�(��c{Z�8��n�ԸQ\_�ꊧK��������>ĥ��A�ڜn��<t�j�X� \`-��\`ZO�A��oy��i���Py�� \_�v�� ?7^�޺\\�TE�x��b��$��H Q����H�jԜ�:���ZF��Ы&o�U���\_��ћ���"X��?���U�нڠ�j��^dum��K��kD��{.�\_�\\�V{v=|�&e�^!�Xw!����)�\*aQ��BBCi7�h�r$�ߜ"�Ǻ�5��ft�̭;�x.�\]�)�y��Ōi�Hf$���� y(cb�3 7 ��s�'�PדK"T%�j,f��<�k�d�e�0�k�N����TKW��"� .-(�t��.����1Ɵ�{�%'Ԧ�~ ����Neg���� �v~s�"c"�{���%��}) ���2 ~u���)\*�.����7mRUƢ�=�z��A(X���ڬ��\`!w�ܙ�'�\`;\]l��DŽ�t�j�����~���1N �~2�v�)�,(��T���� �H>�J�'���3�s;��5���4R��3Xg��Us�!3�l�q�@�nc�\];��.F�p�Y��Hc����7Q��t�IX�=�5��� ��쩞�L0�\*&�a��-�$��(�\_b��f����Q���#/aۜ���z�E�8c\[�'�h�o��^'� �:a0�h�i�:��C��I:���aL�7��2~O�rf�JH��A~b.���~��E ��՟/��6��<ɦ�\\����k�>��d��i�Ԁ���R�T��֡f�%�����B~�\`�\[ܫ����/ �-�9�{� ��6Y� <���g;\_J�$f~��~@�y^��&�d�4�g��Q�����Z����Vٶz�>\[a��/Rj+\\�}��ޟ��3��L\*W�pLnW=vT�-5ރ��Е���w�}GZ���p�r�i#�q}�P\*s���4+�J'L&I�\*���-�\*���+��� ���==��M��\*ϺXt\`uCFsM8=mB��?�#;+�4d�!d��b�8)�����=��R�� ?��b����\_y�O�a� ��G�wɏp9)@̿�'�l�S�൳�\_Da������z���~q��~a~��>ћUk���%穳���=M��pB�i)߬X��a�����嬕k��C3�ɡ�S��� Vm����Y}D��!ڙ\\l2˽i�������v�Y�����TE�R<�R�GH?(H�$��Q:qv����9B�;�k�HO�6�nP�z8 Ů�\]���t��q�^����3�Ю�z�t3�;��m��sv2\_�|\_~"�B� �L����T�@A~��K�s<����H���C� ����&���3)i���t��"�����8�p��8b:�#ߤ��o,�V���� E��c���/PZ��c�t��=����SoU\`�c:‹��潊F�7�x�=i�̋yf�\`�z߻���&��P�O�\_E➾����4G�'�ib8�p���E.:z�d��؃�O�,UK�I�Ae���m��8}SBO'��8ma8C�LZ�!-3xA �W��/��А9��\\��. ��T���<΀�5FVX��P�w� ���m�Xl(�W�����5�t�;vG/1�yk����!�e�j,%v�Hz�� �B���A.���Uq�t��8V���\\�J��t�H� Q��R o�(�c.>��L��Ʒ��ì����Xߢ9~ F�R-ė���(����T�K�)���l�h����\[��\]'7mK����������PyB\\�d��Czd<\_y�Nt\[J��,�D�Q�����=;\]a����w��.��C:��B\`���Q�e0\*px��6&�Β¢��r\`q�������0�ܟ��>.+O���ј6K�q'�PKx�>�~���=n�º7 |�u�%)|�v�)G/FC�pg5��V�����#v����з����a�{�L��\\�Su#��4$� �YW>��\*�t(J�~�iH�r��,�;j�SȮ�v�1��E������p�3۸���,�~K!L|h'��!S�� bA�\[�)n�uu��R��6r�.=C��3ͦ2'C�?�|����ŷ�iGԛ��+=z���\[��k�ej\*���BE{�W.B���4��R�oK򚴘���@H�y�ڐ�d^\_M4����!& &�&��3�0/ eE��8MV�1/���b����jGV��e� �g� ��(nd�Hi˙�rb��>=Z�� �� �)�a��f�%����O�/�kM8RJ�q�/un\`��q�P?n�V�R�g��0���{�8�w���ٷ&�C�|a$��<�B�u���S�T��N��e� �ؚ9L\[�\\U �\]M\]���\]��R��j���CV(~��z��@�)\`�#ݴ3������f}�p\`P��B��SR5��X{0��LC��K��{���l\[��n��K�D�x��Ϳ��;C\_5@gmIR�y�Ḳ{�y�c<�f��.j/x�T�ҫ�ڧ���!��\*�a -=|��g>b�f��o���/�H�����w�� 5%�7�����齲�GMN��$�i,� A�����OD��W\_N;\`�����$g���Gज़a�JӝK�˨װ�/� !�Cd}��\`�z'{Ļ�3E�D�Z��\`8�y�+���6�s/������S1(����U�n�F�\`���y�������k4�� ��\*7�m@�'�J\*�9��Q��\\2�K�w�5�L\]y�9=Pk.X����\\�\]�}Y�R- t�5F����W�jc|�Vb��C �۾3��;8N��D��m�%��m�����Pr�6DI'u��ښ���HT�1�P6+�)T��?�F�bbs�, uSoCM\_�\\.���j��\`%U:��O�\]dXR@������ߴ�(s5M&ʈcg��t�d��2�>@���$�6�c���r�\[��s�? �f�d�N#��K�ݼi��e8|b��}?��\[��)C��ζ���Yw a\_F�Sц\_��I�h����3Ч����Z�� \]C�xs>�jjw&\\q�kP���n{� �==�ڎ�� ��─4 r��\`T���!��g ���H̸�T��-^Q}�Dl�aXdq�N@�e ��/��H�q:i�#7��I�r�X��� �MZ�w��,��CK42zM;����ϻ�1d��>^��{w����pj�̉��co�� U !����/�܍nQǔKt��� �'����BanK�̜�vp�����3�Տ| v�롻S<��4p�H�Gs�� h\['�w�c�y���R�a����W\]�5n���rr����3��z�b�H�k�.�\\ &������� �9�U���w���v�����IL0���: f��Ҟ�(Z ȅ;-Q59���I��A��U�Otu?���������I�EB��a3��)��Ēk�Y\*\_��e�EJ簦��7�\*v�z1o�^�c�us!8�ݛ�Gl�s������z6e\]�f�����?�!�� ��5�8�R4���j\*o�Zq��g�f�� t��1ZR'�� �� ȍ��օ 3�\\�;�2\]�ٛ�A�1�5�����M�G�\]B���O)��'�K�� � I���զ\\~���X��x+���>����"&�\[�x��x��Oˆ�Gs���^�"Pv�r$���U�a����S�!����������9;��Z���.��m�s01��(�����M�s!��K=�g���@���.y�zx<�w���/7���T����K��/������q�ț%�w؝\\����Z�q�4� � ���lN.�!�jX��u ��\`�r��5�%�E��J�g�����7�ͭ�#@t�� ��k9�{9���(�-�6�ȴ�q�pA� 1Q/q���-N׵F��b\\���en��Qi��+�\[)t�q�ӏ"�\`�u�Ў��u") fG�C�����vH�\\y����L�dOK� d�-��oű���'"X:=6xȎ^�l���>� �\]!|��>t}���f�K�b�R�� ����3��B5B��P�K��\`��d�����؅�ł�6��|Q @��l��\[h�JQ��0�+�V�qY0'^�t��H1�Zb験 j�ϗ��NXM�}搙(BC�!�)�-��\`��3\*p�'�}P숊��\]����=��zѥ�< ��1������Ȓ4S��g�j:�<(S?.� �wCF����R��د8�9���&秷��g�mL�u2��)�3�o�k ��'��M��=�\\T��V ����z���2��2%��E���� �t��>�$�\\���.�,�o��.�Y@�rW�}B��K���Q��깤���CVn���C0���I����"���A���&�՞�pF6T��)���+��ד%\*"J�%�i�5�L�A�3Ɂ��q6rߟ��f0v��v�� �\\�0\*�n��y �3s@��ߍ�c"1NrY 7���͛ l��\`�5%X�'}K�@�y3A$FD5���Id�˥ ����\`��Z�TV��A>����u���a\_o�x�Cb�ѥ^)t�(y�B���t\_��\_�� �G�cH���v�}�5��ՌN�Gu2��s��,�ѷء� F�i $|���G��ԋi���6̷��\*���\\��0ݽ�K���w'9:~��!��i�͍8��u�o�thkiA?9�ޱw�N�M����6��\*��&�笢�-��/�Qn�����Lv����|��h@�e��KZ��Qu -�\`^�u�<�L��\`�%�m�u���=m�uR�J� ٿER�sG�y\[g�7��DU|��\`zϗ���A8�\`���2a��K1�y�&q+>���41)���6��M�O\_�h�|�U�����<�6������L�JvT(��B ��ʱ� �������r����V0��,�������7lm�)f A��h 6��z0�9|�L�M�� �{y{s�%@�SѶ�{�l�z.����Z9�ι�-}���9�u ���RR��<�z�z ���ʃ\[��x���pE-�%{�}x/2�������#�>���O�$��#�QN���k�o�� �y�����8�J�q��9�,�?����+��W����W�^G��Z�5�u���� }W���z��W#�@$�v�9�-�1���y&�l�O!�ŇY8� ��g��72�ȅ�N \_v\]���x������ �5�FTC%�M&|�#���n��W ���h��p~�%��g�uUΩ���r�bb�,��J߷��m2�XN6g���$���G�\*�~{8yo���i�)0t�Eõ{?! ġ\]ĔO<� A XuPs\`n� < �H2��:DŽ��3O���� A#/�p������k� ~g�el������fl���0��%p&?�.z�a�X%���\*w�e����΁�{����rR �� #\]� ��u �e�7 --- # Unknown BZh91AY&SY��0�ݼ��\`@����\`PMץ3�H�h�hha4ި\`�F�a12\` i�OUJ�4� �����S#S�4z&�����a4d�U�O$��<�Y����� ���&L�l�"�KS36��"��&fVHBffHBH�m��R"6�b������f\`gM5�$�\*̢�6�(�b�7e����p 4V4��e�.�vC���E�5�65���F�ƪڙ�����ffL̙2d���馜.�e� ��DI�\`\]--�DD�l��(�4�R�G!�����J�D�QK�G!��+��JQ4�R�T4�"#�DQAE$D^|�����SPyu�;��\*��R� R��P�!�{6ek�����Z(�"""�Ջ�����kVō�Z�Q��ƨխ�@��h\*�i�\*PALqԊ��;̳יFza�c�^�5fđ���K�c�g��vf���"5�1g��e5�K���h���G�/�D��\`�6�(Ԕ��RJyzVfa�\[kkl���I\*Ka�S\[$�4ѭkYde�,�2�͛ "6�1�����m.D�m��i6665���4봚I&ҹ��i�ڛM����\]���9'Xlٲ�l�-�Ȣأl\[m�b����$��$��$�I$�Mwkf�v�Wu\\wU�rB�Vj���պ"D�ú�����mm�Wb%��T�wp�-�ٶ�;T��w�wpОBQ,��\[\[f���A�J��e����ƪ��lj1�cUXѭ��c�5m�,DUX���j���ڌ\`i�\[l֦��E�"�E��/�(+$�k7�m�X\[�\_n�\*��%4RRz�I(�J���P��HP\*��PVI��b���k����T��-@\`mR�MII�MIMHG��J�!(�BP/�����e5����:�M�G�1:$v��Cj�6����)$��ڭ�{\[$�%���Ml�T����I\*i�Zֲ��Yde��6Dm�cccg\]$�\\��ۭ�6�lllk4i�i4�M�sm��i�6�Më�:����rN�ٳe��l\[-�E�Fض(��kU�I�I�I$�I$����r"�����̬�+��t(+�m6�6lٳeV�(�cQ�cڪƍm�c�1�m��b ڪ���(+V�����c �J6� ��ܔ��H�����͛kc�)9\]�C��j�f��ͫ��twQ\*��'� R-�P��;5��m^�P���AT�Qb6�Tl�G��e5�����G�/�Š ����@&���$��"��Y��em���\]ӄ&fVd\*(�&��\]�L��Cd��vJ(.���؈�E;vfV\\�.�5�\*��V(�ŋ;u�ffef� 4h�4i�v�e�.�a0QIC,\[,X�lTj6\*5�V���33��32dɓ+�:Էi�0�\]�˰�:n��v\\�؈��\[�񱳵���b�Z��b�0mj����-�b�P����V�B��C+r\*�� ME.�r l��M5�)��J�wuYR��f���J��I$�HA��;II(��u !W�Z��I\_�PVI��c������V��:$ 6�()M�I%4�)�L̬�TQR'rv�I\*Ka�S\[$�4����asV(�����κI����\[�,m�����664hӮ�i$�J�X�b�b��uwm�Wv���9'Xlٲ�l�-��(�(��bأ��\[$�I�I$�I$����r"������:��S�5�j�\[V�\*��6�6lٳeR (��c�6���\[c�1�j�j,X�6����� բ-�m���(�l2VO:�j�R��R٭��j�D�;�����٭��j�"\]�J�����J�H��Zf��ͫ�Ԋ� "��T��)B��(��b��L��gp����� ����� I�jJOP)MPx��ҳ3 ��\[\[d�֦fm��Ds\]jffۻ2:뮶�L\[L������"#a(�,DX���3+9�\]jk�4U�QX��,��e����p1�F��N�m�U�wUv�ֺֻ\*"(�bرcF�b�Q�Q��ݪ��f\`)�� �ə�&L�\]�֥��� ��V�V�:�<鮹�˔\[kv�66v���lZ�"""+V�,X� �V4Q�lQ��\[cjֈT֨jX����� k6m���"ѵm��'s�뻝պ�NWp���\[Y�mlv�\*�Qm���T�i �=����E�JT�%9$!�(+$�k?K���� �cjj���@���eɄ&fVd\*(��;\[$�%��)��J�L��C 0�9�+Dm�cccg\]$�\\��ۭ�6�lllk4i�i4�M�p,X�X�gm���۫�l����6l�l�\[�m\[m�b��lQ�ڭ�H $��$�I$�Mwkf�v�Wp��\]ȩ��5v��p�R��M�6lٲ�lj1�cUXѭ��c�5m�,DUX���j���ڌ\`h�m�+'�E5{���l��ٵv"\]�Hww Rl��ٵv.��%\]��ByTU$\[R��\[\[f���$>�R|)�$dUQb�?����)��X�/����< }�PmP\`OT�RRz�I(JaQQ�(+$�k,�^��ҕk�cn� �6�(Ԕ��RIM�\*F�H�+�\_n���T�RTځI%4���ff�����i�d����5�ISMֵ�9Ӌ�r�6���(�����κI��2�l�,X�M����4봚I&Ҹ�,����۫�:���u�͛6\[-�dh�b��lQ�-�-�m\[$�$�$�I�&L�wkf�v�\]ë�uw4R{N��t�cQ�cֺ�QN�U�Q���c�5��4llմX�mQ�\[UU�\*ՌaJ��6b�z{�N��٭��j�D\\��wuX+k6m���B����5!�\[6��h%\]�Y����JD>Њh�5E�\*��-J�����)�ˑ{�\[̬< ������T��6�RIM9E98BfefB���6M��J��c�kd����30��� !1X�c����F\`gM6ɤ�����k4i�i4�M��)))JJK�\` �Wp�a�f�e�ضF�-�6űFض(��V�$�@I&I$�I5����E�i\]ë�uw�Xk(j�E;m�j׌cƪ��lj1�cUXѭ��c�5m�,DUX���j���ڌ\`i�Y��d��:��{���(�4�R�G!�����T�5��m\]�K���WwpО(Rk"F��5��m^�b��"mT� |��ESR��� �2�̚uj�{���#��T� �\`mR��RPIM=�\[���I\*Ka�-f��$�-�1Ml�T�!33�a ��a�(�0�c�"#lcY�� ��i)( �F�ƍu�M$� �����i),v˰\]��&\`(���4E�Fض(��궭�H �H �I$ɓ&M����\\��a��˰v���ft�\\���1�k\]\_(�U��(���1�c�j�66 j�,X� ��ح�\*���f�j�0�V��+'r�m��ME.�E��wwU���f���T���"���\*�͛kc���,�JO� �D��-RUR����!U-"��d�Md!� �\`@� mУA��ڥ���� IA�//J��3+mmm��\]��J��c�kd����ֵ��2ŖY}f��c����I6�"v��rō����ƍu�M$�i\\ ,V,Y�n��������:�f͖�e�l�\[m�b��lQ�V��H $��L�I$�k�;\[5ȋ����.j�EX�����^1�c�\[Q���1�mUcF��1�cն�X�mUccc�D\[\` �j1���f�a��xp��Τ;��&�l�\[�I�� Wwp�&�mm�Wa��GwuZ�|@% V���ٶ�>�EB^��J-�Ty%���'�(+$�k%��YBp{�\_n�E�\`mR�MISjIM�JLJ�O�B��/�(+$�k6��Q\`� K� k�^�� \`OT��)%Tԩ�<͞y��ieRfLC$ؑ��S �@���f+2Y��L�̉�H�d��D�l����-�lb�(�Vء���el��\[)����Sj�Vڛ�+ LV2�,�bmI���M��2fY�ff+�M�"V�t56P+lS5 L+b�Kb6�,�f+���+2� v�J�v��UѸ�$$�!!#1�YWh���HږҶ��s���&K���)0̳C�V��G,�S�9���YYMLumV�����V���Dڦ��٫Y����l��mZ��s�Zj�h��\[m��RRJY)��� Q��R��)�$�1AY&SY<W�A XxP>�%O \`mRSA5%M�5\*mOl�� �M,��e�l��L� �bF$�@�m#i3Hi6H�1 �iL��H�CQ썡�mUmF�d���Vն�(��L���j1Y�1M��\[j�2��e1i-�ؖԶ��-�m s$�4�I&m�cfݱ� \[e�S5cS fQ�l�b�4�)Whl�֍j��r:N�;�ְ�ֱ+U�I-�ю��ڰSb��mu��#t���vfp�Qcc���J9fꜙ�gn���jc��V��6�ڱ��ڪl��m�^{l��X����v�յ���h ֭m��JG��(v�R��"� \]DHD��� �2�Θm�('���#���w���P��?�T��T�RTځI%4��Tѓ���M���kB� j�Z�U l+ak ���hZ�P���P��wWkn�ݪ�\*�ll� ���چ�b6\[HڶQ�ڣh�M�h\[�\[Sb\[&�\[Fȶ�eM�j���-��-���ڋi6%���c1�6�ڦ� �d��ح��M��V�mI�\[�r��1Whl�l�b����Xʹ\]-���cU�f�� #�\[A�TmKjP��Q�V��Ԏ�Fʹ�R��Th�ݕq���gv�\]��n�(�%��m�D������Ґm��m6�"�{u�P�-"�I�v�m��i��AE�A\_( Bڑ\*�z��Y&!\_�PVI��Z(���\*�v߸��Ճ��%4RJh����S�-�h�Vl��m j2��I�\[IF֓�cmF5����F6�ƶ1��j�6эh��cTclcX�эF61���m/��Qiּ��h�+QY��mCh؍�iT\[QX�Q�U��գl�6 �6��6FқQ���ک����Bڦȶ�\[E�-��Kbmj6ll�\[\[X����\]�8��\[D�VĶ��ӭ����Wg\*kD��,+���l�V�1e�Be��f�Gh6�چ݄���7.�DQ厶j�6�U͇i�&ӵGd�;6�ۥj��Yj�� �bE���#�̎�' j���)���J^�e �E�P�u�aI ���b��L������ �H� ����cj��0'�P �J�P)%'������������V�b���ML��͖��̶)�m2�Fe��U%�l�-d���F��mI�Q���cF5�ƌc!�W�l���쭋j����cTb�EQ����Q�6��,V��lkEX���X�m��3V�FQ���+e �VجSb�f+m��6հ�ژQ�\`����kkm�Wr��Qȝ��\[cmET\[�Vj;\]E�-��S��Um١����������A� �m؎�H�Q���DDDDDDZ�\[5��m�7A���5uFn��6گ+-�V���$�uO�Q� kn�Z��ڭW��R�Je�C��-� T�S��� �2���jh�6��#��� f� \`OT�RRz�SR��^�i�e�f�cm�b6l�f�l�&͊f5X��F5�ckF5h�ڌkQ��(��رk%���+IcT�-���X�,i,Id�&C�˛vΛQ�mY�)�a\[jb��+ 2���ئ�2��̠�-��\[I�M���Q�&��bm ���6l��V ��jaM@��mX(� a\]���U\[m��v;���mF՘�ڶ���\*�Z���mU6�d\]��l2��V,�lȢ��X:�\[ �WF;��َ�ի��DDDD\[s���Ͷ�s �\[;B� n���Km��6�\[l6�Ȇ����W �����Ml���mo6�\*��,��K�B�J�W��B+(�����)��LV\\���<(��\*������T��ԔI%4��ozejm�,���� �j&Z�3\[DʬLڛY�mfQ����\`m2�Eh�h����؊��Eb-��,DA\[6�\[��չ��EAh�c�lMV���6�i�6&�e�kVƬm��U�b�al�)�\[Sj�%�d���+dl+hmQ�V�6�6�jM�ͫ�-X���c�X���\]�����\\+��B�&�\[���b4�ꍄlCAګjM��6ɦ�I���2���-��-��+�8�%sk�V����1�c��K\[p-�b��\[;J�J���s�m�j���6A6����T{�H���m��cJ��Œ(���D��\*�A�PD�䦒)~b��L��(\\QD4�� ���A�R�5X���I� �(�P{kk��m�6c�T3�e6f�6lSfj����kh�֌jђڌ�Td�����k6ђ�d�-��FJ��1���^j-:۶�lV6ƱT\[�V��6�hlM���P\[Eh�F�U�hՍ�\[���5�X�-�X�j���6 �ڍ����Z�lkF�m�m�T�M��\[)�t�mmm���� MC�\\Uʴj�kFՊ�-��'j\]Q���h;R6�U�4546֔Q���\`� e;T�S���r���Xl�9Z.ԭ�j�mvC�#��s�m��W����Bڦ��Q��o�ն���Mm�m�T�\*��J���M"�W� RF���j\*@�/�PVI��s4�pV߸�5\*��0��@&�F�TڞJ{�бdن��E��-�EV"Ո�b-�E\[kb+lE��Uj"�E�EX�X�؋lEDZ"��Q3baZm�Z5v�V��6�h�kF��TelJ�\*�UmF�؛A��!��Pڦ�l-�m�6Kj��mRڍ��F�\[Cb\[ b�iM��Sd6\]Y�1�͛UwRGP��\[Jl��j��l6Q� �Ge\[\]I��a�յ&�u�lX­f�5���\[�Զ��v.ҷ(�vV� f1�ܷme�U�6��C�F��vQ��j�\[�j�m���\[jV��m��VګTZ5Z��:�U����ݪJ\_UZ�+�D�v�i����\*W쨀��PVI��{yq&� \[V׸\`F� BJh&�F�Ji�O���œ6�6����TӤSN�M:A4�PӤCN� :@4�WN�t�GN�t�SN�4�CN�4�\]:GN�Ӥ4�t�4�ӭ4T��4-+r�UUUpB��J�!@4�e6���%�6�j6�ڦ��M�i6��&�lV�m+j\*؍����6Q�Tl��mCh��M�mI���mQ� ��i�f��:1MUأb�6����ԛa���6a5Gk�3̑��&ɵ �\]\[1j�Ȳ̳Y�F�� ��چ��jN�#�ܒ:\[Tv7��Q��GkeN�;6��Q\*���tUR �C�I)O�$A��m��M� ���®���p�⨁=�m�ҽ��\\D���R�P����e5�K�V :U�G�5�z5� \`mR�MI@��ҧ�h�5m��-�a�-��-��-�̶�̶I�mQ�lFe��U3b�ȭmV"�"�"�Q��E�E��# f�ճ5�j��ƣh��F���بѭ��(��l6&��6�j6M�؛Sj�A��Ch����R�&�\[Q�V��M�l&�\[D�$� �6Cb�P�Sb6Tۦ�kkm�Wr��2\]�lV�6U�F��v�f�Mk���SJv%��N�d�l���͙ h6�� ��T�\[w\[m����1�c� nv�����\`�v��ݭ��:�U�����J���Ji2H����U��\[yյ�8��Q ��!�� ���d�Me�Љ Q�\`@�@ qǓYU5�0� =R�oJP$��=�o�����SJV��T���(e)�J6R�e(5)�\[Vf����Z�ƪK�,md�VK�X�%��X��5&���&1��ѲnWS��-��\[��U�6 ��dѵ�Z6ѭ%X��V6��V1��X��fSSeea\[Pƶ\*\*���Tm�hC5&�6�؍�6����mU܀�2��¶��j-����-Ƃ� �u��\*iN�l����CSCmiE-Sh6�� ��T�ʪ�l�������:٭\\�i7A� N��a��3u�j�mW�Y�kj�ڣJ�Z �B;$��J�ګY��;m�.�A%{�SUP=�"Q �b��L���n� � ����x�Zj\`�$�J5$����/��1�6�V�e�\*�ɭc%�6�2VыZ1V�\[Q��b���\[c��X�XŴb�b�b�űFѢ�ƒĖJIiW�E�\[v�-��� b��cTclZ�\*��Ѫ+E�Q\[Q���m�M��S\`mSh6�m�6Kb6�����FҶ��V�mU�M����6�Z��1�Q�R:�r�v����CZ�mj-ƃV�k������h�Ķ ��� M kJ(��J��6I�M�Rw �;��ɰ�-�����Z5ʵڥ���¶��+�"Hm���mQ T�ii��B�J���m��mY��\*^�$uw�|� "l�\*I|�Ċ�1AY&SYď�Y XxP\[�D���� \`OT��ԔI(>���\[f��Lb�͈؍�(ٰ�6I�b�d��Mm-hɶ�%j2j�%�2mc%Xɭ-FM��cF5�ƌc!�W��N�.ѵ6&�\[Fʶj\*�����kI��V��F�U�\[F�Q��X�V6�cV+cm��U�6��ڍ�il��dm �lBڪ6�5QU�Tv���wBN��U�; �-�lSj-�l�ڕ�vp�؆��Kam\]\*h546֔Q�Mms\[\[�\[v�wr�uB�f """"""k�l�nmQ�mWj\]��ݶ��Ҋ6�-M���� �wDEp;�Tm��m��m��am+%e(iB�O�%�U\*W����\_����)��p\]���<(�^lִٰ\`z�OH&�� IA���b��B���YQ1mc�bՌUc�1\[F-h�Z1mF+Q�Tb�1m�V�-ccѓj2U5����(�bK^m�C��+dmV�l��j-�j6�Z\*�\[)�\[I��&�6�e6F�m ��mF�V�mʶCj��l �ƪ5�-QmDmQ����ɤ��r�U�tF\*�+�v\[El���Hգb֋q\`��ֹ�Tڑ���ڃbںT�K ���"V�Ur�-���d\[v�ܕ�wJ�qRDDDJ��Sm�kQ�i�m�؎�f������m\*�Y�&�m��l����B�K�6���Z��5�Z��-�Z�\*�I)E� E�SA��d�Meei���\`@�@ �zOlk6�0'�PJ��6��ը�|�Uf��J�F\*���ŵ�V��V1U�\[h�m��hŵ�F-Q��Ŷ1\[��U�\[F+F-F\*�\[�bъ��6Y{V�t��� h�6�X�Ƭ\[��6�TVƨ�Ũ�ƍ��&��M��M�ڦ�6�m�6Ke�J�� H��P� �m��Q����wEGA��ح��\*ڶ��M��3fI�v��L�6H�v�Ccdʉ,R0@)$CM2F�� ��6����Q�kWV�umZ�hm�֛j�B 4��)�Ġi�6��UٷB$b4�UUM���\*�M�D��oJ�U@ ��CmCIMP�)�H��1�E�%J-�b��L��� ���� �\`�0BThԔ� IP4��/~�G��C�G5(�GB:��t�\*iSB��&�4 �M1ڎ������;U�t.��:G@�)�4&��C@h .�C��i4 :thֶ ��H�+�:��0v�k\*�m��0n�ur PC\]� C�(���J�0J\*�~U�\]��BC�5�� --- # Unknown Optimization Guide High Performance Computing Intel® Xeon® Scalable Processor HPC Cluster Tuning on 3 rd Generation Intel® Xeon® Scalable Processors Revision Record ................................................................................................................................. 2 1. Introduction ............................................................................................................. 3 1.1. 3rd Generation Intel® Xeon Scalable Processors ....................................................... 3 2. Hardware Configuration ........................................................................................... 3 2.1. DIMM Slot Configuration ...................................................................................................... 3 2.2. Memory Size ............................................................................................................................... 4 2.3. Memory Errors ........................................................................................................................... 4 2.4. BIOS Settings ............................................................................................................................. 4 3. Linux Optimizations .................................................................................................. 5 3.1. Virtual Machines ....................................................................................................................... 5 3.2. Network Configuration .......................................................................................................... 6 3.3. Disk Configuration ................................................................................................................... 6 3.4. CPU Configuration ................................................................................................................... 6 3.5. Services......................................................................................................................................... 6 4. Application Settings .................................................................................................. 6 4.1. Development Environment ................................................................................................. 6 4.2. User Environment .................................................................................................................... 7 4.3. Benchmark Optimization ...................................................................................................... 7 5. References ................................................................................................................... 9 5.1. numactl ......................................................................................................................................... 9 5.2. PCM ................................................................................................................................................ 9 Product Brief | Title Revision 1.0 Page 2 | Total 10 Revision Record Date Rev. Description 04/06/2021 1.0 Initial public release. Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 3 | Total 10 Optimizing performance of servers used for the high-performance computing (HPC) applications may require different configuration options than for servers used with other enterprise applications. For HPC clusters, the goal is to reduce workload runtimes for applications using MPI libraries and high-performance fabrics This guide will cover many system and software configuration options that have been demonstrated to improve application performance in internal controlled tests. All configuration settings are intended for multi-node HPC clusters running 2-socket 3 rd Generation Intel® Xeon processor-based servers. No other hardware has been evaluated for this guide. The objective of this guide is to provide an environment optimized for typical multi-user production clusters. Configuration settings should be beneficial to a broad list of multi-node applications using MPI libraries. However, HPC applications may be affected differently by settings using in this guide; therefor performance improvement for any single application cannot be guaranteed. 3rd Generation Intel® Xeon® Scalable processors (former codename “Ice Lake”) deliver industry-leading, workload-optimized platforms with built-in AI acceleration, providing a seamless performance foundation to help speed data’s transformative impact, from the multi-cloud to the intelligent edge and back. Here are some of the features in these new processors: • Enhanced performance • Enhanced Intel® Deep Learning Boost with VNNI • More Intel® Ultra Path Interconnect (UPI) links • Increased DDR4 memory speed and capacity (2 integrated memory controllers; 4 channels per controller) • Intel® Advanced Vector Extensions (Intel® AVX) • Intel® Security Essentials supporting Intel® Security Libraries for Data Center (Intel® SecL-DC) • Intel® Speed Select Technology (Intel® SST) • Support for Intel® Optane™ Persistent Memory 200 series  Populate all memory channels with the fastest DIMM speed supported by the platform. Intel 3rd Generation Xeon Scalable processors supports 8 memory channels per processor. Every memory channel should be occupied by at least one DIMM. Use identical dual-rank, registered DIMMS for all memory slots. Dual-rank DIMMs will perform better than single rank DIMMs. DIMM speed should be the fasted speed supported by the platform. At the same memory speed, 2 DIMMs per channel may perform slightly better than 1 DIMM per channel, if memory speed is not reduced by using more than 1 DIMM. Do not use Intel® Optane™ memory for HPC benchmarks. Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 4 | Total 10 Most HPC applications and benchmarks will benefit from larger physical memory size. Specific requirements will be determined by the application. As an example, each cluster node running the GROMACS or LAMMPS molecular dynamics codes should have a minimum 96 GB of RAM installed. Following memory population guidelines and using one 16-GB DDR4 DIMM in each memory channel would provide a total 256GB of RAM for a 2-socket system. A repeating, corrected memory error will reduce performance. If memory correction is enabled, check dmsg or the system event log to confirm there are no corrected memory events and replace any DIMMs that show repeated memory errors.  Enable Sub-NUMA Clusters, enable One-way IMC Interleave, and set power profiles to “Performance” CPU Power and Performance Policies and Fan Profiles should always be set to “Performance”. Enable Turbo Mode. It is unlikely to improve HPC application results due to high CPU utilization, but it will not reduce performance. You may disable it if performance metrics for a benchmark run must be consistent with previous runs. The recommended setting for hyper-threading (SMT or Symmetric Multi-Threading) is enabled; however, performance benefits will vary for each application. For some applications, a small decrease in performance may be observed. It is recommended to evaluate hyper-threading performance for the applications that you will use. For the STREAM benchmark specifically, the recommended setting for hyper-threading is Disabled. Use optimized configuration settings and recommended values. Default values are noted by an asterisk (\*). Configuration Item Recommended Value Hyper-Threading (SMT) Enabled\* (see text) Core Prefetchers Enabled\* Turbo Boost Technology Enabled\* Intel® SpeedStep® (P-States) Disabled SNC (Sub-NUMA Clusters) Enabled IMC Interleave One-way UPI Prefetch Enabled\* XPT Prefetch Enabled\* Total Memory Encryption (TME) Disabled Memory controller page policy Static closed Autonomous Core C-State Disabled\* CPU C6 Report Disabled\* Enhanced Halt State (C1E) Disabled\* Package C State C0/C1 State\* Relax Ordering Disabled\* Intel VT for Directed I/O (Intel VT-D) Disabled\* CPU Power Policy Performance Local/Remote Threshold Auto\* LLC Prefetch Disabled\* Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 5 | Total 10 Configuration Item Recommended Value LLC Dead Line Alloc. Enabled\* Directory AToS Disabled\* Direct-to-UPI (D2U) Enabled DBP-for-F Enabled Sub-NUMA Cluster (SNC) SNC is a feature that provides similar localization benefits as Cluster-On-Die (COD), a feature found in previous processor families, without some of COD’s downsides. SNC breaks up the last level cache (LLC) into disjoint clusters based on address range, with each cluster bound to a subset of the memory controllers in the system. SNC improves average latency to the LLC and is a replacement for the COD feature found in previous processor families. For all HPC applications, both SNC and XPT/UPI prefetch should be enabled. This will set two clusters per socket and utilize LLC capacity more efficiently and reduces latency due to core/IMC proximity. Integrated Memory Controller (IMC) Interleaving This controls the interleaving between the Integrated Memory Controllers (IMCs). If SNC is enabled, IMC Interleaving is set to one-way, and there will be no interleaving. XPT (eXtended Prediction Table) Prefetch Extended prediction table (XPT) Prefetch is a new capability that is designed to reduce local memory access latency. XPT Prefetch is an “LLC miss predictor” in each core that will issue a speculative DRAM read request in parallel to an LLC lookup, but only when XPT predicts a “miss” from the LLC lookup. Ultra-Path Interconnect (UPI) Prefetch UPI Prefetch is another new capability that is designed to reduce remote memory access latency. The UPI controller issues a UPI Prefetch, also in parallel to an LLC lookup, to the memory controller when a remote read arrives to the home socket. Direct-to-UPI (D2U or D2K) D2U is a latency-saving feature for remote read transactions. With D2U enabled, the IMC will send the data directly to the UPI instead of going through the Caching and Home Agent (CHA), reducing latency. Keep enabled, although workloads that are highly NUMA-optimized or that use high levels of memory bandwidth are less likely to be affected by disabling D2U. DBP-for-F DBP-for-F is a new feature that can benefit multi-threaded workloads, but workloads that are single-threaded could experience lower performance. Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 6 | Total 10 Running applications inside of a virtual machine will reduce performance, although the reduction may be small. Performance impact is dependent on the hypervisor and the configuration used. Do not execute HPC benchmarks in a virtual machine. Many HPC applications are fine-grained, requiring more frequent inter-process communication with smaller payloads. As a result, performance is dependent more on communication latency than on bandwidth. High-performance fabrics are used to minimize message latency. For maximum performance, use one fabric host controller per CPU. In a server, each PCIe expansion slot is associated with one of the CPUs. Install the fabric host controllers so that each CPU is associated with its own fabric host controller. You will need to consult the system board technical specification to determine PCIe lane assignment. Fabric performance may be further enhanced using multiple links (dual-rail or multi-rail). For many, bandwidth needs are limited and using multi-rail does not benefit performance. Disk configuration has little to no impact on benchmark performance, including HPL, HPCG, or STREAM. Storage requirements for other HPC applications Before running benchmarks, all CPUs should be set to performance mode. For example, to use the cpupower utility, run cpupower -c all frequency-set --governor performance  Disable all unnecessary services and cron jobs. When running HPC benchmarks, do not run them as a job using a batch scheduler or resource manager, and those services should be disabled until the benchmark is complete. Make certain that no other users are logged into any systems that will be used during the benchmark.  Build and execute applications using the latest Intel® oneAPI HPC Toolkit. Intel® oneAPI Toolkits enable the development with a unified toolset, allowing developers to deliver applications and solutions across CPU, GPU, and FPGA architectures. The Intel® oneAPI HPC Toolkit delivers what’s needed to build, analyze, optimize, and scale HPC applications with the latest techniques in vectorization, multithreading, multi-node parallelization, and memory optimization. The HPC toolkit is an add-on to the Intel® oneAPI Base Toolkit, which is required. The oneAPI Toolkits are available for installation using a local installer, or through online APT and YUM repositories. To Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 7 | Total 10 install the latest toolkit, go to https://software.intel.com/content/www/us/en/develop/tools/oneapi/hpc-toolkit/download.html The oneMKL provides enhanced math routines and libraries, such as BLAS, LAPACK, sparse solvers, fast Fourier transforms (FFT), random number generator functions (RNG), summary statistics, data fitting, and vector math. Use of oneMKL is recommended for optimal performance of HPC applications and benchmarks. The library is included in the Intel® oneAPI Base Toolkit, but oneMKL support for Intel® MPI library or Intel® Fortran Compilers requires the Intel® oneAPI HPC Toolkit. Use Intel MPI Library minimum version 2021.2.0. MPI applications should be compiled with this version using compilers and libraries from Intel oneAPI toolkit 2021.2.0 or later. For OpenMP based applications that do not benefit from using simultaneous multi-threaded cores, make sure that the number of OpenMP threads do not exceed the available number of physical cores on the system. OpenMP threads are controlled by setting the OMP\_NUM\_THREADS environment variable. Also set the appropriate thread to core affinity, based on how Hyper-Threading is enabled on the server. If Hyper-Threading is enabled: export KMP\_AFFINITY=granularity=fine,compact,1,0 If Hyper-Threading is disabled: export KMP\_AFFINITY=compact If your system is configured with Omni-Path fabric and multiple links, enable multi-rail communication. Set the variable PSM2\_MULTIRAIL equal to the number of cable links on each host controller. By default, it is set to 1. Optimized settings for HPC benchmarks may differ from applications. Optimum tuning of the HP LINPACK benchmark uses custom configuration that may impact performance of other benchmarks and applications. For that reason, it is not included here. For information on tuning the HP LINPACK benchmark, contact your Intel representative.  Use the Intel® Optimized HPCG benchmark included with the oneAPI Math Kernel Library Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 8 | Total 10 The High Performance Conjugate Gradients (HPCG) Benchmark project (http://hpcg-benchmark.org) is designed to complement the HP LINPACK (HPL) benchmark by providing metrics that more closely match a different and broad set of important applications. It is designed to measure the performance of • Sparse matrix-vector multiplication. • Vector updates. • Global dot products. • Local symmetric Gauss-Seidel smoother. • Sparse triangular solver The Intel® Optimized HPCG benchmark provides an implementation of the HPCG benchmark optimized for Intel® Xeon® processors with support for the latest processor technologies, including Intel® Advanced Vector Extensions (Intel® AVX), Intel® Advanced Vector Extensions 2 (Intel® AVX2), Intel® Advanced Vector Extensions 512 (Intel® AVX-512). The benchmark can be found in the “/benchmarks/hpcg” subdirectory under the oneAPI MKL installation. To prepare the benchmark, follow the instructions at https://software.intel.com/content/www/us/en/develop/documentation/onemkl-linux-developer-guide/top/intel-math -kernel-library-benchmarks/intel-optimized-high-performance-conjugate-gradient-benchmark/getting-started-with-int el-optimized-hpcg.html Use the Scalable processor optimized binary xhpcg\_skx. The Intel Optimized HPCG package also includes the source code necessary to build these versions of the benchmark for other MPI implementations. Also note that: • Small problem sizes will produce very good results. However, the problem size must be large enough so that it will not fit into cache; otherwise it is considered an invalid run. It should occupy a minimum 25% of physical memory. Optimum local dimension grid size will need to determine through practical evaluation. • Longer runtimes beyond the required 3600s runtime do not appear to impact performance. • Best results are obtained when using from 1 to 1.25 MPI process per total core count and 12 to 16 OpenMP threads per MPI process. The optimal configuration will need to be determined through experimentation. Skip SMT cores when assigning threads. The STREAM benchmark is a simple, synthetic benchmark designed to measure sustainable memory bandwidth (in MB/s) and a corresponding computation rate for four simple vector kernels (Copy, Scale, Add and Triad). It is also part of the HPCC benchmark suite. Its source code is freely available from http://www.cs.virginia.edu/stream/. It measures: • Sustainable memory bandwidth • Corresponding computation rate for a simple vector kernel The general rule for STREAM is that each array must be at least four times (4×) the sum of all last-level caches used in the run. STREAM may be run in its standard form, or it may be optimized. When optimized, results must be identified as such (see the STREAM FAQ at http://www.cs.virginia.edu/stream/ref.html). For instructions on how to obtain the best performance of the standard STREAM benchmark on Intel processors, see https://software.intel.com/content/www/us/en/develop/articles/optimizing-memory-bandwidth-on-stream-triad.html Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 9 | Total 10 The numactl tool can be used to view the configuration and status of the NUMA node of the current server. For example, CPU core count, memory size of each node, and the distance between different nodes. The process can be bound to the specified CPU core through this tool, and the specified CPU core will run the corresponding process. Figure: numactl sample output You can view the of the status of current NUMA nodes using numastat. This includes local and remote memory access by CPU cores. Figure: numastat sample output The Processor Count Monitor (PCM) can be used to monitor performance indicators of the Intel CPU core. PCM is often used to monitor the bandwidth of persistent memory. The tool can be downloaded from https://github.com/opcm/pcm. Figure: PCM example of monitoring persistent memory bandwidth Optimization Guide | HPC Cluster Tuning for 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 10 | Total 10 Notices & Disclaimers Intel technologies may require enabled hardware, software or service activation. No product or component can be absolutely secure. Your costs and results may vary. Code names are used by Intel to identify products, technologies, or services that are in development and not publicly available. These are not "commercial" names and not intended to function as trademarks The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. © Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others. --- # Unknown Optimization Guide Intel® Deep Learning Boost Intel® Xeon® Scalable Processor Deep Learning with Intel® AVX512 and Intel® Deep Learning Boost Tuning Guide on 3rd Generation Intel® Xeon® Scalable Processors Revision Record ................................................................................................................................. 3 1. Overview ...................................................................................................................... 4 2. Introducing Intel® AVX-512 and Intel® Deep Learning Boost ........................... 5 3. BIOS Settings and Hardware Configurations ....................................................... 6 3.1. BIOS Settings ............................................................................................................................. 6 3.2. Recommended Hardware Configurations ..................................................................... 6 4. Linux System Optimization ...................................................................................... 6 4.1. OpenMP Parameter Settings ............................................................................................... 6 4.2. Number of CPU cores ............................................................................................................. 6 4.3. Impact of NUMA Configuration .......................................................................................... 7 4.4. Configuration of Linux Performance Governor ........................................................... 7 4.5. CPU C-States Settings ............................................................................................................ 7 5. Using Intel® Optimization for TensorFlow\* Deep Learning Framework ........ 7 5.1. Deploying Intel® Optimization for TensorFlow\* Deep Learning Framework . 7 5.2. Inferencing using Intel® Optimization for TensorFlow\* DL Model with FP32/INT8 support .......................................................................................................................... 8 5.3. Training using Intel® Optimization for TensorFlow\* DL Model with FP32/ INT8 Support ...................................................................................................................................... 8 5.4. Applications – Inferencing and Training Using Intel Optimized TensorFlow Wide & Deep Model ......................................................................................................................... 9 5.5. Intel® Math Kernel Library (MKL) Threadpool-Based TensorFlow (Optional) ................................................................................................................................................................. 10 6. Using PyTorch\*, a Deep Learning Framework .................................................. 10 6.1. Deploying PyTorch ................................................................................................................ 10 6.2. Optimization Recommendations for Training and Inferencing PyTorch- based Deep Learning Models .................................................................................................... 11 6.3. Introducing and Using Intel® Extension for PyTorch .............................................. 11 7. Accelerating Vector Recall in the Recommendation System with Intel® Deep Learning Boost VNNI ........................................................................................ 11 8. AI Neural Network Model Quantization ............................................................. 12 Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 2 | Total 24 8.1. AI neural network quantization process ...................................................................... 12 8.2. Intel® AI Quantization Tools for TensorFlow ............................................................. 14 8.3. Installing Intel® AI Quantization Tools for TensorFlow ......................................... 16 8.4. Using Intel® AI Quantization Tools for TensorFlow ................................................ 17 8.4.1. Dataset preparation: ............................................................................................................................. 17 8.4.2. Model preparation: ................................................................................................................................ 17 8.4.3. Run Tuning: ............................................................................................................................................... 17 8.4.4. Run Benchmark: ...................................................................................................................................... 17 9. Using Intel® Distribution of OpenVINO™ Toolkit for Inference Acceleration ......................................................................................................................................... 18 9.1. Intel® Distribution of OpenVINO™ Toolkit ................................................................... 18 9.2. Deploying the Intel® Distribution of OpenVINO™ Toolkit ..................................... 19 9.3. Using Deep Learning Deployment Toolkit (DLDT) of the Intel® Distribution of OpenVINO Toolkit ..................................................................................................................... 19 9.4. Using the Intel® Distribution of OpenVINO™ Toolkit for INT8 Inference Acceleration ...................................................................................................................................... 19 10. Using Intel® DAAL for Accelerated Machine Learning .................................. 21 10.1. Intel® Distribution for Python\* ....................................................................................... 21 10.2. Intel® DAAL ............................................................................................................................. 22 10.3. Installing Intel® Distribution for Python & Intel® DAAL ....................................... 23 10.4. Using Intel® DAAL ................................................................................................................ 23 11. References .............................................................................................................. 24 Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 3 | Total 24 Revision Record Date Rev. Description 04/06/2021 1.0 Initial public release. Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 4 | Total 24 This user guide is intended to explain how the 3 rd Gen Intel® Xeon® Scalable Processor platform ((codename Ice Lake/Whitley) is used for machine learning and deep learning-related tasks. Executing machine learning and deep learning workloads on the Intel® Xeon® Scalable Processor platform has the following advantages: • The platform is very suitable for processing memory-intensive workloads and 3D-CNN topologies used in medical imaging, GAN, seismic analysis, genome sequencing, etc. • The simple numactl command can be used for flexible core control; it is still very suitable for real-time inference even when the number of batches is small. • It is supported by a powerful ecosystem and can be used for distributed training (such as for computations directly at the data source) directly on large-scale clusters. This avoids the additional costs for large storage capacity and expensive cache mechanisms that are usually required for the training of scaled architecture. • Multiple types of workloads (HPC/BigData/AI) are supported on the same cluster to achieve better TCO. • It satisfies the computing requirements in many real deep learning applications via SIMD acceleration. • The same infrastructure can be used directly for training and inference. The development and deployment of typical deep learning applications involve the following stages: These different stages require the allocation of following resources, and choosing the right resources can greatly accelerate the efficiency of your AI services: • Computational power • Memory • Storage for datasets • Communication link between compute nodes • Optimized software All the processes including dataset preparation, model training, model optimization, and model deployment, can be done on an Intel® Xeon® Scalable Processor platform-based infrastructure which also supports machine learning/deep learning platforms for training and inference. The proposed infrastructure is shown in the figure below: Dataset preparation Model training and tuning Model optimization and migrated learning Model deployment Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 5 | Total 24 Intel® Advanced Vector Extensions 512 (Intel® AVX-512) is a “single instruction, multiple data” (SIMD) instruction set based on x86 processors. Compared to traditional “single instruction, single data” instructions, a SIMD instruction allows for executing multiple data operations with a single instruction. As the name implies, Intel® AVX-512 has a register width of 512 bytes, and it supports 16 32-byte single-precision floating-point numbers or 64 8-byte integers. Intel® Xeon® Scalable Processors support multiple types of workloads, including complex AI workloads, and improve AI computation performance with the use of Intel® Deep Learning Boost (Intel® DL Boost). Intel Deep Learning Boost includes Intel® AVX-512 VNNI (Vector Neural Network Instructions) which is an extension to the Intel® AVX-512 instruction set. It can combine three instructions into one for execution, which further unleashes the computing potential of next-generation Intel® Xeon® Scalable Processors and increases the inference performance of the INT8 model. Both 2nd-Generation and 3rd-Generation Intel® Xeon® Scalable Processors support VNNI. Platforms not using VNNI require the vpmaddubsw, vpmaddwd and vpaddd instructions to complete the multiply- accumulate operations in INT8 convolution operation: Platforms using VNNI require only one instruction, “vpdpbusd”, to complete the INT8 convolution operation: Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 6 | Total 24 The configuration items that can be optimized in BIOS and their recommended values are as follows: Configuration item Recommended value Hyper-Threading Enable SNC (Sub NUMA) Disable Boot performance mode Max Performance Turbo Mode Enable Hardware P-State Native Mode Machine learning workloads, and in particular deep learning workloads, are usually used for compute-intensive applications. Hence, they require a selection of suitable types of memory, CPU, hard drives, and other computing resources to achieve optimal performance. In summary, the following common configurations are recommended: Memory configuration The utilization of all memory channels is recommended so that the bandwidth of all memory channels can be utilized. CPU configuration FMA, the Intel AVX-512 acceleration module in Intel processors, is an important component in unleashing computational performance, and artificial intelligence-related workloads are usually part of compute-intensive applications. In order to achieve better computing performance, it is recommended to use the Intel Xeon® Scalable Processors Gold 6 series (or above) which have two Intel AVX512 computational modules per core. Network configuration If cross-node training clusters are required, then it is recommended to choose high-speed networking such as 25G/100G networks for better scalability. Hard drive configuration For high IO efficiency for workloads, SSDs and drives with higher read and write speeds are recommended. The recommended configuration for the main parameters is as follows: • OMP\_NUM\_THREADS = “number of cpu cores in container” • KMP\_BLOCKTIME = 1 or 0 (set according to actual type of model) • KMP\_AFFINITY=granularity=fine, verbose, compact,1,0 The main impact of the number of CPU cores on inference performance is as follows: • When batchsize is small (in online services for instance), the increase in inference throughput gradually weakens as the number of CPU cores increases; in practice, 8-16 CPU cores is recommended for service deployment depending on the model used. • When batchsize is large (in offline services for instance), the inference throughput can increase linearly as the number of CPU cores increases; in practice, more than 20 CPU cores is recommended for service deployment. # taskset -C xxx-xxx –p pid (limits the number of CPU cores used in service) Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 7 | Total 24 For NUMA-based servers, there is usually a 5-10% increase in performance when configuring NUMA on the same node compared to using it on different nodes. #numactl -N NUMA\_NODE -l command args ... (controls NUMA nodes running in service) Performance: As the name suggests, efficiency is the only consideration and the CPU frequency is set to its peak to achieve the best performance. # cpupower frequency-set -g performance CPU C-States: To reduce power consumption when the CPU is idle, the CPU can be placed in the low-power mode. There are several power modes available for each CPU which are collectively referred to as C-states or C-modes. Disabling C-States can increase performance. #cpupower idle-set -d 2,3 TensorFlow\* is one of the most popular deep learning frameworks used in large-scale machine learning (ML) and deep learning (DL) applications. Since 2016, Intel and Google\* engineers have been working together to use Intel® oneAPI Deep Neural Network Library (Intel® oneDNN) to optimize TensorFlow\* performance and accelerate its training and inference performance on the Intel® Xeon® Scalable Processor platform. Reference: https://www.intel.com/content/www/us/en/develop/articles/intel-optimization-for-tensorflow-installation- guide.html Step 1: Install a Python3.x environment. Here is an example to illustrate how to build Python\* 3.6 with Anaconda\* Reference: https://www.anaconda.com/products/individual Download and install the latest version of Anaconda # wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86\_64.sh # sh Anaconda3-2020.02-Linux-x86\_64.sh # source /root/.bashrc # conda install python=3.6 (create a Python3.6 environment) #(base) \[root@xx\]# python -V Python 3.6.10 Step 2: Install the Intel optimation for TensorFlow\*: intel-tensorflow. Install the latest version (2.x) # pip install intel-tensorflow If you need to install tensorflow1.x, we recommend installing the following version to take advantage of the performance acceleration on the 3 rd Gen Intel® Xeon® Scalable Processor platform: # pip install https://storage.googleapis.com/intel-optimized-tensorflow/intel\_tensorflow- 1.15.0up2-cp36-cp36m-manylinux2010\_x86\_64.whl Step 3: Set run-time optimization parameters. Reference: https://github.com/IntelAI/models/blob/master/docs/general/tensorflow/GeneralBestPractices.md Usually, the following two methods are used for inference, which use different optimization settings Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 8 | Total 24 Batch inference: Batch Size >1, measures the number of input tensors that can be processed per second. Usually, all the physical cores in the same CPU socket can be used for batch inference to achieve the best performance. On-line inference (also known as real-time inference): Batch Size=1, a measure of time needed to process one input tensor (when the batch size is 1). In real-time inference, multiple instances are run concurrently to achieve the optimal throughput. 1: Obtaining the number of physical cores in the system: To confirm the current number of physical cores, we recommend using the following command: # lscpu | grep "Core(s) per socket" | cut -d':' -f2 | xargs In this example, we assume 8 physical cores. 2: Setting optimization parameters: Optimization parameters are configured using the two following methods. Please choose the configuration method according to your needs. Method 1: Configure environment parameters directly: export OMP\_NUM\_THREADS=physical cores export KMP\_AFFINITY="granularity=fine,verbose,compact,1,0" export KMP\_BLOCKTIME=1 export KMP\_SETTINGS=1 Method 2: Add environment variables in the Python code that is running: import os os.environ\["KMP\_BLOCKTIME"\] = "1" os.environ\["KMP\_SETTINGS"\] = "1" os.environ\["KMP\_AFFINITY"\]= "granularity=fine,verbose,compact,1,0" if FLAGS.num\_intra\_threads > 0: os.environ\["OMP\_NUM\_THREADS"\]= # config = tf.ConfigProto() config.intra\_op\_parallelism\_threads = # config.inter\_op\_parallelism\_threads = 1 tf.Session(config=config) This section mainly explains how to run the inference benchmark on ResNet50. You can refer to the following reference to inference using your machine learning/deep learning model. Reference: https://github.com/IntelAI/models/blob/master/docs/image\_recognition/tensorflow/Tutorial.md Taking inference benchmarking for ResNet50\* as an example, FP32, BFloat16, and Int8 are supported for model inference. Reference: https://github.com/IntelAI/models/blob/master/benchmarks/image\_recognition/tensorflow/resnet50v1\_5/README.m d FP32-based model inference: https://github.com/IntelAI/models/blob/master/benchmarks/image\_recognition/tensorflow/resnet50v1\_5/README.m d#fp32-inference-instructions INT8-based model inference: https://github.com/IntelAI/models/blob/master/benchmarks/image\_recognition/tensorflow/resnet50v1\_5/README.m d#int8-inference-instructions This section mainly explains how to run a training benchmark on ResNet50. You can refer to the following reference to run your machine learning/deep learning model training. FP32-based training: https://github.com/IntelAI/models/blob/master/benchmarks/image\_recognition/tensorflow/resnet50v1\_5/README.m d#fp32-training-instructions Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 9 | Total 24 Among the many operations in the data center, it is a typical application to use recommendation systems to match users with content they are interested in. Recommendation system is a type of information filtering system that learns about users' interests according to their profiles and past behavior records and predict their ratings or preferences for a given item. It changes the way a business communicates with users and enhances the interaction between the business and its users. When using deep learning, we find, from a large amount of complex raw data, the deep interactions between features that are difficult to be expressed with traditional machines using artificial feature engineering. Related study outcomes include Wide & Deep, DeepFM, FNN, DCN, and other models. Using the Wide & Deep model as an example, the core idea is to take advantage of both the memorization capability of a linear model and the generalization capability of the DNN model and optimize the parameters in these models at the same time during training. This will result in better overall model prediction capabilities. Its structure is shown in the figure below: Wide "Wide" is a generalized linear model, and its inputs mainly include original and interactive features. We can use cross- product transformation to build the interactive features of K-group: Deep “Deep” is a DNN model, and the calculation for each layer is as follows: Co-training The Wide & Deep model uses co-training instead of integration. The difference is that co-training shares a loss function, then updates the parameters in either part of the model at the same time, while integration trains N models independently and fuses them together afterwards. Therefore, the output of the model is: The above is the background information on the Wide & Deep model. Next, we will describe how to run inference benchmarking for the Wide & Deep model. Reference: https://github.com/IntelAI/models/blob/master/docs/recommendation/tensorflow/Tutorial.md Dataset preparation: https://github.com/IntelAI/models/tree/master/benchmarks/recommendation/tensorflow/wide\_deep\_large\_ds#Prepar e-dataset FP32-based model inference: https://github.com/IntelAI/models/tree/master/benchmarks/recommendation/tensorflow/wide\_deep\_large\_ds#fp32- inference-instructions INT8-based model inference: Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 10 | Total 24 https://github.com/IntelAI/models/tree/master/benchmarks/recommendation/tensorflow/wide\_deep\_large\_ds#int8- inference-instructions FP32-based training:https://github.com/IntelAI/models/tree/master/benchmarks/recommendation/tensorflow/wide\_deep\_large\_ds #fp32-training-instructions Starting with TensorFlow 2.3.0, a new feature has been added. You can choose Eigen Threadpool for TensorFlow multi- threading support instead of OpenMP, by using the compiling option --config=mkl\_threadpool instead of -- config=mkl, when compiling the Tensorflow source code. If the user wants to try this feature with TensorFlow 1.15, they need to download the source code that has been ported and optimized by Intel and compile it (it should be particularly pointed out that Bazel\* 0.24.1 needs to be installed for the purpose): # git clone https://github.com/Intel-tensorflow/tensorflow.git # git checkout -b tf-1.15-maint remotes/origin/tf-1.15-maint # bazel --output\_user\_root=$BUILD\_DIR build --config=mkl\_threadpool -c opt --copt=-O3 //tensorflow/tools/pip\_package:build\_pip\_package bazel-bin/tensorflow/tools/pip\_package/build\_pip\_package $BUILD\_DIR After successfully completing the steps above, the TensorFlow wheel file can be found under the $BUILD\_DIR path. For example: tensorflow-1.15.0up2-cp36-cp36m-linux\_x86\_64.whl. The installation steps are as follows: # pip uninstall tensorflow # pip install $BUILD\_DIR/.whl --user Reference: https://software.intel.com/content/www/us/en/develop/articles/getting-started-with-intel-optimization-of- pytorch.html Environment: Python3.6 or above Step 1: Visit the official PyTorch website: https://pytorch.org/ Step 2: Select CPU Currently, Intel oneDNN is integrated into the official version of PyTorch, so there is no need for additional installation to have accelerated performance on the Intel® Xeon® Scalable Processor platform. Select “None” for CUDA. See the figure below for details. Step 3: Installation Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 11 | Total 24 pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch\_stable.html You may refer to the following website to learn more about optimization parameter settings for PyTorch\* on the Intel® Xeon® Scalable Processor platform. Reference: https://software.intel.com/content/www/us/en/develop/articles/how-to-get-better-performance-on- pytorchcaffe2-with-intel-acceleration.html Intel® Extension for PyTorch is a Python extension of PyTorch that aims to improve the computational performance of PyTorch on Intel® Xeon® Processors. Not only does this extension includes additional functions, but it also provides performance optimizations for new Intel hardware. The Github links to the Intel Extension for PyTorch are: https://github.com/intel/intel-extension-for-pytorch https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Features-and- Functionality/IntelPyTorch\_Extensions\_AutoMixedPrecision A problem that needs to be resolved in the recommendation system is how to generate a recommendation list with the length of K for a given user that matches their interests and needs as much as possible (high accuracy) and as fast as possible (low latency)? Conventional recommendation systems include two components: vector recall and ranking. “Vector recall” roughly filters out hundreds or thousands of items from a huge recommendation pool that will most likely interest the user, passes the results on to the ranking module for further sorting before the final recommendation results are obtained. Vector recall can be converted into a high-dimensional vector similarity search problem. The Hierarchical Navigable Small World (HNSW) algorithm is a type of Approximate Nearest Neighbor (ANN) vector similarity search algorithm based on graph structures. It is also one of the fastest and most precise algorithms. Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 12 | Total 24 Usually, the data type of the raw vector data is FP32. For many applications (such as image search), vector data can be expressed in INT8/INT6 and the impact of quantization error on the final search result is limited. The “VNNI intrinsic” instruction can be used for inner product calculations for INT8/INT6 vectors. Many experiments have shown that QPS Performance has greatly improved, and that recall rate remains virtually unchanged. The reason for the improvement in QPS performance is that the memory–bandwidth ratio for INT8/INT16 is smaller than for FP32, and VNNI instructions accelerate the distance calculations in addition. Currently, optimized source code is implemented based on the HNSWLib\[10\] open source project. We have already ported it to the Faiss\[9\] framework, which is widely used in the industry. To achieve the optimal performance, the following deployment steps are recommended: 1. Bind NUMA 2. Each physical CPU core executes a single query process Reference command (using 1 socket and 24 cores as an example): # numactl -C 0-23 When the dataset is large (in the range of 100 million to billions for example), the traditional approach is to slice the dataset into several smaller datasets to get the topK for each dataset separately before merging them back together at the end. Since the amount of communication between multiple machines has increased, latency also increases while the QPS performance decreases. Our experience with HNSW on large datasets show that it is better not to slice datasets if possible, but rather establish indices and execute searches on complete datasets to get the best performance. When a dataset is too large and there is not enough DDR space (e.g. local memory space), you can consider using PMem (Intel® Optane™ persistent memory) By saving the HNSW layer0 data on PMEM, the size of the dataset that can be supported has greatly increased (a single socket can support an INT8 database with up to 4 billion records @ d=100). The persistence feature allows you to skip the loading process for a large amount of data, which greatly reduces the time it takes to initialize. Computations in neural networks are mainly concentrated in the convolutional layer and the fully connected layer. The computations on these two layers can be expressed as: Y = X \* Weights + Bias. Therefore, it is natural to focus on matrix Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 13 | Total 24 multiplication to optimize performance. The way to begin neural network model quantization is by trading-off precision (limited) for performance improvement. By replacing 32-bit floating-point numbers with low-precision integers for matrix operations, it not only speeds up calculations, but also compresses the model, thus saving memory bandwidth. There are three approaches to the quantization of neural network models: • Post-Training Quantization, which is supported by most AI frameworks. • Quantization-Aware-Training, which inserts the FakeQuantization node into the FP32 model when the training converges. It increases the quantization-induced noise. During the backpropagation stage of the training, the model weights fall into a finite interval which results in better quantization precision. • Dynamic Quantization is very similar to PTQ. They are both quantization methods used on post-trained models. The difference lies in that the quantization factor in the activation layer is dynamically decided by the data range used when the neural network model is run, while for PTQ samples from a small-scale pre-processed dataset are used to obtain data distribution and range information in the activation layer, then records it permanently in the newly generated quantization model. Of the Intel® AI Quantization Tools for TensorFlow which we will talk about later on, onnxruntime supports this method at the backend only. The basic procedure for the post-training quantization of neural networks is as follows: 1. Fuse FP32 OP to INT8 OP. For example, MatMul, BiasAdd and ReLU can be fused into a single quantized OP at the fully connected layer, QuantizedMatMulWithBiasAndRelu. Different neural network frameworks support different fuse-able OPs. For Intel® AI Quantization Tools for TensorFlow, which will be discussed later on, below we can see a list of fuse-able OPs supported by TensorFlow: https://github.com/intel/lpot/blob/master/lpot/adaptor/tensorflow.yaml#L190. For fuse-able OPs supported by pyTorch, please see : https://github.com/intel/lpot/blob/master/lpot/adaptor/pytorch\_cpu.yaml#L124 2. Quantize weights and save them in the quantized model. 3. Quantize the input/activation layer by sampling the calibration dataset to acquire the distribution and range information of the data in the activation layer, which is then recorded in the newly generated quantized model. 4. The Requantize operation is fused into its corresponding INT8 OP to generate the final quantized model. Using a simple model which includes two layers of MatMul as an example, we can observe the quantization process as follows: Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 14 | Total 24 Intel® AI Quantization Tools for TensorFlow is an open source Python library which provides API access for low- precision quantization for cross-neural network development frameworks. It is intended to provide simple, easy-to-use and precision-driven auto tuning tools for the quantization of models for accelerating the inference performance of low-precision models on the 3rd Gen Intel® Xeon® Scalable Processor platform. Reference: https://github.com/intel/lpot Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 15 | Total 24 Intel® AI Quantization Tools for TensorFlow currently support the following Intel optimized deep learning frameworks: • Tensorflow\* • PyTorch\* • Apache\* MXNet • ONNX Runtime Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 16 | Total 24 The frameworks and their versions that have already been verified are shown below: OS Python Framework Version CentOS 7.8 Ubuntu 18.04 3.6 3.7 TensorFlow 2.2.0 1.15.0 UP1 1.15.0 UP2 2.3.0 2.1.0 1.15.2 PyTorch 1.5.0+cpu Apache\* MXNet 1.7.0 1.6.0 ONNX Runtime 1.6.0 The tuning strategies supported by Intel® AI Quantization Tools for Tensorflow include: • Basic • Bayesian • Exhaustive • MSE • Random • TPE The workflow for Intel® AI Quantization Tools for TensorFlow is shown below. The model quantization parameters matching the precision loss target are automatically selected according to the set tuning strategy, and the quantized model is generated: For details on installation, refer to: https://github.com/intel/lpot/blob/master/README.md Step 1: Use Anaconda to create a Python3.x virtual environment with the name of lpot. We are using Python 3.7 here as an example: # conda create -n lpot python=3.7 # conda activate lpot Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 17 | Total 24 Step 2: Install lpot; the two following installation methods are available: Installing with the binary file: # pip install lpot Install from the source code # git clone https://github.com/intel/lpot.git # cd lpot # pip install –r requirements.txt # python setup.py install We are using ResNet50 v1.0 as an example to explain how to use this tool for quantization. Step 1: Download and decompress the ImageNet validation dataset: # mkdir –p img\_raw/val && cd img\_raw # wget http://www.image- net.org/challenges/LSVRC/2012/dd31405981ef5f776aa17412e1f0c112/ILSVRC2012\_img\_val.tar # tar –xvf ILSVRC2012\_img\_val.tar -C val Step 2: Move the image files to the child directories sorted by label: # cd val # wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash Step 3: Use the script, prepare\_dataset.sh, to convert raw data to the TFrecord format: # cd examples/tensorflow/image\_recognition # bash prepare\_dataset.sh --output\_dir=./data --raw\_dir=/PATH/TO/img\_raw/val/ -- subset=validation Reference: https://github.com/intel/lpot/tree/master/examples/tensorflow/image\_recognition#2-prepare-dataset # wget https://storage.googleapis.com/intel-optimized- tensorflow/models/v1\_6/resnet50\_fp32\_pretrained\_model.pb Edit the file: examples/tensorflow/image\_recognition/resnet50\_v1.yaml, making sure the dataset path for quantization\\calibration, evaluation\\accuracy and evaluation\\performance is the user's real local path. It should be where the TFrecord data generated previously during the data preparation stage, is located. # cd examples/tensorflow/image\_recognition # bash run\_tuning.sh --config=resnet50\_v1.yaml \\ --input\_model=/PATH/TO/resnet50\_fp32\_pretrained\_model.pb \\ --output\_model=./lpot\_resnet50\_v1.pb Reference: https://github.com/intel/lpot/tree/master/examples/tensorflow/image\_recognition#1-resnet50-v10 # bash run\_benchmark.sh --input\_model=./lpot\_resnet50\_v1.pb --config=resnet50\_v1.yaml The output is shown below. The performance data is for reference only: Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 18 | Total 24 accuracy mode benchmark result: Accuracy is 0.739 Batch size = 32 Latency: 1.341 ms Throughput: 745.631 images/sec performance mode benchmark result: Accuracy is 0.000 Batch size = 32 Latency: 1.300 ms Throughput: 769.302 images/sec Intel® Distribution of OpenVINO TM toolkit’s official website and download websites: https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html Online documentation: https://docs.openvinotoolkit.org/latest/index.html Online documentation in Simplified Chinese: https://docs.openvinotoolkit.org/cn/index.html The Intel® Distribution of OpenVINO TM toolkit is used to accelerate the development of computer vision and deep learning applications. It supports deep learning applications with various accelerators, including CPUs, GPUs, FPGAs, and Intel® Movidius™ CPUs on the Intel® Xeon® Processor platform, and it also directly supports heterogenous execution. The Intel® Distribution of OpenVINO TM toolkit is designed to improve the performance and reduce the development time of computer vision processing and deep learning inference solutions. It includes two components: computer vision and deep learning development kits. The Deep Learning Deployment Toolkit (DLDT) is a cross-platform tool for accelerating deep learning inference performance, and includes the following components: • Model Optimizer: converts models trained with Caffe\*, TensorFlow, Mxnet, and other frameworks into Intermediate Representations (IR). • Inference Engine: executes the IR on CPU, GPU, FPGA, VPU, and other hardware. It automatically calls the Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 19 | Total 24 hardware acceleration kit to accelerate inference performance. The Intel® Distribution of OpenVINO TM toolkit Workflow: You can refer to the installation documentation in Simplified Chinese: Installing the Intel ® Distribution of OpenVINO™ toolkit for Linux\*: https://docs.openvinotoolkit.org/downloads/cn/I03030-5-Install Intel\_ Distribution of OpenVINO\_ toolkit for Linux – OpenVINO\_ Toolkit.pdf Introduction to the Intel ® Deep Learning Deployment toolkit: https://docs.openvinotoolkit.org/downloads/cn/I03030-9-Introduction to Intel\_ Deep Learning Deployment Toolkit – OpenVINO\_ Toolkit.pdf Image Classification C++ Sample (Async): https://docs.openvinotoolkit.org/downloads/cn/I03030-10-Image Classification Cpp Sample Async – OpenVINO\_ Toolkit.pdf Object Detection C++ Sample (SSD): https://docs.openvinotoolkit.org/downloads/cn/I03030-11-Object Detection Cpp Sample SSD - OpenVINO\_ Toolkit.pdf Automatic Speech Recognition C++ Sample: https://docs.openvinotoolkit.org/downloads/cn/I03030-12-Automatic Speech Recognition Cpp Sample - OpenVINO\_ Toolkit.pdf Action Recognition Python\* Demo: https://docs.openvinotoolkit.org/downloads/cn/I03030-13-Action Recognition Python Demo - OpenVINO\_ Toolkit.pdf Crossroad Camera C++ Demo: https://docs.openvinotoolkit.org/downloads/cn/I03030-14-Crossroad Camera Cpp Demo - OpenVINO\_ Toolkit.pdf Human Pose Estimation C++ Demo: https://docs.openvinotoolkit.org/downloads/cn/I03030-15-Human Pose Estimation Cpp Demo - OpenVINO\_ Toolkit.pdf Interactive Face Detection C++ Demo: https://docs.openvinotoolkit.org/downloads/cn/I03030-16-Interactive Face Detection Cpp Demo - OpenVINO\_ Toolkit.pdf By inferencing on an INT8-based model and using Intel DL Boost on the Intel® Xeon® Scalable Processor platform for acceleration, you can greatly increase inference efficiency. At the same time, it saves computing resources and reduces Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 20 | Total 24 power consumption. The 2020 version and later versions of OpenVINO™ all provide INT8 quantization tools which support the quantization of FP32-based models. The INT8-based model quantization tool provided by OpenVINO is a Post-training Optimization Toolkit (POT) is used to optimize and quantize trained models. There is no need to re-train or fine-tune models or to modify model structures. The figure below shows the process of how OpenVINO is used to optimize new models. Step 0: Acquire the trained model, Step 1: POT generation and optimization, Step 2: Optional operation (Whether to fine-tune the model will be determined according to the actual situation for better accuracy), and Step 3: Use OpenVINO IE for model inference. POT provides an independent command line tool and Python API and it mainly supports the following features: ➢ Two types of post-training INT8 quantization algorithms: fast DefaultQuantization and precise AccuracyAwareQuantization. ➢ Uses the Tree-structured Parzen Estimator for global optimization of post-training quantization parameters ➢ Supports both symmetrical and asymmetrical quantization ➢ Supports compression for multiple hardware platforms (CPU, GPU) ➢ Quantizes all channels at the convolutional layer and full connection layer ➢ Supports multiple applications: computer vision, recommendation system ➢ Provides customized optimization methods through provided API Please refer to the following websites for instructions of operations and use: Introduction to the Post-Training Optimization Toolkit https://docs.openvinotoolkit.org/latest/pot\_README.html Low Precision Optimization Guide: https://docs.openvinotoolkit.org/latest/pot\_docs\_LowPrecisionOptimizationGuide.html Post-training Optimization Toolkit Best Practices https://docs.openvinotoolkit.org/latest/pot\_docs\_BestPractices.html Post-training Optimization Toolkit Frequently Asked Questions https://docs.openvinotoolkit.org/latest/pot\_docs\_FrequentlyAskedQuestions.html INT8 quantization and optimization using DL Workbench’s web interface https://docs.openvinotoolkit.org/latest/workbench\_docs\_Workbench\_DG\_Int\_8\_Quantization.html Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 21 | Total 24 Intel ® Data Analytics Acceleration Library (Intel® DAAL) As a branch of artificial intelligence, machine learning is currently attracting a huge amount of attention. Machine learning-based analytics is also getting increasingly popular. The reason is that, when compared to other analytics, machine learning can help IT staff, data scientists, and various business teams and their organizations to quickly unleash the strengths of AI. Furthermore, machine learning offers many new commercial and open-source solutions, providing a vast ecosystem for developers. In addition, developers can choose from a variety of open-source machine learning libraries such as Scikit-learn, Cloudera\* and Spark\* MLlib. Intel® Distribution for Python\* is a Python development toolkit for artificial intelligence software developers. It can be used to accelerate computational speed of Python on the Intel® Xeon® Scalable Processor platform. It is available at Anaconda\*, and it can also be installed and used with Conda\*, PIP\*, APT GET, YUM, Docker\*, among others. Reference and download site: https://software.intel.com/content/www/us/en/develop/tools/distribution-for-python.html Intel® Distribution for Python\* features: ✓ Out-of-the-box: no or little change to source code required to achieve faster Python application performance. ✓ The Integrated Intel® performance libraries: Intel® Math Kernel Library (MKL) and Intel® Data Analytics Acceleration Library (Intel® DAAL), for example, can be used to accelerate NumPy, SciPy, and scikit-learn\* ✓ Latest vector and multithread instructions: Numba\* and Cython can be combined to improve concurrency and vectorization efficiency. Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 22 | Total 24 Intel® Data Analytics Acceleration Library (DAAL) is designed for data scientists to accelerate data analytics and prediction efficiency. In particular, it can take full advantage of vectorization and multithreading for applications with huge amount of data, as well as utilize other technologies to increase the overall performance of machine learning on the Intel® Xeon® Scalable Processor platform. Intel® DAAL is a complete end-to-end software solution designed to help data scientists and analysts quickly build everything from data pre-processing, to data feature engineering, data modeling and deployment. It provides various data analytics needed to develop machine learning and analytics as well as high-performance building blocks required by algorithms. It currently supports linear regression, logic regression, LASSO, AdaBoost, Bayesian classifiers, support vector machines, k-nearest neighbors, k-means clustering, DBSCAN clustering, various types of decision trees, random forest, gradient boosting, and other classic machine learning algorithms. These algorithms are highly optimized to achieve high performance on Intel® processors. For example, a leading big data analytics technology and service provider has used these resources to improve the performance of data mining algorithms by several times. To make it easier for developers to use Intel® DAAL in machine learning applications in Intel-based environments, Intel has open-sourced the entire project (https://github.com/intel/daal), and provides full-memory, streaming and distributed algorithm support for different big data scenarios. For example, DAAL Kmeans can be combined with Spark to perform multi-node clustering on a Spark cluster. In addition, DAAL provides interfaces for C++, Java\*, and Python. Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 23 | Total 24 DAAL4py In order to provide better support for Scikitlearn, which is the most widely used with Python, Intel® DAAL provides a very simple Python interface, DAAL4py (please see the open source website for more details: https://github.com/IntelPython/daal4py). It can be used seamlessly with Scikitlearn and provides acceleration for machine learning algorithms at the underlying layer. Developers do not need to modify the Scikitlearn source code to benefit from the advantages of automatic vectorization and multithreading. DAAL4py currently supports the following algorithms in Scikitlearn: • Sklearn linear regression, Sklearn ridge regression and logic regression • PCA • KMeans • pairwise\_distance • SVC (SVM classification) Download and install Intel® Distribution for Python\* (Intel® DAAL already included): FYI: https://software.intel.com/en-us/distribution-for-python Installing Intel® DAAL separately: FYI: https://software.intel.com/en-us/daal. Intel® DAAL Developer Guide: FYI:https://software.intel.com/en-us/daal-programming-guide There are two ways to use Intel® DAAL to accelerate scikit-learn: Method 1: Using the command line # python -m daal4py Method 2: Adding it to source code import daal4py.sklearn daal4py.sklearn.patch\_sklearn('kmeans') Deep Learning with Intel® AVX512 and Intel® DL Boost on 3rd Generation Intel® Xeon® Scalable Processors Revision 1.0 Page 24 | Total 24 \[1\] Intel® AVX-512 info: https://colfaxresearch.com/skl-avx512/ \[2\] Intel® Optimized AI Frameworks: https://software.intel.com/en-us/frameworks \[3\] Intel® Distribution of OpenVINO™ toolkit: https://docs.openvinotoolkit.org/ \[4\] Intel® Analytics Zoo: https://github.com/intel-analytics/analytics-zoo \[5\] Hands-on IDP and Intel® DAAL : https://software.intel.com/en-us/videos/get-your-hands-dirty-with-intel- distribution-for-python \[6\] IDP benchmarks: https://software.intel.com/en-us/distribution-for-python/benchmarks \[7\] Intel® DL Boost : https://www.intel.ai/increasing-ai-performance-intel-dlboost/#gs.3cxhiw \[8\] Intel® DL Boost: https://www.intel.com/content/dam/www/public/us/en/documents/product-overviews/dl-boost- product-overview.pdf https://software.intel.com/content/www/us/en/develop/articles/lower-numerical-precision-deep-learning-inference- and-training.html \[9\] Open source of Faiss project: https://github.com/facebookresearch/faiss \[10\] Open source of HNSWLib project: https://github.com/nmslib/hnswlib Notices & Disclaimers Intel technologies may require enabled hardware, software or service activation. No product or component can be absolutely secure. Your costs and results may vary. Code names are used by Intel to identify products, technologies, or services that are in development and not publicly available. These are not "commercial" names and not intended to function as trademarks The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. © Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others. --- # Unknown Cineca-hpyc and Cineca-ai modules ================================= Cineca-hpyc and Cineca-ai are collection of python and artifitial intelligence packages, respectively, optimized for Cineca's clusters. .. \_cineca-hpyc\_card: The cineca-hpyc module ---------------------- CINECA-HPyC is a collection of Python scientific packages optimized for CINECA HPC clusters. List of principal packages includes: \* numpy \* scipy \* pandas \* numexpr \* mpi4py \* Cython \* pythran \* joblib \* matplotlib \* IPython \* notebook To see the complete list of all the python packages available and check their versions: .. code-block:: bash $ module load cineca-hpyc/ # list all available installations $ python -m pip list # To use a specific pakage $ python -c "import " # To install an additional package $ pip install Once you loaded the cince-hpyc module, some of the main common packages of HPC enviroment are automatically loaded (e.g. cuda, openMPI). Here there is an example about how to import and use pandas package present in cineca-hpyc module: .. code-block:: bash $ module load cineca-hpyc/ $ python -c "import pandas" # You can start a python interactive section and use pandas $ python Python 3.8.12 (default, Jul 29 2022, 16:25:49) \[GCC 10.2.0\] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pandas as pd >>> df = pd.DataFrame({"col1": \["sap", "hi"\], "col2": \[3, 4\]}) >>> print (df) col1 col2 0 sap 3 1 hi 4 .. Note:: \* On Galileo 100 you need to use autoload in order to load the cineca-hpyc module \`\`module load autoload cineca-hpyc/\`\`. If you wish to install additional python packages to use together with the cineca-hpyc suite, you can create a personal virtual environment and install what you need in the following way: .. code-block:: bash $ module load cineca-hpyc/ $ python -m venv my\_env --system-site-packages $ source my\_env/bin/activate $ pip install # Once you finish to work with your env $ deactivate .. Note:: \* my\_env: choose an arbitrary name for your personal virtual env. \* It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. \* the --system-site-packages flag gives the virtual environment access to the system site-packages directory (otherwise you cannot access the cineca-hpyc environment). .. \_cineca-ai\_card: The cineca-ai module -------------------- The bulk of the "cineca-ai" package, provided by the deeplrn profile, includes (for example) Tensorflow, Pytorch, XGBoost, and other related packages and dependencies. This module has been personalized by CINECA AI experts. You can find different cineca-ai modules in profile/deeplrn. For a complete list load the module and launch the "python -m pip list" command. The CINECA AI project can be used in several ways, depending on the method more suited to your needs and on the availability of conda/pip packages. The way to use the installations of the cineca-ai environment goes through the loading of the module: .. code-block:: bash $ module load profile/deeplrn # To see the available versions $ module av cineca-ai # Select a verion and load it $ module load cineca-ai/ # list all available python installations of cineca-ai $ python -m pip list # use a specific package $ python -c "import " If you need to use a package not included in the list of those provided by the cineca-ai modules, you can always rely on the cineca-ai environment for the dependencies and install what you need within a personal virtual environment and/or a conda environment. \*\*Installing additional packages within a python virtual environment\*\* If you wish to install additional python packages to use together with the cineca-ai suite, you can create a personal virtual env and install what you need in the following way : .. code-block:: bash # create the virtual env loading cineca-ai module $ module load profile/deeplrn $ module av cineca-ai $ module load cineca-ai/ $ python -m venv --system-site-packages # activate the created virtual env to install your python packages. $ source my\_env/bin/activate $ pip list $ pip install # Once you finish to work with your env $ deactivate .. Note:: \* my\_env: choose an arbitrary name for your personal virtual env. \* It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. \* the --system-site-packages flag gives the virtual environment access to the system site-packages directory (otherwise you cannot access the cineca-ai environment). \* to test the installation launch: python -c "import ". \* to use the installed package, just source your env (source /bin/activate): you will access your packages AND those of the cineca-ai environment. \*\*Installing additional packages within a conda virtual environment\*\* If you wish to install additional conda packages to use together with the cineca-ai suite, you can create your conda virtual environment and install what you need in the following way (we strongly suggest using a python venv if possible) : .. code-block:: bash # create the conda virtual env loading cineca-ai module $ module load anaconda3/deeplrn $ module load profile/deeplrn $ module av cineca-ai $ module load cineca-ai/ $ conda create -p /my\_env -c conda-forge --override-channels # activate the created conda virtual env to access cineca packages and install your conda packages. $ conda activate /my\_env $ python -m pip list $ python -m pip install # Once you finish to work with your env $ conda deactivate .. Note:: \* my\_env: choose an arbitrary name for your personal virtual env. \* It is advised to create your personal envs in your $WORK area, since the $HOME disk quota is limited to 50 GB. \* the --system-site-packages flag gives the virtual environment access to the system site-packages directory (otherwise you cannot access the cineca-ai environment). --- # Horizon Dashboard — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Horizon Dashboard * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/management_tools/dashboard.rst.txt) * * * Horizon Dashboard[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard "Link to this heading") ============================================================================================================================================ The Horizon Dashboard is the main interface to interact with OpenStack. Note Every cloud infrastructure at CINECA has a separate instance of the Horizon Dashboard, please refer to the [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics) to know how to access a specific system. As a first step, you are asked to login using your HPC credentials and 2FA (see [HPC credentials](https://docs.hpc.cineca.it/general/users_account.html#hpc-credentials) ). The dashboard gives users a graphical interface to monitor and manage the cloud resources associated to your projects. ![../../../_images/openstack_dashboard_overview.png](https://docs.hpc.cineca.it/_images/openstack_dashboard_overview.png) You find the current active project on the top left of the screen, next to the CINECA logo. By clicking on the drop-down menu, you can choose the project you want to interact with, while the overview panel shows available quotas and the used resources for the selected project. On the left menu, you have various pages to interact with each OpenStack component. In each page, you have the list of all the resources of the corresponding type that have been created inside the project. For example, the _“Instances”_ page shows all the virtual machine instances that are currently in the project, with some information on their status: ![../../../_images/openstack_instances_overview.png](https://docs.hpc.cineca.it/_images/openstack_instances_overview.png) For each of these pages, you usually find on top of the page a search toolbar and a button to create new resources. The main tool to interact with OpenStack resources is the drop-down menu in the **“Actions”** column (on the right side). This gives you all the possible actions that can be performed on the specific resource. Clicking on the name of the resource opens a dedicated page that allows you to inspect all the resource’s properties. --- # Command Line Interface — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Command Line Interface * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/management_tools/command_line.rst.txt) * * * Command Line Interface[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface "Link to this heading") ========================================================================================================================================================= The OpenStack Command Line Interface (CLI) serves as a powerful tool for users to interact with their OpenStack cloud environment directly from the terminal or command prompt. The CLI allows Users to perform the same operations that can be usually done via OpenStack dashboard, such as creating Instances, Volumes, Networks etc and many more, and it offers a convenient and efficient way to manage resources, automate tasks, and perform various operations within an OpenStack deployment. Access to the OpenStack CLI is granted thanks to a feature called Application Credentials (in the following AC) that is available on CINECA OpenStack infrastructure. Important By default, the OpenStack CLI is not enabled to external users. In order to have access to the OpenStack CLI service, you need to provide to CINECA a static IP address of the machine from which you will launch the OpenStack commands. Please contact to CINECA support team requesting to add the static IP to the ones allowed to use this service. Installation[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#installation "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- To install the OpenStack CLI, you need to have a working installation of Python in your _PATH_. If you do not have it, you can download it from the official [Python website](https://www.python.org/downloads/) Using a python virtual environment will help you manage dependencies and avoid conflicts with other Python packages. The recommended method to install the CLI is using the python package in the PyPI repository. pip install python-openstackclient\==#.# Copy to clipboard Important * Check that the **version #.#** is the one compatible with the specific cloud system your project is allocated on in [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics) . With greater versions some commands might not work. * For some services, you might need to install additional packages, please see the [OpenStack CLI Documentation](https://docs.openstack.org/newton/user-guide/common/cli-install-openstack-command-line-clients.html) Alternatively, the CLI is installable via standard package manager on Linux machines. Check your own distribution’s repositories to check if the package is available. Configuration[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#configuration "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------- To use the CLI, you need cloud-specific elements: 1. System **SSL certificate chain**: visit the [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics) page of infrastructure where your project is allocated, to download the certificate 2. Valid **OpenStack Application Credentials**: see section [Application credentials creation](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#application-credentials-creation) 3. **Environmental variables**: see section [Setting the Environment Variables](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#setting-the-environment-variables) ### Application credentials creation[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#application-credentials-creation "Link to this heading") Application Credentials (ACs) are used to securely authenticate users with the OpenStack CLI without exposing user passwords. ACs use a unique **“Application Credential ID”** and corresponding **“secret string”** for authentication, ensuring privacy. Users can also delegate role assignments to ACs, controlling authorization levels. Each tenant has its own set of ACs, requiring separate generation ACs for each tenant. Each user has to generate their own application credentials. To generate an application credential, you need to login into the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) , select the project for which the Application Credentials are needed, then go to _“Identity → Application credentials”_ page. You then need to select _“Create Application Credential”_ and fill out the mandatory fields: * **name**: provide a name for your AC * **expiration date**: set an expiration for your AC -> you can create AC also without expiration date, but please be aware that these will be automatically cancelled at midnight of the creation day. * **roles**: If you want the AC to have all your available roles, please do not select anything. If by accident, you selected an item in the roles list, you have to re-create from scratch the AC. Warning In CINECA HPC Cloud infrastructure, it is possible to create AC with duration up to **7 days**. Application Credentials with expiration time greater that 1 week will be **automatically removed**. After selecting the button _“Create Application Credentials”_, the interface shows you the ID and secret of the generated Application Credential. Download the Application Credential file by clicking on the button _“Download openrc file”_ or, in alternative, _“Download cloud.yaml file”_. Important Remember that the secret will not be available after closing the page, so you must capture it or download it. If you have not saved the AC secret, you have to re-create from scratch the AC. ### Setting the Environment Variables[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#setting-the-environment-variables "Link to this heading") The OpenStack CLI uses environment variables to store authentication details, such as the OpenStack Identity (Keystone) service endpoint, authentication method, and credentials. By setting these environment variables, you can configure the OpenStack CLI to authenticate with your cloud environment and perform various operations. In particular, you need to define an environment variable called _“OS\_CACERT”_ with the the full path to your SSL certificate chain file (visit the [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics) page of infrastructure where your project is allocated, to download it). You can set this variable in different ways depending if you have downloaded the AC in the previous step as _openrc_ or _yaml_ file. openrc file Linux Each time you would like to use the AC, you have to source the openrc file and export the _“OS\_CACERT”_ variable: source app-cred-...-openrc.sh export OS\_CACERT\=/ Copy to clipboard Windows In order to use the app-cred-…-openrc.sh work in Windows Command Prompt, some changes are needed: * remove the first line with _#!/usr/bin/env bash_ * replace all _export_ commands in the file with _set_ commands * remove all double quotes (_”_) * rename the file app-cred-…-openrc.bat Original app-cred-…-openrc.sh for Linux #!/usr/bin/env bash export OS\_AUTH\_TYPE\=v3applicationcredential export OS\_AUTH\_URL\=https://clouddev.hpc.cineca.it:5000 export OS\_IDENTITY\_API\_VERSION\=3 export OS\_REGION\_NAME\="RegionOne" export OS\_INTERFACE\=public export OS\_APPLICATION\_CREDENTIAL\_ID\= export OS\_APPLICATION\_CREDENTIAL\_SECRET\= Copy to clipboard New app-cred-…-openrc.bat for Windows set OS\_AUTH\_TYPE\=v3applicationcredential set OS\_AUTH\_URL\=https://clouddev.hpc.cineca.it:5000 set OS\_IDENTITY\_API\_VERSION\=3 set OS\_REGION\_NAME\=RegionOne set OS\_INTERFACE\=public set OS\_APPLICATION\_CREDENTIAL\_ID\= set OS\_APPLICATION\_CREDENTIAL\_SECRET\= Copy to clipboard Then, each time you would like to use the AC, you have to source the openrc file and export the _“OS\_CACERT”_ variable: call .\\path\\to\\app-cred-...-openrc.bat set OS\_CACERT\=\\ Copy to clipboard If you prefer to use Windows PowerShell, you have to replace all _export_ commands in the app-cred-…-openrc.sh file with _$env:_ commands and remove the extension of the file. clouds.yaml > You have to edit the _yaml_ file and add the “cacert” line with the correct indentation as in the following: > > clouds: > : > > auth: > > auth\_url: > > application\_credential\_id: "" > application\_credential\_secret: "" > > region\_name: "RegionOne" > > interface: "public" > identity\_api\_version: 3 > auth\_type: "v3applicationcredential" > cacert: "/" > > Copy to clipboard `` is the name that will allow your system to refer to that specific authentication, while `` is the URL of the CINECA HPC Cloud infrastructure where the AC have been created. These two fields will be automatically compiled when you download the yaml file from OpenStack together with the AC secret and ID. Finally you must set the environment variable `OS_CLOUD=` or the flag `--os-cloud=` in openstack command to use this AC. Using OpenStack CLI[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#using-openstack-cli "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------- Once you have set up the OpenStack CLI and configured the necessary environment variables, you can start using the CLI to interact with your cloud environment. The OpenStack CLI provides a wide range of commands for managing resources, performing operations, and automating tasks within an OpenStack deployment. Here are two basic OpenStack CLI commands that you can use to test if the CLI is working correctly: openstack project list \# List all the project associated with the user openstack server list \# List all the servers in the project Copy to clipboard More OpenStack commands can be found at the following link: [OpenStack CLI Commands](https://docs.openstack.org/python-openstackclient/latest/cli/command-list.html) For CLI documentation refer to the following link: [OpenStack CLI Documentation](https://docs.openstack.org/python-openstackclient/latest/) --- # Download RCM Software — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Access to the Systems](https://docs.hpc.cineca.it/general/access.html) * Download RCM Software * [View page source](https://docs.hpc.cineca.it/_sources/rcm/rcm.rst.txt) * * * Download RCM Software[](https://docs.hpc.cineca.it/rcm/rcm.html#download-rcm-software "Link to this heading") =============================================================================================================== Requirements: > * Microsoft Windows 7, 8, 10, 11. > > * Linux: Ubuntu from 16.04, CentOS 7 (other distributions have been not tested) > > * Apple MacOS Moajave or newer (both version, intel and apple silicon cpu are supported) > > > Note > > On Linux O.S. the user must modify the permission of the precompiled executable RCM with the command: `chmod +x ` > > **Download:** > > > * [GitHub repository](https://github.com/RemoteConnectionManager/RCM/releases/tag/v1.2-rc4) > > (github) > > > > * [Old version](https://hpc-forge.cineca.it/svn/RemoteGraph/branch/multivnc/build/dist/Releases/?p=817) > > Basic Usage of RCM[](https://docs.hpc.cineca.it/rcm/rcm.html#basic-usage-of-rcm "Link to this heading") ========================================================================================================= **Client/Server Interaction** RCM operates as a client/server application. Every interaction with the application requires the server to execute specific tasks, which may take some time depending on your Internet connection’s bandwidth and latency, as well as the current workload of the clusters. To start **RCM** application, follow the described steps: > 1. **Launch the RCM executable** > > 2. Input the **Connection Details**: provide the required hostname (see the table below) and your username. In case you already have a previous saved session, select it from the drop-down menu. > > 3. **Authenticate**: If you have a valid autentication certificate ([How to manage authentication certificates](https://docs.hpc.cineca.it/general/access.html#how-to-manage-authentication-certificates) > ), leave the \[Password\] field empty. Then, press the Login button. > > > ![../_images/rcm1.png](https://docs.hpc.cineca.it/_images/rcm1.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) **Hostname Table** | **Cluster** | **Hostname** | | --- | --- | | Leonardo | rcm.leonardo.cineca.it | | Galileo100 | rcm.g100.cineca.it | RCM can manage multiple sessions. To start a new one, i.e. one for each different cluster, just click on :bdg-black-line:\` + \` button in the tab bar (see the figure). Then, input the connection details for the new session. > ![../_images/rcm2.png](https://docs.hpc.cineca.it/_images/rcm2.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) Once logged, a new tab will show the list of available remote display sessions. In case you did not create a display session yet, the list will be empty. > ![../_images/rcm3.png](https://docs.hpc.cineca.it/_images/rcm3.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) How to create a new Display[](https://docs.hpc.cineca.it/rcm/rcm.html#how-to-create-a-new-display "Link to this heading") =========================================================================================================================== To create a new display, inside a session, just click on :bdg-black-line:\` + \` button as shown in the figure below. > ![../_images/rcm4.png](https://docs.hpc.cineca.it/_images/rcm4.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) Once the connection is estabilished, a new window for the setting of connection will appear (see the figure below). The configuration options are: | **Option** | **Description** | | --- | --- | | Scheduler | Choose between **SSH** (no scheduler) or **Slurm**. | | Account | Select one of your project account | | Queue | Specify the SLURM partition where the job will run. | | QoS | Choose the proper quality of service (QoS) wich affects the job wall-time limits and permissions. | | Memory | Define the RAM memory (GB) required for your job. | | Time | Set the job walltime (up to 24 h). | | GPU | Specify the number of GPUs | | Service Type | The default serive is the latest version of **VNC**. | | Window Manager | Choose a window manager for the VNC session:

> * Openbox GPU only: Supported exclusively on GPU nodes. THis preloads the visualization stack (e.g. Paraview, VTK, and other pakages).
>
> * Openbox/Fluxbox: Supported on all nodes. To use visualization applications, load the required module (e.g.: module load paraview).
> | Note Hover your cursor over any configuration element for additional information about its functionality. ![../_images/rcm5.png](https://docs.hpc.cineca.it/_images/rcm5.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) ![../_images/rcm6.png](https://docs.hpc.cineca.it/_images/rcm6.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) Click on :bdg-black-line:\` Ok \` button in the dialog window, a remote display session will be created and the user will automatically attached to it. Also, a Turbo VNC window will be open (see the figure below). ![../_images/rcm7.png](https://docs.hpc.cineca.it/_images/rcm7.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) Note The creation of a display can take some time depending on the workload of the cluster. When a new display is created, a new item, showing the display details, will be added to the list of available remote display session on the RCM tab. Especially, the display details are: * **Name:** name of the display session * **Status:** the condition of the remote display session, pending (starting up the display session), valid (display session is running), killing (deleting the display session). * **Time:** the remaining time before the session will ends (if the display session has no time limit, this value is replaced by the symbol “~” ). Note that each display session has a time limit: over that time limit, the display will be automatically killed and not saved data will be lost. * **Resources:** the node of the cluster on which the remote display session has been created. ![../_images/rcm8.png](https://docs.hpc.cineca.it/_images/rcm8.png) ![../_images/spacer.png](https://docs.hpc.cineca.it/_images/spacer.png) How to share a Display[](https://docs.hpc.cineca.it/rcm/rcm.html#how-to-share-a-display "Link to this heading") ================================================================================================================= Sharing a remote display means to give to another user the possibility to access to a specific remote display session you have created. The sharing of a remote display session is done by means of a `.vnc` file that as to be saved by the owner of the display session and opened by the user who has to access to the shared display session. To share a display session: > * click on SHARE THE REMOTE DISPLAY SESSION VIA FILE button related to the remote display session you want to share. > > * A dialog will prompt the user to select a location for saving a file. > > * Send the saved file to the users who need to access to the shared display session. > To connect to a shared display session click on Open button from the File menu and select the received `.vnc` file. How to kill a Display[](https://docs.hpc.cineca.it/rcm/rcm.html#how-to-kill-a-display "Link to this heading") =============================================================================================================== Display sessions can be killed by pressing the KILL THE REMOTE DISPLAY SESSION button. Just press the x button in the row associated with the remote display session you don’t want to use anymore, and it will be removed from the list of the available displays. This operation can take some time, depending on the workload of the clusters. Note that by pressing it, the relative display will be not reachable anymore and you will lose not saved data. Running a GUI-based Software[](https://docs.hpc.cineca.it/rcm/rcm.html#running-a-gui-based-software "Link to this heading") ============================================================================================================================= To execute a GUI-based application you have multiple options: On Standard Resources 1. Load the module for the required software (i.e. paraview) \* open the terminal within the graphical session of RCM module load rcm module load paraview Copy to clipboard 2. Launch the software form the terminal, i.e.: paraview Copy to clipboard On GPU Resources If you are using GPU resources (i.e. using the setting **SSH** or **SLURM**), additional steps are required: 1. Load all the required modules module load rcm module load paraview module load virtualgl Copy to clipboard 2. Launch the software form the terminal by using `vlgrun`, i.e.: vlgrun paraview Copy to clipboard For **Openbox GPU Only** Window Manager If you selected the Openbox GPU only window manager when creating the display, you only need to execute the visualization software using the vglrun command (i.e. for paraview): vlgrun paraview Copy to clipboard These steps ensure proper execution of visualization tools, particularly when utilizing GPU resources for accelerated performance. --- # Infrastructure as a Code — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Infrastructure as a Code * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/management_tools/infrastructure_as_code.rst.txt) * * * Infrastructure as a Code[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/infrastructure_as_code.html#infrastructure-as-a-code "Link to this heading") ======================================================================================================================================================================= The term Infrastructure as Code (IaC) refers to a methodology for the provisioning and management of cloud resources. In particular, it consists on treating infrastructure somewhat as software instead of relying on manual operations. The key advantages of IaC can be summarize as follows: * Automation. * Idempotency. * Version control. * CI/CD. * Documentation. Visit the section [Terraform/OpenTofu/Ansible repositories](https://docs.hpc.cineca.it/cloud/tutorials/index_tutorials_and_repos.html#terraform-opentofu-ansible-repositories) to get further insight on the usage of IaC to manage resources on CINECA HPC cloud infrastructures. Declarative and procedural approaches to IaC[](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/infrastructure_as_code.html#declarative-and-procedural-approaches-to-iac "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- There are two primary approaches to Infrastructure as Code (IaC): declarative and procedural (or imperative). The **declarative approach** relies on the description of the desired final end state of the infrastructure, which is gathered in configuration files. Then, the tools used (e.g., Terraform, OpenTofu) are in charge of interpreting the configuration and apply all the actions needed to put it in place. This methodology is especially useful for tasks like provisioning cloud resources (e.g., servers, networks, load balancers). Terraform is an open-source infrastructure as code software tool created by HashiCorp. It enables users to define and provision datacenter infrastructure using a declarative configuration language known as HashiCorp Configuration Language (HCL), or optionally JSON. Terraform manages external resources such as public cloud infrastructure, private cloud infrastructure, network appliances, software as a service, and platform as a service with a code. For more information, visit the official [Terraform](https://www.terraform.io/) website. The **procedural approach** involves outlining the sequence of steps required to achieve the final end state of the infrastructure, rather of a description of the state itself. It is frequently used in configuration management tasks, such as installing software packages on newly provisioned servers. The most popular tools in this category include Ansible, Puppet, and Chef. In particular, Ansible is an open-source automation tool that simplifies IT tasks such as configuration management, application deployment, and cloud provisioning. Its agentless architecture makes it highly efficient for managing infrastructure. For more information about ansible, visit the official [Ansible](https://www.redhat.com/en/ansible-collaborative) website. Each method entails distinct advantages, making the choice among them dependent on project needs and team preferences. It is worth noting that, while presented as alternatives, these approaches are not mutually exclusive and can complement each other within different aspects of a single project. --- # MEGARIDE — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html) * MEGARIDE * [View page source](https://docs.hpc.cineca.it/_sources/cloud/systems/megaride.rst.txt) * * * MEGARIDE[](https://docs.hpc.cineca.it/cloud/systems/megaride.html#megaride "Link to this heading") ==================================================================================================== **PAGE UNDER CONSTRUCTION** Now under configuration. The HPC cloud infrastructure, named MEGARIDE, is built using OpenStack Overview (version to be confirmed). Note * It is possible to access MEGARIDE via Horizon Dashboard at **TO BE ADDED** * For CLI access, the certificate chain can be downloaded here **TO BE ADDED** ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Openstack Overview** [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#os-overview-card) ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Operative Manual** [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html#operative-manual-card) System architecture[](https://docs.hpc.cineca.it/cloud/systems/megaride.html#system-architecture "Link to this heading") -------------------------------------------------------------------------------------------------------------------------- ![../../_images/megaride.png](https://docs.hpc.cineca.it/_images/megaride.png) **CPU nodes** * 150 Interactive OpenStack Nodes * 50 nodes, x2 Intel 8592V, 64c, 2 TB DDR5 * 100 nodes, x2 Intel 6766E, 144c, 512 GB DDR5 **GPU nodes** * 28 Interactive OpenStack Nodes (AMD 9534, 64c) * equipped with: * 96 GPUs Nvidia L40s (AI/Graph 3D) * 16 GPUs Nvidia H100 NVL2 (AI+HPC) **Storage** * 2 PB Capacity-optimized/HDD Ceph Storage System timeline[](https://docs.hpc.cineca.it/cloud/systems/megaride.html#system-timeline "Link to this heading") ------------------------------------------------------------------------------------------------------------------ * to be announced: Early Availability * to be announced: Start of Pre-Production * to be announced: Start of Production Flavors/Images/Openstack services[](https://docs.hpc.cineca.it/cloud/systems/megaride.html#flavors-images-openstack-services "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------ Flavors TO BE ANNOUNCED Images TO BE ANNOUNCED OpenStack Services TO BE ANNOUNCED --- # GAIA — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html) * GAIA * [View page source](https://docs.hpc.cineca.it/_sources/cloud/systems/gaia.rst.txt) * * * GAIA[](https://docs.hpc.cineca.it/cloud/systems/gaia.html#gaia "Link to this heading") ======================================================================================== **PAGE UNDER CONSTRUCTION** Now under configuration, it is composed of 420 general purpose nodes and 80 GPU nodes, also built on OpenStack (version to be defined). It is not only the biggest of the two infrastructures, but also the only accelerated one, mounting Nvidia GPUs models: A30, L40s and H100NVL. The HPC cloud infrastructure, named GAIA, is built using OpenStack Overview (version to be confirmed). Note * It is possible to access GAIA via Horizon Dashboard at **TO BE ADDED** * For CLI access, the certificate chain can be downloaded here **TO BE ADDED** ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Openstack Overview** [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#os-overview-card) ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Operative Manual** [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html#operative-manual-card) System architecture[](https://docs.hpc.cineca.it/cloud/systems/gaia.html#system-architecture "Link to this heading") ---------------------------------------------------------------------------------------------------------------------- ![../../_images/gaia_architecture.png](https://docs.hpc.cineca.it/_images/gaia_architecture.png) **CPU nodes** * 420 Interactive OpenStack Nodes * Each node consist of: * 2x CPU Intel Xeon Sierra Forrest 144 cores 2.2GHz * 1 TiB of DDR5 6400 MT/s **GPU nodes** * 80 Interactive OpenStack Nodes * Each node consist of: * 2x CPU Intel Xeon Emerald Rapids Platinum 8592+ 64 cores 1.9GHz * 1 TiB of DDR5 4800 MT/s * some equipped with 256 GPUs: * 80 GPUs Nvidia A30 (HPC) * 112 GPUs Nvidia L40s (AI/Graph 3D) * 64 GPUs Nvidia H100 NVL (AI+HPC) **Storage** * 2 PB IOPS-optimized/SSD Ceph Storage * 8 PB Capacity-optimized/HDD Ceph Storage System timeline[](https://docs.hpc.cineca.it/cloud/systems/gaia.html#system-timeline "Link to this heading") -------------------------------------------------------------------------------------------------------------- * to be announced: Early Availability * to be announced: Start of Pre-Production * to be announced: Start of Production Flavors/Images/Openstack services[](https://docs.hpc.cineca.it/cloud/systems/gaia.html#flavors-images-openstack-services "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------------- Flavors TO BE ANNOUNCED Images TO BE ANNOUNCED OpenStack Services TO BE ANNOUNCED --- # Interactive Computing — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Services and Tools](https://docs.hpc.cineca.it/services/services_and_tools.html) * Interactive Computing * [View page source](https://docs.hpc.cineca.it/_sources/services/interactive_computing.rst.txt) * * * Interactive Computing[](https://docs.hpc.cineca.it/services/interactive_computing.html#interactive-computing "Link to this heading") ====================================================================================================================================== Basic concepts[](https://docs.hpc.cineca.it/services/interactive_computing.html#basic-concepts "Link to this heading") ------------------------------------------------------------------------------------------------------------------------ The interactive computing service provides an alternative approach to computational resources. The service is accessible via a web browser, with an extended JupyterLab interface based on the [ICE4HPC suite](https://www.e4company.com/ice4hpc/) developed by E4 analytics. The resources requested through the browser interface are allocated on a dedicated set of nodes, all of which are equipped with GPUs. These GPUs are not shared among users, and their allocation is exclusively granted upon request. On the other hand, the allocated CPUs can be shared if the system is fully utilized. This allows for near-immediate access to the system without waiting time. Once the resources are allocated, the browser session can be closed and quickly restored by accessing the service URL. At the moment the service is available on Galileo100. How to get access[](https://docs.hpc.cineca.it/services/interactive_computing.html#how-to-get-access "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------ Every user with computational resources on the cluster hosting the service can access to it. The service can be reached thought the following web address: [https://jupyter.g100.cineca.it](https://jupyter.g100.cineca.it/) Requested resources and releases[](https://docs.hpc.cineca.it/services/interactive_computing.html#requested-resources-and-releases "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------ At the opening page, the user will be asked to login with the CINECA cluster credentials. After the login, the system prompts the user to a form where he/she can request the resources needed during the interactive session. The form appears as follow: [![Form to submit resources requests on the cluster](https://docs.hpc.cineca.it/_images/iac-form.png)](https://docs.hpc.cineca.it/_images/iac-form.png) In the form, in analogy to a sbatch SLURM script on cluster login nodes, the user has to select: | | | | --- | --- |Form[](https://docs.hpc.cineca.it/services/interactive_computing.html#id1 "Link to this table") | Field | How to fill | | --- | --- | | **Slurm reservation** | you can leave it to “None” unless you are assigned to some specific reservation; | | **Slurm account** | the active account you want to be billed for the session; please note that during the pre-production phase, the accounting is inactive. | | **Number of cores** | the number of cores requested for the interactive computing session; please note that cores are assigned in over-subscription, which means that in the unlucky scenario in which all the cores of the system are allocated, the user may share the same core with other users (currently maximum five users on the same CPU); | | **Memory** | the amount of RAM memory requested for the session; | | **GPU configuration** | the number of GPUs which the user requests; differently from CPUs, they are not shared among users and are assigned exclusively; the number of GPUs is limited, thus please be careful to release the resources you requested when you finish your work (see the session “Logout vs Session shutdown” here below) to let the other users to use them. You can check the availability of resources, in particular GPU ones, by looking at the table at the bottom of the page, wherein each row is displayed the number of nodes with no free GPUs, the ones with a single free GPU and the ones with both the GPUs available; | | **Time** | the wall time of your interactive session; during this time, you can close and reopen your browser tab/windows with no issues: the session will stay active, and you can re-attach it simply by accessing to the Interactive Computing web url once again; | | **ICE4HPC Backend environment** | the suite of tools you expect to find during the session execution; see “[Tools and functionalities](https://docs.hpc.cineca.it/services/interactive_computing.html#tools-and-functionalities)
” section for details; | | **User interface** | only the JupyterLab interface is available so far, so please ignore this menu for now; | Once you have filled out the form with your preferred parameters, click the Start button at the bottom. This action will redirect you to the JupyterLab interface, which runs on the cluster’s compute nodes where the user can select the tool or functionality among the available. Tools and functionalities[](https://docs.hpc.cineca.it/services/interactive_computing.html#tools-and-functionalities "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------- The tools you will see in the JupyterLab interface are “packed” in releases: each tool in each release is pinned at the same version to guarantee compatibility. It is possible to choose the release in the initial form displayed after the login phase: in the drop-down menu, they are labelled with a release date, thus the more the date is recent, the more the tool versions are updated, so as a rule of thumb, you might want to test the most recent release with your code. Currently, the following services are up and running on the interface displayed after your session starts: [![JupyterLab launcher](https://docs.hpc.cineca.it/_images/iac-launcher.png)](https://docs.hpc.cineca.it/_images/iac-launcher.png) depending on the release you chose at the login phase. ### Default kernels[](https://docs.hpc.cineca.it/services/interactive_computing.html#default-kernels "Link to this heading") Some default Python/Julia/R/C/C++ environments are provided by default in each release; the packages versions in each release are fixed, in order to guarantee retrocompatibility; updated versions of the environments will be added in new releases. Here below you can find the default kernels provided in the current releases: Release 2024.04 Python Several Python environments are provided by default. You can click here below to their name if you need to check the versions of the main packages contained in each of them: Python 3.11 python 3.11 Ray ray 2.21.0 Dask python 3.11 dask 2024.04 Tensorflow python 3.11 tensorflow 2.12 Pytorch python 3.11 pytorch 2.3.0 Rapids python 3.11 rapids 24.04 Transformers python 3.11 huggingface\_hub 0.24.0 transformers 4.40.2 MDAnalysis python 3.11 mdanalysis 2.7 You can obtain the complete list inside the environment by running `!mamba list` in a Jupyter notebook (after selecting the corresponding kernel in the top-right menu). You can also add your custom environments, as described in the section “[User custom Python environments](https://docs.hpc.cineca.it/services/interactive_computing.html#user-custom-python-environments) ”. Julia Julia kernel version 1.9.4 is automatically installed in the user’s home directory (in the hidden path `~/.julia`) in background at the first login to the service, thus it won’t be visible in the very first login. If you need Julia in the very first login of the service please wait some minutes and refresh the page to make it visible. Being installed in your home directory, you can freely install your packages and in general manage your Julia environment as usual via Pkg package manager: check [Pkg documentation](https://docs.julialang.org/en/v1/stdlib/Pkg/) for the details. C/C++ C/C++ kernel implementation is based on [Xeus](https://github.com/jupyter-xeus/xeus) ; you can use C/C++ instructions inline, like with Jupyter notebooks. You can check the Xeus documentation [here](https://xeus.readthedocs.io/en/stable/) . [![Xeus C/C++ Jupyter kernel example](https://docs.hpc.cineca.it/_images/iac-xeus.png)](https://docs.hpc.cineca.it/_images/iac-xeus.png) R R version 4.3.3 is currently provided via [rlang](https://github.com/r-lib/rlang) v1.1.4 package. ### Visual Studio Code[](https://docs.hpc.cineca.it/services/interactive_computing.html#visual-studio-code "Link to this heading") Visual Studio Code (VSCode) is a very common code editor developed by Microsoft, which offers many advanced features for programming. You can find some tutorials for beginners in the [official documentation](https://code.visualstudio.com/docs/introvideos/basics) . From the interactive Computing interface, you can see a VSCode entry in the launcher after you have started the session; clicking on Visual Studio Code, a new tab/window (depending on your browser settings) will be opened with a web interface containing the VSCode. You can work with VSCode as long as your JupyterLab session is running and your resources are allocated; to stop your session in advance and release the resources, you need to stop the JupyterLab session in the original tab: see “Logout vs Session Shutdown” section for details. ### Monitoring tools[](https://docs.hpc.cineca.it/services/interactive_computing.html#monitoring-tools "Link to this heading") On the (very) left side of your dashboard there is a vertical menu, which allows the user to access some additional functions; one of the buttons is called “GPU Dashboards”. From here, you can monitor your resources usage in real-time; in particular: * in the “Machine Resources” section, you can monitor CPUs, memory utilization, and network and I/O bandwidth. * if you requested GPUs from the initial form, you would see several additional menu to monitor, for instance, GPUs utilization, memory bandwidth and occupation, PCI throughput. > [![Resources monitoring dashboard example](https://docs.hpc.cineca.it/_images/iac-gpu_monitoring.png)](https://docs.hpc.cineca.it/_images/iac-gpu_monitoring.png) The dashboard is developed by nVidia with the [jupyterlab-nvdashboard](https://github.com/rapidsai/jupyterlab-nvdashboard) plugin. User custom Python environments[](https://docs.hpc.cineca.it/services/interactive_computing.html#user-custom-python-environments "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------- You can create your own customized Python environment and display them on the Jupyter launcher. You can proceed as follows: * open a terminal **from your launcher** Warning Be careful to run the following commands from the terminal in the launcher, i.e. clicking on Terminal in the Jupyter web interface) and not via ssh, in order not to have versions mismatches and/or errors in some cases. * in this terminal a conda instance is already active, specific for the release you chose at the login phase (see section “[Requested resources and releases](https://docs.hpc.cineca.it/services/interactive_computing.html#requested-resources-and-releases) ”), thus you can run the commands below: source $CONDA\_PREFIX/etc/profile.d/conda.sh conda init bash conda create \-n ipykernel \-c conda-forge \--override-channels conda activate conda install \-c conda-forge \--override-channels python \-m ipykernel install \--user \--name \--display-name Copy to clipboard After you will refresh the page, a new environment will appear in the launcher of the dashboard; the new kernel is also listed in the drop-down menu of every new Jupyter notebook so that you can use the packages you installed when you created the environment. Note In case you need to delete the environment from your launcher, you can click on Terminal in your Jupyter launcher and run the following commands (**N.B. from the terminal in the Jupyter launcher**): \# optional: delete the environment from your home directory source $CONDA\_PREFIX/etc/profile.d/conda.sh conda remove \--name \--all \# remove kernel from Jupyter launcher jupyter kernelspec uninstall Copy to clipboard You can also further customize your kernels running specific bash script along with your custom Python environments; for instance, after the procedure described so far, a new json file is created in your home in the following path: $HOME/.local/share/jupyter/kernels//kernel.json Copy to clipboard whose content is similar to the following: { "argv": \[\ "/.conda/envs//bin/python",\ "-m",\ "ipykernel\_launcher",\ "-f",\ "{connection\_file}"\ \], "display\_name": "", "language": "python", "metadata": { "debugger": true } } Copy to clipboard This JSON file describes what Jupyterlab runs when clicking on the related button in the launcher page; you can create a bash script to be added in this JSON file to be properly run before your environment execution; this allows for instance, to load any module from the cluster and make it visible during your Python environment execution. For instance, in the following example, we create a file called `wrapper.sh` in the same folder of the JSON file (you can choose the path and the name you prefer): * click on Terminal on your launcher; * move inside the folder you want and open a new file with your favorite text editor (e.g. `wrapper.sh` using vim or nano or emacs); * write a bash script as the following: #!/bin/bash \### You can add here whatever bash commands you like, like for example, "module load" commands module load module load \### Remember the next line! exec "$@" Copy to clipboard * Make this file executable: chmod +x wrapper.sh Copy to clipboard * Edit your `kernel.json` file (e.g. `nano kernel.json`) by adding the following line: "/.local/share/jupyter/kernels//wrapper.sh", Copy to clipboard as the first entry in your argv JSON field. Thus, in the end, your file should look like the following: { "argv": \[\ "/.local/share/jupyter/kernels//wrapper.sh",\ "/.conda/envs//bin/python",\ "-m",\ "ipykernel\_launcher",\ "-f",\ "{connection\_file}"\ \], "display\_name": "", "language": "python", "metadata": { "debugger": true } } Copy to clipboard Warning You need to replace ``, `` and `` with your specific paths, without using environment variables (e.g. you cannot replace `` with `$HOME` since environment variables won’t be expanded in the json file, thus you need to write the full path explicitely). * Now all the bash commands (and variables) you added inside `wrapper.sh` are visible by the Python kernel of your environment. You can check it by running bash commands directly from your Python environment in Jupyter notebooks using your kernel. Note Bash commands can be run from Jupyter notebooks/consoles using “!” at the beginning of the line. Access to work and scratch areas[](https://docs.hpc.cineca.it/services/interactive_computing.html#access-to-work-and-scratch-areas "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------ In the left column of the principal page, you can see the content of your **home** area on the cluster. Your **work** and **scratch** areas are not visible from your default Interactive Computing interface. To make them visible and reachable, you need to create a symbolic link in your HOME directory pointing those areas. In the following example, we are creating a link to the **scratch area** and a link to the **work** area inside our home directory by launching the following commands in a terminal on the cluster: ln \-s $CINECA\_SCRATCH $HOME/scratch ln \-s $WORK $HOME/work Copy to clipboard Thus now you can see a work and a scratch icons in the file manager on the left side of our interface, and you can access them. Note It is strongly suggested to creating such links to make all the storage available to the Interactive computing sessions (and not just the home storage). Please remember that the $WORK variable refers to the work area of your current default account, so you should create different links for different accounts and keep them updated over time. Logout vs Session shutdown[](https://docs.hpc.cineca.it/services/interactive_computing.html#logout-vs-session-shutdown "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------ By default, your session is not shut down when you close your browser window (or tab); as long as the session is active (until the walltime is reached), the requested resources are not released to other users. In this way, by opening a new browser, the user can restore a still active session simply by re-accessing the [Interactive Computing URL](https://jupyter.g100.cineca.it/) . .. commented so far since accounting is still disabled on IAC .. Until the session is finished by reaching the walltime or closed manually by the user, the requested resources will be billed on the budget account indicated by the user in the form. If you have finished and you want to close your session manually you need to click on File, then select Hub Control Panel and finally Stop my server. > [![Hub control panel from "File" dropdown menu](https://docs.hpc.cineca.it/_images/iac-shutdown_01.png)](https://docs.hpc.cineca.it/_images/iac-shutdown_01.png) > [!["Stop my server" button](https://docs.hpc.cineca.it/_images/iac-shutdown_02.png)](https://docs.hpc.cineca.it/_images/iac-shutdown_02.png) Troubleshooting[](https://docs.hpc.cineca.it/services/interactive_computing.html#troubleshooting "Link to this heading") -------------------------------------------------------------------------------------------------------------------------- “Kernel died unexpectedly” error message. Unfortunately, Jupyter kernels are not very verbose, but in many cases, this error can be related to a buffer overflow; please consider testing the code once again in a new session (see “Logout vs Session shutdown”) requesting a larger amount of memory. Spawning job message hangs after requesting resources. The problem might occur when requesting an unavailable amount of resources, which might be the case of jobs requesting GPUs; you can take a look to the table in the bottom of the form for resources allocation, which lists all the nodes available with zero, one or two GPUs; if you requested, for instance, a session with 2 GPUs but there are no nodes with 2 GPUs currently available, then this issue might occur. Please consider to fit your requests to the available resources to have the session starts in short time. --- # Shares — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Shares * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/os_components/shares.rst.txt) * * * Shares[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/shares.html#shares "Link to this heading") ================================================================================================================ The [OpenStack Manila service](https://docs.openstack.org/manila/latest/) allows the creation of a filesystem that can be shared among virtual machines in the same tenant (intra-tenant) or in different tenants (extra-tenant). This setup is particularly useful for applications that require consistent and simultaneous access to data across different instances. In CINECA HPC Cloud infrastructure, it is possible to create shares of the following types: * Generic type (NFS protocol) * Cephfs type (CEPHFS protocol) We suggest the users that need a shared filesystem to use the **generic type**. Note **Only intra-tenant** shares are allowed on CINECA HPC Cloud infrastructures. --- # Storage — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Storage * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/os_components/storage.rst.txt) * * * Storage[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/storage.html#storage "Link to this heading") =================================================================================================================== This component, based on the [OpenStack Cinder Service](https://docs.openstack.org/cinder/latest/) , allows the management of the storage space in cloud projects. Storage in CINECA HPC Cloud machines is based on Ceph platform. Ceph provides reliable distributed and scalable storage without a single point of failure. Within OpenStack, it is possible to create different block storage devices, that can be used by instances (virtual machines) to store data persistently. Volumes[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/storage.html#volumes "Link to this heading") ------------------------------------------------------------------------------------------------------------------- A volume is a block storage device that can be attached to instances. It provides persistent storage for data. Snapshots[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/storage.html#snapshots "Link to this heading") ----------------------------------------------------------------------------------------------------------------------- A snapshot is a point-in-time copy of a volume. It can be used to create new volumes or restore existing volumes. Snapshots use the Copy On Write (COW) technique to create lightweight snapshots of volumes. Backups[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/storage.html#backups "Link to this heading") ------------------------------------------------------------------------------------------------------------------- A volume backup is a copy of a volume that can be used to restore data in case of data loss or corruption (**NOT IN FULL PRODUCTION**). --- # Database — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Database * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/os_components/database.rst.txt) * * * Database[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/database.html#database "Link to this heading") ====================================================================================================================== [Trove](https://docs.openstack.org/trove/latest/) is a database as a service (DBaaS) component of OpenStack. It is a pluggable service that supports multiple database engines. Database as a service (DBaaS) is a cloud computing managed service offering that provides access to a database without requiring the setup of physical hardware, the installation of software or the need to configure the database. The Database service provides scalable and reliable cloud provisioning functionality for both relational and non-relational database engines. Users can quickly and easily use database features without the burden of handling complex administrative tasks. Cloud users and database administrators can provision and manage multiple database instances as needed. The Database service provides resource isolation at high-performance levels, and automates complex administrative tasks such as deployment, configuration, patching, backups, restorations, and monitoring. --- # Network — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Network * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/os_components/network.rst.txt) * * * Network[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#network "Link to this heading") =================================================================================================================== [OpenStack Neutron service](https://docs.openstack.org/neutron/latest/) is a core component of the OpenStack cloud computing platform, which provides “networking as a service” between interface devices managed by other OpenStack components such as Nova (compute) and Cinder (block storage) services. Neutron allows users to create and manage various networking services like VLANs, SDNs (Software Defined Networks), and other complex network topologies. ![../../../_images/openstack_network_topology.png](https://docs.hpc.cineca.it/_images/openstack_network_topology.png) Project networks are isolated and private. These networks are not accessible from outside the OpenStack environment unless routed through an external network. The **external network** is the public-facing network that allows VMs within the tenant networks to access the internet. To operate, a network will need at least a **subnet** and a **router**. A subnet is a range of IP addresses in your project’s network, while a router is the virtual device that forwards traffic between your project and the external network. Once the project has a network, it is possible to link to it virtual machines and other resources in order to connect them between each other and to the external network. Subnets[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#subnets "Link to this heading") ------------------------------------------------------------------------------------------------------------------- A subnet is a range of IP addresses in your project’s network. You can create subnets to group instances according to security and operational needs. When you create a subnet, you specify the CIDR block for the subnet, which is a subset of the network CIDR block. Note Typical CIDR reserved for private networks are: 192.168.0.0/16, 10.0.0.0/8 or 172.16.0.0/12. Routers[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#routers "Link to this heading") ------------------------------------------------------------------------------------------------------------------- With OpenStack routers, you can create and manage multiple networks, define routing rules, and control the flow of traffic between subnets. This flexibility allows you to design and implement complex network topologies to meet the specific requirements of your cloud infrastructure. In addition to basic routing functionality, OpenStack routers support advanced features like **floating IPs**, which allow you to associate a public IP address with a private IP one to enable external access to instances on your private network. The **gateway** is the IP address of the router that connects the subnet to the external network. The gateway IP address is typically the first or last IP address in the subnet range. Floating IPs[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#floating-ips "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------- In OpenStack, instances are by default assigned private IP addresses from the internal network. These private IPs are not reachable from outside the OpenStack environment. However, by assigning a floating IP to an instance, the instance becomes reachable from the external network, enabling inbound and outbound communication via the internet. When a floating IP is allocated, it is reserved from a pool of available public IP addresses. This pool is configured by the cloud administrator and can be customized based on the organization’s requirements. Each cloud project has a limited number of floating IPs controlled by the project quota and these need to be allocated to the project from the central pool prior to their use. Security Groups[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#security-groups "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------- A security group acts as a virtual firewall for instances and other resources on a network controlling the inbound and outbound traffic to instances. One security group can be associated with one or more instances and a single instance can have multiple security groups associated to it. It is always possible to modify, add and remove security groups in a virtual machine after its creation. The security group contains one or more security rule which specify the network access. A security rule defines how traffic can enter/exit the VM instance via: * an IP address or CIDR block * an EtherType (IPv4 or IPv6) * a direction (INGRESS or EGRESS) * a port the traffic will pass through For example the rule: INGRESS IPv4 TCP 22(SSH) 131.175.44.1 will imply that the virtual machine will be reachable via SSH on port 22 using TCP protocol from the IP 131.175.44.1 --- # Compute — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html) * Compute * [View page source](https://docs.hpc.cineca.it/_sources/cloud/os_overview/os_components/compute.rst.txt) * * * Compute[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#compute "Link to this heading") =================================================================================================================== This component, based on the [OpenStack Nova Service](https://docs.openstack.org/nova/latest/) , allows the management of the computing resources. Instances[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#instances "Link to this heading") ----------------------------------------------------------------------------------------------------------------------- An instance (or virtual machine) is a software-based emulation of a physical computer. It runs an operating system and applications, just like a physical computer. With instances, users can run applications and workloads in the cloud. Instances can be created from an image, or from a snapshot. In the corresponding part of the OpenStack Horizon dashboard _“Compute → Instances”_, you can: * visualize the quota of the selected project and its usage in the _“Overview”_ page * access the list of instances * create a new instance * perform operations (**Actions**) on the selected instance, like attaching/de-attaching volumes, suspend or shut off, creation of snapshots Virtual machines can be of two types: **Ephemeral** Virtual Machines are booted from an OpenStack image. These images have fixed root volume dimension, dictated from the used flavor. Creating an ephemeral virtual machine is more lightweight for tenant resources, since the root volume won’t be accounted on the tenant storage quota. **Bootable** Virtual Machines are created using a pre-existing volume as root volume. Users can choose freely the size of the root volume of the virtual machine. Creating a virtual machine from a bootable volume has two main advantages: * customizable root volume size * the data of the root volume won’t be deleted when deleting the instance (if appropriately configured during creation) Note **Instance affinity and anti-affinity groups** In some cases, users might want to be sure two or more instances run specifically on the same hypervisor (e.g. for ensuring better communication) or on different ones (e.g. to implement high availability). In OpenStack, it is possible to create affinity or anti-affinity groups for this purpose. When creating an instance, it is possible to define whether this will be part of one of the groups available in the tenant. Key Pairs[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#key-pairs "Link to this heading") ----------------------------------------------------------------------------------------------------------------------- Key Pairs are a method for securing SSH access to instances by using public-key cryptography. When a Key Pair is associated with an instance, it allows users to SSH into the instance securely without using a password. This ensures a higher level of security compared to traditional password-based access. Public-Key Cryptography is a cryptographic system that uses pairs of keys - **public keys** (which can be shared) and **private keys** (which are kept secret). The public key is used to encrypt data, and the private key is used to decrypt it. The public key is injected into the instance at the time of creation. The private key is kept by the user and used to establish an SSH connection to the instance. Image Snapshots[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#image-snapshots "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------- A snapshot is a point-in-time copy of data capturing the current state of a virtual machine. It preserves the state and the data of the VM including its power state (on, off, or suspended) and all its files (such as disks, memory, and network interfaces). Snapshots are generally used to restore a VM after a system failure, bad update, or error. You can find instances snapshots under the _“Compute → Images”_ section in the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) . Warning * Before taking a snapshot, log in to the virtual machine and shutdown the instance. * Snapshots are stored on the same cloud infrastructure of the instances, but can be downloaded to be stored at a different location. Images[](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#images "Link to this heading") ----------------------------------------------------------------------------------------------------------------- An image is a template which contains a specific operating system and pre-installed software. Images hosted on the cloud platform have the **“visibility”** attribute which can assume 4 different values that define which users can use the image for the instance creation and provides information about who uploaded the image. | **Visibility** | **Characteristics** | | --- | --- | | Public | * Uploaded only by CINECA HPC Cloud infrastructure administrators.

* Usable by all the users on the cloud platform and appears in the default image list. | | Private | * Uploaded by any user on the cloud platform (image owner).

* Usable only by the image owner. | | Shared | * Uploaded by any user on the cloud platform (image owner).

* By default, usable only by all the users which work on the same project of the image owner.

* The image owner can explicitly chose a user or a group of users to share the image to: the user or the group of user have to accept the sharing. | | Community | * Uploaded by any user on the cloud platform (image owner).

* Usable by all the users on the cloud platform and appears in the default image list. | Warning Since no control can be performed on _Community_ images by the CINECA HPC Cloud infrastructure administrators, **Community images are inhibited**. Trying to **upload a Community image** will results in a **blocking error** both from Horizon dashboard and OpenStack CLI. Default images for common operating systems are provided by CINECA and available to be used. The updated list of all the images uploaded by CINECA HPC Cloud infrastructure administrators on each HPC cloud infrastructure can be found in the systems specifics pages at [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics) . Alternatively, users can import their own images, as described in the [Image: upload](https://docs.hpc.cineca.it/cloud/operative/compute_ops/image_upload.html#image-upload) Warning On CINECA HPC Cloud machines, users are not allowed building windows virtual machines, even if they have their own windows license. --- # LoadBalancer operations — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * LoadBalancer operations * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/lb_ops/index_lb_ops.rst.txt) * * * LoadBalancer operations[](https://docs.hpc.cineca.it/cloud/operative/lb_ops/index_lb_ops.html#loadbalancer-operations "Link to this heading") =============================================================================================================================================== For general information on the compute component, visit the [Load Balancer](https://docs.hpc.cineca.it/cloud/os_overview/os_components/load_balancers.html#load-balancer) page. ![osoctavia](https://docs.hpc.cineca.it/_images/octavia_logo.png) **Load balancer: create** [LoadBalancer: create](https://docs.hpc.cineca.it/cloud/operative/lb_ops/lb_create.html#lb-create-card) ![osoctavia](https://docs.hpc.cineca.it/_images/octavia_logo.png) **Load balancer: troubleshooting** [LoadBalancer: troubleshooting](https://docs.hpc.cineca.it/cloud/operative/lb_ops/lb_troubleshooting.html#lb-troubleshooting-card) --- # ADA — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html) * ADA * [View page source](https://docs.hpc.cineca.it/_sources/cloud/systems/ada.rst.txt) * * * ADA[](https://docs.hpc.cineca.it/cloud/systems/ada.html#ada "Link to this heading") ===================================================================================== In production since 27 September 2021, it is the smaller of the two machines and is composed of 71 nodes. Built on OpenStack (version Zed) can host virtual machines up to 96 vCPUs. The HPC cloud infrastructure, named ADA, is built using OpenStack Overview ([version Zed](https://releases.openstack.org/) ). Note * It is possible to access ADA via Horizon Dashboard at [https://adacloud.hpc.cineca.it](https://adacloud.hpc.cineca.it/) * For CLI access: * the certificate chain can be downloaded here [`ADA certificate`](https://docs.hpc.cineca.it/_downloads/9ddd073947579734ee9630b939c22978/ada_ssl_2026.pem) * check that the **version installed is 6.5 or 6.6** with openstack –version command. With greater versions some commands might not work. ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Openstack Overview** [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#os-overview-card) ![oslogo](https://docs.hpc.cineca.it/_images/openstack_logo.png) **Operative Manual** [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html#operative-manual-card) System architecture[](https://docs.hpc.cineca.it/cloud/systems/ada.html#system-architecture "Link to this heading") --------------------------------------------------------------------------------------------------------------------- ![../../_images/ada_architecture.png](https://docs.hpc.cineca.it/_images/ada_architecture.png) ### Hardware Details - nodes[](https://docs.hpc.cineca.it/cloud/systems/ada.html#hardware-details-nodes "Link to this heading") | **Type** | **Specific** | | --- | --- | | Models | Dual-socket Dell PowerEdge | | Nodes | 71 Interactive OpenStack Nodes | | Processors | 2xCPU Intel CascadeLake 8260 24 cores 2.4GHz with hyperthreading | | Cores | 48 cores/node | | RAM | 768 GB | | Internal Network | 100Gbs Ethernet interconnection | | Storage | 2TB SSD | ### Disks and Filesystems[](https://docs.hpc.cineca.it/cloud/systems/ada.html#disks-and-filesystems "Link to this heading") * 1 PB NVMe/SSD Ceph Storage * This cloud infrastructure is tightly connected both to the LUSTRE storage of 20 PB raw capacity, and to the GSS storage of 6 PB seen by all other infrastructure. System timeline[](https://docs.hpc.cineca.it/cloud/systems/ada.html#system-timeline "Link to this heading") ------------------------------------------------------------------------------------------------------------- * **05 Aug 2021**: Early Availability * **01 Sept 2021**: Start of Pre-Production * **27 Sept 2021**: Start of Production Flavors/Images/Openstack services[](https://docs.hpc.cineca.it/cloud/systems/ada.html#flavors-images-openstack-services "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------- Flavors | **Flavor Name** | **vCPUs** | **RAM (GB)** | **Disk (GB)** | **Available** | | --- | --- | --- | --- | --- | | fl.ada.xxs | 1 | 7.5 | 10 | Yes | | fl.ada.xs | 2 | 15 | 30 | Yes | | fl.ada.s | 4 | 30 | 30 | Yes | | fl.ada.m | 8 | 60 | 30 | Yes | | fl.ada.l | 16 | 120 | 30 | Yes | | fl.ada.xl | 24 | 180 | 30 | On-demand | | fl.ada.xxl | 48 | 360 | 30 | On-demand | | fl.ada.full | 96 | 7200 | 30 | On-demand | Images | **Image Name** | **Information** | **Default User** | **Default Access** | | --- | --- | --- | --- | | CentOS-7-x86\_64-GenericCloud-2009 | CentOS-7-x86\_64-GenericCloud-2009.qcow2, last modified 2020-11-12 [Source](http://cloud.centos.org/centos/7/images/) | centos | SSH keypair | | CentOS-8-GenericCloud-8.4.2105-20210603.0.x86\_64 | CentOS-8-GenericCloud-8.4.2105-20210603.0.x86\_64, last modified 2021-06-03 [Source](https://cloud.centos.org/centos/8/) | centos | SSH keypair | | CentOS-Stream-GenericCloud-8-20220913 | CentOS-Stream-GenericCloud-8-20220913.x86\_64, last modified 2022-09-13 [Source](https://cloud.centos.org/centos/8-stream/) | centos | SSH keypair | | Ubuntu 18.04 LTS (Bionic Beaver) | Ubuntu server 18.04 (Bionic Beaver) LTS for cloud [Source](https://cloud-images.ubuntu.com/) | ubuntu | SSH keypair | | Ubuntu Server 20.04 LTS (Focal Fossa) | focal-server-cloudimg-amd64.img, last modified 2021-07-20 [Source](https://cloud-images.ubuntu.com/) | ubuntu | SSH keypair | | Ubuntu Server 21.04 (Hirsute Hippo) | hirsute-server-cloudimg-amd64.img, last modified 2021-07-20 [Source](https://cloud-images.ubuntu.com/) | ubuntu | SSH keypair | | Ubuntu Server 22.04 LTS (Jammy Jellyfish) | jammy-server-cloudimg-amd64.img, last modified 2022-09-02 [Source](https://cloud-images.ubuntu.com/) | ubuntu | SSH keypair | | Ubuntu Server 24.04 LTS (Noble Numbat) | noble-server-cloudimg-amd64.img, last modified 2025-03-13 [Source](https://cloud-images.ubuntu.com/noble/) | ubuntu | SSH keypair | | Rocky Linux 8.9 | File description: [https://wiki.rockylinux.org/rocky/image/#about-cloud-images](https://wiki.rockylinux.org/rocky/image/#about-cloud-images)
[Source](https://dl.rockylinux.org/pub/rocky/8/images/x86_64/Rocky-8-GenericCloud-Base-8.9-20231119.0.x86_64.qcow2) | rocky | SSH keypair | | Rocky Linux 9.3 | File description: [https://wiki.rockylinux.org/rocky/image/#about-cloud-images](https://wiki.rockylinux.org/rocky/image/#about-cloud-images)
[Source](https://dl.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud-Base-9.3-20231113.0.x86_64.qcow2) | rocky | SSH keypair | | Rocky Linux 9.4 | File description: [https://wiki.rockylinux.org/rocky/image/#about-cloud-images](https://wiki.rockylinux.org/rocky/image/#about-cloud-images)
[Source](https://dl.rockylinux.org/pub/rocky/9/images/x86_64/) | rocky | SSH keypair | | Debian 12 generic 64-bit AMD/Intel | debian-12-generic-amd64.qcow2 [Source](https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.qcow2) | debian | SSH keypair | OpenStack Services > In addition to the core OpenStack Components [OpenStack Overview](https://docs.hpc.cineca.it/cloud/os_overview/index_openstack_overview.html#openstack-overview) > , on ADA we have > > * Barbican, for secure storage, provisioning and management > > * Manila, for shared filesystem management > > * Octavia, for load balancers deployment > > * Trove, for database management > In ADA, the following [Shares](https://docs.hpc.cineca.it/cloud/os_overview/os_components/shares.html#shares) types are available: * generic\_type (NFS protocol) * cephfs\_type (CEPHFS protocol) --- # Unknown 1 | E U R O f u s i o n G a t e w a y U s e r A g r e e m e n t Application for an account on the Gateway Computing Cluster Applicant Information \* Name \* Surname \* EUROfusion beneficiary Other (e.g. for applicants employed by Affiliated Entities etc.) Please specify name of entity, country \* phone (incl. country code) Account Information \* Requested account type (see Annex below) USER: This category of users has access to all installed codes, private and public data bases available on the Gateway. EXTERNAL: This category of users has access to a limited number of publicly released codes and parts of physics data bases. Involvement within EUROfusion activities indicate Work Packages or Projects you are involved in Codes / Tools planned to be used \*Other information please give indications of planned activities Place and Date Signature \* Applicant Please print this form, sign it and send it to the PMU Coordination Officer by email: (Denis Kalupin: Denis.Kalupin@EURO-fusion.org). Request approval Place and Date Signature \* by PMU Coordination Officer ☐ \* I, the applicant, have read, understood and agreed with the above terms and conditions of Gateway access Contact Information \* business e-mail address 2 | E U R O f u s i o n G a t e w a y U s e r A g r e e m e n t Terms and Conditions for Gateway access and use: As Gateway account holder of any of the above categories: ▪ I acknowledge and accept that the setting up of my gateway account does not constitute a (legal) right of access and use of the software and codes stored therein. Access rights to use the software/codes are subject to either the acceptance of a licence agreement attached to the software, applicable cooperation agreements or the granting of access rights by the owner following my written request pursuant to the EUROfusion Consortium Agreement and/or EUROfusion Grant Agreement; ▪ I am aware that for some software additional requirements may apply if so explicitly stated; ▪ I will not install or run any software on the Gateway computing cluster which cannot be directly attributed to my role as a user or external; ▪ By uploading content to the Gateway, I affirm that such content complies with all applicable laws and license conditions (if any) and I shall hold the Gateway operator free and harmless from any related liability. ▪ I accept that all content of the Gateway is provided “as is” and I shall hold the Gateway operator and any content provider free and harmless from any related liability in connection with my use of such content. ▪ I understand that access to the Gateway is granted through individual accounts which must not be transferred or shared. ▪ I have understood and accepted that I have to re-apply for a gateway user account, if my employer changes. ▪ I have understood and accepted that I have to inform the PMU Coordination Officer if my involvement in EUROfusion activities ends. ▪ Access rights to the Gateway computing cluster may be revoked if a user is found to be in breach of the terms of the user agreement as set out in this document. --- # Shares operations — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * Shares operations * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/shares_ops/index_shares_ops.rst.txt) * * * Shares operations[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/index_shares_ops.html#shares-operations "Link to this heading") =========================================================================================================================================== For general information on the Shares component, visit the [Shares](https://docs.hpc.cineca.it/cloud/os_overview/os_components/shares.html#shares) page. ![osmanila](https://docs.hpc.cineca.it/_images/manila_logo.png) **Generic shares: create** [Create and use a GENERIC\_TYPE share](https://docs.hpc.cineca.it/cloud/operative/shares_ops/generic_share_create.html#shares-generic-create-card) ![osmanila](https://docs.hpc.cineca.it/_images/manila_logo.png) **CEPHFS shares: create** [Create and use a CEPHFS\_TYPE share](https://docs.hpc.cineca.it/cloud/operative/shares_ops/cephfs_share_create.html#shares-cephfs-create-card) --- # Database operations — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * Database operations * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/db_ops/index_db_ops.rst.txt) * * * Database operations[](https://docs.hpc.cineca.it/cloud/operative/db_ops/index_db_ops.html#database-operations "Link to this heading") ======================================================================================================================================= For general information on the Database component, visit the [Database](https://docs.hpc.cineca.it/cloud/os_overview/os_components/database.html#database) page. ![ostrove](https://docs.hpc.cineca.it/_images/trove_logo.png) **Database: create** [Database: create](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_create.html#db-create-card) ![ostrove](https://docs.hpc.cineca.it/_images/trove_logo.png) **Database: access** [Database: access](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_access.html#db-access-card) --- # Network operations — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * Network operations * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/network_ops/index_network_ops.rst.txt) * * * Network operations[](https://docs.hpc.cineca.it/cloud/operative/network_ops/index_network_ops.html#network-operations "Link to this heading") =============================================================================================================================================== For general information on the Network component, visit the [Network](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#network) page. ![osneutron](https://docs.hpc.cineca.it/_images/neutron_logo.png) **Network: create** [Network: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/network_create.html#network-create-card) ![osneutron](https://docs.hpc.cineca.it/_images/neutron_logo.png) **floating IP: association** [Floating IP: allocate and associate](https://docs.hpc.cineca.it/cloud/operative/network_ops/fip_association.html#fip-associate-card) ![osneutron](https://docs.hpc.cineca.it/_images/neutron_logo.png) **Security groups: create** [Security groups: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/secgroups_create.html#secgroups-create-card) --- # Storage operations — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * Storage operations * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/storage_ops/index_storage_ops.rst.txt) * * * Storage operations[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/index_storage_ops.html#storage-operations "Link to this heading") =============================================================================================================================================== For general information on the storage component, visit the [Storage](https://docs.hpc.cineca.it/cloud/os_overview/os_components/storage.html#storage) page. ![oscinder](https://docs.hpc.cineca.it/_images/cinder_logo.png) **Volume: create and attach** [Volume: create and attach](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_create.html#volume-create-card) ![oscinder](https://docs.hpc.cineca.it/_images/cinder_logo.png) **Volume: format and mount** [Volume: format and mount](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_mount.html#volume-mount-card) ![oscinder](https://docs.hpc.cineca.it/_images/cinder_logo.png) **Volume: resize** [Volume: resize](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_resize.html#volume-resize-card) --- # Compute operations — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * Compute operations * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/index_compute_ops.rst.txt) * * * Compute operations[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html#compute-operations "Link to this heading") =============================================================================================================================================== For general information on the compute component, visit the [Compute](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#compute) page. ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: create** [Instance: create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#compute-inst-create-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: manage** [Instance: manage and monitor](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_manage.html#compute-inst-manage-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: resize** [Instance: resize](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_resize.html#compute-inst-resize-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: snapshot create** [Instance: snapshot create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#compute-inst-snap-create-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: download** [Instance: snapshot download](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_download.html#compute-inst-download-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: delete** [Instance: delete](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_deletion.html#compute-inst-delete-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: rescue** [Instance: rescue](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_rescue.html#compute-inst-rescue-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Instance: root storage increase** [Instance: root storage increase](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_root_storage_increase.html#compute-inst-root-storage-increase-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **Image: upload** [Image: upload](https://docs.hpc.cineca.it/cloud/operative/compute_ops/image_upload.html#compute-inst-img-upload-card) ![osnova](https://docs.hpc.cineca.it/_images/nova_logo.png) **KeyPair: create** [Key Pair: create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/keypair_create.html#compute-keypair-create-card) --- # Unknown .. \_matlab\_card: Matlab ====== The following guide describes how to use MATLAB to submit jobs on CINECA clusters, retrieve results and debug errors. MATLAB is available on :ref:\`hpc/leonardo:Leonardo\` and :ref:\`hpc/galileo:Galileo100\` clusters. Pre-requisites ^^^^^^^^^^^^^^ To use MATLAB on CINECA HPC's environment, please check the following pre-requisites: 1. You and your collaborators have \*\*a valid account\*\* defined on HPC cluster, see :ref:\`general/users\_account:How to become a User\`. 2. You have access to a \*\*valid MATLAB license\*\* to be used on CINECA HPC clusters. Thanks to an agreement with MathWorks, \*\*CINECA provides several MATLAB licenses\*\* through its internal license server that can be used on CINECA clusters. Usage of the CINECA MATLAB licenses is allowed \*\*exclusively for Open Science\*\* (non-commercial) activities. In case you are interested in using those licenses and you declare us that your activity is devoted to Open Science, please write to superc@cineca.it to be enabled to use CINECA licenses. For all the other cases, or in case you would like to use your personal/department/university license, we need to connect your license server, where the Flex-LM license is installed, with the CINECA compute nodes of the cluster. Detailed instructions can be found in :ref:\`hpc/hpc\_software:How to connect your license server\`. Configuration ^^^^^^^^^^^^^ It is possible to configure MATLAB to submit jobs on CINECA clusters directly from your local MATLAB installation (Running MATLAB from your Desktop) or from login nodes of CINECA clusters (Running MATLAB on the HPC Cluster). Running MATLAB from your Desktop """""""""""""""""""""""""""""""" It is possible to submit MATLAB jobs to the compute nodes of a CINECA cluster directly from your local MATLAB installation. This setup needs to be done once per cluster, per version of MATLAB installed on your machine. \*\*Supported versions:\*\* R2022b, R2023a, R2023b, R2024a, R2024b, R2025a Please check that your local MATLAB installed version \*\*matches one of the supported versions\*\*. Moreover, you need to to have the "\*\*Parallel Computing Toolbox\*\*" installed on your computer. To check for it, you can run the following commands on MATLAB: .. code-block:: matlabsession >> license('test','distrib\_computing\_toolbox') >> ~isempty(ver('parallel')) if answer is 1 for both, it means that the Parallel toolbox is correctly installed. Otherwise you need to install it. \*\*Download\*\* the CINECA MATLAB support package from here (:download:\`cineca.Desktop.zip <../../files/cineca.Desktop.zip>\`) (\*\*Last update: 01 August 2025\*\*). It contains a set of scripts (Integration Scripts) needed to configure MATLAB to launch jobs remotely. Unzip the file in the location returned by the MATLAB command: .. code-block:: matlabsession >> userpath For different solutions you can refer to this \`MathWorks dedicated User Guide page \`\_. \*\*Only in the case you are going to use your personal/department/university license\*\*, you will also have to modify the following files inside the cluster folder you find in the Integration Scripts: \`\`communicatingSubmitFcn.m\`\` and \`\`independentSubmitFcn.m\`\` by adding a MLM\_LICENSE\_FILE line indicating the port and the IP of your license server as in the following: .. code-block:: bash 'PARALLEL\_SERVER\_DEBUG', enableDebug; ... 'MLM\_LICENSE\_FILE', '@'; ... 'MLM\_WEB\_LICENSE', environmentProperties.UseMathworksHostedLicensing; ... This step is \*\*not needed\*\* if you are going to use CINECA licenses. Finally open MATLAB and create a new cluster profile launching the command .. code-block:: matlabsession >> configCluster Submission to the cluster requires SSH credentials. You will be prompted for the cluster, your username and password. Jobs will run on the cluster rather than on the local machine. .. important:: You can access only clusters where you have an active budget account. To manage the local cluster configuration in the top menu select "Parallel", then "Create and Manage Clusters..." A window will be opened where you can modify the Additional Properties of your configuration based on your needs (See :ref:\`hpc/software/matlab:Configuring Jobs\` Section about a description of the Available Properties). Running MATLAB on the HPC Cluster """"""""""""""""""""""""""""""""" Alternatively to Remote submission, you can also launch MATLAB jobs directly from login nodes of CINECA clusters. Log-in to the cluster and load the MATLAB module: .. code-block:: bash $ module load profile/eng $ module load matlab/ There may be available more than one MATLAB version. You can check for it through :ref:\`hpc/hpc\_enviroment:The modmap command\` section. Take care to select the last valid release for your license. Configure MATLAB to run parallel jobs. This only needs to be called once per version of MATLAB and once per user. .. code-block:: bash $ configCluster.sh in alternative you can start MATLAB without desktop .. code-block:: bash $ matlab -nodisplay then launch the command .. code-block:: matlabsession >> configCluster A new profile will be created (i.e. 'galileo100 R2024b' on Galileo100). Jobs will run across multiple nodes on the cluster rathen than on the host machine. You can check the list of available profiles: .. code-block:: matlabsession >> \[ ALLPROFILES,DEFAULTPROFILE\] = parallel.clusterProfiles ALLPROFILES = 1x2 cell array {'galileo100 R2024b'} {'local'} DEFAULTPROFILE= 'galileo100 R2024b' Please check that the \`\`DEFAULTPROFILE\`\` is not set to 'local'. The 'local' profile is not allowed on our cluster, so don't use it. If it is set to 'local' you have to set for example .. code-block:: matlabsession >> DEFAULTPROFILE='galileo100 R2024b' on Galileo100 and similarly on Leonardo. Configuring Jobs ^^^^^^^^^^^^^^^^ Prior to submitting the job, various parameters have to be specified in order to be passed to jobs, such as queue, username, e-mail, etc. .. note:: Any parameters specified using the below workflow will be persistent between MATLAB sessions if saved at the end of the configuration. Before specifying any parameters, you will need to obtain a handle to the cluster object. .. code-block:: matlabsession >> % Get a handle to the cluster >> c = parcluster; You are now \*\*required\*\* to specify an Account Name, a Queue Name and the Wall Time (visit :ref:\`hpc/hpc\_intro:Budget and Accounting\` to see how to retrieve your Budget Account Name using the saldo command) .. code-block:: matlabsession >> % Specify an Account to use for MATLAB jobs >> c.AdditionalProperties.AccountName = 'account\_name'; >> % Specify a queue to use for MATLAB jobs >> c.AdditionalProperties.Partition = 'partition-name'; >> % Specify the walltime (e.g. 5 hours) >> c.AdditionalProperties.WallTime = '05:00:00'; On Leonardo cluster there are two partitions: 'boost\_usr\_prod' to access GPU nodes and 'dcgp\_usr\_prod' to access CPU nodes. You can find additional info on the :ref:\`hpc/leonardo:Leonardo\` dedicated pages. For Galileo100 cluster the main partition is 'g100\_usr\_prod'. In :ref:\`hpc/galileo:Galileo100\` dedicated page you can find other possible Partitions and QOS available allowing for different combinations of nodes, walltime and priority. You can specify other \*\*additional\*\* (not-mandatory) parameters along with your job. .. code-block:: matlabsession >> % Specify QoS >> c.AdditionalProperties.QoS = 'name-of-qos'; >> % Specify processor cores per node. Default is 32 for Leonardo GPU nodes and 112 on Leonardo CPU nodes; 18 for Marconi and 48 for Galileo100. >> c.AdditionalProperties.ProcsPerNode = 18; >> % specify the number of GPUsPerNode. Valid only on Leonardo GPU partition >> c.AdditionalProperties.GPUsPerNode = 1; >> % Specify memory to use for MATLAB jobs, per core (default: 4gb) >> c.AdditionalProperties.MemUsage = '6gb'; >> % Require node exclusivity >> c.AdditionalProperties.RequireExclusiveNode = true; >> % Request to use a reservation >> c.AdditionalProperties.Reservation = 'name-of-reservation'; >> % Specify e-mail address to receive notifications about your job >> c.AdditionalProperties.EmailAddress = ‘test@foo.com’; >> % Turn onthe Debug Message. Default is off (logical boolean true/false). >> c.AdditionalProperties.DebugMessagesTurnedOn = true; >> % Specify the tmpfs dimension, for Leonardo CPU partition (default: 10GB) >> c.AdditionalProperties.Tmpfs = '20G'; To check for the values of the current configuration options, call the AdditionalProperties without semicolon .. code-block:: matlabsession >> % To view current configurations >> c.AdditionalProperties To clear a value, assign the property an empty value (‘’, \[\], or false). .. code-block:: matlabsession >> % To clear a configuration that takes a string as input >> c.AdditionalProperties.EmailAddress = ‘ ’; To save a profile, with your configuration so you will find it in future sessions .. code-block:: matlabsession >> c.saveProfile; Serial Jobs ^^^^^^^^^^^ Interactive Jobs """""""""""""""" To run an interactive pool job on the cluster, continue to use \`parpool\` as before. .. note:: This is valid ONLY when running MATLAB on the cluster .. code-block:: matlabsession >> % Get a handle to the cluster >> c = parcluster; Rather than running a local pool on the host machine, the pool can now run across multiple nodes on the cluster. .. code-block:: matlabsession >> % Run a parfor over 1000 iterations >> parfor idx = 1:1000 >> a(idx) = rand; >> end Delete the pool when it’s no longer needed. .. code-block:: matlabsession >> %Delete the pool >> pool.delete Independent Batch Jobs """""""""""""""""""""" Use the batch command to submit asynchronous jobs to the cluster. The batch command will return a job object which is used to access the output of the submitted job. See the MATLAB documentation for more help on \`batch \`\_.. .. code-block:: matlabsession >> % Get a handle to the cluster >> c = parcluster; Submit job to query where MATLAB is running on the cluster .. code-block:: matlabsession >> j = c.batch(@pwd, 1, {}); Query job for state: queued | running | finished .. code-block:: matlabsession >> j.State If state is finished, fetch results .. code-block:: matlabsession >> j.fetchOutputs{:} or .. code-block:: matlabsession >> fetchOutputs(j) Display the diary .. code-block:: matlabsession >> diary(j) Delete the job after results are no longer needed .. code-block:: matlabsession >> j.delete; To retrieve a list of currently running or completed jobs, call parcluster to retrieve the cluster object. The cluster object stores an array of jobs that were run, are running, or are queued to run. This allows us to fetch the results of completed jobs. Retrieve and view the list of jobs as shown below. .. code-block:: matlabsession >> c = parcluster; >> jobs = c.Jobs; >> % Get a handle to the second job in the list >> job2 = c.Jobs(2); Once we’ve identified the job we want, we can retrieve the results as we’ve done previously. \`\`fetchOutputs\`\` is used to retrieve function output arguments; if using batch with a script, use load instead. Data that has been written to files on the cluster needs be retrieved directly from the file system. To view results of a previously completed job: .. code-block:: matlabsession >> % Fetch results for job with ID 2 >> j2.fetchOutputs{:} .. note:: You can view a list of your jobs, as well as their IDs, using the above c.Jobs command. If the job produces an error view the error log file .. code-block:: matlabsession >> c.getDebugLog(j.Tasks(1)) .. note:: When submitting independent jobs, with multiple tasks, you will have to specify the task number. Parallel Jobs ^^^^^^^^^^^^^ Interactive Jobs """""""""""""""" To run an interactive pool job on the cluster, you can use parpool. Valid only on login nodes. .. code-block:: matlabsession >> % Get a handle to the cluster >> c = parcluster; Open a pool of 64 workers on the cluster .. code-block:: matlabsession >> pool = c.parpool(64); Rather than running local on the local machine, the pool can now run across multiple nodes on the cluster. .. code-block:: matlabsession >> % Run a parfor over 1000 iterations >> parfor idx = 1:1000 a(idx) = rand(); end Once we’re done with the pool, delete it. .. code-block:: matlabsession >> % Delete the pool >> pool.delete; Batch Jobs """""""""" Users can also submit parallel workflows remotely from their local MATLAB installation on their PC with batch. The following example are available at the following directory available after loaded the module .. code-block:: bash CIN\_EXAMPLE=/cineca/prod/opt/tools/matlab/CINECA\_example \*\*Parallel\_example.m\*\* Let’s use the following example for a parallel job. .. code-block:: matlab function t = parallel\_example(iter) if nargin==0, iter = 16; end disp('Start sim') t0 = tic; parfor idx = 1:iter A(idx) ) idc; pause(2) end t = toc(t0); disp('Sim completed.') We will use the batch command again, but since we’re running a parallel job, we’ll also specify a MATLAB Pool. .. code-block:: matlabsession >> % Get a handle to the cluster >> c = parcluster; >> % Submit a batch pool job using 4 workers for 16 iterations >> j = c.batch(@parallel\_example, 1, {}, ‘Pool’, 4); For more info on the batch commands, please see the \`MATLAB on-line guide \`\_. .. code-block:: matlabsession >> % View current job status >> j.State >> % Fetch the results after a finished state is retrieved >> j.fetchOutputs{:} ans = 15.5328 >> % Display the diary >> diary(j) The job ran in 15.53 sec. using 4 workers. \*\*Note that these jobs will always request N+1 cores for your job\*\*, since one worker is required to manage the batch job and pool of workers. For example, a job that needs eight workers will consume nine CPU cores. We’ll run the same simulation, but increase the Pool size. This time, to retrieve the results at a later time, we’ll keep track of the job ID. \*\*NOTE:\*\* For some applications, there will be a diminishing return when allocating too many workers, as the overhead may exceed computation time. .. code-block:: matlabsession >> % Get a handle to the cluster >> c = parcluster; >> % Submit a batch pool job using 8 workers for 16 simulations >> j = c.batch(@parallel\_example, 1, {}, ‘Pool’, 8); >> % Get the job ID >> id = j.ID Id = 4 >> % Clear workspace, as though we quit MATLAB >> clear j Once we have a handle to the cluster, we’ll call the \`\`findJob\`\` method to search for the job with the specified job ID. .. code-block:: matlabsession >> % Get a handle to the cluster >> c = parcluster; >> % Find the old job >> j = c.findJob(‘ID’, 4); >> % Retrieve the state of the job >> j.State ans = finished >> % Fetch the results >> j.fetchOutputs{:} ans = 6.4488 >> % If necessary, retrieve output/error log file >> c.getDebugLog(j) The job now runs 6.4488 seconds using 8 workers. Run code with different number of workers to determine the ideal number to use. \*\*hpccLinpack.m\*\* This example is taken from .. code-block:: bash $MATLAB\_HOME/toolbox/distcomp/examples/benchmark/hpcchallenge/ It is an implementation of the HPCC Global HPL benchmark .. code-block:: matlabsession >> function perf = hpccLinpack( m ) The function input is the size of the real matrix m-by-m to be inverted. The outputs is perf, performance in gigaflops Start to submit on 1 core, with m=1024: .. code-block:: matlabsession >> j = c.batch(@hpccLinpack, 1, {1024}, 'Pool', 1) Data size: 0.007812 GB Performance: 1.576476 GFlops Repeat on one full node on Marconi .. code-block:: matlabsession >> j = c.batch(@hpccLinpack, 1, {1024}, 'Pool', 35) Data size: 0.007812 GB Performance: 0.311111 GFlops Increase the size of the matrix, .. code-block:: matlabsession >> j = c.batch(@hpccLinpack, 1, {2048}, 'Pool', 35) Data size: 0.031250 GB Performance: 2.466961 GFlops >> j = c.batch(@hpccLinpack, 1, {4096}, 'Pool', 35) Data size: 0.125000 GB Performance: 47.951919 GFlops >> j = c.batch(@hpccLinpack, 1, {8192}, 'Pool', 71) Data size: 0.500000 GB Performance: 86.003520 GFlops .. .. >> j = c.batch(@hpccLinpack, 1, {16384}, 'Pool', 35) Data size: 2.000000 GB Performance: 356.687648 GFlops Debugging ^^^^^^^^^ If a serial job produces an error, we can call the getDebugLog method to view the error log file. .. code-block:: matlabsession >> j.Parent.getDebugLog(j.Tasks(1)) When submitting independent jobs, with multiple tasks, you will have to specify the task number. For Pool jobs, do not deference into the job object. .. code-block:: matlabsession >> j.Parent.getDebugLog(j) The scheduler job ID can be derived by calling schedID .. code-block:: matlabsession >> schedID(j) ans = 25539 To learn More ^^^^^^^^^^^^^ To learn more about the MATLAB Parallel Computing Toolbox, check out these resources: \* :download:\`Hands-On Wokshop@CINECA <../../files/MATLAB\_PCT\_handsOnWorkshop.pdf>\` \* :download:\`Exercises Workshop Day 1 <../../files/Matlab\_PCT\_Workshop.7z>\` \* :download:\`Exercises Workshop Day 2 <../../files/MATLAB\_exercise.7z>\` \* \`Parallel Computing Coding Examples \`\_ \* \`Parallel Computing Documentation \`\_ \* \`Parallel Computing Overview \`\_ \* \`Parallel Computing Tutorials \`\_ \* \`Parallel Computing Videos \`\_ \* \`Parallel Computing Webinars \`\_ Parallel Computing Benchmark and Performance ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ \* \`Benchmarking Parfor Performance Using a Simple Example: Game of Blackjack \`\_ \* \`Resource Contention in Task Parallel Problems \`\_ \* \`Benchmarking Distributed Jobs (Task Parallel Applications on the Cluster) \`\_ \* \`Benchmarking Parallel "\\" Operator (A\\b) \`\_ \* \`Profiling Load Unbalanced Distributed Arrays in Data Parallel Applications \`\_ \* \`Profiling Explicit Parallel Communication while using Message Passing Functions in MATLAB \`\_ --- # Unknown .. \_quantum\_espresso\_card: QuantumESPRESSO =============== The following guide describes how to load, configure and use QuantumESPRESSO @ CINECA's cluster. QuantumESPRESSO is available on :ref:\`hpc/leonardo:Leonardo\` and :ref:\`hpc/galileo:Galileo100\` clusters. Relevant links ^^^^^^^^^^^^^^ - QE repository: https://gitlab.com/QEF/q-e.git - MaX benchmarks: https://gitlab.com/max-centre/benchmarks-max3.git - JUBE xmls: https://gitlab.com/max-centre/JUBE4MaX.git - spack recipe: https://gitlab.com/spack/spack/-/blob/develop/var/spack/repos/builtin.mock/packages/quantum-espresso/package.py Modules ^^^^^^^ CPU-based and GPU-based machines deploy QuantumESPRESSO with different software stacks, to fully exploit the underlying hardware. In particular: - \*\*Intel/Oneapi\*\* compiler and MPI implementation on G100 and Leonardo DCGP, plus \*\*MKL\*\* for FFT, BLAS/LAPACK and SCALAPACK - \*\*NVHPC\*\* compiler and \*\*OpenMPI/HPCX-MPI\*\* on Leonardo Booster, plus \*\*OpenBLAS\*\* and \*\*FFTW\*\* libraries. Installations based on gcc compiler do not provide performance, and are provided for postprocessing executables. Alternative Installations ^^^^^^^^^^^^^^^^^^^^^^^^^ If you wish installing your own version of QuantumESPRESSO, we suggesting using CMake and the options provided in the \`Wiki of the official repository \`\_ for the CINECA cluster in use. Parallelization strategies ^^^^^^^^^^^^^^^^^^^^^^^^^^ QuantumESPRESSO supports different parallelization strategies. - R&G (\`-npw\` or no options) processes to distribute real/reciprocal spaces - pools (\`-nk\`) to distribute k-points - images (\`-ni\`) to distribute irreducible representations or q-points in a dispersion - band processes (\`-nbnd\`) to distribute the Kohn-Sham states - linear algebra processes (auto) to distribute diagonalization, via scalapack or custom algorithm. For GPU installations, the diagonalization is done on a single GPU (scalapack are not used We suggest the following for optimal performance on Leonardo Booster: - prioritize pools over R&G , in particular for workloads with hundreds of planes or less in the z-direction, also for intra-node distribution. - The minimum number of k-points per pool (kunit) in PWSCF is the number of k-points (\`kunit=1\`), while in phonon is usually \`kunit=2\`, except the following cases: (i) lgamma and not noncolin or domag: \`kunit=1\` (ii) not lgamma but noncolin and domag: \`kunit=4\`. - Images implements independent calculations but they might be affected by imbalanced workload, so a mixture of images and poolsusually provides best performances. More detailed information about parallelization strategies can be found on this \`link \`\_ GPU performance considerations and troubleshooting ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Mapping and binding """"""""""""""""""" If you are using the node in exclusive mode, distribute the resources among MPI tasks (usually 1 MPI task per GPU) as follows 1. Set SLURM options to ask for 8 cpus per task .. code-block:: #SLURM --nodes= #SLURM --ntasks-per-node=4 #SLURM --cpus-per-task=8 2. Launch the MPI application by mapping the tasks over the full node with 8 cpus per task. The following code snippets show the command line for mpirun and srun: .. code-block:: mpirun --map-by node:PE=$SLURM\_CPUS\_PER\_TASK --rank by core pw.x .. code-block:: srun --cpus-per-task=$SLURM\_CPUS\_PER\_TASK --cpu-bind=cores pw.x 3. MPI-GPU binding is done by source code, so an external binding with \`CUDA\_VISIBLE\_DEVICES\` is not needed. Multi-node multi-GPU runs """"""""""""""""""""""""" If your workload requires \*\*multi-node distribution\*\* due to memory constrains on GPUs, we suggest testing the following environment variables to improve performnaces. \*\*Memory issues during SCF loop (OOM)\*\* Your code crashes after some iterations steps in the SCF loop. Instead of increasing the number of nodes, try adding the folliwing environment variable in your jobscript. .. code-block:: matlabsession export UCX\_TLS=^cuda\_ipc This error is due to handles automatically created by the MPI library when calling Isend+Irecv\*Waitall. .. note:: Do not export this environment variable if using a number of R&G processes less or equal to 4. \*\*Increase multi-node BW\*\* If your code distributes FFTs across multiple nodes, the MPI installations might not use all the NICs available for inter-node communications. Try interposing this script between the mpi launcher and the executable .. code-block:: #!/bin/bash # Replace with OMPI\_COMM\_WORLD\_LOCAL\_RANK if using mpirun case $(( ${SLURM\_LOCALID} )) in 0) export UCX\_NET\_DEVICES=mlx5\_0:1 CUDA\_VISIBLE\_DEVICES=0 ;; 1) export UCX\_NET\_DEVICES=mlx5\_1:1 CUDA\_VISIBLE\_DEVICES=1 ;; 2) export UCX\_NET\_DEVICES=mlx5\_2:1 CUDA\_VISIBLE\_DEVICES=2 ;; 3) export UCX\_NET\_DEVICES=mlx5\_3:1 CUDA\_VISIBLE\_DEVICES=3 ;; esac echo Launching on $UCX\_NET\_DEVICES .. note:: The environment variable depends on the mpi launcher, srun (\`SLURM\_LOCALID\`) or mpirun (\`OMPI\_COMM\_WORLD\_LOCAL\_RANK\`) \*\*Improve multi-node scaling with FFT distribution\*\* If you need to distribute FFTs over multiple-nodes and achieve the so called 'eager' regime, with small messages exchanged among processes, try reducing the threshold for the rendez-vous algorithm, which can be more efficient on GPUs .. code-block:: export UCX\_RNDV\_THRESH=8192 --- # Unknown .. \_pitagora\_card: Pitagora ======== .. .. figure:: ../img/warning3.png .. :align: center .. :class: no-scaled-link .. :height: 150px .. .. .. figure:: ../img/spacer.png .. :align: center .. :class: no-scaled-link .. :height: 20px Pitagora is the new EUROfusion supercomputer hosted by \*\*CINECA\*\* and currently built in the CINECA's headquarter in Casalecchio di Reno, Bologna, Italy. The cluster is supplied by Lenovo corp. and is composed of two partitions: a general purpose partition cpu-based named \*\*DCPG\*\* and an accelerated partition based on NVIDIA H100 accelerators named \*\*Booster\*\*. The specific guide for the \*\*Pitagora\*\* cluster contains unique information that deviates from the general behavior described in the HPC Clusters sections. Access to the System -------------------- The machine is reachable via \`\`ssh\`\` (secure Shell) protocol at hostname point: \*\*login.pitagora.cineca.it\*\*. The connection is established, automatically, to one of the available login nodes. It is possible to connect to \*\*Pitagora\*\* using one the specific login hostname points: \* login01-ext.pitagora.cineca.it \* login02-ext.pitagora.cineca.it \* login03-ext.pitagora.cineca.it \* login04-ext.pitagora.cineca.it \* login05-ext.pitagora.cineca.it \* login06-ext.pitagora.cineca.it .. warning:: \*\*The mandatory access to Pitagora is the two-factor authetication (2FA)\*\*. Get more information at section :ref:\`general/access:Access to the Systems\`. .. note:: \*\*Even-numbered login nodes\*\* have the same architecture of \*\*Booster\*\* parition's compute nodes while \*\*odd-numbered\*\* have the same architecture of \*\*DCGP\*\* parition's compute nodes \* \*\*login-boost.pitagora.cineca.it\*\* will allow users to log on one of the \*\*even-numbered login nodes\*\* in a round robin fashion. \* \*\*login-dcgp.pitagora.cineca.it\*\* will allow users to log on one of the \*\*odd-numbered login nodes\*\* in a round robin fashion. System Architecture ------------------- The system, supplied by Lenovo, is based on two new specifically-designed compute blades, which are available throught two distinct SLURM partitios on the Cluster: \* \*\*GPU\*\* blade based on NVIDIA NVIDIA H100 accelerators - \*\*Booster\*\* partition. \* \*\*CPU\*\*-only blade based on AMD Turin 128c processors - \*\*Data Centric General Purpose (DCGP)\*\* partition. The overall system architecture uses NVIDIA Mellanox InfiniBand High Data Rate (HDR) connectivity, with smart in-network computing acceleration engines that enable extremely low latency and high data throughput to provide the highest AI and HPC application performance and scalability. Hardware Details ^^^^^^^^^^^^^^^^ .. tab-set:: .. tab-item:: Booster .. list-table:: :widths: 30 50 :header-rows: 1 \* - \*\*Type\*\* - \*\*Specific\*\* \* - Models - Lenovo SD650-N V3 \* - Racks - 7 \* - Nodes - 168 \* - Processors/node - 2x Intel Emerald Rapids Xeon Gold 6548Y+ 32c 2.5 GHz \* - CPU/node - 64 \* - Accelerators/node - 4x NVIDIA H100 SXM 80GB HBM2e \* - Local Storage/node (tmfs) - \* - RAM/node - 512 GiB DDR5 5600 Mhz \* - Rmax - 27.27 PFlop/s (\`top500 \`\_) \* - Internal Network - Nvidia ConnectX-7 NDR200 \* - Storage (raw capacity) - 2 x 7.68 GiB SSDs (HW RAID 1) .. tab-item:: DCGP .. list-table:: :widths: 30 50 :header-rows: 1 \* - \*\*Type\*\* - \*\*Specific\*\* \* - Models - Lenovo SD665 V3 \* - Racks - 14 \* - Nodes - 1008 \* - Processors/node - 2x AMD Turin EPYC 9745 128c 2.4 GHz - Zen5 microarch \* - CPU/node - 256 \* - Accelerators/node - (none) \* - Local Storage/node (tmfs) - \* - RAM/node - 768 GiB DDR5 6400 Mhz \* - Rmax - 17 Pflop/s (\`top500 \`\_) \* - Internal Network - Nvidia ConnectX-7 NDR SharedIO 200Gbit/s \* - Storage (raw capacity) - Diskless nodes Job Managing and SLURM Partitions --------------------------------- In the following table you can find informations about the SLURM partitions for \*\*Booster\*\* and \*\*DCGP\*\* partitions of the production environment. Please note that the slurm email service is not active yet. .. seealso:: Further information about job submission are reported in the general section :ref:\`hpc/hpc\_scheduler:Scheduler and Job Submission\`. .. tab-set:: .. tab-item:: Booster +------------------+------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ | \*\*Partition\*\* | \*\*QOS\*\* | \*\*#Nodes/#per job\*\* | \*\*Walltime\*\* | \*\*#Max Nodes/#per user\*\* | \*\*Priority\*\* | \*\*Notes\*\* | +==================+========================+===========================+==============+============================+==============+=====================================+ | boost\_fua\_prod | normal | max = 16 | 24:00:00 | 32 | 40 | | + +------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ | | boost\_qos\_fuabprod | min = 17 (full nodes) | 24:00:00 | 32 | 60 | runs on 96 nodes (GrpTRES) | | | | max = 32 | | | | | +------------------+------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ | boost\_fua\_dbg | normal | max = 2 | 00:30:00 | | 40 | runs on 2 nodes (GrpTRES) | +------------------+------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ .. tab-item:: DCGP +------------------+------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ | \*\*Partition\*\* | \*\*QOS\*\* | \*\*#Nodes/#per job\*\* | \*\*Walltime\*\* | \*\*#Max Nodes/#per user\*\* | \*\*Priority\*\* | \*\*Notes\*\* | +==================+========================+===========================+==============+============================+==============+=====================================+ | dcgp\_fua\_prod | normal | max = 64 | 24:00:00 | 64 | 40 | | + +------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ | | dcgp\_qos\_fuabprod | min = 65 (full nodes) | 24:00:00 | 128 | 60 | runs on 640 nodes (GrpTRES) | | | | max = 128 | | | | | + +------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ | | dcgp\_qos\_fualprod | max = 3 | 4-00:00:00 | 3 | 40 | | +------------------+------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ | dcgp\_fua\_dbg | normal | max = 2 | 00:30:00 | 2 | 40 | runs on 8 nodes (GrpTRES) | +------------------+------------------------+---------------------------+--------------+----------------------------+--------------+-------------------------------------+ Processes/Threads Binding/Affinity ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ \*\*Processes Binding\*\* \* By default, srun (SLURM launcher) performs an automatic binding. For multi-threaded application request the proper --cpus-per-task and bind the processes to cores (srun --cpu-bind=cores). \* By default, OpenMPI libraries (mpirun launcher) bind processes to core. For multi-threaded applications this causes the cpu overallocation. Ensure that you are either not bound at all (by specifying --bind-to none) or bound to multiple cores using an appropriate binding level or specific number of processing elements per application process (--map-by socket:PE=$SLURM\_CPUS\_PER\_NODE). \* By default, IntelMPI libraries (mpirun launcher with hydra process manager) performs a correct binding. If you opt for IntelMPI mpirun as launcher, unset the I\_MPI\_PMI\_LIBRARY (meant for using Intelmpi with srun) defined when loading the module to avoid the verbose warnings. \*\*Threads Affinity\*\* All present compilers (gcc, nvhpc, aocc, intel) by default don't bind threads to cores. You can act on the threads affinity with the standard OMP\_PLACES/OMP\_PROC\_BIND variables. Known Issues ------------ This section collects currently known issues affecting PITAGORA. The list below is intended as a quick reference for users who may experience problems on the system. We strongly encourage all users to report any issues they encounter - whether listed here or not - to the user support team. .. card:: Internode GPUDirect Communication: UCX GPUDirect RDMA Error +++++ \*\*Status:\*\* :bdg-danger:\`Open\` | :octicon:\`calendar\` \*\*Last Update:\*\* 2025-07-31 | :octicon:\`cache\` \*\*Partition:\*\* Booster .. dropdown:: :octicon:\`info\` \*\*Description\*\* The \*\*UCX GPUDirect RDMA\*\* feature is currently not functioning for point-to-point communications. This is likely due to an incompatibility between \*\*Intel UPI\*\* (intersocket connection) and \*\*UCX\*\*, as referenced in \`NVIDIA issue 2235234 \`\_. \*\*Case 1\*\*: The mpi job fails with errors like the following one: .. code-block:: bash \[r310c04s01:2918358:0:2918358\] ib\_mlx5\_log.c:179 Local protection error on mlx5\_0:1/IB (synd 0x4 vend 0x51 hw\_synd 0/2) \[r310c04s01:2918358:0:2918358\] ib\_mlx5\_log.c:179 RC QP 0xb232 wqe\[20\]: SEND s-e \[inl len 10\] \[va 0x14986e800000 len 1 lkey 0x63a2\] \[rqpn 0xc526 dlid=2992 sl=0 port=1 src\_path\_bits=0\] \[r310c03s04:1011253:0:1011253\] ib\_mlx5\_log.c:179 Local protection error on mlx5\_0:1/IB (synd 0x4 vend 0x51 hw\_synd 0/2) \[r310c03s04:1011253:0:1011253\] ib\_mlx5\_log.c:179 RC QP 0xc526 wqe\[25\]: SEND s-e \[inl len 10\] \[va 0x14fa6a800000 len 1 lkey 0x688b\] \[rqpn 0xb232 dlid=5592 sl=0 port=1 src\_path\_bits=0\] ==== backtrace (tid:2918358) ==== 0 0x00000000000129b0 uct\_ib\_mlx5\_completion\_with\_err() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/ib\_mlx5\_log.c:179 1 0x00000000000279ec uct\_rc\_mlx5\_iface\_handle\_failure() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:235 2 0x00000000000279ec uct\_rc\_iface\_arbiter\_dispatch() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/rc/base/rc\_iface.h:455 3 0x00000000000279ec uct\_rc\_mlx5\_iface\_handle\_failure() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:238 4 0x0000000000013a25 uct\_ib\_mlx5\_check\_completion() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/ib\_mlx5.c:477 5 0x0000000000028d97 uct\_ib\_mlx5\_poll\_cq() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/ib\_mlx5.inl:148 6 0x0000000000028d97 uct\_rc\_mlx5\_iface\_poll\_tx() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5.inl:1891 7 0x0000000000028d97 uct\_rc\_mlx5\_iface\_progress() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:127 8 0x0000000000028d97 uct\_rc\_mlx5\_iface\_progress\_cyclic() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/ib/mlx5/rc/rc\_mlx5\_iface.c:132 9 0x000000000004dc4a ucs\_callbackq\_dispatch() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/ucs/datastruct/callbackq.h:215 10 0x000000000004dc4a uct\_worker\_progress() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/uct/api/uct.h:2813 11 0x000000000004dc4a ucp\_worker\_progress() /build-result/src/hpcx-v2.22.1-gcc-doca\_ofed-redhat9-cuda12-x86\_64/ucx-b00d4c4b05219b73a7d764f03644b5296ba9dfdb/src/ucp/core/ucp\_worker.c:3033 12 0x0000000000031f73 opal\_progress() ???:0 13 0x0000000000054955 ompi\_request\_default\_wait\_all() ???:0 14 0x000000000009ce07 MPI\_Waitall() ???:0 15 0x0000000000403e33 main() ???:0 16 0x0000000000029590 \_\_libc\_start\_call\_main() ???:0 17 0x0000000000029640 \_\_libc\_start\_main\_alias\_2() :0 18 0x0000000000404cc5 \_start() ???:0 ================================= \*\*TEMPORARY SOLUTION:\*\* UCX GPUDirect RDMA has been disabled by default through the following environment variable, set automatically in all OpenMPI modules: .. code-block:: bash export UCX\_IB\_GPU\_DIRECT\_RDMA=no This can be verified by inspecting OpenMPI modules. In the example below, unrelated lines have been omitted for clarity. .. code-block:: bash \[@ ~\]$ module show openmpi/4.1.6--gcc--12.3.0 ------------------------------------------------------------------- /pitagora/prod/opt/modulefiles/base/libraries/openmpi/4.1.6--gcc--12.3.0: module-whatis {An open source Message Passing Interface implementation.} \[...\] setenv UCX\_IB\_GPU\_DIRECT\_RDMA no \[...\] append-path MANPATH {} ------------------------------------------------------------------- .. important:: - \*\*No action is required by the user to apply this workaround.\*\* - If users wish to test UCX GPUDirect RDMA manually, they can unset the variable after loading the module to re-enable the feature. - \*\*No significant performance degradation has been observed\*\* in synthetic benchmarks (e.g., OSU) or selected real-world applications. \*\*Case 2\*\*: Pytorch with NCCL backend jobs hang. \*\*TEMPORARY SOLUTION:\*\* Disable NCCL GPU Direct RDMA as follows: .. code-block:: bash export NCCL\_NET\_GDR\_LEVEL=LOC .. card:: IntelMPI Provider/Fabric Compatibility on AMD Processors +++++ \*\*Status:\*\* :bdg-danger:\`Open\` | :octicon:\`calendar\` \*\*Last Update:\*\* 2025-12-18 | :octicon:\`cache\` \*\*Partition:\*\* DCGP .. dropdown:: :octicon:\`info\` \*\*Description\*\* \*\*DESCRIPTION:\*\* The \*\*Mellanox (MLX) provider for IntelMPI does not work correctly with several applications and libraries on the AMD-based DCGP partition\*\*, resulting in job crashes or significantly reduced performance. This is related to the UCX Transport Layer behind the communication, expecially to the TLS "rc" (Reliable Connection) that presents some bugs. \*\*Known affected codes:\*\* \* STARWALL \* ASCOT5 \* PETSc-based software \* GENE \*\*TEMPORARY SOLUTION:\*\* The \*\*rc TLS has been automatically discarded for IntelMPI\*\*. This is done automatically when loading the IntelMPI module by exporting the following environment variable: .. code-block:: bash export UCX\_TLS=^rc This can be confirmed by inspecting the IntelMPI module. In the example below, unrelated lines have been omitted for clarity. .. code-block:: bash \[@ ~\]$ module show intel-oneapi-mpi/2021.12.1 ------------------------------------------------------------------- /pitagora/prod/opt/modulefiles/base/libraries/intel-oneapi-mpi/2021.12.1: module-whatis {Intel MPI Library is a multifabric message-passing library that implements the open-source MPICH specification. Use the library to create, maintain, and test advanced, complex applications that perform better on high-performance computing (HPC) clusters based on Intel processors.} conflict intel-oneapi-mpi \[...\] setenv UCX\_TLS=^rc \[...\] setenv MPIFC mpiifx ------------------------------------------------------------------- .. important:: - \*\*No action is required from the user to enable this configuration\*\* as it is now the default for IntelMPI modules. - For some applications (e.g., ONIX), this configuration may be cause of slowdowns. In these cases, it is recommendable to test other provider and TLS choices: in the case of ONIX, switching the provider to Verbs may improve the performance (it is not case for most other applications). \*\*To use Verbs\*\*: .. code-block:: bash export FI\_PROVIDER=verbs - If switching providers manually, ensure that you export the variables \*\*after\*\* loading the IntelMPI module. .. \*\*NOTES:\*\* .. - The \*\*VERBS provider\*\* uses the same fabric as MLX and is fully capable of utilizing the \*\*InfiniBand (IB) network\*\*. Switching from MLX to VERBS simply means that IntelMPI will use the API provided by the VERBS library instead of the one from the MLX library, while the underlying communication fabric (i.e. the low-level communication layer that enables data transfer between processes across nodes) remains the same - namely, \*\*OFI (OpenFabrics Interfaces)\*\*. .. - The \*\*TCP provider\*\* does not support the InfiniBand network; it relies exclusively on the \*\*Ethernet network\*\*. .. Preloaded Modules Prevent Job Submission .. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. .. \*\*Status:\*\* Open .. .. \*\*Last Update:\*\* 2025-07-29 .. .. \*\*Impacted partitions:\*\* All .. .. DESCRIPTION: .. ++++++++++++ .. .. Submitting jobs with modules preloaded in the shell environment can cause the following error: .. .. .. code-block:: bash .. .. sbatch: error: Batch job submission failed: Unexpected message received .. .. TEMPORARY SOLUTION: .. +++++++++++++++++++ .. .. Before submitting your job, run: .. .. .. code-block:: bash .. .. module purge .. .. Then load all necessary modules \*\*inside your job script\*\*. --- # Unknown .. \_leonardo\_card: Leonardo ======== Leonardo is the \*pre-exascale\* Tier-0 supercomputer of the EuroHPC Joint Undertaking (JU), hosted by \*\*CINECA\*\* and currently located at the Bologna DAMA-Technopole in Italy. This guide provides specific information about the \*\*Leonardo\*\* cluster, including details that differ from the general behavior described in the broader HPC Clusters section. .. |ico2| image:: img/leonardo\_logo.png :height: 55px :class: no-scaled-link Access to the System -------------------- The machine is reachable via \`\`ssh\`\` (secure Shell) protocol at hostname point: \*\*login.leonardo.cineca.it\*\*. The connection is established, automatically, to one of the available login nodes. It is possible to connect to \*\*Leonardo\*\* using one the specific login hostname points: \* login01-ext.leonardo.cineca.it \* login02-ext.leonardo.cineca.it \* login05-ext.leonardo.cineca.it \* login07-ext.leonardo.cineca.it .. warning:: \*\*The mandatory access to Leonardo si the two-factor authetication (2FA)\*\*. Get more information at section :ref:\`general/access:Access to the Systems\`. System Architecture ------------------- The cluster, supplied by EVIDEN ATOS, is based on two new specifically-designed compute blades, which are available throught two distinc Slurm partitios on the Cluster: \* X2135 \*\*GPU\*\* blade based on NVIDIA Ampere A100-64 accelerators - \*\*Booster\*\* partition. \* X2140 \*\*CPU\*\*-only blade based on Intel Sapphire Rapids processors - \*\*Data Centric General Purpose (DCGP)\*\* partition. The overall system architecture uses NVIDIA Mellanox InfiniBand High Data Rate (HDR) connectivity, with smart in-network computing acceleration engines that enable extremely low latency and high data throughput to provide the highest AI and HPC application performance and scalability. The \*\*Booster\*\* partition entered pre-production in May 2023 and moved to \*\*full production in July 2023\*\*. The \*\*DCGP\*\* partition followed, starting pre-production in January 2024 and reaching \*\*full production in February 2024\*\*. Hardware Details ^^^^^^^^^^^^^^^^ .. tab-set:: .. tab-item:: Booster .. list-table:: :widths: 30 50 :header-rows: 1 \* - \*\*Type\*\* - \*\*Specific\*\* \* - Models - Atos BullSequana X2135, Da Vinci single-node GPU \* - Racks - 116 \* - Nodes - 3456 \* - Processors/node - 1x \`Intel Ice Lake Intel Xeon Platinum 8358 \`\_ \* - CPU/node - 32 \* - Accelerators/node - 4x \`NVIDIA Ampere100 custom \`\_, 64GiB HBM2e NVLink 3.0 (200 GB/s) \* - Local Storage/node (tmfs) - (none) \* - RAM/node - 512 GiB DDR4 3200 MHz \* - Rmax - 241.2 PFlop/s (\`top500 \`\_) \* - Internal Network - 200 Gbps NVIDIA Mellanox HDR InfiniBand - Dragonfly+ Topology \* - Storage (raw capacity) - 106 PiB based on DDN ES7990X and Hard Drive Disks (Capacity Tier) 5.7 PiB based on DDN ES400NVX2 and Solid State Drives (Fast Tier) .. tab-item:: DCGP .. list-table:: :widths: 30 50 :header-rows: 1 \* - \*\*Type\*\* - \*\*Specific\*\* \* - Models - Atos BullSequana X2140 three-node CPU blade \* - Racks - 22 \* - Nodes - 1536 \* - Processors/node - 2x \`Intel Sapphire Rapids Intel Xeon Platinum 8480+ \`\_ \* - CPU/node - 112 cores/node \* - Accelerators - (none) \* - Local Storage/node (tmfs) - 3 TiB \* - RAM/node - 512(8x64) GiB DDR5 4800 MHz \* - Rmax - 7.84 PFlop/s (\`top500 \`\_) \* - Internal Network - 200 Gbps NVIDIA Mellanox HDR InfiniBand - Dragonfly+ Topology \* - Storage (raw capacity) - 106 PiB based on DDN ES7990X and Hard Drive Disks (Capacity Tier) 5.7 PiB based on DDN ES400NVX2 and Solid State Drives (Fast Tier) File Systems and Data Managment ------------------------------- The storage organization conforms to \*\*CINECA\*\* infrastructure. General information are reported in :ref:\`hpc/hpc\_data\_storage:File Systems and Data Management\` section. In the following, only differences with respect to general behavior are listed and explained. .. dropdown:: \*\*$TMPDIR\*\* \* on the local SSD disks on login nodes (14 TB of capacity), mounted as \`\`/scratch\_local\`\` (\`\`TMPDIR=/scratch\_local\`\`). This is a shared area with no quota, remove all the files once they are not requested anymore. A cleaning procedure will be enforced in case of improper use of the area. \* on the local SSD disks on the serial node (\`\`lrd\_all\_serial\`\`, 14TB of capacity), managed via the Slurm \`\`job\_container/tmpfs plugin\`\`. This plugin provides a \*job-specific\*, private temporary file system space, with private instances of \`\`/tmp\`\` and \`\`/dev/shm\`\` in the job's user space (\`\`TMPDIR=/tmp\`\`, visible via the command \`\`df -h\`\`), removed at the end of the serial job. You can request the resource via sbatch directive or srun option \`\`--gres=tmpfs:XX\`\` (for instance: \`\`--gres=tmpfs:200G\`\`), with a maximum of 1 TB for the serial jobs. If not explicitly requested, the \`\`/tmp\`\` has the default dimension of 10 GB. \* on the local SSD disks on DCGP nodes (3 TB of capacity). As for the serial node, the local \`\`/tmp\`\` and \`\`/dev/shm\`\` areas are managed via plugin, which at the start of the jobs mounts private instances of \`\`/tmp\`\` and \`\`/dev/shm\`\` in the job's user space (\`\`TMPDIR=/tmp\`\`, visible via the command \`\`df -h /tmp\`\`), and unmounts them at the end of the job (all data will be lost). You can request the resource via sbatch directive or srun option \`\`--gres=tmpfs:XX\`\`, with a maximum of all the available 3 TB for DCGP nodes. As for the serial node, if not explicitly requested, the \`\`/tmp\`\` has the default dimension of 10 GB. Please note: for the DCGP jobs the requested amount of \`\`gres/tmpfs\`\` resource contributes to the consumed budget, changing the number of accounted equivalent core hours, see the dedicated section on the Accounting. \* on RAM on the diskless booster nodes (with a fixed size of 10 GB, no increase is allowed, and the \`\`gres/tmpfs\`\` resource is disabled). Job Managing and Slurm Partitions --------------------------------- In the following table you can find informations about the Slurm partitions for \*\*Booster\*\* and \*\*DCGP\*\* partitions. .. seealso:: Further information about job submission are reported in the general section :ref:\`hpc/hpc\_scheduler:Scheduler and Job Submission\`. .. tab-set:: .. tab-item:: Booster +----------------+--------------------+-------------------------+--------------+---------------------------------+--------------+-------------------------------------+ | \*\*Partition\*\* | \*\*QOS\*\* | \*\*TRES Limits per Job\*\* | \*\*Walltime\*\* | \*\*MaxTRES per User\*\* | \*\*Priority\*\* | \*\*Notes\*\* | +================+====================+=========================+==============+=================================+==============+=====================================+ | lrd\_all\_serial | normal | Max = 4 cores | 04:00:00 | 1 node / 4 cores | 40 | No GPUs, Hyperthreading x 2 | | | | | | | | | | (\*\*default\*\*) | | (8 logical cores) | | (30800 MB RAM) | | \*\*Budget Free\*\* | +----------------+--------------------+-------------------------+--------------+---------------------------------+--------------+-------------------------------------+ | boost\_usr\_prod | normal | Max = 64 nodes | 24:00:00 | | 40 | | + +--------------------+-------------------------+--------------+---------------------------------+--------------+-------------------------------------+ | | boost\_qos\_dbg | Max = 4 nodes | 00:30:00 | 4 nodes / 128 cores / 16 GPUs | 80 | Max 1 job running and/or pending | | | | | | | | | | | | | | | | per User Account | + +--------------------+-------------------------+--------------+---------------------------------+--------------+-------------------------------------+ | | boost\_qos\_bprod | Min = 65 full nodes | 24:00:00 | 256 nodes | 60 | | | | | | | | | | | | | Max = 256 nodes | | | | | + +--------------------+-------------------------+--------------+---------------------------------+--------------+-------------------------------------+ | | boost\_qos\_lprod | Max = 8 nodes | 4-00:00:00 | 8 nodes / 32 GPUs | 40 | Max resources per Project Account | +----------------+--------------------+-------------------------+--------------+---------------------------------+--------------+-------------------------------------+ .. tab-item:: DCGP +----------------+--------------------+-------------------------+--------------+---------------------------------------+--------------+-------------------------------------+ | \*\*Partition\*\* | \*\*QOS\*\* | \*\*TRES Limits per Job\*\* | \*\*Walltime\*\* | \*\*MaxTRES per User or Proj. Account\*\* | \*\*Priority\*\* | \*\*Notes\*\* | +================+====================+=========================+==============+=======================================+==============+=====================================+ | lrd\_all\_serial | normal | Max = 4 cores | 04:00:00 | 1 node / 4 cores | 40 | Hyperthreading x 2 | | | | | | | | | | (\*\*default\*\*) | | (8 logical cores) | | (30800 MB RAM) | | \*\*Budget Free\*\* | +----------------+--------------------+-------------------------+--------------+---------------------------------------+--------------+-------------------------------------+ | dcgp\_usr\_prod | normal | Max = 16 nodes | 24:00:00 | 512 nodes per Prj. Account | 40 | | + +--------------------+-------------------------+--------------+---------------------------------------+--------------+-------------------------------------+ | | dcgp\_qos\_dbg | Max = 2 nodes | 00:30:00 | 2 nodes / 224 cores per User Account | 80 | Max 1 job running and/or pending | | | | | | | | | | | | | | 512 nodes per Prj. Account | | per User Account | + +--------------------+-------------------------+--------------+---------------------------------------+--------------+-------------------------------------+ | | dcgp\_qos\_bprod | Min = 17 full nodes | 24:00:00 | 128 nodes per User Account | 60 | GrpTRES = 1536 nodes | | | | | | | | | | | | Max = 128 nodes | | 512 nodes per Prj. Account | | Min is 17 FULL nodes | + +--------------------+-------------------------+--------------+---------------------------------------+--------------+-------------------------------------+ | | dcgp\_qos\_lprod | Max = 3 nodes | 4-00:00:00 | 3 nodes / 336 cores per user Account | 40 | | | | | | | | | | | | | | | 512 nodes per Prj. Account | | | +----------------+--------------------+-------------------------+--------------+---------------------------------------+--------------+-------------------------------------+ Network Architecture -------------------- .. raw:: html

Leonardo features a state-of-the-art interconnect system tailored for high-performance computing (HPC). It delivers low latency and high bandwidth by leveraging NVIDIA Mellanox InfiniBand HDR (High Data Rate) technology, powered by NVIDIA QUANTUM QM8700 Smart Switches, and a Dragonfly+ topology. Below is an overview of its architecture and key features:

  • Hierarchical Cell Structure: The system is structured into multiple cells, each comprising a group of interconnected compute nodes.
  • Inter-cell Connectivity: As illustrated in the figure below, cells are connected via an all-to-all topology. Each pair of distinct cells is linked by 18 independent connections, each passing through a dedicated Layer 2 (L2) switch. This design ensures high availability and reduces congestion.
  • Intra-cell Topology: Inside each cell, a non-blocking two-layer fat-tree topology is used, allowing scalable and efficient intra-cell communication.
  • System Composition:
    • 19 cells dedicated to the Booster partition.
    • 2 cells for the DCGP (Data-Centric General Purpose) partition.
    • 1 hybrid cell with both accelerated (36 Booster nodes) and conventional (288 DCGP nodes) compute resources.
    • 1 cell allocated for management, storage, and login services.
  • Adaptive Routing: The network employs adaptive routing, dynamically optimizing data paths to alleviate congestion and maintain performance under load.
.. figure:: img/leo-net-all2all.png :height: 350px :align: center :class: no-scaled-link .. image:: img/spacer.png :align: center :class: no-scaled-link .. dropdown:: Cell Configuration and Intra-cell Connectivity :animate: fade-in-slide-down :chevron: down-up .. tab-set:: .. tab-item:: Booster .. raw:: html

Each Booster cell is composed of:

  • 6 × Atos BullSequana XH2000 racks, each containing:
    • 3 × Level 2 (L2) switches
    • 3 × Level 1 (L1) switches
    • 30 compute nodes — each equipped with 4 GPUs, each connected via a dedicated 100 Gbps port

Total per Booster cell: 18 L2 switches, 18 L1 switches, and 180 compute nodes.

Connectivity Overview

Level 2 (L2) Switches:

  • UP: 22 × 200 Gbps ports connecting to L2 switches in other cells
  • DOWN: 18 × 200 Gbps ports connecting to L1 switches within the cell
  • Oversubscription: 0.8:1

Level 1 (L1) Switches:

  • UP: 18 × 200 Gbps ports connected to all L2 switches in the cell
  • DOWN: 40 × 100 Gbps ports connected to GPUs across 10 compute nodes
  • Oversubscription: 1.11:1
.. figure:: img/leo-net-booster\_cell.png :height: 750px :align: center .. tab-item:: DCGP .. raw:: html .. raw:: html

Each DCGP cell is composed of:

  • 8 × Atos BullSequana XH2000 racks, each containing:
    • 3 or 0 Level 2 (L2) switches
    • 2 × Level 1 (L1) switches
    • 78 compute nodes — each connected via a dedicated 100 Gbps port

Total per DCGP cell: 18 L2 switches, 16 L1 switches, and 624 compute nodes.

Connectivity Overview

Level 2 (L2) Switches:

  • UP: 22 × 200 Gbps ports connecting to L2 switches in other cells
  • DOWN: 18 × 200 Gbps ports connecting to L1 switches within the same cell
  • Oversubscription ratio: 0.8:1

Level 1 (L1) Switches: (divided into two groups):

  • 9 switches with 40 downlinks:
    • UP: 18 × 200 Gbps ports connected to all L2 switches in the cell
    • DOWN: 40 × 100 Gbps ports connected to compute nodes
    • Oversubscription ratio: 1.11:1
  • 9 switches with 38 downlinks:
    • UP: 18 × 200 Gbps ports connected to all L2 switches in the cell
    • DOWN: 38 × 100 Gbps ports connected to compute nodes
    • Oversubscription ratio: 1.05:1
.. figure:: img/leo-net-dcgp\_cell.png :height: 750px :align: center Advanced Information ^^^^^^^^^^^^^^^^^^^^ .. dropdown:: Network Topology - Map :animate: fade-in-slide-down :chevron: down-up The topology is presented in a table format, where each row corresponds to a compute node. For each node, the table specifies the associated L1 switch and cell, providing a clear overview of the physical and logical network layout within the cluster. :download:\`Network Topology - Map <../files/ntopology.dat>\` .. dropdown:: Network Topology - Distance Matrix :animate: fade-in-slide-down :chevron: down-up The attached compressed CSV file contains the distance matrix of all compute nodes in the cluster. The matrix uses the following metric to represent the network distance between any two nodes: \* \*\*0\*\* – Same nodes \* \*\*1\*\* – Same L1 switch, same cell. \* \*\*2\*\* – Different L1 switch, same cell. \* \*\*3\*\* – Different L1 switch and different cell. This matrix can be used to analyze communication locality and optimize node selection for distributed workloads. :download:\`Distance Matrix <../files/ntopology-dst\_mtx.tar.bz2>\` .. dropdown:: Switch Naming Format :animate: fade-in-slide-down :chevron: down-up .. code-block:: isw where \`\`\`\` is a 5- or 6-digits number varies based on the location and type of the switch. Specifically: \* \`\`RR\`\` = region number (1 or 2 digits) \* \`\`rr\`\` = rack number (2 digits) \* \`\`SS\`\` = switch id (2 digits) .. note:: If \`\`SS\`\` is an even number, it refers to an L1 switch; if it is an odd number, it refers to an L2 switch. Documents --------- \* Article on Leonardo architecture and the technologies adopted for its GPU-accelerated partition: CINECA Supercomputing Centre, SuperComputing Applications and Innovation Department. (2024). “LEONARDO: A Pan-European Pre-Exascale Supercomputer for HPC and AI applications.”, Journal of large-scale research facilities, 8, A186. https://doi.org/10.17815/jlsrf-8-186 \* Details about new technologies included in the Witley platform with Intel Xeon Icelake contained in the Leonardo pre-exascale system (\`link \`\_) \* Additional documents (\`link \`\_) Some tuning guides for dedicated enviroments (ML/DL or HPC Clusters): \* :download:\`Tuning Guide <../files/Tuning\_guide.pdf>\` \* :download:\`Deep Learning <../files/Deep\_learning.pdf>\` --- # Unknown .. \_galileo\_card: Galileo100 ========== Galileo100 is a new infrastructure co-funded by the European ICEI (Interactive Computing e-Infrastructure) project and engineered by DELL. It is the national Tier-1 system for scientific research and is available to the Italian public and industrial researchers since September 2021. It also features 77 cloud computing servers and was expanded in November 2022 with 82 additional nodes. \*\*Galileo100\*\* is used for high-end technical and industrial HPC projects, as well as meteorology and environmental studies. The specific guide for the \*\*Galileo100\*\* cluster contains unique information that deviates from the general behavior described in the HPC Clusters sections. Access to the System -------------------- The machine is reachable via \`\`ssh\`\` (secure Shell) protocol at hostname point: \*\*login.g100.cineca.it\*\*. The connection is established, automatically, to one of the available login nodes. It is possible to connect to \*\*Galileo100\*\* using one the specific login hostname points: \* login01-ext.g100.cineca.it \* login02-ext.g100.cineca.it \* login03-ext.g100.cineca.it .. warning:: \*\*The mandatory access to Galileo100 is the two-factor authetication (2FA)\*\*. Get more information at section :ref:\`general/access:Access to the Systems\`. System Architecture ------------------- Hardware Details ^^^^^^^^^^^^^^^^ .. list-table:: :widths: 30 50 :header-rows: 1 \* - \*\*Type\*\* - \*\*Specific\*\* \* - Models - Dual-soket Dell PowerEdge \* - Nodes - 630 \* - Processors/node - 2xCPU x86 Intel Xeon Platinum 8276/L 2.4GHz \* - CPU/node - 48 \* - Accelerators/node - 2xGPU Nvidia V100 PCIe3 with 32 GB Ram on 36 Viz Nodes \* - RAM/node - 384 GiB (+ 3.0 TiB Optane on 180 fat nodes) \* - Peak Performance - 2 PFlop/s (3.53 TFlop/s in single node) \* - Internal Network - Mellanox Infiniband 100GbE Disks and Filesystems --------------------- The storage organization conforms to \*\*CINECA\*\* infrastructure. General information are reported in :ref:\`hpc/hpc\_data\_storage:File Systems and Data Management\` section. In the following, only differences with respect to general behavior are listed and explained. Job Managing and SLURM Partitions --------------------------------- .. list-table:: :widths: 10 10 20 10 10 10 10 20 :header-rows: 1 :class: tight-table \* - \*\*Partition\*\* - \*\*QOS\*\* - \*\*#Cores per job\*\* - \*\*Walltime\*\* - \*\*Max jobs/resources per user\*\* - \*\*Max memory per node (MB)\*\* - \*\*Priority\*\* - \*\*Notes\*\* \* - g100\_all\_serial (default) - noQOS - 4 cores - 04:00:00 - 4 cores 120 submitted jobs - 31,200 (30 GB) - 40 - on two login nodes \*\*budget free\*\* \* - g100\_all\_serial (default) - qos\_install - 16 cores - 04:00:00 - 16 cores 1 running job - 100 GB - 40 - request to superc@cineca.it \* - g100\_usr\_dbg - noQOS - 2 nodes - 01:00:00 - - 375,300 (366 GB) - 40 - \* - g100\_usr\_dbg - qos\_ind - Depending on the specific agreement - Depending on the specific agreement - - 375,300 (366 GB) - 90 - Partition dedicated to specific kinds of users. \* - g100\_usr\_prod \*g100\_usr\_smem\* \*\*g100\_usr\_pmem\*\* - noQOS - min = 1 max = 32 nodes - 24:00:00 - 100 running jobs 120 submitted jobs - 375,300 (366 GB) - 40 - runs on thin and persistent memory nodes \*runs only on thin nodes\* \*\*runs only on persistent memory nodes\*\* \* - g100\_usr\_prod \*g100\_usr\_smem\* \*\*g100\_usr\_pmem\*\* - g100\_qos\_bprod - min = 1537 (33 nodes) max = 3072 (64 nodes) - 24:00:00 - 100 running jobs 120 submitted jobs - 375,300 (366 GB) - 60 - runs on thin and persistent memory nodes \*runs only on thin nodes\* \*\*runs only on persistent memory nodes\*\* \* - g100\_usr\_prod \*g100\_usr\_smem\* \*\*g100\_usr\_pmem\*\* - g100\_qos\_lprod - min = 1 max = 2 nodes - 4-00:00:00 - 2 nodes 100 running jobs 120 submitted jobs - 375,300 (366 GB) - 40 - runs on thin and persistent memory nodes \*runs only on thin nodes\* \*\*runs only on persistent memory nodes\*\* \* - g100\_usr\_prod \*g100\_usr\_smem\* \*\*g100\_usr\_pmem\*\* - qos\_special - > 32 nodes - > 24:00:00 - - 375,300 (366 GB) - 40 - request to superc@cineca.it \* - g100\_usr\_bmem - noQOS - 25 nodes - 24:00:00 - 100 running jobs 120 submitted jobs - 3,036,000 (3 TB) - 40 - runs on fat nodes \* - g100\_usr\_interactive - noQOS - max = 0.5 node - 8:00:00 - 100 running jobs 120 submitted jobs - 375,300 (366 GB) - 40 - on nodes with GPUs --gres=gpu:N (N=1) \* - g100\_meteo\_prod - qos\_meteo - - 24:00:00 - - 375,300 (366 GB) - 40 - Partition reserved to meteo services, \*\*NOT opened to production.\*\* Runs on thin nodes Dedicated Services ------------------ Interactive Computing ^^^^^^^^^^^^^^^^^^^^^ Galileo 100 resources are also accessible via web browser on a Jupyter-based interface at the following link: https://jupyter.g100.cineca.it/ Further details are reported at the following :ref:\`services/interactive\_computing:interactive Computing\`. Please note that the service is considered in pre-production, thus the resources are not accounted from the budget and the service is provided with no warranty. --- # Unknown .. \_miniconda\_card: Miniconda ========= Miniconda is a minimal installer for Conda, a popular package manager for Python and other languages. Unlike the full Anaconda distribution, which includes hundreds of preinstalled packages and tools, Miniconda provides only the Conda package manager and Python. This lightweight installer allows users to create customized environments by installing only the packages they need. Miniconda is ideal for: - Users who want a smaller installation footprint. - Environments where storage space or bandwidth is limited. - Developers and researchers who prefer full control over package versions and dependencies. With Miniconda, users can: - Create and manage isolated environments with different Python versions. - Install packages from multiple channels such as conda-forge or bioconda. - Ensure reproducibility and compatibility across systems. How to Install Miniconda ------------------------ .. important:: \*\*Cleaning Up Anaconda3 previous Configuration from the Home Directory\*\* Sometimes, previous Anaconda installations may interfere with the correct installation of Miniconda. For this reason, it is recommended to perform the following cleanup steps before proceeding with the installation: - \*\*Delete the Conda configuration file\*\*:: rm -f $HOME/.condarc - \*\*Delete the Conda data directory\*\*:: rm -rf $HOME/.conda - \*\*Remove Anaconda initialization from your shell configuration file\*\*: Open your \`\`$HOME/.bashrc\`\` and remove all lines related to Anaconda3. These lines are usually located at the end of the file and enclosed between the following markers: .. code-block:: bash # >>> conda initialize >>> ... # <<< conda initialize <<< You can safely delete this entire block. To \`install Miniconda3 \`\_, you have to download the installation script by running: .. code-block:: bash wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86\_64.sh Execute the downloaded script with: .. code-block:: bash bash $HOME/Miniconda3-latest-Linux-x86\_64.sh During the installation, reply \*\*"yes"\*\* or \*\*"ENTER"\*\* to all prompts. At the end of the installation, reload the bash session to apply the modifications introduced by the installer: .. code-block:: bash exec bash This command will start a new bash session in which the \`\`(base)\`\` Conda environment will be automatically activated. Configure Channels ^^^^^^^^^^^^^^^^^^ Conda channels are configured at multiple levels: - Global configuration: \`\`~/.condarc\`\` - Environment-specific configuration: \`\`/conda-meta/.condarc\`\` To configure Conda to work correctly: - Disable automatic activation of the base environment: .. code-block:: bash conda config --set auto\_activate\_base false - Reload the bash session again: .. code-block:: bash exec bash This will start a new session where the \`\`(base)\`\` environment will no longer be automatically activated. - Enable strict channel priority: .. code-block:: bash conda config --set channel\_priority strict - Add the \`conda-forge\` channel: .. code-block:: bash conda config --add channels conda-forge - Edit the following configuration files: - \`\`$HOME/.condarc\`\` - \`\`$HOME/miniconda3/.condarc\`\` In both files, comment out (by adding a \`\`#\`\` at the beginning of the line) any lines containing: .. code-block:: yaml - https://repo.anaconda.com/pkgs/main - https://repo.anaconda.com/pkgs/r .. note:: In some versions of Conda, you may instead see: .. code-block:: yaml - defaults In that case, comment out the \`\`- defaults\`\` line. .. important:: Removing defaults from your Conda configuration will not break Conda. Conda will continue to function as expected. You can use conda-forge as your only channel without issues. These last three steps ensure that only the \`conda-forge\` channel is used, and disable the default Anaconda channels which may cause timeout errors or access errors when downloading packages on our HPC systems. For full documentation on this, see: https://conda-forge.org/docs/user/transitioning\_from\_defaults/ https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually Create a New Environment ^^^^^^^^^^^^^^^^^^^^^^^^ To create a new environment: .. code-block:: bash conda create -n new\_env Activate the environment: .. code-block:: bash conda activate new\_env Install Packages from Conda-Forge ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If the desired package is available in the \`conda-forge\` channel, install it \*inside\* the environment: .. code-block:: bash conda install -n new\_env Example: .. code-block:: bash conda install -n new\_env pytorch To check availability: .. code-block:: bash conda search pytorch --- # Unknown .. \_what\_is\_cloud\_card: What is Cloud Computing ======================= Cloud computing is a \*\*virtualization-based technology\*\* that allows users remote access to a pool of virtual resources (typically CPUs, memory and storage) completely isolated one from another that can be used to create and operate one or more virtual machines depending on the needs. \*\*HPC cloud\*\*, or High-Performance Computing cloud, integrates high-performance computing resources and capabilities with cloud computing infrastructure. It combines the computational power and scalability of traditional HPC systems with the flexibility and on-demand nature of cloud services. Features of HPC Cloud ----------------------- HPC Cloud users are able to exploit the computational power of HPC machines with the added benefit of extreme flexibility. Users can organize their pool of computational resources how they see fit for their personal workflow, creating a single virtual machine that harnesses the full capability of all the resources at once, or smaller virtual machines that can communicate and interact with each other. More in details, within HPC Cloud the users can: - decide how to exploit computational and storage resources, creating their personal virtual infrastructure (\*\*flexibility\*\*) - manage their virtual machines, for example by deciding the operative system and software environments (\*\*flexibility\*\*) - scale the computational resources based on their needs allowing to handle varying workloads efficiently (\*\*scalability\*\*) - optimize the use of the resources without idle time (\*\*resources optimization\*\*) - access the resources remotely (\*\*accessibility\*\*) What to apply for: HPC or Cloud computing? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. csv-table:: :header: " ", "HPC", "Cloud Computing" :widths: 5 5 5 :class: tight-table "\*\*Performance\*\*", "Target the highest possible.", "Depends on workload, but generally, virtualization has a small impact." "\*\*User access\*\*", "HPC site staff authorization.", "Once a project is granted, it is managed by the user." "\*\*Operating System\*\*", "It is chosen by HPC site staff given the HW constraints. Security updates are managed by HPC site.", "Selected by the user. Security patches and updates are managed by the user." "\*\*Software Stack\*\*", "Mostly installed by HPC site staff. Users can install their own without \*root\* privilege. The environment is provided \*as is\*.", "The user is root on the VMs and can install all the required software stack. Users can modify the environment to suit their needs." "\*\*Snapshots of the environment\*\*", "Cannot be done", "User can save snapshot images of the VMs." "\*\*Running simulations\*\*", "Users are provided with a job scheduler (SLURM) and have to wait for resources to free in order for their job to run", "Users are able to run serial or parallel jobs how/when they need." "\*\*Very large simulations\*\*", "Users are provided with a job scheduler and have to wait for resources to free in order for their job to run.", "Users can plan their workloads execution." Service Models ^^^^^^^^^^^^^^ Cloud computing resources can be provided using different service models: - \*\*Infrastructure as a Service (IaaS)\*\*: Provides virtualized computing resources over the internet. Users are able to deploy those resources how they see fit. - \*\*Platform as a Service (PaaS)\*\*: Offers a platform allowing developers to build, deploy, and manage applications without worrying about the underlying infrastructure. - \*\*Software as a Service (SaaS)\*\*: Delivers software applications over the internet, accessible via a web browser, without the need for local installation or maintenance. In CINECA, we provide resources using the IaaS model (see section :ref:\`cloud/general/cineca\_cloud\_model:cineca hpc cloud model\` for more information). --- # Unknown .. \_cineca\_cloud\_model\_card: CINECA HPC Cloud Model ======================= This page describes how CINECA provides HPC Cloud resources to its users. .. note:: CINECA HPC Cloud infrastructure is certified ISO 27001 since 2022 for \*\*"Servizi informatici HPC in cloud per la ricerca in ambito life science"\*\* and since 2025 for \*\*"Erogazione di servizi IaaS per ricerca e innovazione su HPC Cloud"\*\*. Details can be found \`here \`\_. CINECA Service Model ^^^^^^^^^^^^^^^^^^^^ CINECA HPC Cloud infrastructure is provided via an \*\*Infrastructure as a Service (IaaS)\*\* model. In IaaS model, the Cloud Provider administrates the hardware and virtualization layers of the infrastructure and provides both computing resources (virtual CPUs, storage, network, GPUs...) and high-level APIs (dashboards, command line (CLI) tools) that users can employ to control the resources they were granted. HPC Cloud use cases ^^^^^^^^^^^^^^^^^^^ Cloud computing means \*\*paramount flexibility\*\*. With a cloud IaaS model, users are able to setup their project environment as they see fit, using all the infrastructure tools and resources and with the support provided by CINECA to meet their specific needs. CINECA users rely on the HPC Cloud infrastructure to address different use cases. The list below is not meant to be exhaustive, but to provide examples of scenarios where HPC Cloud can be particularly useful. .. grid:: 3 .. grid-item-card:: Hosting of data processing and analysis services (typical Infrastructure as a Service, IaaS).​ .. grid-item-card:: Hosting of HPC mini-cluster with adequate performance.​ .. grid-item-card:: Hosting of data management services receiving or exposing data from/to web.​ .. grid:: 3 .. grid-item-card:: Hosting of data management services receiving or exposing data from/to internal CINECA HPC infrastructure.​ .. grid-item-card:: Hosting of workload processing sensitive data.​ .. grid-item-card:: Bridging HPC Infrastructure, e.g. hosting front-end services for management of workloads on CINECA HPC system.​ .. grid:: 3 .. grid-item-card:: Flexible and automated deployment via Kubernetes on top of OpenStack of containerized workflows.​ .. grid-item-card:: Collaborative infrastructure deployment within a user tenant (Infrastructure as Code, IaC).​ .. grid-item-card:: Everything that requires performance and flexibility (respect to the HPC cluster).​ Responsibilities ^^^^^^^^^^^^^^^^^ While CINECA is responsible for the provisioning and maintenance of the hardware and virtualization layer (OpenStack), the users are responsible for anything they set up and install on their project (e.g. network setup, OS and applications on virtual machines, access to services and VMs). A clear separation of roles in using the service is represented in the scheme below: .. image:: ../\_img/cloud\_model.png .. list-table:: Roles and responsibilities :widths: 1 1 1 1 :header-rows: 1 :class: tight-table \* - \*\*Name\*\* - \*\*Description\*\* - \*\*Role\*\* - \*\*Responsibilities\*\* \* - \*\*CINECA\*\* - Cloud provider - - Administers physical infrastructure - Provides virtualization layer and API tools - - Maintaining hardware and virtualization layer - User support \* - \*\*User Admin\*\* - - Users with granted budget on CINECA HPC cloud - Project PIs and collaborators in \`CINECA UserDB \`\_ - - Create and manage cloud resources via the provided APIs (dashboard or CLI) - Responsible for all the resources they create (VMs, storage, networks,...) - - Administer of the resources - Maintain VMs for which they have admin privileges - Implement security measures - Backups/snapshots of resources during the project and at the end of the validity period \* - \*\*User\*\* - Users with granted access to the project VMs by User Admins. - Can utilize VMs and services they have been granted access to by User Admins. - Maintain the VMs for which they have admin privileges Any user (\*“User Admins"\* or \*“Users”\*) with administration privileges on IaaS resources (VMs) has the responsibility to maintain the security (security patch, fix) on those resources. In particular, they have the responsibility to perform VMs and volume data backups. See the dedicated page for :ref:\`cloud/tenant\_adm/security\_guidelines:security guidelines\` information. .. warning:: Currently snapshots and backups of resources are stored in the same HPC Cloud infrastructure. From the project management perspective, CINECA will interact only with \*“User Admins"\*. At the end of the project validity, the \*“User Admins”\* will receive communication from CINECA staff that the project as expired with the date by when the resources will be removed. It is \*“User Admins”\* responsibility to make copy of the necessary VMs or data before that date. --- # Unknown .. \_budget\_accounting\_card: Budget and accounting ====================== Following the :ref:\`cloud/general/cineca\_cloud\_model:cineca hpc cloud model\`, the users manage autonomously the Cloud projects (tenants), one or more, to which they are associated. To access Cloud resources, you need to get a CINECA HPC account. For detailed instructions on creating an account, please refer to the :ref:\`general/users\_account:How to become a User\` section. Each tenant is then composed by a pool of virtual resources (project quota/budget) which are defined in terms of: - Number of vCPUs - GB of RAM - GB of storage - Number of public IP addresses (floating IPs) On request, you can be also granted - Number of GPUs - Additional storage for shares Upon accessing the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` with HPC credentials, within the assigned tenants, you can autonomously create and manage all components of the virtual infrastructure (VMs, Networks, Security Policies, Load Balancer Policies, and so on) in compliance with the Access Policies accepted at the time of registration on UserDB (see the :ref:\`cloud/operative/index\_operative\_manual:operative manual\` for specific operations details). When resource consuming operations, such as virtual machine creation, are performed, the request is validated against the maximum quota permitted for the current project. The users can also autonomously monitor the usage of the assigned resources in the OpenStack Horizon Dashboard, see :ref:\`cloud/operative/compute\_ops/instance\_manage:instance: manage and monitor\` page. --- # Unknown EFGW Gateway ============ EFGW is the new EUROfusion Gateway system hosted by CINECA in the headquarter Casalecchio di Reno, Bologna, Italy. The cluster is supplied by Lenovo Corp. and is equipped with 15 AMD nodes, including 4 nodes with fat memory (1.5 TB), and 1 node with 4 H100 GPUs and local SSD storage. How to get a User Account ------------------------- Users of the old EUROfusion Gateway were migrated on the new system with the same username. For new users: to get access to EFGW the following steps are required: \* Register on \`UserDB portal \`\_ \* complete the registration filling your affiliation in the \*\*Institution\*\* page and uploading a valid Identity Document in \*\*Documents for HPC\*\* page. \* Download the :download:\`Gateway User Agreement <../files/eurofusion\_gateway\_user\_agreement\_26\_10\_2022.pdf>\` (GUA) \* Fill and sign the GUA, send it via email to EUROfusion Coordination Officer Denis Kalupin (Denis.Kalupin-at-euro-fusion.org) \* After the GUA is signed by the EUROfusion Coordination Officer, the user will receive an email from CINECA with final instructions. Access to the System -------------------- The machine is reachable via \`\`ssh\`\` (secure Shell) protocol at hostname point: \*\*login.eufus.eu\*\*. The connection is established, automatically, to one of the available login nodes. It is also possible to connect to \*\*EFGW\*\* using one the specific login hostname points: \* \*\*viz05-ext.efgw.cineca.it\*\* \* \*\*viz06-ext.efgw.cineca.it\*\* \* \*\*viz07-ext.efgw.cineca.it\*\* \* \*\*viz08-ext.efgw.cineca.it\*\* Each login node is equipped with two AMD EPIC 9254 24-Core Processors. An alias hostname pointing to all the login nodes in a round-robin fashion, will be set-up in the next weeks. .. warning:: \*\*The mandatory access to EFGW is the two-factor authentication (2FA) via the dedicated provisioner efgw\*\*. Get more information at section :ref:\`general/access:Access to the Systems\`. \*\*Please note\*\*: EFGW users have to obtain the ssh certificate from the \*\*efgw\*\* provisioner. \* In the section :ref:\`general/access:How to activate the \*\*2FA\*\* and the \*\*OTP\*\* generator\` use the \`step-CA EFGW client \`\_ in the place of the step-CA CINECA-HPC client \* In the section :ref:\`general/access:How to configure \*smallstep\* client\` \*\*Step 3\*\*, obtain the ssh certificate from the efgw provisioner .. code-block:: bash step ssh login 'username' --provisioner efgw \* In the section :ref:\`general/access:How to manage authentication \*certificates\*\`, use the efgw provisioner in all the key \*step\* commands (certificate re-generation, certificate creation in file format) How to access EFGW with NX -------------------------- 1. Get a ssh key with \*step\* and put it in your $HOME/.ssh folder .. code-block:: bash step ssh certificate 'username' --provisioner efgw ~/.ssh/gw\_key 2. Configure a NX session as follows: .. image:: ../img/nx1.png :width: 600px :align: center .. image:: ../img/nx2.png :width: 600px :align: center .. image:: ../img/nx3.png :width: 600px :align: center .. image:: ../img/nx4.png :width: 600px :align: center You can freely install software on your NX desktop using the "flatpak" package. Please refer to the \`official documentation \`\_ for instructions on how to use it. \*\*Please keep in mind that you need to use it with the --user flag.\*\* System Architecture ------------------- The cluster, supplied by Lenovo, is based on AMD processors: \* 10 nodes with two AMD EPYC 9745 128-Core Processors and 738 GB DDR5 RAM per node \* 4 nodes with two AMD EPYC 9745 128-Core Processors and 1511 GB DDR5 RAM per node \* 1 node with two AMD EPIC 9354 32-Core Processors, 4 H100 GPUs, and 738 GB DDR5 RAM per node File Systems and Data Management -------------------------------- The storage organization conforms to \*\*CINECA\*\* infrastructure. General information are reported in :ref:\`hpc/hpc\_data\_storage:File Systems and Data Management\` section. In the following, only differences with respect to general behavior are listed and explained. The storage is organized as a replica of the previous Gateway cluster with the data of \*\*/afs\*\* and \*\*/pfs\*\* areas copied on the new lustre storage system (no afs available, only the data were copied). Please notice that the path \*\*/gss\_efgw\_work\*\*, linked to the /pfs areas on the old Gateway, does not exist on the new Gateway. The TMPDIR is defined: \* on the local SSD disks on login nodes (2.5 TB of capacity), mounted as \`\`/scratch\_local\`\` (\`\`TMPDIR=/scratch\_local\`\`). This is a shared area with no quota, remove all the files once they are not requested anymore. A cleaning procedure will be enforced in case of improper use of the area. \* on the local SSD disk on the GPU node (850 GB of capacity, default size 10 GB ) \* on RAM on all the 14 cpu-only, diskless compute nodes (with a fixed size of 10 GB) On the GPU node, a larger local TMPDIR area can be requested, if needed, with the slurm directive: .. code-block:: bash $ SBATCH --gres=tmpfs:XXG up to a maximum of 212.5 GB. Environment and Customization ----------------------------- The main tools and compilers are available through the module command when logging into the cluster: .. code-block:: bash $ module av To have all the modules of aocc, gcc, and OneAPI stacks installed from CINECA staff, you need to load the "cineca-modules" module and execute "module av" command: .. code-block:: bash $ module load cineca-modules $ module av For getting information about "module" usage, compilers, and mpi libraries you can consult the :ref:\`hpc/hpc\_enviroment:The module command\` and :ref:\`hpc/hpc\_enviroment:Compilers\` You can install any additional software you may need with \`flatpack \`\_ or :ref:\`hpc/hpc\_enviroment:SPACK\` . How to make your $HOME/public open to all users ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In order to configure on the new EFGW your $HOME/public as it was on the old EFGW afs filesystem, please add the proper ACL to your $HOME directory as follows: .. code-block:: bash $ setfacl -m g:g2:x $HOME Job Managing and Slurm Partitions --------------------------------- In the following table you can find informations about the Slurm partitions on the EFGW cluster. +----------------+--------------------+--------------------------------+--------------+------------------------------+-------------------------+--------------+--------------------------+ | \*\*Partition\*\* | \*\*QOS\*\* | \*\*#Cores per job\*\* | \*\*Walltime\*\* | \*\*Max jobs/res. per user\*\* | \*\*Max memory per node\*\* | \*\*Priority\*\* | \*\*Notes\*\* | +================+====================+================================+==============+==============================+=========================+==============+==========================+ | gw | noQOS | max=768 cores | 48:00:00 | 2000 submitted jobs | 735 GB / 1511 GB | 40 | Four fat memory nodes | + +--------------------+--------------------------------+--------------+------------------------------+-------------------------+--------------+--------------------------+ | | qos\_dbg | max=128 cores | 00:30:00 | Max 128 cores | 735 GB / 1511 GB | 80 | Can run on max 128 cores | + +--------------------+--------------------------------+--------------+------------------------------+-------------------------+--------------+--------------------------+ | | qos\_gwlong | max=256 cores | 144:00:00 | Max 128 cores, 2 running jobs| 735 GB / 1511 GB | 40 | Four fat memory nodes | +----------------+--------------------+--------------------------------+--------------+------------------------------+-------------------------+--------------+--------------------------+ | gwgpu | noQOS | max=16 cores/1 gpu / 188250 MB | 08:00:00 | 1 running job | 735 GB | 40 | | +----------------+--------------------+--------------------------------+--------------+------------------------------+-------------------------+--------------+--------------------------+ .. note:: In the new Gateway the debug partition has been replaces by a QoS. How to request support ---------------------- For general support: Please write a mail to superc@cineca.it specifying EFGW in the Subject. For problems related to IMAS-ITER software and installations: please refer to the ACH-04 (PSNC) support: https://confluence.eufus.psnc.pl/spaces/PSNCACH04/overview --- # Unknown ##------ Network Topology ------## # Nodes IswL1s Cells # lrdn0001 isw10100 cell01 lrdn0002 isw10100 cell01 lrdn0003 isw10100 cell01 lrdn0004 isw10100 cell01 lrdn0005 isw10100 cell01 lrdn0006 isw10100 cell01 lrdn0007 isw10100 cell01 lrdn0008 isw10100 cell01 lrdn0009 isw10104 cell01 lrdn0010 isw10104 cell01 lrdn0011 isw10104 cell01 lrdn0012 isw10104 cell01 lrdn0013 isw10100 cell01 lrdn0014 isw10100 cell01 lrdn0015 isw10102 cell01 lrdn0016 isw10102 cell01 lrdn0017 isw10104 cell01 lrdn0018 isw10104 cell01 lrdn0019 isw10104 cell01 lrdn0020 isw10104 cell01 lrdn0021 isw10102 cell01 lrdn0022 isw10102 cell01 lrdn0023 isw10102 cell01 lrdn0024 isw10102 cell01 lrdn0025 isw10102 cell01 lrdn0026 isw10102 cell01 lrdn0027 isw10102 cell01 lrdn0028 isw10102 cell01 lrdn0029 isw10104 cell01 lrdn0030 isw10104 cell01 lrdn0031 isw10200 cell01 lrdn0032 isw10200 cell01 lrdn0033 isw10200 cell01 lrdn0034 isw10200 cell01 lrdn0035 isw10200 cell01 lrdn0036 isw10200 cell01 lrdn0037 isw10200 cell01 lrdn0038 isw10200 cell01 lrdn0039 isw10204 cell01 lrdn0040 isw10204 cell01 lrdn0041 isw10204 cell01 lrdn0042 isw10204 cell01 lrdn0043 isw10200 cell01 lrdn0044 isw10200 cell01 lrdn0045 isw10202 cell01 lrdn0046 isw10202 cell01 lrdn0047 isw10204 cell01 lrdn0048 isw10204 cell01 lrdn0049 isw10204 cell01 lrdn0050 isw10204 cell01 lrdn0051 isw10202 cell01 lrdn0052 isw10202 cell01 lrdn0053 isw10202 cell01 lrdn0054 isw10202 cell01 lrdn0055 isw10202 cell01 lrdn0056 isw10202 cell01 lrdn0057 isw10202 cell01 lrdn0058 isw10202 cell01 lrdn0059 isw10204 cell01 lrdn0060 isw10204 cell01 lrdn0061 isw10300 cell01 lrdn0062 isw10300 cell01 lrdn0063 isw10300 cell01 lrdn0064 isw10300 cell01 lrdn0065 isw10300 cell01 lrdn0066 isw10300 cell01 lrdn0067 isw10300 cell01 lrdn0068 isw10300 cell01 lrdn0069 isw10304 cell01 lrdn0070 isw10304 cell01 lrdn0071 isw10304 cell01 lrdn0072 isw10304 cell01 lrdn0073 isw10300 cell01 lrdn0074 isw10300 cell01 lrdn0075 isw10302 cell01 lrdn0076 isw10302 cell01 lrdn0077 isw10304 cell01 lrdn0078 isw10304 cell01 lrdn0079 isw10304 cell01 lrdn0080 isw10304 cell01 lrdn0081 isw10302 cell01 lrdn0082 isw10302 cell01 lrdn0083 isw10302 cell01 lrdn0084 isw10302 cell01 lrdn0085 isw10302 cell01 lrdn0086 isw10302 cell01 lrdn0087 isw10302 cell01 lrdn0088 isw10302 cell01 lrdn0089 isw10304 cell01 lrdn0090 isw10304 cell01 lrdn0091 isw10400 cell01 lrdn0092 isw10400 cell01 lrdn0093 isw10400 cell01 lrdn0094 isw10400 cell01 lrdn0095 isw10400 cell01 lrdn0096 isw10400 cell01 lrdn0097 isw10400 cell01 lrdn0098 isw10400 cell01 lrdn0099 isw10404 cell01 lrdn0100 isw10404 cell01 lrdn0101 isw10404 cell01 lrdn0102 isw10404 cell01 lrdn0103 isw10400 cell01 lrdn0104 isw10400 cell01 lrdn0105 isw10402 cell01 lrdn0106 isw10402 cell01 lrdn0107 isw10404 cell01 lrdn0108 isw10404 cell01 lrdn0109 isw10404 cell01 lrdn0110 isw10404 cell01 lrdn0111 isw10402 cell01 lrdn0112 isw10402 cell01 lrdn0113 isw10402 cell01 lrdn0114 isw10402 cell01 lrdn0115 isw10402 cell01 lrdn0116 isw10402 cell01 lrdn0117 isw10402 cell01 lrdn0118 isw10402 cell01 lrdn0119 isw10404 cell01 lrdn0120 isw10404 cell01 lrdn0121 isw10500 cell01 lrdn0122 isw10500 cell01 lrdn0123 isw10500 cell01 lrdn0124 isw10500 cell01 lrdn0125 isw10500 cell01 lrdn0126 isw10500 cell01 lrdn0127 isw10500 cell01 lrdn0128 isw10500 cell01 lrdn0129 isw10504 cell01 lrdn0130 isw10504 cell01 lrdn0131 isw10504 cell01 lrdn0132 isw10504 cell01 lrdn0133 isw10500 cell01 lrdn0134 isw10500 cell01 lrdn0135 isw10502 cell01 lrdn0136 isw10502 cell01 lrdn0137 isw10504 cell01 lrdn0138 isw10504 cell01 lrdn0139 isw10504 cell01 lrdn0140 isw10504 cell01 lrdn0141 isw10502 cell01 lrdn0142 isw10502 cell01 lrdn0143 isw10502 cell01 lrdn0144 isw10502 cell01 lrdn0145 isw10502 cell01 lrdn0146 isw10502 cell01 lrdn0147 isw10502 cell01 lrdn0148 isw10502 cell01 lrdn0149 isw10504 cell01 lrdn0150 isw10504 cell01 lrdn0151 isw10600 cell01 lrdn0152 isw10600 cell01 lrdn0153 isw10600 cell01 lrdn0154 isw10600 cell01 lrdn0155 isw10600 cell01 lrdn0156 isw10600 cell01 lrdn0157 isw10600 cell01 lrdn0158 isw10600 cell01 lrdn0159 isw10604 cell01 lrdn0160 isw10604 cell01 lrdn0161 isw10604 cell01 lrdn0162 isw10604 cell01 lrdn0163 isw10600 cell01 lrdn0164 isw10600 cell01 lrdn0165 isw10602 cell01 lrdn0166 isw10602 cell01 lrdn0167 isw10604 cell01 lrdn0168 isw10604 cell01 lrdn0169 isw10604 cell01 lrdn0170 isw10604 cell01 lrdn0171 isw10602 cell01 lrdn0172 isw10602 cell01 lrdn0173 isw10602 cell01 lrdn0174 isw10602 cell01 lrdn0175 isw10602 cell01 lrdn0176 isw10602 cell01 lrdn0177 isw10602 cell01 lrdn0178 isw10602 cell01 lrdn0179 isw10604 cell01 lrdn0180 isw10604 cell01 lrdn0181 isw10700 cell02 lrdn0182 isw10700 cell02 lrdn0183 isw10700 cell02 lrdn0184 isw10700 cell02 lrdn0185 isw10700 cell02 lrdn0186 isw10700 cell02 lrdn0187 isw10700 cell02 lrdn0188 isw10700 cell02 lrdn0189 isw10704 cell02 lrdn0190 isw10704 cell02 lrdn0191 isw10704 cell02 lrdn0192 isw10704 cell02 lrdn0193 isw10700 cell02 lrdn0194 isw10700 cell02 lrdn0195 isw10702 cell02 lrdn0196 isw10702 cell02 lrdn0197 isw10704 cell02 lrdn0198 isw10704 cell02 lrdn0199 isw10704 cell02 lrdn0200 isw10704 cell02 lrdn0201 isw10702 cell02 lrdn0202 isw10702 cell02 lrdn0203 isw10702 cell02 lrdn0204 isw10702 cell02 lrdn0205 isw10702 cell02 lrdn0206 isw10702 cell02 lrdn0207 isw10702 cell02 lrdn0208 isw10702 cell02 lrdn0209 isw10704 cell02 lrdn0210 isw10704 cell02 lrdn0211 isw10800 cell02 lrdn0212 isw10800 cell02 lrdn0213 isw10800 cell02 lrdn0214 isw10800 cell02 lrdn0215 isw10800 cell02 lrdn0216 isw10800 cell02 lrdn0217 isw10800 cell02 lrdn0218 isw10800 cell02 lrdn0219 isw10804 cell02 lrdn0220 isw10804 cell02 lrdn0221 isw10804 cell02 lrdn0222 isw10804 cell02 lrdn0223 isw10800 cell02 lrdn0224 isw10800 cell02 lrdn0225 isw10802 cell02 lrdn0226 isw10802 cell02 lrdn0227 isw10804 cell02 lrdn0228 isw10804 cell02 lrdn0229 isw10804 cell02 lrdn0230 isw10804 cell02 lrdn0231 isw10802 cell02 lrdn0232 isw10802 cell02 lrdn0233 isw10802 cell02 lrdn0234 isw10802 cell02 lrdn0235 isw10802 cell02 lrdn0236 isw10802 cell02 lrdn0237 isw10802 cell02 lrdn0238 isw10802 cell02 lrdn0239 isw10804 cell02 lrdn0240 isw10804 cell02 lrdn0241 isw10900 cell02 lrdn0242 isw10900 cell02 lrdn0243 isw10900 cell02 lrdn0244 isw10900 cell02 lrdn0245 isw10900 cell02 lrdn0246 isw10900 cell02 lrdn0247 isw10900 cell02 lrdn0248 isw10900 cell02 lrdn0249 isw10904 cell02 lrdn0250 isw10904 cell02 lrdn0251 isw10904 cell02 lrdn0252 isw10904 cell02 lrdn0253 isw10900 cell02 lrdn0254 isw10900 cell02 lrdn0255 isw10902 cell02 lrdn0256 isw10902 cell02 lrdn0257 isw10904 cell02 lrdn0258 isw10904 cell02 lrdn0259 isw10904 cell02 lrdn0260 isw10904 cell02 lrdn0261 isw10902 cell02 lrdn0262 isw10902 cell02 lrdn0263 isw10902 cell02 lrdn0264 isw10902 cell02 lrdn0265 isw10902 cell02 lrdn0266 isw10902 cell02 lrdn0267 isw10902 cell02 lrdn0268 isw10902 cell02 lrdn0269 isw10904 cell02 lrdn0270 isw10904 cell02 lrdn0271 isw11000 cell02 lrdn0272 isw11000 cell02 lrdn0273 isw11000 cell02 lrdn0274 isw11000 cell02 lrdn0275 isw11000 cell02 lrdn0276 isw11000 cell02 lrdn0277 isw11000 cell02 lrdn0278 isw11000 cell02 lrdn0279 isw11004 cell02 lrdn0280 isw11004 cell02 lrdn0281 isw11004 cell02 lrdn0282 isw11004 cell02 lrdn0283 isw11000 cell02 lrdn0284 isw11000 cell02 lrdn0285 isw11002 cell02 lrdn0286 isw11002 cell02 lrdn0287 isw11004 cell02 lrdn0288 isw11004 cell02 lrdn0289 isw11004 cell02 lrdn0290 isw11004 cell02 lrdn0291 isw11002 cell02 lrdn0292 isw11002 cell02 lrdn0293 isw11002 cell02 lrdn0294 isw11002 cell02 lrdn0295 isw11002 cell02 lrdn0296 isw11002 cell02 lrdn0297 isw11002 cell02 lrdn0298 isw11002 cell02 lrdn0299 isw11004 cell02 lrdn0300 isw11004 cell02 lrdn0301 isw11100 cell02 lrdn0302 isw11100 cell02 lrdn0303 isw11100 cell02 lrdn0304 isw11100 cell02 lrdn0305 isw11100 cell02 lrdn0306 isw11100 cell02 lrdn0307 isw11100 cell02 lrdn0308 isw11100 cell02 lrdn0309 isw11104 cell02 lrdn0310 isw11104 cell02 lrdn0311 isw11104 cell02 lrdn0312 isw11104 cell02 lrdn0313 isw11100 cell02 lrdn0314 isw11100 cell02 lrdn0315 isw11102 cell02 lrdn0316 isw11102 cell02 lrdn0317 isw11104 cell02 lrdn0318 isw11104 cell02 lrdn0319 isw11104 cell02 lrdn0320 isw11104 cell02 lrdn0321 isw11102 cell02 lrdn0322 isw11102 cell02 lrdn0323 isw11102 cell02 lrdn0324 isw11102 cell02 lrdn0325 isw11102 cell02 lrdn0326 isw11102 cell02 lrdn0327 isw11102 cell02 lrdn0328 isw11102 cell02 lrdn0329 isw11104 cell02 lrdn0330 isw11104 cell02 lrdn0331 isw11200 cell02 lrdn0332 isw11200 cell02 lrdn0333 isw11200 cell02 lrdn0334 isw11200 cell02 lrdn0335 isw11200 cell02 lrdn0336 isw11200 cell02 lrdn0337 isw11200 cell02 lrdn0338 isw11200 cell02 lrdn0339 isw11204 cell02 lrdn0340 isw11204 cell02 lrdn0341 isw11204 cell02 lrdn0342 isw11204 cell02 lrdn0343 isw11200 cell02 lrdn0344 isw11200 cell02 lrdn0345 isw11202 cell02 lrdn0346 isw11202 cell02 lrdn0347 isw11204 cell02 lrdn0348 isw11204 cell02 lrdn0349 isw11204 cell02 lrdn0350 isw11204 cell02 lrdn0351 isw11202 cell02 lrdn0352 isw11202 cell02 lrdn0353 isw11202 cell02 lrdn0354 isw11202 cell02 lrdn0355 isw11202 cell02 lrdn0356 isw11202 cell02 lrdn0357 isw11202 cell02 lrdn0358 isw11202 cell02 lrdn0359 isw11204 cell02 lrdn0360 isw11204 cell02 lrdn0361 isw20100 cell03 lrdn0362 isw20100 cell03 lrdn0363 isw20100 cell03 lrdn0364 isw20100 cell03 lrdn0365 isw20100 cell03 lrdn0366 isw20100 cell03 lrdn0367 isw20100 cell03 lrdn0368 isw20100 cell03 lrdn0369 isw20104 cell03 lrdn0370 isw20104 cell03 lrdn0371 isw20104 cell03 lrdn0372 isw20104 cell03 lrdn0373 isw20100 cell03 lrdn0374 isw20100 cell03 lrdn0375 isw20102 cell03 lrdn0376 isw20102 cell03 lrdn0377 isw20104 cell03 lrdn0378 isw20104 cell03 lrdn0379 isw20104 cell03 lrdn0380 isw20104 cell03 lrdn0381 isw20102 cell03 lrdn0382 isw20102 cell03 lrdn0383 isw20102 cell03 lrdn0384 isw20102 cell03 lrdn0385 isw20102 cell03 lrdn0386 isw20102 cell03 lrdn0387 isw20102 cell03 lrdn0388 isw20102 cell03 lrdn0389 isw20104 cell03 lrdn0390 isw20104 cell03 lrdn0391 isw20200 cell03 lrdn0392 isw20200 cell03 lrdn0393 isw20200 cell03 lrdn0394 isw20200 cell03 lrdn0395 isw20200 cell03 lrdn0396 isw20200 cell03 lrdn0397 isw20200 cell03 lrdn0398 isw20200 cell03 lrdn0399 isw20204 cell03 lrdn0400 isw20204 cell03 lrdn0401 isw20204 cell03 lrdn0402 isw20204 cell03 lrdn0403 isw20200 cell03 lrdn0404 isw20200 cell03 lrdn0405 isw20202 cell03 lrdn0406 isw20202 cell03 lrdn0407 isw20204 cell03 lrdn0408 isw20204 cell03 lrdn0409 isw20204 cell03 lrdn0410 isw20204 cell03 lrdn0411 isw20202 cell03 lrdn0412 isw20202 cell03 lrdn0413 isw20202 cell03 lrdn0414 isw20202 cell03 lrdn0415 isw20202 cell03 lrdn0416 isw20202 cell03 lrdn0417 isw20202 cell03 lrdn0418 isw20202 cell03 lrdn0419 isw20204 cell03 lrdn0420 isw20204 cell03 lrdn0421 isw20300 cell03 lrdn0422 isw20300 cell03 lrdn0423 isw20300 cell03 lrdn0424 isw20300 cell03 lrdn0425 isw20300 cell03 lrdn0426 isw20300 cell03 lrdn0427 isw20300 cell03 lrdn0428 isw20300 cell03 lrdn0429 isw20304 cell03 lrdn0430 isw20304 cell03 lrdn0431 isw20304 cell03 lrdn0432 isw20304 cell03 lrdn0433 isw20300 cell03 lrdn0434 isw20300 cell03 lrdn0435 isw20302 cell03 lrdn0436 isw20302 cell03 lrdn0437 isw20304 cell03 lrdn0438 isw20304 cell03 lrdn0439 isw20304 cell03 lrdn0440 isw20304 cell03 lrdn0441 isw20302 cell03 lrdn0442 isw20302 cell03 lrdn0443 isw20302 cell03 lrdn0444 isw20302 cell03 lrdn0445 isw20302 cell03 lrdn0446 isw20302 cell03 lrdn0447 isw20302 cell03 lrdn0448 isw20302 cell03 lrdn0449 isw20304 cell03 lrdn0450 isw20304 cell03 lrdn0451 isw20400 cell03 lrdn0452 isw20400 cell03 lrdn0453 isw20400 cell03 lrdn0454 isw20400 cell03 lrdn0455 isw20400 cell03 lrdn0456 isw20400 cell03 lrdn0457 isw20400 cell03 lrdn0458 isw20400 cell03 lrdn0459 isw20404 cell03 lrdn0460 isw20404 cell03 lrdn0461 isw20404 cell03 lrdn0462 isw20404 cell03 lrdn0463 isw20400 cell03 lrdn0464 isw20400 cell03 lrdn0465 isw20402 cell03 lrdn0466 isw20402 cell03 lrdn0467 isw20404 cell03 lrdn0468 isw20404 cell03 lrdn0469 isw20404 cell03 lrdn0470 isw20404 cell03 lrdn0471 isw20402 cell03 lrdn0472 isw20402 cell03 lrdn0473 isw20402 cell03 lrdn0474 isw20402 cell03 lrdn0475 isw20402 cell03 lrdn0476 isw20402 cell03 lrdn0477 isw20402 cell03 lrdn0478 isw20402 cell03 lrdn0479 isw20404 cell03 lrdn0480 isw20404 cell03 lrdn0481 isw20500 cell03 lrdn0482 isw20500 cell03 lrdn0483 isw20500 cell03 lrdn0484 isw20500 cell03 lrdn0485 isw20500 cell03 lrdn0486 isw20500 cell03 lrdn0487 isw20500 cell03 lrdn0488 isw20500 cell03 lrdn0489 isw20504 cell03 lrdn0490 isw20504 cell03 lrdn0491 isw20504 cell03 lrdn0492 isw20504 cell03 lrdn0493 isw20500 cell03 lrdn0494 isw20500 cell03 lrdn0495 isw20502 cell03 lrdn0496 isw20502 cell03 lrdn0497 isw20504 cell03 lrdn0498 isw20504 cell03 lrdn0499 isw20504 cell03 lrdn0500 isw20504 cell03 lrdn0501 isw20502 cell03 lrdn0502 isw20502 cell03 lrdn0503 isw20502 cell03 lrdn0504 isw20502 cell03 lrdn0505 isw20502 cell03 lrdn0506 isw20502 cell03 lrdn0507 isw20502 cell03 lrdn0508 isw20502 cell03 lrdn0509 isw20504 cell03 lrdn0510 isw20504 cell03 lrdn0511 isw20600 cell03 lrdn0512 isw20600 cell03 lrdn0513 isw20600 cell03 lrdn0514 isw20600 cell03 lrdn0515 isw20600 cell03 lrdn0516 isw20600 cell03 lrdn0517 isw20600 cell03 lrdn0518 isw20600 cell03 lrdn0519 isw20604 cell03 lrdn0520 isw20604 cell03 lrdn0521 isw20604 cell03 lrdn0522 isw20604 cell03 lrdn0523 isw20600 cell03 lrdn0524 isw20600 cell03 lrdn0525 isw20602 cell03 lrdn0526 isw20602 cell03 lrdn0527 isw20604 cell03 lrdn0528 isw20604 cell03 lrdn0529 isw20604 cell03 lrdn0530 isw20604 cell03 lrdn0531 isw20602 cell03 lrdn0532 isw20602 cell03 lrdn0533 isw20602 cell03 lrdn0534 isw20602 cell03 lrdn0535 isw20602 cell03 lrdn0536 isw20602 cell03 lrdn0537 isw20602 cell03 lrdn0538 isw20602 cell03 lrdn0539 isw20604 cell03 lrdn0540 isw20604 cell03 lrdn0541 isw20700 cell04 lrdn0542 isw20700 cell04 lrdn0543 isw20700 cell04 lrdn0544 isw20700 cell04 lrdn0545 isw20700 cell04 lrdn0546 isw20700 cell04 lrdn0547 isw20700 cell04 lrdn0548 isw20700 cell04 lrdn0549 isw20704 cell04 lrdn0550 isw20704 cell04 lrdn0551 isw20704 cell04 lrdn0552 isw20704 cell04 lrdn0553 isw20700 cell04 lrdn0554 isw20700 cell04 lrdn0555 isw20702 cell04 lrdn0556 isw20702 cell04 lrdn0557 isw20704 cell04 lrdn0558 isw20704 cell04 lrdn0559 isw20704 cell04 lrdn0560 isw20704 cell04 lrdn0561 isw20702 cell04 lrdn0562 isw20702 cell04 lrdn0563 isw20702 cell04 lrdn0564 isw20702 cell04 lrdn0565 isw20702 cell04 lrdn0566 isw20702 cell04 lrdn0567 isw20702 cell04 lrdn0568 isw20702 cell04 lrdn0569 isw20704 cell04 lrdn0570 isw20704 cell04 lrdn0571 isw20800 cell04 lrdn0572 isw20800 cell04 lrdn0573 isw20800 cell04 lrdn0574 isw20800 cell04 lrdn0575 isw20800 cell04 lrdn0576 isw20800 cell04 lrdn0577 isw20800 cell04 lrdn0578 isw20800 cell04 lrdn0579 isw20804 cell04 lrdn0580 isw20804 cell04 lrdn0581 isw20804 cell04 lrdn0582 isw20804 cell04 lrdn0583 isw20800 cell04 lrdn0584 isw20800 cell04 lrdn0585 isw20802 cell04 lrdn0586 isw20802 cell04 lrdn0587 isw20804 cell04 lrdn0588 isw20804 cell04 lrdn0589 isw20804 cell04 lrdn0590 isw20804 cell04 lrdn0591 isw20802 cell04 lrdn0592 isw20802 cell04 lrdn0593 isw20802 cell04 lrdn0594 isw20802 cell04 lrdn0595 isw20802 cell04 lrdn0596 isw20802 cell04 lrdn0597 isw20802 cell04 lrdn0598 isw20802 cell04 lrdn0599 isw20804 cell04 lrdn0600 isw20804 cell04 lrdn0601 isw20900 cell04 lrdn0602 isw20900 cell04 lrdn0603 isw20900 cell04 lrdn0604 isw20900 cell04 lrdn0605 isw20900 cell04 lrdn0606 isw20900 cell04 lrdn0607 isw20900 cell04 lrdn0608 isw20900 cell04 lrdn0609 isw20904 cell04 lrdn0610 isw20904 cell04 lrdn0611 isw20904 cell04 lrdn0612 isw20904 cell04 lrdn0613 isw20900 cell04 lrdn0614 isw20900 cell04 lrdn0615 isw20902 cell04 lrdn0616 isw20902 cell04 lrdn0617 isw20904 cell04 lrdn0618 isw20904 cell04 lrdn0619 isw20904 cell04 lrdn0620 isw20904 cell04 lrdn0621 isw20902 cell04 lrdn0622 isw20902 cell04 lrdn0623 isw20902 cell04 lrdn0624 isw20902 cell04 lrdn0625 isw20902 cell04 lrdn0626 isw20902 cell04 lrdn0627 isw20902 cell04 lrdn0628 isw20902 cell04 lrdn0629 isw20904 cell04 lrdn0630 isw20904 cell04 lrdn0631 isw21000 cell04 lrdn0632 isw21000 cell04 lrdn0633 isw21000 cell04 lrdn0634 isw21000 cell04 lrdn0635 isw21000 cell04 lrdn0636 isw21000 cell04 lrdn0637 isw21000 cell04 lrdn0638 isw21000 cell04 lrdn0639 isw21004 cell04 lrdn0640 isw21004 cell04 lrdn0641 isw21004 cell04 lrdn0642 isw21004 cell04 lrdn0643 isw21000 cell04 lrdn0644 isw21000 cell04 lrdn0645 isw21002 cell04 lrdn0646 isw21002 cell04 lrdn0647 isw21004 cell04 lrdn0648 isw21004 cell04 lrdn0649 isw21004 cell04 lrdn0650 isw21004 cell04 lrdn0651 isw21002 cell04 lrdn0652 isw21002 cell04 lrdn0653 isw21002 cell04 lrdn0654 isw21002 cell04 lrdn0655 isw21002 cell04 lrdn0656 isw21002 cell04 lrdn0657 isw21002 cell04 lrdn0658 isw21002 cell04 lrdn0659 isw21004 cell04 lrdn0660 isw21004 cell04 lrdn0661 isw21100 cell04 lrdn0662 isw21100 cell04 lrdn0663 isw21100 cell04 lrdn0664 isw21100 cell04 lrdn0665 isw21100 cell04 lrdn0666 isw21100 cell04 lrdn0667 isw21100 cell04 lrdn0668 isw21100 cell04 lrdn0669 isw21104 cell04 lrdn0670 isw21104 cell04 lrdn0671 isw21104 cell04 lrdn0672 isw21104 cell04 lrdn0673 isw21100 cell04 lrdn0674 isw21100 cell04 lrdn0675 isw21102 cell04 lrdn0676 isw21102 cell04 lrdn0677 isw21104 cell04 lrdn0678 isw21104 cell04 lrdn0679 isw21104 cell04 lrdn0680 isw21104 cell04 lrdn0681 isw21102 cell04 lrdn0682 isw21102 cell04 lrdn0683 isw21102 cell04 lrdn0684 isw21102 cell04 lrdn0685 isw21102 cell04 lrdn0686 isw21102 cell04 lrdn0687 isw21102 cell04 lrdn0688 isw21102 cell04 lrdn0689 isw21104 cell04 lrdn0690 isw21104 cell04 lrdn0691 isw21200 cell04 lrdn0692 isw21200 cell04 lrdn0693 isw21200 cell04 lrdn0694 isw21200 cell04 lrdn0695 isw21200 cell04 lrdn0696 isw21200 cell04 lrdn0697 isw21200 cell04 lrdn0698 isw21200 cell04 lrdn0699 isw21204 cell04 lrdn0700 isw21204 cell04 lrdn0701 isw21204 cell04 lrdn0702 isw21204 cell04 lrdn0703 isw21200 cell04 lrdn0704 isw21200 cell04 lrdn0705 isw21202 cell04 lrdn0706 isw21202 cell04 lrdn0707 isw21204 cell04 lrdn0708 isw21204 cell04 lrdn0709 isw21204 cell04 lrdn0710 isw21204 cell04 lrdn0711 isw21202 cell04 lrdn0712 isw21202 cell04 lrdn0713 isw21202 cell04 lrdn0714 isw21202 cell04 lrdn0715 isw21202 cell04 lrdn0716 isw21202 cell04 lrdn0717 isw21202 cell04 lrdn0718 isw21202 cell04 lrdn0719 isw21204 cell04 lrdn0720 isw21204 cell04 lrdn0721 isw30100 cell05 lrdn0722 isw30100 cell05 lrdn0723 isw30100 cell05 lrdn0724 isw30100 cell05 lrdn0725 isw30100 cell05 lrdn0726 isw30100 cell05 lrdn0727 isw30100 cell05 lrdn0728 isw30100 cell05 lrdn0729 isw30104 cell05 lrdn0730 isw30104 cell05 lrdn0731 isw30104 cell05 lrdn0732 isw30104 cell05 lrdn0733 isw30100 cell05 lrdn0734 isw30100 cell05 lrdn0735 isw30102 cell05 lrdn0736 isw30102 cell05 lrdn0737 isw30104 cell05 lrdn0738 isw30104 cell05 lrdn0739 isw30104 cell05 lrdn0740 isw30104 cell05 lrdn0741 isw30102 cell05 lrdn0742 isw30102 cell05 lrdn0743 isw30102 cell05 lrdn0744 isw30102 cell05 lrdn0745 isw30102 cell05 lrdn0746 isw30102 cell05 lrdn0747 isw30102 cell05 lrdn0748 isw30102 cell05 lrdn0749 isw30104 cell05 lrdn0750 isw30104 cell05 lrdn0751 isw30200 cell05 lrdn0752 isw30200 cell05 lrdn0753 isw30200 cell05 lrdn0754 isw30200 cell05 lrdn0755 isw30200 cell05 lrdn0756 isw30200 cell05 lrdn0757 isw30200 cell05 lrdn0758 isw30200 cell05 lrdn0759 isw30204 cell05 lrdn0760 isw30204 cell05 lrdn0761 isw30204 cell05 lrdn0762 isw30204 cell05 lrdn0763 isw30200 cell05 lrdn0764 isw30200 cell05 lrdn0765 isw30202 cell05 lrdn0766 isw30202 cell05 lrdn0767 isw30204 cell05 lrdn0768 isw30204 cell05 lrdn0769 isw30204 cell05 lrdn0770 isw30204 cell05 lrdn0771 isw30202 cell05 lrdn0772 isw30202 cell05 lrdn0773 isw30202 cell05 lrdn0774 isw30202 cell05 lrdn0775 isw30202 cell05 lrdn0776 isw30202 cell05 lrdn0777 isw30202 cell05 lrdn0778 isw30202 cell05 lrdn0779 isw30204 cell05 lrdn0780 isw30204 cell05 lrdn0781 isw30300 cell05 lrdn0782 isw30300 cell05 lrdn0783 isw30300 cell05 lrdn0784 isw30300 cell05 lrdn0785 isw30300 cell05 lrdn0786 isw30300 cell05 lrdn0787 isw30300 cell05 lrdn0788 isw30300 cell05 lrdn0789 isw30304 cell05 lrdn0790 isw30304 cell05 lrdn0791 isw30304 cell05 lrdn0792 isw30304 cell05 lrdn0793 isw30300 cell05 lrdn0794 isw30300 cell05 lrdn0795 isw30302 cell05 lrdn0796 isw30302 cell05 lrdn0797 isw30304 cell05 lrdn0798 isw30304 cell05 lrdn0799 isw30304 cell05 lrdn0800 isw30304 cell05 lrdn0801 isw30302 cell05 lrdn0802 isw30302 cell05 lrdn0803 isw30302 cell05 lrdn0804 isw30302 cell05 lrdn0805 isw30302 cell05 lrdn0806 isw30302 cell05 lrdn0807 isw30302 cell05 lrdn0808 isw30302 cell05 lrdn0809 isw30304 cell05 lrdn0810 isw30304 cell05 lrdn0811 isw30400 cell05 lrdn0812 isw30400 cell05 lrdn0813 isw30400 cell05 lrdn0814 isw30400 cell05 lrdn0815 isw30400 cell05 lrdn0816 isw30400 cell05 lrdn0817 isw30400 cell05 lrdn0818 isw30400 cell05 lrdn0819 isw30404 cell05 lrdn0820 isw30404 cell05 lrdn0821 isw30404 cell05 lrdn0822 isw30404 cell05 lrdn0823 isw30400 cell05 lrdn0824 isw30400 cell05 lrdn0825 isw30402 cell05 lrdn0826 isw30402 cell05 lrdn0827 isw30404 cell05 lrdn0828 isw30404 cell05 lrdn0829 isw30404 cell05 lrdn0830 isw30404 cell05 lrdn0831 isw30402 cell05 lrdn0832 isw30402 cell05 lrdn0833 isw30402 cell05 lrdn0834 isw30402 cell05 lrdn0835 isw30402 cell05 lrdn0836 isw30402 cell05 lrdn0837 isw30402 cell05 lrdn0838 isw30402 cell05 lrdn0839 isw30404 cell05 lrdn0840 isw30404 cell05 lrdn0841 isw30500 cell05 lrdn0842 isw30500 cell05 lrdn0843 isw30500 cell05 lrdn0844 isw30500 cell05 lrdn0845 isw30500 cell05 lrdn0846 isw30500 cell05 lrdn0847 isw30500 cell05 lrdn0848 isw30500 cell05 lrdn0849 isw30504 cell05 lrdn0850 isw30504 cell05 lrdn0851 isw30504 cell05 lrdn0852 isw30504 cell05 lrdn0853 isw30500 cell05 lrdn0854 isw30500 cell05 lrdn0855 isw30502 cell05 lrdn0856 isw30502 cell05 lrdn0857 isw30504 cell05 lrdn0858 isw30504 cell05 lrdn0859 isw30504 cell05 lrdn0860 isw30504 cell05 lrdn0861 isw30502 cell05 lrdn0862 isw30502 cell05 lrdn0863 isw30502 cell05 lrdn0864 isw30502 cell05 lrdn0865 isw30502 cell05 lrdn0866 isw30502 cell05 lrdn0867 isw30502 cell05 lrdn0868 isw30502 cell05 lrdn0869 isw30504 cell05 lrdn0870 isw30504 cell05 lrdn0871 isw30600 cell05 lrdn0872 isw30600 cell05 lrdn0873 isw30600 cell05 lrdn0874 isw30600 cell05 lrdn0875 isw30600 cell05 lrdn0876 isw30600 cell05 lrdn0877 isw30600 cell05 lrdn0878 isw30600 cell05 lrdn0879 isw30604 cell05 lrdn0880 isw30604 cell05 lrdn0881 isw30604 cell05 lrdn0882 isw30604 cell05 lrdn0883 isw30600 cell05 lrdn0884 isw30600 cell05 lrdn0885 isw30602 cell05 lrdn0886 isw30602 cell05 lrdn0887 isw30604 cell05 lrdn0888 isw30604 cell05 lrdn0889 isw30604 cell05 lrdn0890 isw30604 cell05 lrdn0891 isw30602 cell05 lrdn0892 isw30602 cell05 lrdn0893 isw30602 cell05 lrdn0894 isw30602 cell05 lrdn0895 isw30602 cell05 lrdn0896 isw30602 cell05 lrdn0897 isw30602 cell05 lrdn0898 isw30602 cell05 lrdn0899 isw30604 cell05 lrdn0900 isw30604 cell05 lrdn0901 isw30700 cell06 lrdn0902 isw30700 cell06 lrdn0903 isw30700 cell06 lrdn0904 isw30700 cell06 lrdn0905 isw30700 cell06 lrdn0906 isw30700 cell06 lrdn0907 isw30700 cell06 lrdn0908 isw30700 cell06 lrdn0909 isw30704 cell06 lrdn0910 isw30704 cell06 lrdn0911 isw30704 cell06 lrdn0912 isw30704 cell06 lrdn0913 isw30700 cell06 lrdn0914 isw30700 cell06 lrdn0915 isw30702 cell06 lrdn0916 isw30702 cell06 lrdn0917 isw30704 cell06 lrdn0918 isw30704 cell06 lrdn0919 isw30704 cell06 lrdn0920 isw30704 cell06 lrdn0921 isw30702 cell06 lrdn0922 isw30702 cell06 lrdn0923 isw30702 cell06 lrdn0924 isw30702 cell06 lrdn0925 isw30702 cell06 lrdn0926 isw30702 cell06 lrdn0927 isw30702 cell06 lrdn0928 isw30702 cell06 lrdn0929 isw30704 cell06 lrdn0930 isw30704 cell06 lrdn0931 isw30800 cell06 lrdn0932 isw30800 cell06 lrdn0933 isw30800 cell06 lrdn0934 isw30800 cell06 lrdn0935 isw30800 cell06 lrdn0936 isw30800 cell06 lrdn0937 isw30800 cell06 lrdn0938 isw30800 cell06 lrdn0939 isw30804 cell06 lrdn0940 isw30804 cell06 lrdn0941 isw30804 cell06 lrdn0942 isw30804 cell06 lrdn0943 isw30800 cell06 lrdn0944 isw30800 cell06 lrdn0945 isw30802 cell06 lrdn0946 isw30802 cell06 lrdn0947 isw30804 cell06 lrdn0948 isw30804 cell06 lrdn0949 isw30804 cell06 lrdn0950 isw30804 cell06 lrdn0951 isw30802 cell06 lrdn0952 isw30802 cell06 lrdn0953 isw30802 cell06 lrdn0954 isw30802 cell06 lrdn0955 isw30802 cell06 lrdn0956 isw30802 cell06 lrdn0957 isw30802 cell06 lrdn0958 isw30802 cell06 lrdn0959 isw30804 cell06 lrdn0960 isw30804 cell06 lrdn0961 isw30900 cell06 lrdn0962 isw30900 cell06 lrdn0963 isw30900 cell06 lrdn0964 isw30900 cell06 lrdn0965 isw30900 cell06 lrdn0966 isw30900 cell06 lrdn0967 isw30900 cell06 lrdn0968 isw30900 cell06 lrdn0969 isw30904 cell06 lrdn0970 isw30904 cell06 lrdn0971 isw30904 cell06 lrdn0972 isw30904 cell06 lrdn0973 isw30900 cell06 lrdn0974 isw30900 cell06 lrdn0975 isw30902 cell06 lrdn0976 isw30902 cell06 lrdn0977 isw30904 cell06 lrdn0978 isw30904 cell06 lrdn0979 isw30904 cell06 lrdn0980 isw30904 cell06 lrdn0981 isw30902 cell06 lrdn0982 isw30902 cell06 lrdn0983 isw30902 cell06 lrdn0984 isw30902 cell06 lrdn0985 isw30902 cell06 lrdn0986 isw30902 cell06 lrdn0987 isw30902 cell06 lrdn0988 isw30902 cell06 lrdn0989 isw30904 cell06 lrdn0990 isw30904 cell06 lrdn0991 isw31000 cell06 lrdn0992 isw31000 cell06 lrdn0993 isw31000 cell06 lrdn0994 isw31000 cell06 lrdn0995 isw31000 cell06 lrdn0996 isw31000 cell06 lrdn0997 isw31000 cell06 lrdn0998 isw31000 cell06 lrdn0999 isw31004 cell06 lrdn1000 isw31004 cell06 lrdn1001 isw31004 cell06 lrdn1002 isw31004 cell06 lrdn1003 isw31000 cell06 lrdn1004 isw31000 cell06 lrdn1005 isw31002 cell06 lrdn1006 isw31002 cell06 lrdn1007 isw31004 cell06 lrdn1008 isw31004 cell06 lrdn1009 isw31004 cell06 lrdn1010 isw31004 cell06 lrdn1011 isw31002 cell06 lrdn1012 isw31002 cell06 lrdn1013 isw31002 cell06 lrdn1014 isw31002 cell06 lrdn1015 isw31002 cell06 lrdn1016 isw31002 cell06 lrdn1017 isw31002 cell06 lrdn1018 isw31002 cell06 lrdn1019 isw31004 cell06 lrdn1020 isw31004 cell06 lrdn1021 isw31100 cell06 lrdn1022 isw31100 cell06 lrdn1023 isw31100 cell06 lrdn1024 isw31100 cell06 lrdn1025 isw31100 cell06 lrdn1026 isw31100 cell06 lrdn1027 isw31100 cell06 lrdn1028 isw31100 cell06 lrdn1029 isw31104 cell06 lrdn1030 isw31104 cell06 lrdn1031 isw31104 cell06 lrdn1032 isw31104 cell06 lrdn1033 isw31100 cell06 lrdn1034 isw31100 cell06 lrdn1035 isw31102 cell06 lrdn1036 isw31102 cell06 lrdn1037 isw31104 cell06 lrdn1038 isw31104 cell06 lrdn1039 isw31104 cell06 lrdn1040 isw31104 cell06 lrdn1041 isw31102 cell06 lrdn1042 isw31102 cell06 lrdn1043 isw31102 cell06 lrdn1044 isw31102 cell06 lrdn1045 isw31102 cell06 lrdn1046 isw31102 cell06 lrdn1047 isw31102 cell06 lrdn1048 isw31102 cell06 lrdn1049 isw31104 cell06 lrdn1050 isw31104 cell06 lrdn1051 isw31200 cell06 lrdn1052 isw31200 cell06 lrdn1053 isw31200 cell06 lrdn1054 isw31200 cell06 lrdn1055 isw31200 cell06 lrdn1056 isw31200 cell06 lrdn1057 isw31200 cell06 lrdn1058 isw31200 cell06 lrdn1059 isw31204 cell06 lrdn1060 isw31204 cell06 lrdn1061 isw31204 cell06 lrdn1062 isw31204 cell06 lrdn1063 isw31200 cell06 lrdn1064 isw31200 cell06 lrdn1065 isw31202 cell06 lrdn1066 isw31202 cell06 lrdn1067 isw31204 cell06 lrdn1068 isw31204 cell06 lrdn1069 isw31204 cell06 lrdn1070 isw31204 cell06 lrdn1071 isw31202 cell06 lrdn1072 isw31202 cell06 lrdn1073 isw31202 cell06 lrdn1074 isw31202 cell06 lrdn1075 isw31202 cell06 lrdn1076 isw31202 cell06 lrdn1077 isw31202 cell06 lrdn1078 isw31202 cell06 lrdn1079 isw31204 cell06 lrdn1080 isw31204 cell06 lrdn1081 isw40100 cell07 lrdn1082 isw40100 cell07 lrdn1083 isw40100 cell07 lrdn1084 isw40100 cell07 lrdn1085 isw40100 cell07 lrdn1086 isw40100 cell07 lrdn1087 isw40100 cell07 lrdn1088 isw40100 cell07 lrdn1089 isw40104 cell07 lrdn1090 isw40104 cell07 lrdn1091 isw40104 cell07 lrdn1092 isw40104 cell07 lrdn1093 isw40100 cell07 lrdn1094 isw40100 cell07 lrdn1095 isw40102 cell07 lrdn1096 isw40102 cell07 lrdn1097 isw40104 cell07 lrdn1098 isw40104 cell07 lrdn1099 isw40104 cell07 lrdn1100 isw40104 cell07 lrdn1101 isw40102 cell07 lrdn1102 isw40102 cell07 lrdn1103 isw40102 cell07 lrdn1104 isw40102 cell07 lrdn1105 isw40102 cell07 lrdn1106 isw40102 cell07 lrdn1107 isw40102 cell07 lrdn1108 isw40102 cell07 lrdn1109 isw40104 cell07 lrdn1110 isw40104 cell07 lrdn1111 isw40200 cell07 lrdn1112 isw40200 cell07 lrdn1113 isw40200 cell07 lrdn1114 isw40200 cell07 lrdn1115 isw40200 cell07 lrdn1116 isw40200 cell07 lrdn1117 isw40200 cell07 lrdn1118 isw40200 cell07 lrdn1119 isw40204 cell07 lrdn1120 isw40204 cell07 lrdn1121 isw40204 cell07 lrdn1122 isw40204 cell07 lrdn1123 isw40200 cell07 lrdn1124 isw40200 cell07 lrdn1125 isw40202 cell07 lrdn1126 isw40202 cell07 lrdn1127 isw40204 cell07 lrdn1128 isw40204 cell07 lrdn1129 isw40204 cell07 lrdn1130 isw40204 cell07 lrdn1131 isw40202 cell07 lrdn1132 isw40202 cell07 lrdn1133 isw40202 cell07 lrdn1134 isw40202 cell07 lrdn1135 isw40202 cell07 lrdn1136 isw40202 cell07 lrdn1137 isw40202 cell07 lrdn1138 isw40202 cell07 lrdn1139 isw40204 cell07 lrdn1140 isw40204 cell07 lrdn1141 isw40300 cell07 lrdn1142 isw40300 cell07 lrdn1143 isw40300 cell07 lrdn1144 isw40300 cell07 lrdn1145 isw40300 cell07 lrdn1146 isw40300 cell07 lrdn1147 isw40300 cell07 lrdn1148 isw40300 cell07 lrdn1149 isw40304 cell07 lrdn1150 isw40304 cell07 lrdn1151 isw40304 cell07 lrdn1152 isw40304 cell07 lrdn1153 isw40300 cell07 lrdn1154 isw40300 cell07 lrdn1155 isw40302 cell07 lrdn1156 isw40302 cell07 lrdn1157 isw40304 cell07 lrdn1158 isw40304 cell07 lrdn1159 isw40304 cell07 lrdn1160 isw40304 cell07 lrdn1161 isw40302 cell07 lrdn1162 isw40302 cell07 lrdn1163 isw40302 cell07 lrdn1164 isw40302 cell07 lrdn1165 isw40302 cell07 lrdn1166 isw40302 cell07 lrdn1167 isw40302 cell07 lrdn1168 isw40302 cell07 lrdn1169 isw40304 cell07 lrdn1170 isw40304 cell07 lrdn1171 isw40400 cell07 lrdn1172 isw40400 cell07 lrdn1173 isw40400 cell07 lrdn1174 isw40400 cell07 lrdn1175 isw40400 cell07 lrdn1176 isw40400 cell07 lrdn1177 isw40400 cell07 lrdn1178 isw40400 cell07 lrdn1179 isw40404 cell07 lrdn1180 isw40404 cell07 lrdn1181 isw40404 cell07 lrdn1182 isw40404 cell07 lrdn1183 isw40400 cell07 lrdn1184 isw40400 cell07 lrdn1185 isw40402 cell07 lrdn1186 isw40402 cell07 lrdn1187 isw40404 cell07 lrdn1188 isw40404 cell07 lrdn1189 isw40404 cell07 lrdn1190 isw40404 cell07 lrdn1191 isw40402 cell07 lrdn1192 isw40402 cell07 lrdn1193 isw40402 cell07 lrdn1194 isw40402 cell07 lrdn1195 isw40402 cell07 lrdn1196 isw40402 cell07 lrdn1197 isw40402 cell07 lrdn1198 isw40402 cell07 lrdn1199 isw40404 cell07 lrdn1200 isw40404 cell07 lrdn1201 isw40500 cell07 lrdn1202 isw40500 cell07 lrdn1203 isw40500 cell07 lrdn1204 isw40500 cell07 lrdn1205 isw40500 cell07 lrdn1206 isw40500 cell07 lrdn1207 isw40500 cell07 lrdn1208 isw40500 cell07 lrdn1209 isw40504 cell07 lrdn1210 isw40504 cell07 lrdn1211 isw40504 cell07 lrdn1212 isw40504 cell07 lrdn1213 isw40500 cell07 lrdn1214 isw40500 cell07 lrdn1215 isw40502 cell07 lrdn1216 isw40502 cell07 lrdn1217 isw40504 cell07 lrdn1218 isw40504 cell07 lrdn1219 isw40504 cell07 lrdn1220 isw40504 cell07 lrdn1221 isw40502 cell07 lrdn1222 isw40502 cell07 lrdn1223 isw40502 cell07 lrdn1224 isw40502 cell07 lrdn1225 isw40502 cell07 lrdn1226 isw40502 cell07 lrdn1227 isw40502 cell07 lrdn1228 isw40502 cell07 lrdn1229 isw40504 cell07 lrdn1230 isw40504 cell07 lrdn1231 isw40600 cell07 lrdn1232 isw40600 cell07 lrdn1233 isw40600 cell07 lrdn1234 isw40600 cell07 lrdn1235 isw40600 cell07 lrdn1236 isw40600 cell07 lrdn1237 isw40600 cell07 lrdn1238 isw40600 cell07 lrdn1239 isw40604 cell07 lrdn1240 isw40604 cell07 lrdn1241 isw40604 cell07 lrdn1242 isw40604 cell07 lrdn1243 isw40600 cell07 lrdn1244 isw40600 cell07 lrdn1245 isw40602 cell07 lrdn1246 isw40602 cell07 lrdn1247 isw40604 cell07 lrdn1248 isw40604 cell07 lrdn1249 isw40604 cell07 lrdn1250 isw40604 cell07 lrdn1251 isw40602 cell07 lrdn1252 isw40602 cell07 lrdn1253 isw40602 cell07 lrdn1254 isw40602 cell07 lrdn1255 isw40602 cell07 lrdn1256 isw40602 cell07 lrdn1257 isw40602 cell07 lrdn1258 isw40602 cell07 lrdn1259 isw40604 cell07 lrdn1260 isw40604 cell07 lrdn1261 isw40700 cell08 lrdn1262 isw40700 cell08 lrdn1263 isw40700 cell08 lrdn1264 isw40700 cell08 lrdn1265 isw40700 cell08 lrdn1266 isw40700 cell08 lrdn1267 isw40700 cell08 lrdn1268 isw40700 cell08 lrdn1269 isw40704 cell08 lrdn1270 isw40704 cell08 lrdn1271 isw40704 cell08 lrdn1272 isw40704 cell08 lrdn1273 isw40700 cell08 lrdn1274 isw40700 cell08 lrdn1275 isw40702 cell08 lrdn1276 isw40702 cell08 lrdn1277 isw40704 cell08 lrdn1278 isw40704 cell08 lrdn1279 isw40704 cell08 lrdn1280 isw40704 cell08 lrdn1281 isw40702 cell08 lrdn1282 isw40702 cell08 lrdn1283 isw40702 cell08 lrdn1284 isw40702 cell08 lrdn1285 isw40702 cell08 lrdn1286 isw40702 cell08 lrdn1287 isw40702 cell08 lrdn1288 isw40702 cell08 lrdn1289 isw40704 cell08 lrdn1290 isw40704 cell08 lrdn1291 isw40800 cell08 lrdn1292 isw40800 cell08 lrdn1293 isw40800 cell08 lrdn1294 isw40800 cell08 lrdn1295 isw40800 cell08 lrdn1296 isw40800 cell08 lrdn1297 isw40800 cell08 lrdn1298 isw40800 cell08 lrdn1299 isw40804 cell08 lrdn1300 isw40804 cell08 lrdn1301 isw40804 cell08 lrdn1302 isw40804 cell08 lrdn1303 isw40800 cell08 lrdn1304 isw40800 cell08 lrdn1305 isw40802 cell08 lrdn1306 isw40802 cell08 lrdn1307 isw40804 cell08 lrdn1308 isw40804 cell08 lrdn1309 isw40804 cell08 lrdn1310 isw40804 cell08 lrdn1311 isw40802 cell08 lrdn1312 isw40802 cell08 lrdn1313 isw40802 cell08 lrdn1314 isw40802 cell08 lrdn1315 isw40802 cell08 lrdn1316 isw40802 cell08 lrdn1317 isw40802 cell08 lrdn1318 isw40802 cell08 lrdn1319 isw40804 cell08 lrdn1320 isw40804 cell08 lrdn1321 isw40900 cell08 lrdn1322 isw40900 cell08 lrdn1323 isw40900 cell08 lrdn1324 isw40900 cell08 lrdn1325 isw40900 cell08 lrdn1326 isw40900 cell08 lrdn1327 isw40900 cell08 lrdn1328 isw40900 cell08 lrdn1329 isw40904 cell08 lrdn1330 isw40904 cell08 lrdn1331 isw40904 cell08 lrdn1332 isw40904 cell08 lrdn1333 isw40900 cell08 lrdn1334 isw40900 cell08 lrdn1335 isw40902 cell08 lrdn1336 isw40902 cell08 lrdn1337 isw40904 cell08 lrdn1338 isw40904 cell08 lrdn1339 isw40904 cell08 lrdn1340 isw40904 cell08 lrdn1341 isw40902 cell08 lrdn1342 isw40902 cell08 lrdn1343 isw40902 cell08 lrdn1344 isw40902 cell08 lrdn1345 isw40902 cell08 lrdn1346 isw40902 cell08 lrdn1347 isw40902 cell08 lrdn1348 isw40902 cell08 lrdn1349 isw40904 cell08 lrdn1350 isw40904 cell08 lrdn1351 isw41000 cell08 lrdn1352 isw41000 cell08 lrdn1353 isw41000 cell08 lrdn1354 isw41000 cell08 lrdn1355 isw41000 cell08 lrdn1356 isw41000 cell08 lrdn1357 isw41000 cell08 lrdn1358 isw41000 cell08 lrdn1359 isw41004 cell08 lrdn1360 isw41004 cell08 lrdn1361 isw41004 cell08 lrdn1362 isw41004 cell08 lrdn1363 isw41000 cell08 lrdn1364 isw41000 cell08 lrdn1365 isw41002 cell08 lrdn1366 isw41002 cell08 lrdn1367 isw41004 cell08 lrdn1368 isw41004 cell08 lrdn1369 isw41004 cell08 lrdn1370 isw41004 cell08 lrdn1371 isw41002 cell08 lrdn1372 isw41002 cell08 lrdn1373 isw41002 cell08 lrdn1374 isw41002 cell08 lrdn1375 isw41002 cell08 lrdn1376 isw41002 cell08 lrdn1377 isw41002 cell08 lrdn1378 isw41002 cell08 lrdn1379 isw41004 cell08 lrdn1380 isw41004 cell08 lrdn1381 isw41100 cell08 lrdn1382 isw41100 cell08 lrdn1383 isw41100 cell08 lrdn1384 isw41100 cell08 lrdn1385 isw41100 cell08 lrdn1386 isw41100 cell08 lrdn1387 isw41100 cell08 lrdn1388 isw41100 cell08 lrdn1389 isw41104 cell08 lrdn1390 isw41104 cell08 lrdn1391 isw41104 cell08 lrdn1392 isw41104 cell08 lrdn1393 isw41100 cell08 lrdn1394 isw41100 cell08 lrdn1395 isw41102 cell08 lrdn1396 isw41102 cell08 lrdn1397 isw41104 cell08 lrdn1398 isw41104 cell08 lrdn1399 isw41104 cell08 lrdn1400 isw41104 cell08 lrdn1401 isw41102 cell08 lrdn1402 isw41102 cell08 lrdn1403 isw41102 cell08 lrdn1404 isw41102 cell08 lrdn1405 isw41102 cell08 lrdn1406 isw41102 cell08 lrdn1407 isw41102 cell08 lrdn1408 isw41102 cell08 lrdn1409 isw41104 cell08 lrdn1410 isw41104 cell08 lrdn1411 isw41200 cell08 lrdn1412 isw41200 cell08 lrdn1413 isw41200 cell08 lrdn1414 isw41200 cell08 lrdn1415 isw41200 cell08 lrdn1416 isw41200 cell08 lrdn1417 isw41200 cell08 lrdn1418 isw41200 cell08 lrdn1419 isw41204 cell08 lrdn1420 isw41204 cell08 lrdn1421 isw41204 cell08 lrdn1422 isw41204 cell08 lrdn1423 isw41200 cell08 lrdn1424 isw41200 cell08 lrdn1425 isw41202 cell08 lrdn1426 isw41202 cell08 lrdn1427 isw41204 cell08 lrdn1428 isw41204 cell08 lrdn1429 isw41204 cell08 lrdn1430 isw41204 cell08 lrdn1431 isw41202 cell08 lrdn1432 isw41202 cell08 lrdn1433 isw41202 cell08 lrdn1434 isw41202 cell08 lrdn1435 isw41202 cell08 lrdn1436 isw41202 cell08 lrdn1437 isw41202 cell08 lrdn1438 isw41202 cell08 lrdn1439 isw41204 cell08 lrdn1440 isw41204 cell08 lrdn1441 isw50100 cell09 lrdn1442 isw50100 cell09 lrdn1443 isw50100 cell09 lrdn1444 isw50100 cell09 lrdn1445 isw50100 cell09 lrdn1446 isw50100 cell09 lrdn1447 isw50100 cell09 lrdn1448 isw50100 cell09 lrdn1449 isw50104 cell09 lrdn1450 isw50104 cell09 lrdn1451 isw50104 cell09 lrdn1452 isw50104 cell09 lrdn1453 isw50100 cell09 lrdn1454 isw50100 cell09 lrdn1455 isw50102 cell09 lrdn1456 isw50102 cell09 lrdn1457 isw50104 cell09 lrdn1458 isw50104 cell09 lrdn1459 isw50104 cell09 lrdn1460 isw50104 cell09 lrdn1461 isw50102 cell09 lrdn1462 isw50102 cell09 lrdn1463 isw50102 cell09 lrdn1464 isw50102 cell09 lrdn1465 isw50102 cell09 lrdn1466 isw50102 cell09 lrdn1467 isw50102 cell09 lrdn1468 isw50102 cell09 lrdn1469 isw50104 cell09 lrdn1470 isw50104 cell09 lrdn1471 isw50200 cell09 lrdn1472 isw50200 cell09 lrdn1473 isw50200 cell09 lrdn1474 isw50200 cell09 lrdn1475 isw50200 cell09 lrdn1476 isw50200 cell09 lrdn1477 isw50200 cell09 lrdn1478 isw50200 cell09 lrdn1479 isw50204 cell09 lrdn1480 isw50204 cell09 lrdn1481 isw50204 cell09 lrdn1482 isw50204 cell09 lrdn1483 isw50200 cell09 lrdn1484 isw50200 cell09 lrdn1485 isw50202 cell09 lrdn1486 isw50202 cell09 lrdn1487 isw50204 cell09 lrdn1488 isw50204 cell09 lrdn1489 isw50204 cell09 lrdn1490 isw50204 cell09 lrdn1491 isw50202 cell09 lrdn1492 isw50202 cell09 lrdn1493 isw50202 cell09 lrdn1494 isw50202 cell09 lrdn1495 isw50202 cell09 lrdn1496 isw50202 cell09 lrdn1497 isw50202 cell09 lrdn1498 isw50202 cell09 lrdn1499 isw50204 cell09 lrdn1500 isw50204 cell09 lrdn1501 isw50300 cell09 lrdn1502 isw50300 cell09 lrdn1503 isw50300 cell09 lrdn1504 isw50300 cell09 lrdn1505 isw50300 cell09 lrdn1506 isw50300 cell09 lrdn1507 isw50300 cell09 lrdn1508 isw50300 cell09 lrdn1509 isw50304 cell09 lrdn1510 isw50304 cell09 lrdn1511 isw50304 cell09 lrdn1512 isw50304 cell09 lrdn1513 isw50300 cell09 lrdn1514 isw50300 cell09 lrdn1515 isw50302 cell09 lrdn1516 isw50302 cell09 lrdn1517 isw50304 cell09 lrdn1518 isw50304 cell09 lrdn1519 isw50304 cell09 lrdn1520 isw50304 cell09 lrdn1521 isw50302 cell09 lrdn1522 isw50302 cell09 lrdn1523 isw50302 cell09 lrdn1524 isw50302 cell09 lrdn1525 isw50302 cell09 lrdn1526 isw50302 cell09 lrdn1527 isw50302 cell09 lrdn1528 isw50302 cell09 lrdn1529 isw50304 cell09 lrdn1530 isw50304 cell09 lrdn1531 isw50400 cell09 lrdn1532 isw50400 cell09 lrdn1533 isw50400 cell09 lrdn1534 isw50400 cell09 lrdn1535 isw50400 cell09 lrdn1536 isw50400 cell09 lrdn1537 isw50400 cell09 lrdn1538 isw50400 cell09 lrdn1539 isw50404 cell09 lrdn1540 isw50404 cell09 lrdn1541 isw50404 cell09 lrdn1542 isw50404 cell09 lrdn1543 isw50400 cell09 lrdn1544 isw50400 cell09 lrdn1545 isw50402 cell09 lrdn1546 isw50402 cell09 lrdn1547 isw50404 cell09 lrdn1548 isw50404 cell09 lrdn1549 isw50404 cell09 lrdn1550 isw50404 cell09 lrdn1551 isw50402 cell09 lrdn1552 isw50402 cell09 lrdn1553 isw50402 cell09 lrdn1554 isw50402 cell09 lrdn1555 isw50402 cell09 lrdn1556 isw50402 cell09 lrdn1557 isw50402 cell09 lrdn1558 isw50402 cell09 lrdn1559 isw50404 cell09 lrdn1560 isw50404 cell09 lrdn1561 isw50500 cell09 lrdn1562 isw50500 cell09 lrdn1563 isw50500 cell09 lrdn1564 isw50500 cell09 lrdn1565 isw50500 cell09 lrdn1566 isw50500 cell09 lrdn1567 isw50500 cell09 lrdn1568 isw50500 cell09 lrdn1569 isw50504 cell09 lrdn1570 isw50504 cell09 lrdn1571 isw50504 cell09 lrdn1572 isw50504 cell09 lrdn1573 isw50500 cell09 lrdn1574 isw50500 cell09 lrdn1575 isw50502 cell09 lrdn1576 isw50502 cell09 lrdn1577 isw50504 cell09 lrdn1578 isw50504 cell09 lrdn1579 isw50504 cell09 lrdn1580 isw50504 cell09 lrdn1581 isw50502 cell09 lrdn1582 isw50502 cell09 lrdn1583 isw50502 cell09 lrdn1584 isw50502 cell09 lrdn1585 isw50502 cell09 lrdn1586 isw50502 cell09 lrdn1587 isw50502 cell09 lrdn1588 isw50502 cell09 lrdn1589 isw50504 cell09 lrdn1590 isw50504 cell09 lrdn1591 isw50600 cell09 lrdn1592 isw50600 cell09 lrdn1593 isw50600 cell09 lrdn1594 isw50600 cell09 lrdn1595 isw50600 cell09 lrdn1596 isw50600 cell09 lrdn1597 isw50600 cell09 lrdn1598 isw50600 cell09 lrdn1599 isw50604 cell09 lrdn1600 isw50604 cell09 lrdn1601 isw50604 cell09 lrdn1602 isw50604 cell09 lrdn1603 isw50600 cell09 lrdn1604 isw50600 cell09 lrdn1605 isw50602 cell09 lrdn1606 isw50602 cell09 lrdn1607 isw50604 cell09 lrdn1608 isw50604 cell09 lrdn1609 isw50604 cell09 lrdn1610 isw50604 cell09 lrdn1611 isw50602 cell09 lrdn1612 isw50602 cell09 lrdn1613 isw50602 cell09 lrdn1614 isw50602 cell09 lrdn1615 isw50602 cell09 lrdn1616 isw50602 cell09 lrdn1617 isw50602 cell09 lrdn1618 isw50602 cell09 lrdn1619 isw50604 cell09 lrdn1620 isw50604 cell09 lrdn1621 isw50700 cell10 lrdn1622 isw50700 cell10 lrdn1623 isw50700 cell10 lrdn1624 isw50700 cell10 lrdn1625 isw50700 cell10 lrdn1626 isw50700 cell10 lrdn1627 isw50700 cell10 lrdn1628 isw50700 cell10 lrdn1629 isw50704 cell10 lrdn1630 isw50704 cell10 lrdn1631 isw50704 cell10 lrdn1632 isw50704 cell10 lrdn1633 isw50700 cell10 lrdn1634 isw50700 cell10 lrdn1635 isw50702 cell10 lrdn1636 isw50702 cell10 lrdn1637 isw50704 cell10 lrdn1638 isw50704 cell10 lrdn1639 isw50704 cell10 lrdn1640 isw50704 cell10 lrdn1641 isw50702 cell10 lrdn1642 isw50702 cell10 lrdn1643 isw50702 cell10 lrdn1644 isw50702 cell10 lrdn1645 isw50702 cell10 lrdn1646 isw50702 cell10 lrdn1647 isw50702 cell10 lrdn1648 isw50702 cell10 lrdn1649 isw50704 cell10 lrdn1650 isw50704 cell10 lrdn1651 isw50800 cell10 lrdn1652 isw50800 cell10 lrdn1653 isw50800 cell10 lrdn1654 isw50800 cell10 lrdn1655 isw50800 cell10 lrdn1656 isw50800 cell10 lrdn1657 isw50800 cell10 lrdn1658 isw50800 cell10 lrdn1659 isw50804 cell10 lrdn1660 isw50804 cell10 lrdn1661 isw50804 cell10 lrdn1662 isw50804 cell10 lrdn1663 isw50800 cell10 lrdn1664 isw50800 cell10 lrdn1665 isw50802 cell10 lrdn1666 isw50802 cell10 lrdn1667 isw50804 cell10 lrdn1668 isw50804 cell10 lrdn1669 isw50804 cell10 lrdn1670 isw50804 cell10 lrdn1671 isw50802 cell10 lrdn1672 isw50802 cell10 lrdn1673 isw50802 cell10 lrdn1674 isw50802 cell10 lrdn1675 isw50802 cell10 lrdn1676 isw50802 cell10 lrdn1677 isw50802 cell10 lrdn1678 isw50802 cell10 lrdn1679 isw50804 cell10 lrdn1680 isw50804 cell10 lrdn1681 isw50900 cell10 lrdn1682 isw50900 cell10 lrdn1683 isw50900 cell10 lrdn1684 isw50900 cell10 lrdn1685 isw50900 cell10 lrdn1686 isw50900 cell10 lrdn1687 isw50900 cell10 lrdn1688 isw50900 cell10 lrdn1689 isw50904 cell10 lrdn1690 isw50904 cell10 lrdn1691 isw50904 cell10 lrdn1692 isw50904 cell10 lrdn1693 isw50900 cell10 lrdn1694 isw50900 cell10 lrdn1695 isw50902 cell10 lrdn1696 isw50902 cell10 lrdn1697 isw50904 cell10 lrdn1698 isw50904 cell10 lrdn1699 isw50904 cell10 lrdn1700 isw50904 cell10 lrdn1701 isw50902 cell10 lrdn1702 isw50902 cell10 lrdn1703 isw50902 cell10 lrdn1704 isw50902 cell10 lrdn1705 isw50902 cell10 lrdn1706 isw50902 cell10 lrdn1707 isw50902 cell10 lrdn1708 isw50902 cell10 lrdn1709 isw50904 cell10 lrdn1710 isw50904 cell10 lrdn1711 isw51000 cell10 lrdn1712 isw51000 cell10 lrdn1713 isw51000 cell10 lrdn1714 isw51000 cell10 lrdn1715 isw51000 cell10 lrdn1716 isw51000 cell10 lrdn1717 isw51000 cell10 lrdn1718 isw51000 cell10 lrdn1719 isw51004 cell10 lrdn1720 isw51004 cell10 lrdn1721 isw51004 cell10 lrdn1722 isw51004 cell10 lrdn1723 isw51000 cell10 lrdn1724 isw51000 cell10 lrdn1725 isw51002 cell10 lrdn1726 isw51002 cell10 lrdn1727 isw51004 cell10 lrdn1728 isw51004 cell10 lrdn1729 isw51004 cell10 lrdn1730 isw51004 cell10 lrdn1731 isw51002 cell10 lrdn1732 isw51002 cell10 lrdn1733 isw51002 cell10 lrdn1734 isw51002 cell10 lrdn1735 isw51002 cell10 lrdn1736 isw51002 cell10 lrdn1737 isw51002 cell10 lrdn1738 isw51002 cell10 lrdn1739 isw51004 cell10 lrdn1740 isw51004 cell10 lrdn1741 isw51100 cell10 lrdn1742 isw51100 cell10 lrdn1743 isw51100 cell10 lrdn1744 isw51100 cell10 lrdn1745 isw51100 cell10 lrdn1746 isw51100 cell10 lrdn1747 isw51100 cell10 lrdn1748 isw51100 cell10 lrdn1749 isw51104 cell10 lrdn1750 isw51104 cell10 lrdn1751 isw51104 cell10 lrdn1752 isw51104 cell10 lrdn1753 isw51100 cell10 lrdn1754 isw51100 cell10 lrdn1755 isw51102 cell10 lrdn1756 isw51102 cell10 lrdn1757 isw51104 cell10 lrdn1758 isw51104 cell10 lrdn1759 isw51104 cell10 lrdn1760 isw51104 cell10 lrdn1761 isw51102 cell10 lrdn1762 isw51102 cell10 lrdn1763 isw51102 cell10 lrdn1764 isw51102 cell10 lrdn1765 isw51102 cell10 lrdn1766 isw51102 cell10 lrdn1767 isw51102 cell10 lrdn1768 isw51102 cell10 lrdn1769 isw51104 cell10 lrdn1770 isw51104 cell10 lrdn1771 isw51200 cell10 lrdn1772 isw51200 cell10 lrdn1773 isw51200 cell10 lrdn1774 isw51200 cell10 lrdn1775 isw51200 cell10 lrdn1776 isw51200 cell10 lrdn1777 isw51200 cell10 lrdn1778 isw51200 cell10 lrdn1779 isw51204 cell10 lrdn1780 isw51204 cell10 lrdn1781 isw51204 cell10 lrdn1782 isw51204 cell10 lrdn1783 isw51200 cell10 lrdn1784 isw51200 cell10 lrdn1785 isw51202 cell10 lrdn1786 isw51202 cell10 lrdn1787 isw51204 cell10 lrdn1788 isw51204 cell10 lrdn1789 isw51204 cell10 lrdn1790 isw51204 cell10 lrdn1791 isw51202 cell10 lrdn1792 isw51202 cell10 lrdn1793 isw51202 cell10 lrdn1794 isw51202 cell10 lrdn1795 isw51202 cell10 lrdn1796 isw51202 cell10 lrdn1797 isw51202 cell10 lrdn1798 isw51202 cell10 lrdn1799 isw51204 cell10 lrdn1800 isw51204 cell10 lrdn1801 isw60100 cell11 lrdn1802 isw60100 cell11 lrdn1803 isw60100 cell11 lrdn1804 isw60100 cell11 lrdn1805 isw60100 cell11 lrdn1806 isw60100 cell11 lrdn1807 isw60100 cell11 lrdn1808 isw60100 cell11 lrdn1809 isw60104 cell11 lrdn1810 isw60104 cell11 lrdn1811 isw60104 cell11 lrdn1812 isw60104 cell11 lrdn1813 isw60100 cell11 lrdn1814 isw60100 cell11 lrdn1815 isw60102 cell11 lrdn1816 isw60102 cell11 lrdn1817 isw60104 cell11 lrdn1818 isw60104 cell11 lrdn1819 isw60104 cell11 lrdn1820 isw60104 cell11 lrdn1821 isw60102 cell11 lrdn1822 isw60102 cell11 lrdn1823 isw60102 cell11 lrdn1824 isw60102 cell11 lrdn1825 isw60102 cell11 lrdn1826 isw60102 cell11 lrdn1827 isw60102 cell11 lrdn1828 isw60102 cell11 lrdn1829 isw60104 cell11 lrdn1830 isw60104 cell11 lrdn1831 isw60200 cell11 lrdn1832 isw60200 cell11 lrdn1833 isw60200 cell11 lrdn1834 isw60200 cell11 lrdn1835 isw60200 cell11 lrdn1836 isw60200 cell11 lrdn1837 isw60200 cell11 lrdn1838 isw60200 cell11 lrdn1839 isw60204 cell11 lrdn1840 isw60204 cell11 lrdn1841 isw60204 cell11 lrdn1842 isw60204 cell11 lrdn1843 isw60200 cell11 lrdn1844 isw60200 cell11 lrdn1845 isw60202 cell11 lrdn1846 isw60202 cell11 lrdn1847 isw60204 cell11 lrdn1848 isw60204 cell11 lrdn1849 isw60204 cell11 lrdn1850 isw60204 cell11 lrdn1851 isw60202 cell11 lrdn1852 isw60202 cell11 lrdn1853 isw60202 cell11 lrdn1854 isw60202 cell11 lrdn1855 isw60202 cell11 lrdn1856 isw60202 cell11 lrdn1857 isw60202 cell11 lrdn1858 isw60202 cell11 lrdn1859 isw60204 cell11 lrdn1860 isw60204 cell11 lrdn1861 isw60300 cell11 lrdn1862 isw60300 cell11 lrdn1863 isw60300 cell11 lrdn1864 isw60300 cell11 lrdn1865 isw60300 cell11 lrdn1866 isw60300 cell11 lrdn1867 isw60300 cell11 lrdn1868 isw60300 cell11 lrdn1869 isw60304 cell11 lrdn1870 isw60304 cell11 lrdn1871 isw60304 cell11 lrdn1872 isw60304 cell11 lrdn1873 isw60300 cell11 lrdn1874 isw60300 cell11 lrdn1875 isw60302 cell11 lrdn1876 isw60302 cell11 lrdn1877 isw60304 cell11 lrdn1878 isw60304 cell11 lrdn1879 isw60304 cell11 lrdn1880 isw60304 cell11 lrdn1881 isw60302 cell11 lrdn1882 isw60302 cell11 lrdn1883 isw60302 cell11 lrdn1884 isw60302 cell11 lrdn1885 isw60302 cell11 lrdn1886 isw60302 cell11 lrdn1887 isw60302 cell11 lrdn1888 isw60302 cell11 lrdn1889 isw60304 cell11 lrdn1890 isw60304 cell11 lrdn1891 isw60400 cell11 lrdn1892 isw60400 cell11 lrdn1893 isw60400 cell11 lrdn1894 isw60400 cell11 lrdn1895 isw60400 cell11 lrdn1896 isw60400 cell11 lrdn1897 isw60400 cell11 lrdn1898 isw60400 cell11 lrdn1899 isw60404 cell11 lrdn1900 isw60404 cell11 lrdn1901 isw60404 cell11 lrdn1902 isw60404 cell11 lrdn1903 isw60400 cell11 lrdn1904 isw60400 cell11 lrdn1905 isw60402 cell11 lrdn1906 isw60402 cell11 lrdn1907 isw60404 cell11 lrdn1908 isw60404 cell11 lrdn1909 isw60404 cell11 lrdn1910 isw60404 cell11 lrdn1911 isw60402 cell11 lrdn1912 isw60402 cell11 lrdn1913 isw60402 cell11 lrdn1914 isw60402 cell11 lrdn1915 isw60402 cell11 lrdn1916 isw60402 cell11 lrdn1917 isw60402 cell11 lrdn1918 isw60402 cell11 lrdn1919 isw60404 cell11 lrdn1920 isw60404 cell11 lrdn1921 isw60500 cell11 lrdn1922 isw60500 cell11 lrdn1923 isw60500 cell11 lrdn1924 isw60500 cell11 lrdn1925 isw60500 cell11 lrdn1926 isw60500 cell11 lrdn1927 isw60500 cell11 lrdn1928 isw60500 cell11 lrdn1929 isw60504 cell11 lrdn1930 isw60504 cell11 lrdn1931 isw60504 cell11 lrdn1932 isw60504 cell11 lrdn1933 isw60500 cell11 lrdn1934 isw60500 cell11 lrdn1935 isw60502 cell11 lrdn1936 isw60502 cell11 lrdn1937 isw60504 cell11 lrdn1938 isw60504 cell11 lrdn1939 isw60504 cell11 lrdn1940 isw60504 cell11 lrdn1941 isw60502 cell11 lrdn1942 isw60502 cell11 lrdn1943 isw60502 cell11 lrdn1944 isw60502 cell11 lrdn1945 isw60502 cell11 lrdn1946 isw60502 cell11 lrdn1947 isw60502 cell11 lrdn1948 isw60502 cell11 lrdn1949 isw60504 cell11 lrdn1950 isw60504 cell11 lrdn1951 isw60600 cell11 lrdn1952 isw60600 cell11 lrdn1953 isw60600 cell11 lrdn1954 isw60600 cell11 lrdn1955 isw60600 cell11 lrdn1956 isw60600 cell11 lrdn1957 isw60600 cell11 lrdn1958 isw60600 cell11 lrdn1959 isw60604 cell11 lrdn1960 isw60604 cell11 lrdn1961 isw60604 cell11 lrdn1962 isw60604 cell11 lrdn1963 isw60600 cell11 lrdn1964 isw60600 cell11 lrdn1965 isw60602 cell11 lrdn1966 isw60602 cell11 lrdn1967 isw60604 cell11 lrdn1968 isw60604 cell11 lrdn1969 isw60604 cell11 lrdn1970 isw60604 cell11 lrdn1971 isw60602 cell11 lrdn1972 isw60602 cell11 lrdn1973 isw60602 cell11 lrdn1974 isw60602 cell11 lrdn1975 isw60602 cell11 lrdn1976 isw60602 cell11 lrdn1977 isw60602 cell11 lrdn1978 isw60602 cell11 lrdn1979 isw60604 cell11 lrdn1980 isw60604 cell11 lrdn1981 isw60700 cell12 lrdn1982 isw60700 cell12 lrdn1983 isw60700 cell12 lrdn1984 isw60700 cell12 lrdn1985 isw60700 cell12 lrdn1986 isw60700 cell12 lrdn1987 isw60700 cell12 lrdn1988 isw60700 cell12 lrdn1989 isw60704 cell12 lrdn1990 isw60704 cell12 lrdn1991 isw60704 cell12 lrdn1992 isw60704 cell12 lrdn1993 isw60700 cell12 lrdn1994 isw60700 cell12 lrdn1995 isw60702 cell12 lrdn1996 isw60702 cell12 lrdn1997 isw60704 cell12 lrdn1998 isw60704 cell12 lrdn1999 isw60704 cell12 lrdn2000 isw60704 cell12 lrdn2001 isw60702 cell12 lrdn2002 isw60702 cell12 lrdn2003 isw60702 cell12 lrdn2004 isw60702 cell12 lrdn2005 isw60702 cell12 lrdn2006 isw60702 cell12 lrdn2007 isw60702 cell12 lrdn2008 isw60702 cell12 lrdn2009 isw60704 cell12 lrdn2010 isw60704 cell12 lrdn2011 isw60800 cell12 lrdn2012 isw60800 cell12 lrdn2013 isw60800 cell12 lrdn2014 isw60800 cell12 lrdn2015 isw60800 cell12 lrdn2016 isw60800 cell12 lrdn2017 isw60800 cell12 lrdn2018 isw60800 cell12 lrdn2019 isw60804 cell12 lrdn2020 isw60804 cell12 lrdn2021 isw60804 cell12 lrdn2022 isw60804 cell12 lrdn2023 isw60800 cell12 lrdn2024 isw60800 cell12 lrdn2025 isw60802 cell12 lrdn2026 isw60802 cell12 lrdn2027 isw60804 cell12 lrdn2028 isw60804 cell12 lrdn2029 isw60804 cell12 lrdn2030 isw60804 cell12 lrdn2031 isw60802 cell12 lrdn2032 isw60802 cell12 lrdn2033 isw60802 cell12 lrdn2034 isw60802 cell12 lrdn2035 isw60802 cell12 lrdn2036 isw60802 cell12 lrdn2037 isw60802 cell12 lrdn2038 isw60802 cell12 lrdn2039 isw60804 cell12 lrdn2040 isw60804 cell12 lrdn2041 isw60900 cell12 lrdn2042 isw60900 cell12 lrdn2043 isw60900 cell12 lrdn2044 isw60900 cell12 lrdn2045 isw60900 cell12 lrdn2046 isw60900 cell12 lrdn2047 isw60900 cell12 lrdn2048 isw60900 cell12 lrdn2049 isw60904 cell12 lrdn2050 isw60904 cell12 lrdn2051 isw60904 cell12 lrdn2052 isw60904 cell12 lrdn2053 isw60900 cell12 lrdn2054 isw60900 cell12 lrdn2055 isw60902 cell12 lrdn2056 isw60902 cell12 lrdn2057 isw60904 cell12 lrdn2058 isw60904 cell12 lrdn2059 isw60904 cell12 lrdn2060 isw60904 cell12 lrdn2061 isw60902 cell12 lrdn2062 isw60902 cell12 lrdn2063 isw60902 cell12 lrdn2064 isw60902 cell12 lrdn2065 isw60902 cell12 lrdn2066 isw60902 cell12 lrdn2067 isw60902 cell12 lrdn2068 isw60902 cell12 lrdn2069 isw60904 cell12 lrdn2070 isw60904 cell12 lrdn2071 isw61000 cell12 lrdn2072 isw61000 cell12 lrdn2073 isw61000 cell12 lrdn2074 isw61000 cell12 lrdn2075 isw61000 cell12 lrdn2076 isw61000 cell12 lrdn2077 isw61000 cell12 lrdn2078 isw61000 cell12 lrdn2079 isw61004 cell12 lrdn2080 isw61004 cell12 lrdn2081 isw61004 cell12 lrdn2082 isw61004 cell12 lrdn2083 isw61000 cell12 lrdn2084 isw61000 cell12 lrdn2085 isw61002 cell12 lrdn2086 isw61002 cell12 lrdn2087 isw61004 cell12 lrdn2088 isw61004 cell12 lrdn2089 isw61004 cell12 lrdn2090 isw61004 cell12 lrdn2091 isw61002 cell12 lrdn2092 isw61002 cell12 lrdn2093 isw61002 cell12 lrdn2094 isw61002 cell12 lrdn2095 isw61002 cell12 lrdn2096 isw61002 cell12 lrdn2097 isw61002 cell12 lrdn2098 isw61002 cell12 lrdn2099 isw61004 cell12 lrdn2100 isw61004 cell12 lrdn2101 isw61100 cell12 lrdn2102 isw61100 cell12 lrdn2103 isw61100 cell12 lrdn2104 isw61100 cell12 lrdn2105 isw61100 cell12 lrdn2106 isw61100 cell12 lrdn2107 isw61100 cell12 lrdn2108 isw61100 cell12 lrdn2109 isw61104 cell12 lrdn2110 isw61104 cell12 lrdn2111 isw61104 cell12 lrdn2112 isw61104 cell12 lrdn2113 isw61100 cell12 lrdn2114 isw61100 cell12 lrdn2115 isw61102 cell12 lrdn2116 isw61102 cell12 lrdn2117 isw61104 cell12 lrdn2118 isw61104 cell12 lrdn2119 isw61104 cell12 lrdn2120 isw61104 cell12 lrdn2121 isw61102 cell12 lrdn2122 isw61102 cell12 lrdn2123 isw61102 cell12 lrdn2124 isw61102 cell12 lrdn2125 isw61102 cell12 lrdn2126 isw61102 cell12 lrdn2127 isw61102 cell12 lrdn2128 isw61102 cell12 lrdn2129 isw61104 cell12 lrdn2130 isw61104 cell12 lrdn2131 isw61200 cell12 lrdn2132 isw61200 cell12 lrdn2133 isw61200 cell12 lrdn2134 isw61200 cell12 lrdn2135 isw61200 cell12 lrdn2136 isw61200 cell12 lrdn2137 isw61200 cell12 lrdn2138 isw61200 cell12 lrdn2139 isw61204 cell12 lrdn2140 isw61204 cell12 lrdn2141 isw61204 cell12 lrdn2142 isw61204 cell12 lrdn2143 isw61200 cell12 lrdn2144 isw61200 cell12 lrdn2145 isw61202 cell12 lrdn2146 isw61202 cell12 lrdn2147 isw61204 cell12 lrdn2148 isw61204 cell12 lrdn2149 isw61204 cell12 lrdn2150 isw61204 cell12 lrdn2151 isw61202 cell12 lrdn2152 isw61202 cell12 lrdn2153 isw61202 cell12 lrdn2154 isw61202 cell12 lrdn2155 isw61202 cell12 lrdn2156 isw61202 cell12 lrdn2157 isw61202 cell12 lrdn2158 isw61202 cell12 lrdn2159 isw61204 cell12 lrdn2160 isw61204 cell12 lrdn2161 isw70100 cell13 lrdn2162 isw70100 cell13 lrdn2163 isw70100 cell13 lrdn2164 isw70100 cell13 lrdn2165 isw70100 cell13 lrdn2166 isw70100 cell13 lrdn2167 isw70100 cell13 lrdn2168 isw70100 cell13 lrdn2169 isw70104 cell13 lrdn2170 isw70104 cell13 lrdn2171 isw70104 cell13 lrdn2172 isw70104 cell13 lrdn2173 isw70100 cell13 lrdn2174 isw70100 cell13 lrdn2175 isw70102 cell13 lrdn2176 isw70102 cell13 lrdn2177 isw70104 cell13 lrdn2178 isw70104 cell13 lrdn2179 isw70104 cell13 lrdn2180 isw70104 cell13 lrdn2181 isw70102 cell13 lrdn2182 isw70102 cell13 lrdn2183 isw70102 cell13 lrdn2184 isw70102 cell13 lrdn2185 isw70102 cell13 lrdn2186 isw70102 cell13 lrdn2187 isw70102 cell13 lrdn2188 isw70102 cell13 lrdn2189 isw70104 cell13 lrdn2190 isw70104 cell13 lrdn2191 isw70200 cell13 lrdn2192 isw70200 cell13 lrdn2193 isw70200 cell13 lrdn2194 isw70200 cell13 lrdn2195 isw70200 cell13 lrdn2196 isw70200 cell13 lrdn2197 isw70200 cell13 lrdn2198 isw70200 cell13 lrdn2199 isw70204 cell13 lrdn2200 isw70204 cell13 lrdn2201 isw70204 cell13 lrdn2202 isw70204 cell13 lrdn2203 isw70200 cell13 lrdn2204 isw70200 cell13 lrdn2205 isw70202 cell13 lrdn2206 isw70202 cell13 lrdn2207 isw70204 cell13 lrdn2208 isw70204 cell13 lrdn2209 isw70204 cell13 lrdn2210 isw70204 cell13 lrdn2211 isw70202 cell13 lrdn2212 isw70202 cell13 lrdn2213 isw70202 cell13 lrdn2214 isw70202 cell13 lrdn2215 isw70202 cell13 lrdn2216 isw70202 cell13 lrdn2217 isw70202 cell13 lrdn2218 isw70202 cell13 lrdn2219 isw70204 cell13 lrdn2220 isw70204 cell13 lrdn2221 isw70300 cell13 lrdn2222 isw70300 cell13 lrdn2223 isw70300 cell13 lrdn2224 isw70300 cell13 lrdn2225 isw70300 cell13 lrdn2226 isw70300 cell13 lrdn2227 isw70300 cell13 lrdn2228 isw70300 cell13 lrdn2229 isw70304 cell13 lrdn2230 isw70304 cell13 lrdn2231 isw70304 cell13 lrdn2232 isw70304 cell13 lrdn2233 isw70300 cell13 lrdn2234 isw70300 cell13 lrdn2235 isw70302 cell13 lrdn2236 isw70302 cell13 lrdn2237 isw70304 cell13 lrdn2238 isw70304 cell13 lrdn2239 isw70304 cell13 lrdn2240 isw70304 cell13 lrdn2241 isw70302 cell13 lrdn2242 isw70302 cell13 lrdn2243 isw70302 cell13 lrdn2244 isw70302 cell13 lrdn2245 isw70302 cell13 lrdn2246 isw70302 cell13 lrdn2247 isw70302 cell13 lrdn2248 isw70302 cell13 lrdn2249 isw70304 cell13 lrdn2250 isw70304 cell13 lrdn2251 isw70400 cell13 lrdn2252 isw70400 cell13 lrdn2253 isw70400 cell13 lrdn2254 isw70400 cell13 lrdn2255 isw70400 cell13 lrdn2256 isw70400 cell13 lrdn2257 isw70400 cell13 lrdn2258 isw70400 cell13 lrdn2259 isw70404 cell13 lrdn2260 isw70404 cell13 lrdn2261 isw70404 cell13 lrdn2262 isw70404 cell13 lrdn2263 isw70400 cell13 lrdn2264 isw70400 cell13 lrdn2265 isw70402 cell13 lrdn2266 isw70402 cell13 lrdn2267 isw70404 cell13 lrdn2268 isw70404 cell13 lrdn2269 isw70404 cell13 lrdn2270 isw70404 cell13 lrdn2271 isw70402 cell13 lrdn2272 isw70402 cell13 lrdn2273 isw70402 cell13 lrdn2274 isw70402 cell13 lrdn2275 isw70402 cell13 lrdn2276 isw70402 cell13 lrdn2277 isw70402 cell13 lrdn2278 isw70402 cell13 lrdn2279 isw70404 cell13 lrdn2280 isw70404 cell13 lrdn2281 isw70500 cell13 lrdn2282 isw70500 cell13 lrdn2283 isw70500 cell13 lrdn2284 isw70500 cell13 lrdn2285 isw70500 cell13 lrdn2286 isw70500 cell13 lrdn2287 isw70500 cell13 lrdn2288 isw70500 cell13 lrdn2289 isw70504 cell13 lrdn2290 isw70504 cell13 lrdn2291 isw70504 cell13 lrdn2292 isw70504 cell13 lrdn2293 isw70500 cell13 lrdn2294 isw70500 cell13 lrdn2295 isw70502 cell13 lrdn2296 isw70502 cell13 lrdn2297 isw70504 cell13 lrdn2298 isw70504 cell13 lrdn2299 isw70504 cell13 lrdn2300 isw70504 cell13 lrdn2301 isw70502 cell13 lrdn2302 isw70502 cell13 lrdn2303 isw70502 cell13 lrdn2304 isw70502 cell13 lrdn2305 isw70502 cell13 lrdn2306 isw70502 cell13 lrdn2307 isw70502 cell13 lrdn2308 isw70502 cell13 lrdn2309 isw70504 cell13 lrdn2310 isw70504 cell13 lrdn2311 isw70600 cell13 lrdn2312 isw70600 cell13 lrdn2313 isw70600 cell13 lrdn2314 isw70600 cell13 lrdn2315 isw70600 cell13 lrdn2316 isw70600 cell13 lrdn2317 isw70600 cell13 lrdn2318 isw70600 cell13 lrdn2319 isw70604 cell13 lrdn2320 isw70604 cell13 lrdn2321 isw70604 cell13 lrdn2322 isw70604 cell13 lrdn2323 isw70600 cell13 lrdn2324 isw70600 cell13 lrdn2325 isw70602 cell13 lrdn2326 isw70602 cell13 lrdn2327 isw70604 cell13 lrdn2328 isw70604 cell13 lrdn2329 isw70604 cell13 lrdn2330 isw70604 cell13 lrdn2331 isw70602 cell13 lrdn2332 isw70602 cell13 lrdn2333 isw70602 cell13 lrdn2334 isw70602 cell13 lrdn2335 isw70602 cell13 lrdn2336 isw70602 cell13 lrdn2337 isw70602 cell13 lrdn2338 isw70602 cell13 lrdn2339 isw70604 cell13 lrdn2340 isw70604 cell13 lrdn2341 isw70700 cell14 lrdn2342 isw70700 cell14 lrdn2343 isw70700 cell14 lrdn2344 isw70700 cell14 lrdn2345 isw70700 cell14 lrdn2346 isw70700 cell14 lrdn2347 isw70700 cell14 lrdn2348 isw70700 cell14 lrdn2349 isw70704 cell14 lrdn2350 isw70704 cell14 lrdn2351 isw70704 cell14 lrdn2352 isw70704 cell14 lrdn2353 isw70700 cell14 lrdn2354 isw70700 cell14 lrdn2355 isw70702 cell14 lrdn2356 isw70702 cell14 lrdn2357 isw70704 cell14 lrdn2358 isw70704 cell14 lrdn2359 isw70704 cell14 lrdn2360 isw70704 cell14 lrdn2361 isw70702 cell14 lrdn2362 isw70702 cell14 lrdn2363 isw70702 cell14 lrdn2364 isw70702 cell14 lrdn2365 isw70702 cell14 lrdn2366 isw70702 cell14 lrdn2367 isw70702 cell14 lrdn2368 isw70702 cell14 lrdn2369 isw70704 cell14 lrdn2370 isw70704 cell14 lrdn2371 isw70800 cell14 lrdn2372 isw70800 cell14 lrdn2373 isw70800 cell14 lrdn2374 isw70800 cell14 lrdn2375 isw70800 cell14 lrdn2376 isw70800 cell14 lrdn2377 isw70800 cell14 lrdn2378 isw70800 cell14 lrdn2379 isw70804 cell14 lrdn2380 isw70804 cell14 lrdn2381 isw70804 cell14 lrdn2382 isw70804 cell14 lrdn2383 isw70800 cell14 lrdn2384 isw70800 cell14 lrdn2385 isw70802 cell14 lrdn2386 isw70802 cell14 lrdn2387 isw70804 cell14 lrdn2388 isw70804 cell14 lrdn2389 isw70804 cell14 lrdn2390 isw70804 cell14 lrdn2391 isw70802 cell14 lrdn2392 isw70802 cell14 lrdn2393 isw70802 cell14 lrdn2394 isw70802 cell14 lrdn2395 isw70802 cell14 lrdn2396 isw70802 cell14 lrdn2397 isw70802 cell14 lrdn2398 isw70802 cell14 lrdn2399 isw70804 cell14 lrdn2400 isw70804 cell14 lrdn2401 isw70900 cell14 lrdn2402 isw70900 cell14 lrdn2403 isw70900 cell14 lrdn2404 isw70900 cell14 lrdn2405 isw70900 cell14 lrdn2406 isw70900 cell14 lrdn2407 isw70900 cell14 lrdn2408 isw70900 cell14 lrdn2409 isw70904 cell14 lrdn2410 isw70904 cell14 lrdn2411 isw70904 cell14 lrdn2412 isw70904 cell14 lrdn2413 isw70900 cell14 lrdn2414 isw70900 cell14 lrdn2415 isw70902 cell14 lrdn2416 isw70902 cell14 lrdn2417 isw70904 cell14 lrdn2418 isw70904 cell14 lrdn2419 isw70904 cell14 lrdn2420 isw70904 cell14 lrdn2421 isw70902 cell14 lrdn2422 isw70902 cell14 lrdn2423 isw70902 cell14 lrdn2424 isw70902 cell14 lrdn2425 isw70902 cell14 lrdn2426 isw70902 cell14 lrdn2427 isw70902 cell14 lrdn2428 isw70902 cell14 lrdn2429 isw70904 cell14 lrdn2430 isw70904 cell14 lrdn2431 isw71000 cell14 lrdn2432 isw71000 cell14 lrdn2433 isw71000 cell14 lrdn2434 isw71000 cell14 lrdn2435 isw71000 cell14 lrdn2436 isw71000 cell14 lrdn2437 isw71000 cell14 lrdn2438 isw71000 cell14 lrdn2439 isw71004 cell14 lrdn2440 isw71004 cell14 lrdn2441 isw71004 cell14 lrdn2442 isw71004 cell14 lrdn2443 isw71000 cell14 lrdn2444 isw71000 cell14 lrdn2445 isw71002 cell14 lrdn2446 isw71002 cell14 lrdn2447 isw71004 cell14 lrdn2448 isw71004 cell14 lrdn2449 isw71004 cell14 lrdn2450 isw71004 cell14 lrdn2451 isw71002 cell14 lrdn2452 isw71002 cell14 lrdn2453 isw71002 cell14 lrdn2454 isw71002 cell14 lrdn2455 isw71002 cell14 lrdn2456 isw71002 cell14 lrdn2457 isw71002 cell14 lrdn2458 isw71002 cell14 lrdn2459 isw71004 cell14 lrdn2460 isw71004 cell14 lrdn2461 isw71100 cell14 lrdn2462 isw71100 cell14 lrdn2463 isw71100 cell14 lrdn2464 isw71100 cell14 lrdn2465 isw71100 cell14 lrdn2466 isw71100 cell14 lrdn2467 isw71100 cell14 lrdn2468 isw71100 cell14 lrdn2469 isw71104 cell14 lrdn2470 isw71104 cell14 lrdn2471 isw71104 cell14 lrdn2472 isw71104 cell14 lrdn2473 isw71100 cell14 lrdn2474 isw71100 cell14 lrdn2475 isw71102 cell14 lrdn2476 isw71102 cell14 lrdn2477 isw71104 cell14 lrdn2478 isw71104 cell14 lrdn2479 isw71104 cell14 lrdn2480 isw71104 cell14 lrdn2481 isw71102 cell14 lrdn2482 isw71102 cell14 lrdn2483 isw71102 cell14 lrdn2484 isw71102 cell14 lrdn2485 isw71102 cell14 lrdn2486 isw71102 cell14 lrdn2487 isw71102 cell14 lrdn2488 isw71102 cell14 lrdn2489 isw71104 cell14 lrdn2490 isw71104 cell14 lrdn2491 isw71200 cell14 lrdn2492 isw71200 cell14 lrdn2493 isw71200 cell14 lrdn2494 isw71200 cell14 lrdn2495 isw71200 cell14 lrdn2496 isw71200 cell14 lrdn2497 isw71200 cell14 lrdn2498 isw71200 cell14 lrdn2499 isw71204 cell14 lrdn2500 isw71204 cell14 lrdn2501 isw71204 cell14 lrdn2502 isw71204 cell14 lrdn2503 isw71200 cell14 lrdn2504 isw71200 cell14 lrdn2505 isw71202 cell14 lrdn2506 isw71202 cell14 lrdn2507 isw71204 cell14 lrdn2508 isw71204 cell14 lrdn2509 isw71204 cell14 lrdn2510 isw71204 cell14 lrdn2511 isw71202 cell14 lrdn2512 isw71202 cell14 lrdn2513 isw71202 cell14 lrdn2514 isw71202 cell14 lrdn2515 isw71202 cell14 lrdn2516 isw71202 cell14 lrdn2517 isw71202 cell14 lrdn2518 isw71202 cell14 lrdn2519 isw71204 cell14 lrdn2520 isw71204 cell14 lrdn2521 isw80100 cell15 lrdn2522 isw80100 cell15 lrdn2523 isw80100 cell15 lrdn2524 isw80100 cell15 lrdn2525 isw80100 cell15 lrdn2526 isw80100 cell15 lrdn2527 isw80100 cell15 lrdn2528 isw80100 cell15 lrdn2529 isw80104 cell15 lrdn2530 isw80104 cell15 lrdn2531 isw80104 cell15 lrdn2532 isw80104 cell15 lrdn2533 isw80100 cell15 lrdn2534 isw80100 cell15 lrdn2535 isw80102 cell15 lrdn2536 isw80102 cell15 lrdn2537 isw80104 cell15 lrdn2538 isw80104 cell15 lrdn2539 isw80104 cell15 lrdn2540 isw80104 cell15 lrdn2541 isw80102 cell15 lrdn2542 isw80102 cell15 lrdn2543 isw80102 cell15 lrdn2544 isw80102 cell15 lrdn2545 isw80102 cell15 lrdn2546 isw80102 cell15 lrdn2547 isw80102 cell15 lrdn2548 isw80102 cell15 lrdn2549 isw80104 cell15 lrdn2550 isw80104 cell15 lrdn2551 isw80200 cell15 lrdn2552 isw80200 cell15 lrdn2553 isw80200 cell15 lrdn2554 isw80200 cell15 lrdn2555 isw80200 cell15 lrdn2556 isw80200 cell15 lrdn2557 isw80200 cell15 lrdn2558 isw80200 cell15 lrdn2559 isw80204 cell15 lrdn2560 isw80204 cell15 lrdn2561 isw80204 cell15 lrdn2562 isw80204 cell15 lrdn2563 isw80200 cell15 lrdn2564 isw80200 cell15 lrdn2565 isw80202 cell15 lrdn2566 isw80202 cell15 lrdn2567 isw80204 cell15 lrdn2568 isw80204 cell15 lrdn2569 isw80204 cell15 lrdn2570 isw80204 cell15 lrdn2571 isw80202 cell15 lrdn2572 isw80202 cell15 lrdn2573 isw80202 cell15 lrdn2574 isw80202 cell15 lrdn2575 isw80202 cell15 lrdn2576 isw80202 cell15 lrdn2577 isw80202 cell15 lrdn2578 isw80202 cell15 lrdn2579 isw80204 cell15 lrdn2580 isw80204 cell15 lrdn2581 isw80300 cell15 lrdn2582 isw80300 cell15 lrdn2583 isw80300 cell15 lrdn2584 isw80300 cell15 lrdn2585 isw80300 cell15 lrdn2586 isw80300 cell15 lrdn2587 isw80300 cell15 lrdn2588 isw80300 cell15 lrdn2589 isw80304 cell15 lrdn2590 isw80304 cell15 lrdn2591 isw80304 cell15 lrdn2592 isw80304 cell15 lrdn2593 isw80300 cell15 lrdn2594 isw80300 cell15 lrdn2595 isw80302 cell15 lrdn2596 isw80302 cell15 lrdn2597 isw80304 cell15 lrdn2598 isw80304 cell15 lrdn2599 isw80304 cell15 lrdn2600 isw80304 cell15 lrdn2601 isw80302 cell15 lrdn2602 isw80302 cell15 lrdn2603 isw80302 cell15 lrdn2604 isw80302 cell15 lrdn2605 isw80302 cell15 lrdn2606 isw80302 cell15 lrdn2607 isw80302 cell15 lrdn2608 isw80302 cell15 lrdn2609 isw80304 cell15 lrdn2610 isw80304 cell15 lrdn2611 isw80400 cell15 lrdn2612 isw80400 cell15 lrdn2613 isw80400 cell15 lrdn2614 isw80400 cell15 lrdn2615 isw80400 cell15 lrdn2616 isw80400 cell15 lrdn2617 isw80400 cell15 lrdn2618 isw80400 cell15 lrdn2619 isw80404 cell15 lrdn2620 isw80404 cell15 lrdn2621 isw80404 cell15 lrdn2622 isw80404 cell15 lrdn2623 isw80400 cell15 lrdn2624 isw80400 cell15 lrdn2625 isw80402 cell15 lrdn2626 isw80402 cell15 lrdn2627 isw80404 cell15 lrdn2628 isw80404 cell15 lrdn2629 isw80404 cell15 lrdn2630 isw80404 cell15 lrdn2631 isw80402 cell15 lrdn2632 isw80402 cell15 lrdn2633 isw80402 cell15 lrdn2634 isw80402 cell15 lrdn2635 isw80402 cell15 lrdn2636 isw80402 cell15 lrdn2637 isw80402 cell15 lrdn2638 isw80402 cell15 lrdn2639 isw80404 cell15 lrdn2640 isw80404 cell15 lrdn2641 isw80500 cell15 lrdn2642 isw80500 cell15 lrdn2643 isw80500 cell15 lrdn2644 isw80500 cell15 lrdn2645 isw80500 cell15 lrdn2646 isw80500 cell15 lrdn2647 isw80500 cell15 lrdn2648 isw80500 cell15 lrdn2649 isw80504 cell15 lrdn2650 isw80504 cell15 lrdn2651 isw80504 cell15 lrdn2652 isw80504 cell15 lrdn2653 isw80500 cell15 lrdn2654 isw80500 cell15 lrdn2655 isw80502 cell15 lrdn2656 isw80502 cell15 lrdn2657 isw80504 cell15 lrdn2658 isw80504 cell15 lrdn2659 isw80504 cell15 lrdn2660 isw80504 cell15 lrdn2661 isw80502 cell15 lrdn2662 isw80502 cell15 lrdn2663 isw80502 cell15 lrdn2664 isw80502 cell15 lrdn2665 isw80502 cell15 lrdn2666 isw80502 cell15 lrdn2667 isw80502 cell15 lrdn2668 isw80502 cell15 lrdn2669 isw80504 cell15 lrdn2670 isw80504 cell15 lrdn2671 isw80600 cell15 lrdn2672 isw80600 cell15 lrdn2673 isw80600 cell15 lrdn2674 isw80600 cell15 lrdn2675 isw80600 cell15 lrdn2676 isw80600 cell15 lrdn2677 isw80600 cell15 lrdn2678 isw80600 cell15 lrdn2679 isw80604 cell15 lrdn2680 isw80604 cell15 lrdn2681 isw80604 cell15 lrdn2682 isw80604 cell15 lrdn2683 isw80600 cell15 lrdn2684 isw80600 cell15 lrdn2685 isw80602 cell15 lrdn2686 isw80602 cell15 lrdn2687 isw80604 cell15 lrdn2688 isw80604 cell15 lrdn2689 isw80604 cell15 lrdn2690 isw80604 cell15 lrdn2691 isw80602 cell15 lrdn2692 isw80602 cell15 lrdn2693 isw80602 cell15 lrdn2694 isw80602 cell15 lrdn2695 isw80602 cell15 lrdn2696 isw80602 cell15 lrdn2697 isw80602 cell15 lrdn2698 isw80602 cell15 lrdn2699 isw80604 cell15 lrdn2700 isw80604 cell15 lrdn2701 isw80700 cell16 lrdn2702 isw80700 cell16 lrdn2703 isw80700 cell16 lrdn2704 isw80700 cell16 lrdn2705 isw80700 cell16 lrdn2706 isw80700 cell16 lrdn2707 isw80700 cell16 lrdn2708 isw80700 cell16 lrdn2709 isw80704 cell16 lrdn2710 isw80704 cell16 lrdn2711 isw80704 cell16 lrdn2712 isw80704 cell16 lrdn2713 isw80700 cell16 lrdn2714 isw80700 cell16 lrdn2715 isw80702 cell16 lrdn2716 isw80702 cell16 lrdn2717 isw80704 cell16 lrdn2718 isw80704 cell16 lrdn2719 isw80704 cell16 lrdn2720 isw80704 cell16 lrdn2721 isw80702 cell16 lrdn2722 isw80702 cell16 lrdn2723 isw80702 cell16 lrdn2724 isw80702 cell16 lrdn2725 isw80702 cell16 lrdn2726 isw80702 cell16 lrdn2727 isw80702 cell16 lrdn2728 isw80702 cell16 lrdn2729 isw80704 cell16 lrdn2730 isw80704 cell16 lrdn2731 isw80800 cell16 lrdn2732 isw80800 cell16 lrdn2733 isw80800 cell16 lrdn2734 isw80800 cell16 lrdn2735 isw80800 cell16 lrdn2736 isw80800 cell16 lrdn2737 isw80800 cell16 lrdn2738 isw80800 cell16 lrdn2739 isw80804 cell16 lrdn2740 isw80804 cell16 lrdn2741 isw80804 cell16 lrdn2742 isw80804 cell16 lrdn2743 isw80800 cell16 lrdn2744 isw80800 cell16 lrdn2745 isw80802 cell16 lrdn2746 isw80802 cell16 lrdn2747 isw80804 cell16 lrdn2748 isw80804 cell16 lrdn2749 isw80804 cell16 lrdn2750 isw80804 cell16 lrdn2751 isw80802 cell16 lrdn2752 isw80802 cell16 lrdn2753 isw80802 cell16 lrdn2754 isw80802 cell16 lrdn2755 isw80802 cell16 lrdn2756 isw80802 cell16 lrdn2757 isw80802 cell16 lrdn2758 isw80802 cell16 lrdn2759 isw80804 cell16 lrdn2760 isw80804 cell16 lrdn2761 isw80900 cell16 lrdn2762 isw80900 cell16 lrdn2763 isw80900 cell16 lrdn2764 isw80900 cell16 lrdn2765 isw80900 cell16 lrdn2766 isw80900 cell16 lrdn2767 isw80900 cell16 lrdn2768 isw80900 cell16 lrdn2769 isw80904 cell16 lrdn2770 isw80904 cell16 lrdn2771 isw80904 cell16 lrdn2772 isw80904 cell16 lrdn2773 isw80900 cell16 lrdn2774 isw80900 cell16 lrdn2775 isw80902 cell16 lrdn2776 isw80902 cell16 lrdn2777 isw80904 cell16 lrdn2778 isw80904 cell16 lrdn2779 isw80904 cell16 lrdn2780 isw80904 cell16 lrdn2781 isw80902 cell16 lrdn2782 isw80902 cell16 lrdn2783 isw80902 cell16 lrdn2784 isw80902 cell16 lrdn2785 isw80902 cell16 lrdn2786 isw80902 cell16 lrdn2787 isw80902 cell16 lrdn2788 isw80902 cell16 lrdn2789 isw80904 cell16 lrdn2790 isw80904 cell16 lrdn2791 isw81000 cell16 lrdn2792 isw81000 cell16 lrdn2793 isw81000 cell16 lrdn2794 isw81000 cell16 lrdn2795 isw81000 cell16 lrdn2796 isw81000 cell16 lrdn2797 isw81000 cell16 lrdn2798 isw81000 cell16 lrdn2799 isw81004 cell16 lrdn2800 isw81004 cell16 lrdn2801 isw81004 cell16 lrdn2802 isw81004 cell16 lrdn2803 isw81000 cell16 lrdn2804 isw81000 cell16 lrdn2805 isw81002 cell16 lrdn2806 isw81002 cell16 lrdn2807 isw81004 cell16 lrdn2808 isw81004 cell16 lrdn2809 isw81004 cell16 lrdn2810 isw81004 cell16 lrdn2811 isw81002 cell16 lrdn2812 isw81002 cell16 lrdn2813 isw81002 cell16 lrdn2814 isw81002 cell16 lrdn2815 isw81002 cell16 lrdn2816 isw81002 cell16 lrdn2817 isw81002 cell16 lrdn2818 isw81002 cell16 lrdn2819 isw81004 cell16 lrdn2820 isw81004 cell16 lrdn2821 isw81100 cell16 lrdn2822 isw81100 cell16 lrdn2823 isw81100 cell16 lrdn2824 isw81100 cell16 lrdn2825 isw81100 cell16 lrdn2826 isw81100 cell16 lrdn2827 isw81100 cell16 lrdn2828 isw81100 cell16 lrdn2829 isw81104 cell16 lrdn2830 isw81104 cell16 lrdn2831 isw81104 cell16 lrdn2832 isw81104 cell16 lrdn2833 isw81100 cell16 lrdn2834 isw81100 cell16 lrdn2835 isw81102 cell16 lrdn2836 isw81102 cell16 lrdn2837 isw81104 cell16 lrdn2838 isw81104 cell16 lrdn2839 isw81104 cell16 lrdn2840 isw81104 cell16 lrdn2841 isw81102 cell16 lrdn2842 isw81102 cell16 lrdn2843 isw81102 cell16 lrdn2844 isw81102 cell16 lrdn2845 isw81102 cell16 lrdn2846 isw81102 cell16 lrdn2847 isw81102 cell16 lrdn2848 isw81102 cell16 lrdn2849 isw81104 cell16 lrdn2850 isw81104 cell16 lrdn2851 isw81200 cell16 lrdn2852 isw81200 cell16 lrdn2853 isw81200 cell16 lrdn2854 isw81200 cell16 lrdn2855 isw81200 cell16 lrdn2856 isw81200 cell16 lrdn2857 isw81200 cell16 lrdn2858 isw81200 cell16 lrdn2859 isw81204 cell16 lrdn2860 isw81204 cell16 lrdn2861 isw81204 cell16 lrdn2862 isw81204 cell16 lrdn2863 isw81200 cell16 lrdn2864 isw81200 cell16 lrdn2865 isw81202 cell16 lrdn2866 isw81202 cell16 lrdn2867 isw81204 cell16 lrdn2868 isw81204 cell16 lrdn2869 isw81204 cell16 lrdn2870 isw81204 cell16 lrdn2871 isw81202 cell16 lrdn2872 isw81202 cell16 lrdn2873 isw81202 cell16 lrdn2874 isw81202 cell16 lrdn2875 isw81202 cell16 lrdn2876 isw81202 cell16 lrdn2877 isw81202 cell16 lrdn2878 isw81202 cell16 lrdn2879 isw81204 cell16 lrdn2880 isw81204 cell16 lrdn2881 isw90100 cell17 lrdn2882 isw90100 cell17 lrdn2883 isw90100 cell17 lrdn2884 isw90100 cell17 lrdn2885 isw90100 cell17 lrdn2886 isw90100 cell17 lrdn2887 isw90100 cell17 lrdn2888 isw90100 cell17 lrdn2889 isw90104 cell17 lrdn2890 isw90104 cell17 lrdn2891 isw90104 cell17 lrdn2892 isw90104 cell17 lrdn2893 isw90100 cell17 lrdn2894 isw90100 cell17 lrdn2895 isw90102 cell17 lrdn2896 isw90102 cell17 lrdn2897 isw90104 cell17 lrdn2898 isw90104 cell17 lrdn2899 isw90104 cell17 lrdn2900 isw90104 cell17 lrdn2901 isw90102 cell17 lrdn2902 isw90102 cell17 lrdn2903 isw90102 cell17 lrdn2904 isw90102 cell17 lrdn2905 isw90102 cell17 lrdn2906 isw90102 cell17 lrdn2907 isw90102 cell17 lrdn2908 isw90102 cell17 lrdn2909 isw90104 cell17 lrdn2910 isw90104 cell17 lrdn2911 isw90200 cell17 lrdn2912 isw90200 cell17 lrdn2913 isw90200 cell17 lrdn2914 isw90200 cell17 lrdn2915 isw90200 cell17 lrdn2916 isw90200 cell17 lrdn2917 isw90200 cell17 lrdn2918 isw90200 cell17 lrdn2919 isw90204 cell17 lrdn2920 isw90204 cell17 lrdn2921 isw90204 cell17 lrdn2922 isw90204 cell17 lrdn2923 isw90200 cell17 lrdn2924 isw90200 cell17 lrdn2925 isw90202 cell17 lrdn2926 isw90202 cell17 lrdn2927 isw90204 cell17 lrdn2928 isw90204 cell17 lrdn2929 isw90204 cell17 lrdn2930 isw90204 cell17 lrdn2931 isw90202 cell17 lrdn2932 isw90202 cell17 lrdn2933 isw90202 cell17 lrdn2934 isw90202 cell17 lrdn2935 isw90202 cell17 lrdn2936 isw90202 cell17 lrdn2937 isw90202 cell17 lrdn2938 isw90202 cell17 lrdn2939 isw90204 cell17 lrdn2940 isw90204 cell17 lrdn2941 isw90300 cell17 lrdn2942 isw90300 cell17 lrdn2943 isw90300 cell17 lrdn2944 isw90300 cell17 lrdn2945 isw90300 cell17 lrdn2946 isw90300 cell17 lrdn2947 isw90300 cell17 lrdn2948 isw90300 cell17 lrdn2949 isw90304 cell17 lrdn2950 isw90304 cell17 lrdn2951 isw90304 cell17 lrdn2952 isw90304 cell17 lrdn2953 isw90300 cell17 lrdn2954 isw90300 cell17 lrdn2955 isw90302 cell17 lrdn2956 isw90302 cell17 lrdn2957 isw90304 cell17 lrdn2958 isw90304 cell17 lrdn2959 isw90304 cell17 lrdn2960 isw90304 cell17 lrdn2961 isw90302 cell17 lrdn2962 isw90302 cell17 lrdn2963 isw90302 cell17 lrdn2964 isw90302 cell17 lrdn2965 isw90302 cell17 lrdn2966 isw90302 cell17 lrdn2967 isw90302 cell17 lrdn2968 isw90302 cell17 lrdn2969 isw90304 cell17 lrdn2970 isw90304 cell17 lrdn2971 isw90400 cell17 lrdn2972 isw90400 cell17 lrdn2973 isw90400 cell17 lrdn2974 isw90400 cell17 lrdn2975 isw90400 cell17 lrdn2976 isw90400 cell17 lrdn2977 isw90400 cell17 lrdn2978 isw90400 cell17 lrdn2979 isw90404 cell17 lrdn2980 isw90404 cell17 lrdn2981 isw90404 cell17 lrdn2982 isw90404 cell17 lrdn2983 isw90400 cell17 lrdn2984 isw90400 cell17 lrdn2985 isw90402 cell17 lrdn2986 isw90402 cell17 lrdn2987 isw90404 cell17 lrdn2988 isw90404 cell17 lrdn2989 isw90404 cell17 lrdn2990 isw90404 cell17 lrdn2991 isw90402 cell17 lrdn2992 isw90402 cell17 lrdn2993 isw90402 cell17 lrdn2994 isw90402 cell17 lrdn2995 isw90402 cell17 lrdn2996 isw90402 cell17 lrdn2997 isw90402 cell17 lrdn2998 isw90402 cell17 lrdn2999 isw90404 cell17 lrdn3000 isw90404 cell17 lrdn3001 isw90500 cell17 lrdn3002 isw90500 cell17 lrdn3003 isw90500 cell17 lrdn3004 isw90500 cell17 lrdn3005 isw90500 cell17 lrdn3006 isw90500 cell17 lrdn3007 isw90500 cell17 lrdn3008 isw90500 cell17 lrdn3009 isw90504 cell17 lrdn3010 isw90504 cell17 lrdn3011 isw90504 cell17 lrdn3012 isw90504 cell17 lrdn3013 isw90500 cell17 lrdn3014 isw90500 cell17 lrdn3015 isw90502 cell17 lrdn3016 isw90502 cell17 lrdn3017 isw90504 cell17 lrdn3018 isw90504 cell17 lrdn3019 isw90504 cell17 lrdn3020 isw90504 cell17 lrdn3021 isw90502 cell17 lrdn3022 isw90502 cell17 lrdn3023 isw90502 cell17 lrdn3024 isw90502 cell17 lrdn3025 isw90502 cell17 lrdn3026 isw90502 cell17 lrdn3027 isw90502 cell17 lrdn3028 isw90502 cell17 lrdn3029 isw90504 cell17 lrdn3030 isw90504 cell17 lrdn3031 isw90600 cell17 lrdn3032 isw90600 cell17 lrdn3033 isw90600 cell17 lrdn3034 isw90600 cell17 lrdn3035 isw90600 cell17 lrdn3036 isw90600 cell17 lrdn3037 isw90600 cell17 lrdn3038 isw90600 cell17 lrdn3039 isw90604 cell17 lrdn3040 isw90604 cell17 lrdn3041 isw90604 cell17 lrdn3042 isw90604 cell17 lrdn3043 isw90600 cell17 lrdn3044 isw90600 cell17 lrdn3045 isw90602 cell17 lrdn3046 isw90602 cell17 lrdn3047 isw90604 cell17 lrdn3048 isw90604 cell17 lrdn3049 isw90604 cell17 lrdn3050 isw90604 cell17 lrdn3051 isw90602 cell17 lrdn3052 isw90602 cell17 lrdn3053 isw90602 cell17 lrdn3054 isw90602 cell17 lrdn3055 isw90602 cell17 lrdn3056 isw90602 cell17 lrdn3057 isw90602 cell17 lrdn3058 isw90602 cell17 lrdn3059 isw90604 cell17 lrdn3060 isw90604 cell17 lrdn3061 isw90700 cell18 lrdn3062 isw90700 cell18 lrdn3063 isw90700 cell18 lrdn3064 isw90700 cell18 lrdn3065 isw90700 cell18 lrdn3066 isw90700 cell18 lrdn3067 isw90700 cell18 lrdn3068 isw90700 cell18 lrdn3069 isw90704 cell18 lrdn3070 isw90704 cell18 lrdn3071 isw90704 cell18 lrdn3072 isw90704 cell18 lrdn3073 isw90700 cell18 lrdn3074 isw90700 cell18 lrdn3075 isw90702 cell18 lrdn3076 isw90702 cell18 lrdn3077 isw90704 cell18 lrdn3078 isw90704 cell18 lrdn3079 isw90704 cell18 lrdn3080 isw90704 cell18 lrdn3081 isw90702 cell18 lrdn3082 isw90702 cell18 lrdn3083 isw90702 cell18 lrdn3084 isw90702 cell18 lrdn3085 isw90702 cell18 lrdn3086 isw90702 cell18 lrdn3087 isw90702 cell18 lrdn3088 isw90702 cell18 lrdn3089 isw90704 cell18 lrdn3090 isw90704 cell18 lrdn3091 isw90800 cell18 lrdn3092 isw90800 cell18 lrdn3093 isw90800 cell18 lrdn3094 isw90800 cell18 lrdn3095 isw90800 cell18 lrdn3096 isw90800 cell18 lrdn3097 isw90800 cell18 lrdn3098 isw90800 cell18 lrdn3099 isw90804 cell18 lrdn3100 isw90804 cell18 lrdn3101 isw90804 cell18 lrdn3102 isw90804 cell18 lrdn3103 isw90800 cell18 lrdn3104 isw90800 cell18 lrdn3105 isw90802 cell18 lrdn3106 isw90802 cell18 lrdn3107 isw90804 cell18 lrdn3108 isw90804 cell18 lrdn3109 isw90804 cell18 lrdn3110 isw90804 cell18 lrdn3111 isw90802 cell18 lrdn3112 isw90802 cell18 lrdn3113 isw90802 cell18 lrdn3114 isw90802 cell18 lrdn3115 isw90802 cell18 lrdn3116 isw90802 cell18 lrdn3117 isw90802 cell18 lrdn3118 isw90802 cell18 lrdn3119 isw90804 cell18 lrdn3120 isw90804 cell18 lrdn3121 isw90900 cell18 lrdn3122 isw90900 cell18 lrdn3123 isw90900 cell18 lrdn3124 isw90900 cell18 lrdn3125 isw90900 cell18 lrdn3126 isw90900 cell18 lrdn3127 isw90900 cell18 lrdn3128 isw90900 cell18 lrdn3129 isw90904 cell18 lrdn3130 isw90904 cell18 lrdn3131 isw90904 cell18 lrdn3132 isw90904 cell18 lrdn3133 isw90900 cell18 lrdn3134 isw90900 cell18 lrdn3135 isw90902 cell18 lrdn3136 isw90902 cell18 lrdn3137 isw90904 cell18 lrdn3138 isw90904 cell18 lrdn3139 isw90904 cell18 lrdn3140 isw90904 cell18 lrdn3141 isw90902 cell18 lrdn3142 isw90902 cell18 lrdn3143 isw90902 cell18 lrdn3144 isw90902 cell18 lrdn3145 isw90902 cell18 lrdn3146 isw90902 cell18 lrdn3147 isw90902 cell18 lrdn3148 isw90902 cell18 lrdn3149 isw90904 cell18 lrdn3150 isw90904 cell18 lrdn3151 isw91000 cell18 lrdn3152 isw91000 cell18 lrdn3153 isw91000 cell18 lrdn3154 isw91000 cell18 lrdn3155 isw91000 cell18 lrdn3156 isw91000 cell18 lrdn3157 isw91000 cell18 lrdn3158 isw91000 cell18 lrdn3159 isw91004 cell18 lrdn3160 isw91004 cell18 lrdn3161 isw91004 cell18 lrdn3162 isw91004 cell18 lrdn3163 isw91000 cell18 lrdn3164 isw91000 cell18 lrdn3165 isw91002 cell18 lrdn3166 isw91002 cell18 lrdn3167 isw91004 cell18 lrdn3168 isw91004 cell18 lrdn3169 isw91004 cell18 lrdn3170 isw91004 cell18 lrdn3171 isw91002 cell18 lrdn3172 isw91002 cell18 lrdn3173 isw91002 cell18 lrdn3174 isw91002 cell18 lrdn3175 isw91002 cell18 lrdn3176 isw91002 cell18 lrdn3177 isw91002 cell18 lrdn3178 isw91002 cell18 lrdn3179 isw91004 cell18 lrdn3180 isw91004 cell18 lrdn3181 isw91100 cell18 lrdn3182 isw91100 cell18 lrdn3183 isw91100 cell18 lrdn3184 isw91100 cell18 lrdn3185 isw91100 cell18 lrdn3186 isw91100 cell18 lrdn3187 isw91100 cell18 lrdn3188 isw91100 cell18 lrdn3189 isw91104 cell18 lrdn3190 isw91104 cell18 lrdn3191 isw91104 cell18 lrdn3192 isw91104 cell18 lrdn3193 isw91100 cell18 lrdn3194 isw91100 cell18 lrdn3195 isw91102 cell18 lrdn3196 isw91102 cell18 lrdn3197 isw91104 cell18 lrdn3198 isw91104 cell18 lrdn3199 isw91104 cell18 lrdn3200 isw91104 cell18 lrdn3201 isw91102 cell18 lrdn3202 isw91102 cell18 lrdn3203 isw91102 cell18 lrdn3204 isw91102 cell18 lrdn3205 isw91102 cell18 lrdn3206 isw91102 cell18 lrdn3207 isw91102 cell18 lrdn3208 isw91102 cell18 lrdn3209 isw91104 cell18 lrdn3210 isw91104 cell18 lrdn3211 isw91200 cell18 lrdn3212 isw91200 cell18 lrdn3213 isw91200 cell18 lrdn3214 isw91200 cell18 lrdn3215 isw91200 cell18 lrdn3216 isw91200 cell18 lrdn3217 isw91200 cell18 lrdn3218 isw91200 cell18 lrdn3219 isw91204 cell18 lrdn3220 isw91204 cell18 lrdn3221 isw91204 cell18 lrdn3222 isw91204 cell18 lrdn3223 isw91200 cell18 lrdn3224 isw91200 cell18 lrdn3225 isw91202 cell18 lrdn3226 isw91202 cell18 lrdn3227 isw91204 cell18 lrdn3228 isw91204 cell18 lrdn3229 isw91204 cell18 lrdn3230 isw91204 cell18 lrdn3231 isw91202 cell18 lrdn3232 isw91202 cell18 lrdn3233 isw91202 cell18 lrdn3234 isw91202 cell18 lrdn3235 isw91202 cell18 lrdn3236 isw91202 cell18 lrdn3237 isw91202 cell18 lrdn3238 isw91202 cell18 lrdn3239 isw91204 cell18 lrdn3240 isw91204 cell18 lrdn3241 isw100100 cell19 lrdn3242 isw100100 cell19 lrdn3243 isw100100 cell19 lrdn3244 isw100100 cell19 lrdn3245 isw100100 cell19 lrdn3246 isw100100 cell19 lrdn3247 isw100100 cell19 lrdn3248 isw100100 cell19 lrdn3249 isw100104 cell19 lrdn3250 isw100104 cell19 lrdn3251 isw100104 cell19 lrdn3252 isw100104 cell19 lrdn3253 isw100100 cell19 lrdn3254 isw100100 cell19 lrdn3255 isw100102 cell19 lrdn3256 isw100102 cell19 lrdn3257 isw100104 cell19 lrdn3258 isw100104 cell19 lrdn3259 isw100104 cell19 lrdn3260 isw100104 cell19 lrdn3261 isw100102 cell19 lrdn3262 isw100102 cell19 lrdn3263 isw100102 cell19 lrdn3264 isw100102 cell19 lrdn3265 isw100102 cell19 lrdn3266 isw100102 cell19 lrdn3267 isw100102 cell19 lrdn3268 isw100102 cell19 lrdn3269 isw100104 cell19 lrdn3270 isw100104 cell19 lrdn3271 isw100200 cell19 lrdn3272 isw100200 cell19 lrdn3273 isw100200 cell19 lrdn3274 isw100200 cell19 lrdn3275 isw100200 cell19 lrdn3276 isw100200 cell19 lrdn3277 isw100200 cell19 lrdn3278 isw100200 cell19 lrdn3279 isw100204 cell19 lrdn3280 isw100204 cell19 lrdn3281 isw100204 cell19 lrdn3282 isw100204 cell19 lrdn3283 isw100200 cell19 lrdn3284 isw100200 cell19 lrdn3285 isw100202 cell19 lrdn3286 isw100202 cell19 lrdn3287 isw100204 cell19 lrdn3288 isw100204 cell19 lrdn3289 isw100204 cell19 lrdn3290 isw100204 cell19 lrdn3291 isw100202 cell19 lrdn3292 isw100202 cell19 lrdn3293 isw100202 cell19 lrdn3294 isw100202 cell19 lrdn3295 isw100202 cell19 lrdn3296 isw100202 cell19 lrdn3297 isw100202 cell19 lrdn3298 isw100202 cell19 lrdn3299 isw100204 cell19 lrdn3300 isw100204 cell19 lrdn3301 isw100300 cell19 lrdn3302 isw100300 cell19 lrdn3303 isw100300 cell19 lrdn3304 isw100300 cell19 lrdn3305 isw100300 cell19 lrdn3306 isw100300 cell19 lrdn3307 isw100300 cell19 lrdn3308 isw100300 cell19 lrdn3309 isw100304 cell19 lrdn3310 isw100304 cell19 lrdn3311 isw100304 cell19 lrdn3312 isw100304 cell19 lrdn3313 isw100300 cell19 lrdn3314 isw100300 cell19 lrdn3315 isw100302 cell19 lrdn3316 isw100302 cell19 lrdn3317 isw100304 cell19 lrdn3318 isw100304 cell19 lrdn3319 isw100304 cell19 lrdn3320 isw100304 cell19 lrdn3321 isw100302 cell19 lrdn3322 isw100302 cell19 lrdn3323 isw100302 cell19 lrdn3324 isw100302 cell19 lrdn3325 isw100302 cell19 lrdn3326 isw100302 cell19 lrdn3327 isw100302 cell19 lrdn3328 isw100302 cell19 lrdn3329 isw100304 cell19 lrdn3330 isw100304 cell19 lrdn3331 isw100400 cell19 lrdn3332 isw100400 cell19 lrdn3333 isw100400 cell19 lrdn3334 isw100400 cell19 lrdn3335 isw100400 cell19 lrdn3336 isw100400 cell19 lrdn3337 isw100400 cell19 lrdn3338 isw100400 cell19 lrdn3339 isw100404 cell19 lrdn3340 isw100404 cell19 lrdn3341 isw100404 cell19 lrdn3342 isw100404 cell19 lrdn3343 isw100400 cell19 lrdn3344 isw100400 cell19 lrdn3345 isw100402 cell19 lrdn3346 isw100402 cell19 lrdn3347 isw100404 cell19 lrdn3348 isw100404 cell19 lrdn3349 isw100404 cell19 lrdn3350 isw100404 cell19 lrdn3351 isw100402 cell19 lrdn3352 isw100402 cell19 lrdn3353 isw100402 cell19 lrdn3354 isw100402 cell19 lrdn3355 isw100402 cell19 lrdn3356 isw100402 cell19 lrdn3357 isw100402 cell19 lrdn3358 isw100402 cell19 lrdn3359 isw100404 cell19 lrdn3360 isw100404 cell19 lrdn3361 isw100500 cell19 lrdn3362 isw100500 cell19 lrdn3363 isw100500 cell19 lrdn3364 isw100500 cell19 lrdn3365 isw100500 cell19 lrdn3366 isw100500 cell19 lrdn3367 isw100500 cell19 lrdn3368 isw100500 cell19 lrdn3369 isw100504 cell19 lrdn3370 isw100504 cell19 lrdn3371 isw100504 cell19 lrdn3372 isw100504 cell19 lrdn3373 isw100500 cell19 lrdn3374 isw100500 cell19 lrdn3375 isw100502 cell19 lrdn3376 isw100502 cell19 lrdn3377 isw100504 cell19 lrdn3378 isw100504 cell19 lrdn3379 isw100504 cell19 lrdn3380 isw100504 cell19 lrdn3381 isw100502 cell19 lrdn3382 isw100502 cell19 lrdn3383 isw100502 cell19 lrdn3384 isw100502 cell19 lrdn3385 isw100502 cell19 lrdn3386 isw100502 cell19 lrdn3387 isw100502 cell19 lrdn3388 isw100502 cell19 lrdn3389 isw100504 cell19 lrdn3390 isw100504 cell19 lrdn3391 isw100600 cell19 lrdn3392 isw100600 cell19 lrdn3393 isw100600 cell19 lrdn3394 isw100600 cell19 lrdn3395 isw100600 cell19 lrdn3396 isw100600 cell19 lrdn3397 isw100600 cell19 lrdn3398 isw100600 cell19 lrdn3399 isw100604 cell19 lrdn3400 isw100604 cell19 lrdn3401 isw100604 cell19 lrdn3402 isw100604 cell19 lrdn3403 isw100600 cell19 lrdn3404 isw100600 cell19 lrdn3405 isw100602 cell19 lrdn3406 isw100602 cell19 lrdn3407 isw100604 cell19 lrdn3408 isw100604 cell19 lrdn3409 isw100604 cell19 lrdn3410 isw100604 cell19 lrdn3411 isw100602 cell19 lrdn3412 isw100602 cell19 lrdn3413 isw100602 cell19 lrdn3414 isw100602 cell19 lrdn3415 isw100602 cell19 lrdn3416 isw100602 cell19 lrdn3417 isw100602 cell19 lrdn3418 isw100602 cell19 lrdn3419 isw100604 cell19 lrdn3420 isw100604 cell19 lrdn3421 isw110100 cell20 lrdn3422 isw110100 cell20 lrdn3423 isw110100 cell20 lrdn3424 isw110100 cell20 lrdn3425 isw110100 cell20 lrdn3426 isw110100 cell20 lrdn3427 isw110100 cell20 lrdn3428 isw110100 cell20 lrdn3429 isw110104 cell20 lrdn3430 isw110104 cell20 lrdn3431 isw110104 cell20 lrdn3432 isw110104 cell20 lrdn3433 isw110100 cell20 lrdn3434 isw110100 cell20 lrdn3435 isw110102 cell20 lrdn3436 isw110102 cell20 lrdn3437 isw110104 cell20 lrdn3438 isw110104 cell20 lrdn3439 isw110200 cell20 lrdn3440 isw110200 cell20 lrdn3441 isw110200 cell20 lrdn3442 isw110200 cell20 lrdn3443 isw110200 cell20 lrdn3444 isw110200 cell20 lrdn3445 isw110200 cell20 lrdn3446 isw110200 cell20 lrdn3447 isw110204 cell20 lrdn3448 isw110204 cell20 lrdn3449 isw110204 cell20 lrdn3450 isw110204 cell20 lrdn3451 isw110200 cell20 lrdn3452 isw110200 cell20 lrdn3453 isw110202 cell20 lrdn3454 isw110202 cell20 lrdn3455 isw110204 cell20 lrdn3456 isw110204 cell20 lrdn3457 isw110300 cell20 lrdn3458 isw110300 cell20 lrdn3459 isw110300 cell20 lrdn3460 isw110300 cell20 lrdn3461 isw110300 cell20 lrdn3462 isw110300 cell20 lrdn3463 isw110300 cell20 lrdn3464 isw110300 cell20 lrdn3465 isw110300 cell20 lrdn3466 isw110300 cell20 lrdn3467 isw110300 cell20 lrdn3468 isw110300 cell20 lrdn3469 isw110300 cell20 lrdn3470 isw110300 cell20 lrdn3471 isw110300 cell20 lrdn3472 isw110300 cell20 lrdn3473 isw110300 cell20 lrdn3474 isw110300 cell20 lrdn3475 isw110300 cell20 lrdn3476 isw110300 cell20 lrdn3477 isw110300 cell20 lrdn3478 isw110300 cell20 lrdn3479 isw110300 cell20 lrdn3480 isw110300 cell20 lrdn3481 isw110300 cell20 lrdn3482 isw110300 cell20 lrdn3483 isw110302 cell20 lrdn3484 isw110300 cell20 lrdn3485 isw110300 cell20 lrdn3486 isw110302 cell20 lrdn3487 isw110300 cell20 lrdn3488 isw110300 cell20 lrdn3489 isw110302 cell20 lrdn3490 isw110300 cell20 lrdn3491 isw110300 cell20 lrdn3492 isw110302 cell20 lrdn3493 isw110302 cell20 lrdn3494 isw110300 cell20 lrdn3495 isw110300 cell20 lrdn3496 isw110302 cell20 lrdn3497 isw110300 cell20 lrdn3498 isw110300 cell20 lrdn3499 isw110302 cell20 lrdn3500 isw110300 cell20 lrdn3501 isw110300 cell20 lrdn3502 isw110302 cell20 lrdn3503 isw110300 cell20 lrdn3504 isw110300 cell20 lrdn3505 isw110400 cell20 lrdn3506 isw110400 cell20 lrdn3507 isw110400 cell20 lrdn3508 isw110400 cell20 lrdn3509 isw110400 cell20 lrdn3510 isw110400 cell20 lrdn3511 isw110400 cell20 lrdn3512 isw110400 cell20 lrdn3513 isw110400 cell20 lrdn3514 isw110400 cell20 lrdn3515 isw110400 cell20 lrdn3516 isw110400 cell20 lrdn3517 isw110400 cell20 lrdn3518 isw110400 cell20 lrdn3519 isw110400 cell20 lrdn3520 isw110400 cell20 lrdn3521 isw110400 cell20 lrdn3522 isw110400 cell20 lrdn3523 isw110400 cell20 lrdn3524 isw110400 cell20 lrdn3525 isw110400 cell20 lrdn3526 isw110400 cell20 lrdn3527 isw110400 cell20 lrdn3528 isw110400 cell20 lrdn3529 isw110400 cell20 lrdn3530 isw110400 cell20 lrdn3531 isw110402 cell20 lrdn3532 isw110400 cell20 lrdn3533 isw110400 cell20 lrdn3534 isw110402 cell20 lrdn3535 isw110400 cell20 lrdn3536 isw110400 cell20 lrdn3537 isw110402 cell20 lrdn3538 isw110400 cell20 lrdn3539 isw110400 cell20 lrdn3540 isw110402 cell20 lrdn3541 isw110402 cell20 lrdn3542 isw110400 cell20 lrdn3543 isw110400 cell20 lrdn3544 isw110402 cell20 lrdn3545 isw110400 cell20 lrdn3546 isw110400 cell20 lrdn3547 isw110402 cell20 lrdn3548 isw110400 cell20 lrdn3549 isw110400 cell20 lrdn3550 isw110402 cell20 lrdn3551 isw110400 cell20 lrdn3552 isw110400 cell20 lrdn3553 isw110500 cell20 lrdn3554 isw110500 cell20 lrdn3555 isw110500 cell20 lrdn3556 isw110500 cell20 lrdn3557 isw110500 cell20 lrdn3558 isw110500 cell20 lrdn3559 isw110500 cell20 lrdn3560 isw110500 cell20 lrdn3561 isw110500 cell20 lrdn3562 isw110500 cell20 lrdn3563 isw110500 cell20 lrdn3564 isw110500 cell20 lrdn3565 isw110500 cell20 lrdn3566 isw110500 cell20 lrdn3567 isw110500 cell20 lrdn3568 isw110500 cell20 lrdn3569 isw110500 cell20 lrdn3570 isw110500 cell20 lrdn3571 isw110500 cell20 lrdn3572 isw110500 cell20 lrdn3573 isw110500 cell20 lrdn3574 isw110500 cell20 lrdn3575 isw110500 cell20 lrdn3576 isw110500 cell20 lrdn3577 isw110500 cell20 lrdn3578 isw110500 cell20 lrdn3579 isw110502 cell20 lrdn3580 isw110500 cell20 lrdn3581 isw110500 cell20 lrdn3582 isw110502 cell20 lrdn3583 isw110500 cell20 lrdn3584 isw110500 cell20 lrdn3585 isw110502 cell20 lrdn3586 isw110500 cell20 lrdn3587 isw110500 cell20 lrdn3588 isw110502 cell20 lrdn3589 isw110502 cell20 lrdn3590 isw110500 cell20 lrdn3591 isw110500 cell20 lrdn3592 isw110502 cell20 lrdn3593 isw110500 cell20 lrdn3594 isw110500 cell20 lrdn3595 isw110502 cell20 lrdn3596 isw110500 cell20 lrdn3597 isw110500 cell20 lrdn3598 isw110502 cell20 lrdn3599 isw110500 cell20 lrdn3600 isw110500 cell20 lrdn3601 isw110600 cell20 lrdn3602 isw110600 cell20 lrdn3603 isw110600 cell20 lrdn3604 isw110600 cell20 lrdn3605 isw110600 cell20 lrdn3606 isw110600 cell20 lrdn3607 isw110600 cell20 lrdn3608 isw110600 cell20 lrdn3609 isw110600 cell20 lrdn3610 isw110600 cell20 lrdn3611 isw110600 cell20 lrdn3612 isw110600 cell20 lrdn3613 isw110600 cell20 lrdn3614 isw110600 cell20 lrdn3615 isw110600 cell20 lrdn3616 isw110600 cell20 lrdn3617 isw110600 cell20 lrdn3618 isw110600 cell20 lrdn3619 isw110600 cell20 lrdn3620 isw110600 cell20 lrdn3621 isw110600 cell20 lrdn3622 isw110600 cell20 lrdn3623 isw110600 cell20 lrdn3624 isw110600 cell20 lrdn3625 isw110600 cell20 lrdn3626 isw110600 cell20 lrdn3627 isw110602 cell20 lrdn3628 isw110600 cell20 lrdn3629 isw110600 cell20 lrdn3630 isw110602 cell20 lrdn3631 isw110600 cell20 lrdn3632 isw110600 cell20 lrdn3633 isw110602 cell20 lrdn3634 isw110600 cell20 lrdn3635 isw110600 cell20 lrdn3636 isw110602 cell20 lrdn3637 isw110602 cell20 lrdn3638 isw110600 cell20 lrdn3639 isw110600 cell20 lrdn3640 isw110602 cell20 lrdn3641 isw110600 cell20 lrdn3642 isw110600 cell20 lrdn3643 isw110602 cell20 lrdn3644 isw110600 cell20 lrdn3645 isw110600 cell20 lrdn3646 isw110602 cell20 lrdn3647 isw110600 cell20 lrdn3648 isw110600 cell20 lrdn3649 isw110700 cell20 lrdn3650 isw110700 cell20 lrdn3651 isw110700 cell20 lrdn3652 isw110700 cell20 lrdn3653 isw110700 cell20 lrdn3654 isw110700 cell20 lrdn3655 isw110700 cell20 lrdn3656 isw110700 cell20 lrdn3657 isw110700 cell20 lrdn3658 isw110700 cell20 lrdn3659 isw110700 cell20 lrdn3660 isw110700 cell20 lrdn3661 isw110700 cell20 lrdn3662 isw110700 cell20 lrdn3663 isw110700 cell20 lrdn3664 isw110700 cell20 lrdn3665 isw110700 cell20 lrdn3666 isw110700 cell20 lrdn3667 isw110700 cell20 lrdn3668 isw110700 cell20 lrdn3669 isw110700 cell20 lrdn3670 isw110700 cell20 lrdn3671 isw110700 cell20 lrdn3672 isw110700 cell20 lrdn3673 isw110700 cell20 lrdn3674 isw110700 cell20 lrdn3675 isw110702 cell20 lrdn3676 isw110700 cell20 lrdn3677 isw110700 cell20 lrdn3678 isw110702 cell20 lrdn3679 isw110700 cell20 lrdn3680 isw110700 cell20 lrdn3681 isw110702 cell20 lrdn3682 isw110700 cell20 lrdn3683 isw110700 cell20 lrdn3684 isw110702 cell20 lrdn3685 isw110702 cell20 lrdn3686 isw110700 cell20 lrdn3687 isw110700 cell20 lrdn3688 isw110702 cell20 lrdn3689 isw110700 cell20 lrdn3690 isw110700 cell20 lrdn3691 isw110702 cell20 lrdn3692 isw110700 cell20 lrdn3693 isw110700 cell20 lrdn3694 isw110702 cell20 lrdn3695 isw110700 cell20 lrdn3696 isw110700 cell20 lrdn3697 isw110800 cell20 lrdn3698 isw110800 cell20 lrdn3699 isw110800 cell20 lrdn3700 isw110800 cell20 lrdn3701 isw110800 cell20 lrdn3702 isw110800 cell20 lrdn3703 isw110800 cell20 lrdn3704 isw110800 cell20 lrdn3705 isw110800 cell20 lrdn3706 isw110800 cell20 lrdn3707 isw110800 cell20 lrdn3708 isw110800 cell20 lrdn3709 isw110800 cell20 lrdn3710 isw110800 cell20 lrdn3711 isw110800 cell20 lrdn3712 isw110800 cell20 lrdn3713 isw110800 cell20 lrdn3714 isw110800 cell20 lrdn3715 isw110800 cell20 lrdn3716 isw110800 cell20 lrdn3717 isw110800 cell20 lrdn3718 isw110800 cell20 lrdn3719 isw110800 cell20 lrdn3720 isw110800 cell20 lrdn3721 isw110800 cell20 lrdn3722 isw110800 cell20 lrdn3723 isw110802 cell20 lrdn3724 isw110800 cell20 lrdn3725 isw110800 cell20 lrdn3726 isw110802 cell20 lrdn3727 isw110800 cell20 lrdn3728 isw110800 cell20 lrdn3729 isw110802 cell20 lrdn3730 isw110800 cell20 lrdn3731 isw110800 cell20 lrdn3732 isw110802 cell20 lrdn3733 isw110802 cell20 lrdn3734 isw110800 cell20 lrdn3735 isw110800 cell20 lrdn3736 isw110802 cell20 lrdn3737 isw110800 cell20 lrdn3738 isw110800 cell20 lrdn3739 isw110802 cell20 lrdn3740 isw110800 cell20 lrdn3741 isw110800 cell20 lrdn3742 isw110802 cell20 lrdn3743 isw110800 cell20 lrdn3744 isw110800 cell20 lrdn3745 isw120100 cell21 lrdn3746 isw120100 cell21 lrdn3747 isw120100 cell21 lrdn3748 isw120100 cell21 lrdn3749 isw120100 cell21 lrdn3750 isw120100 cell21 lrdn3751 isw120100 cell21 lrdn3752 isw120100 cell21 lrdn3753 isw120100 cell21 lrdn3754 isw120100 cell21 lrdn3755 isw120100 cell21 lrdn3756 isw120100 cell21 lrdn3757 isw120100 cell21 lrdn3758 isw120100 cell21 lrdn3759 isw120100 cell21 lrdn3760 isw120100 cell21 lrdn3761 isw120100 cell21 lrdn3762 isw120100 cell21 lrdn3763 isw120100 cell21 lrdn3764 isw120100 cell21 lrdn3765 isw120100 cell21 lrdn3766 isw120100 cell21 lrdn3767 isw120100 cell21 lrdn3768 isw120100 cell21 lrdn3769 isw120100 cell21 lrdn3770 isw120100 cell21 lrdn3771 isw120102 cell21 lrdn3772 isw120100 cell21 lrdn3773 isw120100 cell21 lrdn3774 isw120102 cell21 lrdn3775 isw120100 cell21 lrdn3776 isw120100 cell21 lrdn3777 isw120102 cell21 lrdn3778 isw120100 cell21 lrdn3779 isw120100 cell21 lrdn3780 isw120102 cell21 lrdn3781 isw120102 cell21 lrdn3782 isw120100 cell21 lrdn3783 isw120100 cell21 lrdn3784 isw120102 cell21 lrdn3785 isw120100 cell21 lrdn3786 isw120100 cell21 lrdn3787 isw120102 cell21 lrdn3788 isw120100 cell21 lrdn3789 isw120100 cell21 lrdn3790 isw120102 cell21 lrdn3791 isw120100 cell21 lrdn3792 isw120100 cell21 lrdn3793 isw120102 cell21 lrdn3794 isw120102 cell21 lrdn3795 isw120102 cell21 lrdn3796 isw120102 cell21 lrdn3797 isw120102 cell21 lrdn3798 isw120102 cell21 lrdn3799 isw120102 cell21 lrdn3800 isw120102 cell21 lrdn3801 isw120102 cell21 lrdn3802 isw120102 cell21 lrdn3803 isw120102 cell21 lrdn3804 isw120102 cell21 lrdn3805 isw120102 cell21 lrdn3806 isw120102 cell21 lrdn3807 isw120102 cell21 lrdn3808 isw120102 cell21 lrdn3809 isw120102 cell21 lrdn3810 isw120102 cell21 lrdn3811 isw120102 cell21 lrdn3812 isw120102 cell21 lrdn3813 isw120102 cell21 lrdn3814 isw120102 cell21 lrdn3815 isw120102 cell21 lrdn3816 isw120102 cell21 lrdn3817 isw120102 cell21 lrdn3818 isw120102 cell21 lrdn3819 isw120102 cell21 lrdn3820 isw120102 cell21 lrdn3821 isw120102 cell21 lrdn3822 isw120102 cell21 lrdn3823 isw120200 cell21 lrdn3824 isw120200 cell21 lrdn3825 isw120200 cell21 lrdn3826 isw120200 cell21 lrdn3827 isw120200 cell21 lrdn3828 isw120200 cell21 lrdn3829 isw120200 cell21 lrdn3830 isw120200 cell21 lrdn3831 isw120200 cell21 lrdn3832 isw120200 cell21 lrdn3833 isw120200 cell21 lrdn3834 isw120200 cell21 lrdn3835 isw120200 cell21 lrdn3836 isw120200 cell21 lrdn3837 isw120200 cell21 lrdn3838 isw120200 cell21 lrdn3839 isw120200 cell21 lrdn3840 isw120200 cell21 lrdn3841 isw120200 cell21 lrdn3842 isw120200 cell21 lrdn3843 isw120200 cell21 lrdn3844 isw120200 cell21 lrdn3845 isw120200 cell21 lrdn3846 isw120200 cell21 lrdn3847 isw120200 cell21 lrdn3848 isw120200 cell21 lrdn3849 isw120202 cell21 lrdn3850 isw120200 cell21 lrdn3851 isw120200 cell21 lrdn3852 isw120202 cell21 lrdn3853 isw120200 cell21 lrdn3854 isw120200 cell21 lrdn3855 isw120202 cell21 lrdn3856 isw120200 cell21 lrdn3857 isw120200 cell21 lrdn3858 isw120202 cell21 lrdn3859 isw120202 cell21 lrdn3860 isw120200 cell21 lrdn3861 isw120200 cell21 lrdn3862 isw120202 cell21 lrdn3863 isw120200 cell21 lrdn3864 isw120200 cell21 lrdn3865 isw120202 cell21 lrdn3866 isw120200 cell21 lrdn3867 isw120200 cell21 lrdn3868 isw120202 cell21 lrdn3869 isw120200 cell21 lrdn3870 isw120200 cell21 lrdn3871 isw120202 cell21 lrdn3872 isw120202 cell21 lrdn3873 isw120202 cell21 lrdn3874 isw120202 cell21 lrdn3875 isw120202 cell21 lrdn3876 isw120202 cell21 lrdn3877 isw120202 cell21 lrdn3878 isw120202 cell21 lrdn3879 isw120202 cell21 lrdn3880 isw120202 cell21 lrdn3881 isw120202 cell21 lrdn3882 isw120202 cell21 lrdn3883 isw120202 cell21 lrdn3884 isw120202 cell21 lrdn3885 isw120202 cell21 lrdn3886 isw120202 cell21 lrdn3887 isw120202 cell21 lrdn3888 isw120202 cell21 lrdn3889 isw120202 cell21 lrdn3890 isw120202 cell21 lrdn3891 isw120202 cell21 lrdn3892 isw120202 cell21 lrdn3893 isw120202 cell21 lrdn3894 isw120202 cell21 lrdn3895 isw120202 cell21 lrdn3896 isw120202 cell21 lrdn3897 isw120202 cell21 lrdn3898 isw120202 cell21 lrdn3899 isw120202 cell21 lrdn3900 isw120202 cell21 lrdn3901 isw120300 cell21 lrdn3902 isw120300 cell21 lrdn3903 isw120300 cell21 lrdn3904 isw120300 cell21 lrdn3905 isw120300 cell21 lrdn3906 isw120300 cell21 lrdn3907 isw120300 cell21 lrdn3908 isw120300 cell21 lrdn3909 isw120300 cell21 lrdn3910 isw120300 cell21 lrdn3911 isw120300 cell21 lrdn3912 isw120300 cell21 lrdn3913 isw120300 cell21 lrdn3914 isw120300 cell21 lrdn3915 isw120300 cell21 lrdn3916 isw120300 cell21 lrdn3917 isw120300 cell21 lrdn3918 isw120300 cell21 lrdn3919 isw120300 cell21 lrdn3920 isw120300 cell21 lrdn3921 isw120300 cell21 lrdn3922 isw120300 cell21 lrdn3923 isw120300 cell21 lrdn3924 isw120300 cell21 lrdn3925 isw120300 cell21 lrdn3926 isw120300 cell21 lrdn3927 isw120302 cell21 lrdn3928 isw120300 cell21 lrdn3929 isw120300 cell21 lrdn3930 isw120302 cell21 lrdn3931 isw120300 cell21 lrdn3932 isw120300 cell21 lrdn3933 isw120302 cell21 lrdn3934 isw120300 cell21 lrdn3935 isw120300 cell21 lrdn3936 isw120302 cell21 lrdn3937 isw120302 cell21 lrdn3938 isw120300 cell21 lrdn3939 isw120300 cell21 lrdn3940 isw120302 cell21 lrdn3941 isw120300 cell21 lrdn3942 isw120300 cell21 lrdn3943 isw120302 cell21 lrdn3944 isw120300 cell21 lrdn3945 isw120300 cell21 lrdn3946 isw120302 cell21 lrdn3947 isw120300 cell21 lrdn3948 isw120300 cell21 lrdn3949 isw120302 cell21 lrdn3950 isw120302 cell21 lrdn3951 isw120302 cell21 lrdn3952 isw120302 cell21 lrdn3953 isw120302 cell21 lrdn3954 isw120302 cell21 lrdn3955 isw120302 cell21 lrdn3956 isw120302 cell21 lrdn3957 isw120302 cell21 lrdn3958 isw120302 cell21 lrdn3959 isw120302 cell21 lrdn3960 isw120302 cell21 lrdn3961 isw120302 cell21 lrdn3962 isw120302 cell21 lrdn3963 isw120302 cell21 lrdn3964 isw120302 cell21 lrdn3965 isw120302 cell21 lrdn3966 isw120302 cell21 lrdn3967 isw120302 cell21 lrdn3968 isw120302 cell21 lrdn3969 isw120302 cell21 lrdn3970 isw120302 cell21 lrdn3971 isw120302 cell21 lrdn3972 isw120302 cell21 lrdn3973 isw120302 cell21 lrdn3974 isw120302 cell21 lrdn3975 isw120302 cell21 lrdn3976 isw120302 cell21 lrdn3977 isw120302 cell21 lrdn3978 isw120302 cell21 lrdn3979 isw120400 cell21 lrdn3980 isw120400 cell21 lrdn3981 isw120400 cell21 lrdn3982 isw120400 cell21 lrdn3983 isw120400 cell21 lrdn3984 isw120400 cell21 lrdn3985 isw120400 cell21 lrdn3986 isw120400 cell21 lrdn3987 isw120400 cell21 lrdn3988 isw120400 cell21 lrdn3989 isw120400 cell21 lrdn3990 isw120400 cell21 lrdn3991 isw120400 cell21 lrdn3992 isw120400 cell21 lrdn3993 isw120400 cell21 lrdn3994 isw120400 cell21 lrdn3995 isw120400 cell21 lrdn3996 isw120400 cell21 lrdn3997 isw120400 cell21 lrdn3998 isw120400 cell21 lrdn3999 isw120400 cell21 lrdn4000 isw120400 cell21 lrdn4001 isw120400 cell21 lrdn4002 isw120400 cell21 lrdn4003 isw120400 cell21 lrdn4004 isw120400 cell21 lrdn4005 isw120402 cell21 lrdn4006 isw120400 cell21 lrdn4007 isw120400 cell21 lrdn4008 isw120402 cell21 lrdn4009 isw120400 cell21 lrdn4010 isw120400 cell21 lrdn4011 isw120402 cell21 lrdn4012 isw120400 cell21 lrdn4013 isw120400 cell21 lrdn4014 isw120402 cell21 lrdn4015 isw120402 cell21 lrdn4016 isw120400 cell21 lrdn4017 isw120400 cell21 lrdn4018 isw120402 cell21 lrdn4019 isw120400 cell21 lrdn4020 isw120400 cell21 lrdn4021 isw120402 cell21 lrdn4022 isw120400 cell21 lrdn4023 isw120400 cell21 lrdn4024 isw120402 cell21 lrdn4025 isw120400 cell21 lrdn4026 isw120400 cell21 lrdn4027 isw120402 cell21 lrdn4028 isw120402 cell21 lrdn4029 isw120402 cell21 lrdn4030 isw120402 cell21 lrdn4031 isw120402 cell21 lrdn4032 isw120402 cell21 lrdn4033 isw120402 cell21 lrdn4034 isw120402 cell21 lrdn4035 isw120402 cell21 lrdn4036 isw120402 cell21 lrdn4037 isw120402 cell21 lrdn4038 isw120402 cell21 lrdn4039 isw120402 cell21 lrdn4040 isw120402 cell21 lrdn4041 isw120402 cell21 lrdn4042 isw120402 cell21 lrdn4043 isw120402 cell21 lrdn4044 isw120402 cell21 lrdn4045 isw120402 cell21 lrdn4046 isw120402 cell21 lrdn4047 isw120402 cell21 lrdn4048 isw120402 cell21 lrdn4049 isw120402 cell21 lrdn4050 isw120402 cell21 lrdn4051 isw120402 cell21 lrdn4052 isw120402 cell21 lrdn4053 isw120402 cell21 lrdn4054 isw120402 cell21 lrdn4055 isw120402 cell21 lrdn4056 isw120402 cell21 lrdn4057 isw120500 cell21 lrdn4058 isw120500 cell21 lrdn4059 isw120500 cell21 lrdn4060 isw120500 cell21 lrdn4061 isw120500 cell21 lrdn4062 isw120500 cell21 lrdn4063 isw120500 cell21 lrdn4064 isw120500 cell21 lrdn4065 isw120500 cell21 lrdn4066 isw120500 cell21 lrdn4067 isw120500 cell21 lrdn4068 isw120500 cell21 lrdn4069 isw120500 cell21 lrdn4070 isw120500 cell21 lrdn4071 isw120500 cell21 lrdn4072 isw120500 cell21 lrdn4073 isw120500 cell21 lrdn4074 isw120500 cell21 lrdn4075 isw120500 cell21 lrdn4076 isw120500 cell21 lrdn4077 isw120500 cell21 lrdn4078 isw120500 cell21 lrdn4079 isw120500 cell21 lrdn4080 isw120500 cell21 lrdn4081 isw120500 cell21 lrdn4082 isw120500 cell21 lrdn4083 isw120502 cell21 lrdn4084 isw120500 cell21 lrdn4085 isw120500 cell21 lrdn4086 isw120502 cell21 lrdn4087 isw120500 cell21 lrdn4088 isw120500 cell21 lrdn4089 isw120502 cell21 lrdn4090 isw120500 cell21 lrdn4091 isw120500 cell21 lrdn4092 isw120502 cell21 lrdn4093 isw120502 cell21 lrdn4094 isw120500 cell21 lrdn4095 isw120500 cell21 lrdn4096 isw120502 cell21 lrdn4097 isw120500 cell21 lrdn4098 isw120500 cell21 lrdn4099 isw120502 cell21 lrdn4100 isw120500 cell21 lrdn4101 isw120500 cell21 lrdn4102 isw120502 cell21 lrdn4103 isw120500 cell21 lrdn4104 isw120500 cell21 lrdn4105 isw120502 cell21 lrdn4106 isw120502 cell21 lrdn4107 isw120502 cell21 lrdn4108 isw120502 cell21 lrdn4109 isw120502 cell21 lrdn4110 isw120502 cell21 lrdn4111 isw120502 cell21 lrdn4112 isw120502 cell21 lrdn4113 isw120502 cell21 lrdn4114 isw120502 cell21 lrdn4115 isw120502 cell21 lrdn4116 isw120502 cell21 lrdn4117 isw120502 cell21 lrdn4118 isw120502 cell21 lrdn4119 isw120502 cell21 lrdn4120 isw120502 cell21 lrdn4121 isw120502 cell21 lrdn4122 isw120502 cell21 lrdn4123 isw120502 cell21 lrdn4124 isw120502 cell21 lrdn4125 isw120502 cell21 lrdn4126 isw120502 cell21 lrdn4127 isw120502 cell21 lrdn4128 isw120502 cell21 lrdn4129 isw120502 cell21 lrdn4130 isw120502 cell21 lrdn4131 isw120502 cell21 lrdn4132 isw120502 cell21 lrdn4133 isw120502 cell21 lrdn4134 isw120502 cell21 lrdn4135 isw120600 cell21 lrdn4136 isw120600 cell21 lrdn4137 isw120600 cell21 lrdn4138 isw120600 cell21 lrdn4139 isw120600 cell21 lrdn4140 isw120600 cell21 lrdn4141 isw120600 cell21 lrdn4142 isw120600 cell21 lrdn4143 isw120600 cell21 lrdn4144 isw120600 cell21 lrdn4145 isw120600 cell21 lrdn4146 isw120600 cell21 lrdn4147 isw120600 cell21 lrdn4148 isw120600 cell21 lrdn4149 isw120600 cell21 lrdn4150 isw120600 cell21 lrdn4151 isw120600 cell21 lrdn4152 isw120600 cell21 lrdn4153 isw120600 cell21 lrdn4154 isw120600 cell21 lrdn4155 isw120600 cell21 lrdn4156 isw120600 cell21 lrdn4157 isw120600 cell21 lrdn4158 isw120600 cell21 lrdn4159 isw120600 cell21 lrdn4160 isw120600 cell21 lrdn4161 isw120602 cell21 lrdn4162 isw120600 cell21 lrdn4163 isw120600 cell21 lrdn4164 isw120602 cell21 lrdn4165 isw120600 cell21 lrdn4166 isw120600 cell21 lrdn4167 isw120602 cell21 lrdn4168 isw120600 cell21 lrdn4169 isw120600 cell21 lrdn4170 isw120602 cell21 lrdn4171 isw120602 cell21 lrdn4172 isw120600 cell21 lrdn4173 isw120600 cell21 lrdn4174 isw120602 cell21 lrdn4175 isw120600 cell21 lrdn4176 isw120600 cell21 lrdn4177 isw120602 cell21 lrdn4178 isw120600 cell21 lrdn4179 isw120600 cell21 lrdn4180 isw120602 cell21 lrdn4181 isw120600 cell21 lrdn4182 isw120600 cell21 lrdn4183 isw120602 cell21 lrdn4184 isw120602 cell21 lrdn4185 isw120602 cell21 lrdn4186 isw120602 cell21 lrdn4187 isw120602 cell21 lrdn4188 isw120602 cell21 lrdn4189 isw120602 cell21 lrdn4190 isw120602 cell21 lrdn4191 isw120602 cell21 lrdn4192 isw120602 cell21 lrdn4193 isw120602 cell21 lrdn4194 isw120602 cell21 lrdn4195 isw120602 cell21 lrdn4196 isw120602 cell21 lrdn4197 isw120602 cell21 lrdn4198 isw120602 cell21 lrdn4199 isw120602 cell21 lrdn4200 isw120602 cell21 lrdn4201 isw120602 cell21 lrdn4202 isw120602 cell21 lrdn4203 isw120602 cell21 lrdn4204 isw120602 cell21 lrdn4205 isw120602 cell21 lrdn4206 isw120602 cell21 lrdn4207 isw120602 cell21 lrdn4208 isw120602 cell21 lrdn4209 isw120602 cell21 lrdn4210 isw120602 cell21 lrdn4211 isw120602 cell21 lrdn4212 isw120602 cell21 lrdn4213 isw120700 cell21 lrdn4214 isw120700 cell21 lrdn4215 isw120700 cell21 lrdn4216 isw120700 cell21 lrdn4217 isw120700 cell21 lrdn4218 isw120700 cell21 lrdn4219 isw120700 cell21 lrdn4220 isw120700 cell21 lrdn4221 isw120700 cell21 lrdn4222 isw120700 cell21 lrdn4223 isw120700 cell21 lrdn4224 isw120700 cell21 lrdn4225 isw120700 cell21 lrdn4226 isw120700 cell21 lrdn4227 isw120700 cell21 lrdn4228 isw120700 cell21 lrdn4229 isw120700 cell21 lrdn4230 isw120700 cell21 lrdn4231 isw120700 cell21 lrdn4232 isw120700 cell21 lrdn4233 isw120700 cell21 lrdn4234 isw120700 cell21 lrdn4235 isw120700 cell21 lrdn4236 isw120700 cell21 lrdn4237 isw120700 cell21 lrdn4238 isw120700 cell21 lrdn4239 isw120702 cell21 lrdn4240 isw120700 cell21 lrdn4241 isw120700 cell21 lrdn4242 isw120702 cell21 lrdn4243 isw120700 cell21 lrdn4244 isw120700 cell21 lrdn4245 isw120702 cell21 lrdn4246 isw120700 cell21 lrdn4247 isw120700 cell21 lrdn4248 isw120702 cell21 lrdn4249 isw120702 cell21 lrdn4250 isw120700 cell21 lrdn4251 isw120700 cell21 lrdn4252 isw120702 cell21 lrdn4253 isw120700 cell21 lrdn4254 isw120700 cell21 lrdn4255 isw120702 cell21 lrdn4256 isw120700 cell21 lrdn4257 isw120700 cell21 lrdn4258 isw120702 cell21 lrdn4259 isw120700 cell21 lrdn4260 isw120700 cell21 lrdn4261 isw120702 cell21 lrdn4262 isw120702 cell21 lrdn4263 isw120702 cell21 lrdn4264 isw120702 cell21 lrdn4265 isw120702 cell21 lrdn4266 isw120702 cell21 lrdn4267 isw120702 cell21 lrdn4268 isw120702 cell21 lrdn4269 isw120702 cell21 lrdn4270 isw120702 cell21 lrdn4271 isw120702 cell21 lrdn4272 isw120702 cell21 lrdn4273 isw120702 cell21 lrdn4274 isw120702 cell21 lrdn4275 isw120702 cell21 lrdn4276 isw120702 cell21 lrdn4277 isw120702 cell21 lrdn4278 isw120702 cell21 lrdn4279 isw120702 cell21 lrdn4280 isw120702 cell21 lrdn4281 isw120702 cell21 lrdn4282 isw120702 cell21 lrdn4283 isw120702 cell21 lrdn4284 isw120702 cell21 lrdn4285 isw120702 cell21 lrdn4286 isw120702 cell21 lrdn4287 isw120702 cell21 lrdn4288 isw120702 cell21 lrdn4289 isw120702 cell21 lrdn4290 isw120702 cell21 lrdn4291 isw120800 cell21 lrdn4292 isw120800 cell21 lrdn4293 isw120800 cell21 lrdn4294 isw120800 cell21 lrdn4295 isw120800 cell21 lrdn4296 isw120800 cell21 lrdn4297 isw120800 cell21 lrdn4298 isw120800 cell21 lrdn4299 isw120800 cell21 lrdn4300 isw120800 cell21 lrdn4301 isw120800 cell21 lrdn4302 isw120800 cell21 lrdn4303 isw120800 cell21 lrdn4304 isw120800 cell21 lrdn4305 isw120800 cell21 lrdn4306 isw120800 cell21 lrdn4307 isw120800 cell21 lrdn4308 isw120800 cell21 lrdn4309 isw120800 cell21 lrdn4310 isw120800 cell21 lrdn4311 isw120800 cell21 lrdn4312 isw120800 cell21 lrdn4313 isw120800 cell21 lrdn4314 isw120800 cell21 lrdn4315 isw120800 cell21 lrdn4316 isw120800 cell21 lrdn4317 isw120802 cell21 lrdn4318 isw120800 cell21 lrdn4319 isw120800 cell21 lrdn4320 isw120802 cell21 lrdn4321 isw120800 cell21 lrdn4322 isw120800 cell21 lrdn4323 isw120802 cell21 lrdn4324 isw120800 cell21 lrdn4325 isw120800 cell21 lrdn4326 isw120802 cell21 lrdn4327 isw120802 cell21 lrdn4328 isw120800 cell21 lrdn4329 isw120800 cell21 lrdn4330 isw120802 cell21 lrdn4331 isw120800 cell21 lrdn4332 isw120800 cell21 lrdn4333 isw120802 cell21 lrdn4334 isw120800 cell21 lrdn4335 isw120800 cell21 lrdn4336 isw120802 cell21 lrdn4337 isw120800 cell21 lrdn4338 isw120800 cell21 lrdn4339 isw120802 cell21 lrdn4340 isw120802 cell21 lrdn4341 isw120802 cell21 lrdn4342 isw120802 cell21 lrdn4343 isw120802 cell21 lrdn4344 isw120802 cell21 lrdn4345 isw120802 cell21 lrdn4346 isw120802 cell21 lrdn4347 isw120802 cell21 lrdn4348 isw120802 cell21 lrdn4349 isw120802 cell21 lrdn4350 isw120802 cell21 lrdn4351 isw120802 cell21 lrdn4352 isw120802 cell21 lrdn4353 isw120802 cell21 lrdn4354 isw120802 cell21 lrdn4355 isw120802 cell21 lrdn4356 isw120802 cell21 lrdn4357 isw120802 cell21 lrdn4358 isw120802 cell21 lrdn4359 isw120802 cell21 lrdn4360 isw120802 cell21 lrdn4361 isw120802 cell21 lrdn4362 isw120802 cell21 lrdn4363 isw120802 cell21 lrdn4364 isw120802 cell21 lrdn4365 isw120802 cell21 lrdn4366 isw120802 cell21 lrdn4367 isw120802 cell21 lrdn4368 isw120802 cell21 lrdn4369 isw130100 cell22 lrdn4370 isw130100 cell22 lrdn4371 isw130100 cell22 lrdn4372 isw130100 cell22 lrdn4373 isw130100 cell22 lrdn4374 isw130100 cell22 lrdn4375 isw130100 cell22 lrdn4376 isw130100 cell22 lrdn4377 isw130100 cell22 lrdn4378 isw130100 cell22 lrdn4379 isw130100 cell22 lrdn4380 isw130100 cell22 lrdn4381 isw130100 cell22 lrdn4382 isw130100 cell22 lrdn4383 isw130100 cell22 lrdn4384 isw130100 cell22 lrdn4385 isw130100 cell22 lrdn4386 isw130100 cell22 lrdn4387 isw130100 cell22 lrdn4388 isw130100 cell22 lrdn4389 isw130100 cell22 lrdn4390 isw130100 cell22 lrdn4391 isw130100 cell22 lrdn4392 isw130100 cell22 lrdn4393 isw130100 cell22 lrdn4394 isw130100 cell22 lrdn4395 isw130102 cell22 lrdn4396 isw130100 cell22 lrdn4397 isw130100 cell22 lrdn4398 isw130102 cell22 lrdn4399 isw130100 cell22 lrdn4400 isw130100 cell22 lrdn4401 isw130102 cell22 lrdn4402 isw130100 cell22 lrdn4403 isw130100 cell22 lrdn4404 isw130102 cell22 lrdn4405 isw130102 cell22 lrdn4406 isw130100 cell22 lrdn4407 isw130100 cell22 lrdn4408 isw130102 cell22 lrdn4409 isw130100 cell22 lrdn4410 isw130100 cell22 lrdn4411 isw130102 cell22 lrdn4412 isw130100 cell22 lrdn4413 isw130100 cell22 lrdn4414 isw130102 cell22 lrdn4415 isw130100 cell22 lrdn4416 isw130100 cell22 lrdn4417 isw130102 cell22 lrdn4418 isw130102 cell22 lrdn4419 isw130102 cell22 lrdn4420 isw130102 cell22 lrdn4421 isw130102 cell22 lrdn4422 isw130102 cell22 lrdn4423 isw130102 cell22 lrdn4424 isw130102 cell22 lrdn4425 isw130102 cell22 lrdn4426 isw130102 cell22 lrdn4427 isw130102 cell22 lrdn4428 isw130102 cell22 lrdn4429 isw130102 cell22 lrdn4430 isw130102 cell22 lrdn4431 isw130102 cell22 lrdn4432 isw130102 cell22 lrdn4433 isw130102 cell22 lrdn4434 isw130102 cell22 lrdn4435 isw130102 cell22 lrdn4436 isw130102 cell22 lrdn4437 isw130102 cell22 lrdn4438 isw130102 cell22 lrdn4439 isw130102 cell22 lrdn4440 isw130102 cell22 lrdn4441 isw130102 cell22 lrdn4442 isw130102 cell22 lrdn4443 isw130102 cell22 lrdn4444 isw130102 cell22 lrdn4445 isw130102 cell22 lrdn4446 isw130102 cell22 lrdn4447 isw130200 cell22 lrdn4448 isw130200 cell22 lrdn4449 isw130200 cell22 lrdn4450 isw130200 cell22 lrdn4451 isw130200 cell22 lrdn4452 isw130200 cell22 lrdn4453 isw130200 cell22 lrdn4454 isw130200 cell22 lrdn4455 isw130200 cell22 lrdn4456 isw130200 cell22 lrdn4457 isw130200 cell22 lrdn4458 isw130200 cell22 lrdn4459 isw130200 cell22 lrdn4460 isw130200 cell22 lrdn4461 isw130200 cell22 lrdn4462 isw130200 cell22 lrdn4463 isw130200 cell22 lrdn4464 isw130200 cell22 lrdn4465 isw130200 cell22 lrdn4466 isw130200 cell22 lrdn4467 isw130200 cell22 lrdn4468 isw130200 cell22 lrdn4469 isw130200 cell22 lrdn4470 isw130200 cell22 lrdn4471 isw130200 cell22 lrdn4472 isw130200 cell22 lrdn4473 isw130202 cell22 lrdn4474 isw130200 cell22 lrdn4475 isw130200 cell22 lrdn4476 isw130202 cell22 lrdn4477 isw130200 cell22 lrdn4478 isw130200 cell22 lrdn4479 isw130202 cell22 lrdn4480 isw130200 cell22 lrdn4481 isw130200 cell22 lrdn4482 isw130202 cell22 lrdn4483 isw130202 cell22 lrdn4484 isw130200 cell22 lrdn4485 isw130200 cell22 lrdn4486 isw130202 cell22 lrdn4487 isw130200 cell22 lrdn4488 isw130200 cell22 lrdn4489 isw130202 cell22 lrdn4490 isw130200 cell22 lrdn4491 isw130200 cell22 lrdn4492 isw130202 cell22 lrdn4493 isw130200 cell22 lrdn4494 isw130200 cell22 lrdn4495 isw130202 cell22 lrdn4496 isw130202 cell22 lrdn4497 isw130202 cell22 lrdn4498 isw130202 cell22 lrdn4499 isw130202 cell22 lrdn4500 isw130202 cell22 lrdn4501 isw130202 cell22 lrdn4502 isw130202 cell22 lrdn4503 isw130202 cell22 lrdn4504 isw130202 cell22 lrdn4505 isw130202 cell22 lrdn4506 isw130202 cell22 lrdn4507 isw130202 cell22 lrdn4508 isw130202 cell22 lrdn4509 isw130202 cell22 lrdn4510 isw130202 cell22 lrdn4511 isw130202 cell22 lrdn4512 isw130202 cell22 lrdn4513 isw130202 cell22 lrdn4514 isw130202 cell22 lrdn4515 isw130202 cell22 lrdn4516 isw130202 cell22 lrdn4517 isw130202 cell22 lrdn4518 isw130202 cell22 lrdn4519 isw130202 cell22 lrdn4520 isw130202 cell22 lrdn4521 isw130202 cell22 lrdn4522 isw130202 cell22 lrdn4523 isw130202 cell22 lrdn4524 isw130202 cell22 lrdn4525 isw130300 cell22 lrdn4526 isw130300 cell22 lrdn4527 isw130300 cell22 lrdn4528 isw130300 cell22 lrdn4529 isw130300 cell22 lrdn4530 isw130300 cell22 lrdn4531 isw130300 cell22 lrdn4532 isw130300 cell22 lrdn4533 isw130300 cell22 lrdn4534 isw130300 cell22 lrdn4535 isw130300 cell22 lrdn4536 isw130300 cell22 lrdn4537 isw130300 cell22 lrdn4538 isw130300 cell22 lrdn4539 isw130300 cell22 lrdn4540 isw130300 cell22 lrdn4541 isw130300 cell22 lrdn4542 isw130300 cell22 lrdn4543 isw130300 cell22 lrdn4544 isw130300 cell22 lrdn4545 isw130300 cell22 lrdn4546 isw130300 cell22 lrdn4547 isw130300 cell22 lrdn4548 isw130300 cell22 lrdn4549 isw130300 cell22 lrdn4550 isw130300 cell22 lrdn4551 isw130302 cell22 lrdn4552 isw130300 cell22 lrdn4553 isw130300 cell22 lrdn4554 isw130302 cell22 lrdn4555 isw130300 cell22 lrdn4556 isw130300 cell22 lrdn4557 isw130302 cell22 lrdn4558 isw130300 cell22 lrdn4559 isw130300 cell22 lrdn4560 isw130302 cell22 lrdn4561 isw130302 cell22 lrdn4562 isw130300 cell22 lrdn4563 isw130300 cell22 lrdn4564 isw130302 cell22 lrdn4565 isw130300 cell22 lrdn4566 isw130300 cell22 lrdn4567 isw130302 cell22 lrdn4568 isw130300 cell22 lrdn4569 isw130300 cell22 lrdn4570 isw130302 cell22 lrdn4571 isw130300 cell22 lrdn4572 isw130300 cell22 lrdn4573 isw130302 cell22 lrdn4574 isw130302 cell22 lrdn4575 isw130302 cell22 lrdn4576 isw130302 cell22 lrdn4577 isw130302 cell22 lrdn4578 isw130302 cell22 lrdn4579 isw130302 cell22 lrdn4580 isw130302 cell22 lrdn4581 isw130302 cell22 lrdn4582 isw130302 cell22 lrdn4583 isw130302 cell22 lrdn4584 isw130302 cell22 lrdn4585 isw130302 cell22 lrdn4586 isw130302 cell22 lrdn4587 isw130302 cell22 lrdn4588 isw130302 cell22 lrdn4589 isw130302 cell22 lrdn4590 isw130302 cell22 lrdn4591 isw130302 cell22 lrdn4592 isw130302 cell22 lrdn4593 isw130302 cell22 lrdn4594 isw130302 cell22 lrdn4595 isw130302 cell22 lrdn4596 isw130302 cell22 lrdn4597 isw130302 cell22 lrdn4598 isw130302 cell22 lrdn4599 isw130302 cell22 lrdn4600 isw130302 cell22 lrdn4601 isw130302 cell22 lrdn4602 isw130302 cell22 lrdn4603 isw130400 cell22 lrdn4604 isw130400 cell22 lrdn4605 isw130400 cell22 lrdn4606 isw130400 cell22 lrdn4607 isw130400 cell22 lrdn4608 isw130400 cell22 lrdn4609 isw130400 cell22 lrdn4610 isw130400 cell22 lrdn4611 isw130400 cell22 lrdn4612 isw130400 cell22 lrdn4613 isw130400 cell22 lrdn4614 isw130400 cell22 lrdn4615 isw130400 cell22 lrdn4616 isw130400 cell22 lrdn4617 isw130400 cell22 lrdn4618 isw130400 cell22 lrdn4619 isw130400 cell22 lrdn4620 isw130400 cell22 lrdn4621 isw130400 cell22 lrdn4622 isw130400 cell22 lrdn4623 isw130400 cell22 lrdn4624 isw130400 cell22 lrdn4625 isw130400 cell22 lrdn4626 isw130400 cell22 lrdn4627 isw130400 cell22 lrdn4628 isw130400 cell22 lrdn4629 isw130402 cell22 lrdn4630 isw130400 cell22 lrdn4631 isw130400 cell22 lrdn4632 isw130402 cell22 lrdn4633 isw130400 cell22 lrdn4634 isw130400 cell22 lrdn4635 isw130402 cell22 lrdn4636 isw130400 cell22 lrdn4637 isw130400 cell22 lrdn4638 isw130402 cell22 lrdn4639 isw130402 cell22 lrdn4640 isw130400 cell22 lrdn4641 isw130400 cell22 lrdn4642 isw130402 cell22 lrdn4643 isw130400 cell22 lrdn4644 isw130400 cell22 lrdn4645 isw130402 cell22 lrdn4646 isw130400 cell22 lrdn4647 isw130400 cell22 lrdn4648 isw130402 cell22 lrdn4649 isw130400 cell22 lrdn4650 isw130400 cell22 lrdn4651 isw130402 cell22 lrdn4652 isw130402 cell22 lrdn4653 isw130402 cell22 lrdn4654 isw130402 cell22 lrdn4655 isw130402 cell22 lrdn4656 isw130402 cell22 lrdn4657 isw130402 cell22 lrdn4658 isw130402 cell22 lrdn4659 isw130402 cell22 lrdn4660 isw130402 cell22 lrdn4661 isw130402 cell22 lrdn4662 isw130402 cell22 lrdn4663 isw130402 cell22 lrdn4664 isw130402 cell22 lrdn4665 isw130402 cell22 lrdn4666 isw130402 cell22 lrdn4667 isw130402 cell22 lrdn4668 isw130402 cell22 lrdn4669 isw130402 cell22 lrdn4670 isw130402 cell22 lrdn4671 isw130402 cell22 lrdn4672 isw130402 cell22 lrdn4673 isw130402 cell22 lrdn4674 isw130402 cell22 lrdn4675 isw130402 cell22 lrdn4676 isw130402 cell22 lrdn4677 isw130402 cell22 lrdn4678 isw130402 cell22 lrdn4679 isw130402 cell22 lrdn4680 isw130402 cell22 lrdn4681 isw130500 cell22 lrdn4682 isw130500 cell22 lrdn4683 isw130500 cell22 lrdn4684 isw130500 cell22 lrdn4685 isw130500 cell22 lrdn4686 isw130500 cell22 lrdn4687 isw130500 cell22 lrdn4688 isw130500 cell22 lrdn4689 isw130500 cell22 lrdn4690 isw130500 cell22 lrdn4691 isw130500 cell22 lrdn4692 isw130500 cell22 lrdn4693 isw130500 cell22 lrdn4694 isw130500 cell22 lrdn4695 isw130500 cell22 lrdn4696 isw130500 cell22 lrdn4697 isw130500 cell22 lrdn4698 isw130500 cell22 lrdn4699 isw130500 cell22 lrdn4700 isw130500 cell22 lrdn4701 isw130500 cell22 lrdn4702 isw130500 cell22 lrdn4703 isw130500 cell22 lrdn4704 isw130500 cell22 lrdn4705 isw130500 cell22 lrdn4706 isw130500 cell22 lrdn4707 isw130502 cell22 lrdn4708 isw130500 cell22 lrdn4709 isw130500 cell22 lrdn4710 isw130502 cell22 lrdn4711 isw130500 cell22 lrdn4712 isw130500 cell22 lrdn4713 isw130502 cell22 lrdn4714 isw130500 cell22 lrdn4715 isw130500 cell22 lrdn4716 isw130502 cell22 lrdn4717 isw130502 cell22 lrdn4718 isw130500 cell22 lrdn4719 isw130500 cell22 lrdn4720 isw130502 cell22 lrdn4721 isw130500 cell22 lrdn4722 isw130500 cell22 lrdn4723 isw130502 cell22 lrdn4724 isw130500 cell22 lrdn4725 isw130500 cell22 lrdn4726 isw130502 cell22 lrdn4727 isw130500 cell22 lrdn4728 isw130500 cell22 lrdn4729 isw130502 cell22 lrdn4730 isw130502 cell22 lrdn4731 isw130502 cell22 lrdn4732 isw130502 cell22 lrdn4733 isw130502 cell22 lrdn4734 isw130502 cell22 lrdn4735 isw130502 cell22 lrdn4736 isw130502 cell22 lrdn4737 isw130502 cell22 lrdn4738 isw130502 cell22 lrdn4739 isw130502 cell22 lrdn4740 isw130502 cell22 lrdn4741 isw130502 cell22 lrdn4742 isw130502 cell22 lrdn4743 isw130502 cell22 lrdn4744 isw130502 cell22 lrdn4745 isw130502 cell22 lrdn4746 isw130502 cell22 lrdn4747 isw130502 cell22 lrdn4748 isw130502 cell22 lrdn4749 isw130502 cell22 lrdn4750 isw130502 cell22 lrdn4751 isw130502 cell22 lrdn4752 isw130502 cell22 lrdn4753 isw130502 cell22 lrdn4754 isw130502 cell22 lrdn4755 isw130502 cell22 lrdn4756 isw130502 cell22 lrdn4757 isw130502 cell22 lrdn4758 isw130502 cell22 lrdn4759 isw130600 cell22 lrdn4760 isw130600 cell22 lrdn4761 isw130600 cell22 lrdn4762 isw130600 cell22 lrdn4763 isw130600 cell22 lrdn4764 isw130600 cell22 lrdn4765 isw130600 cell22 lrdn4766 isw130600 cell22 lrdn4767 isw130600 cell22 lrdn4768 isw130600 cell22 lrdn4769 isw130600 cell22 lrdn4770 isw130600 cell22 lrdn4771 isw130600 cell22 lrdn4772 isw130600 cell22 lrdn4773 isw130600 cell22 lrdn4774 isw130600 cell22 lrdn4775 isw130600 cell22 lrdn4776 isw130600 cell22 lrdn4777 isw130600 cell22 lrdn4778 isw130600 cell22 lrdn4779 isw130600 cell22 lrdn4780 isw130600 cell22 lrdn4781 isw130600 cell22 lrdn4782 isw130600 cell22 lrdn4783 isw130600 cell22 lrdn4784 isw130600 cell22 lrdn4785 isw130602 cell22 lrdn4786 isw130600 cell22 lrdn4787 isw130600 cell22 lrdn4788 isw130602 cell22 lrdn4789 isw130600 cell22 lrdn4790 isw130600 cell22 lrdn4791 isw130602 cell22 lrdn4792 isw130600 cell22 lrdn4793 isw130600 cell22 lrdn4794 isw130602 cell22 lrdn4795 isw130602 cell22 lrdn4796 isw130600 cell22 lrdn4797 isw130600 cell22 lrdn4798 isw130602 cell22 lrdn4799 isw130600 cell22 lrdn4800 isw130600 cell22 lrdn4801 isw130602 cell22 lrdn4802 isw130600 cell22 lrdn4803 isw130600 cell22 lrdn4804 isw130602 cell22 lrdn4805 isw130600 cell22 lrdn4806 isw130600 cell22 lrdn4807 isw130602 cell22 lrdn4808 isw130602 cell22 lrdn4809 isw130602 cell22 lrdn4810 isw130602 cell22 lrdn4811 isw130602 cell22 lrdn4812 isw130602 cell22 lrdn4813 isw130602 cell22 lrdn4814 isw130602 cell22 lrdn4815 isw130602 cell22 lrdn4816 isw130602 cell22 lrdn4817 isw130602 cell22 lrdn4818 isw130602 cell22 lrdn4819 isw130602 cell22 lrdn4820 isw130602 cell22 lrdn4821 isw130602 cell22 lrdn4822 isw130602 cell22 lrdn4823 isw130602 cell22 lrdn4824 isw130602 cell22 lrdn4825 isw130602 cell22 lrdn4826 isw130602 cell22 lrdn4827 isw130602 cell22 lrdn4828 isw130602 cell22 lrdn4829 isw130602 cell22 lrdn4830 isw130602 cell22 lrdn4831 isw130602 cell22 lrdn4832 isw130602 cell22 lrdn4833 isw130602 cell22 lrdn4834 isw130602 cell22 lrdn4835 isw130602 cell22 lrdn4836 isw130602 cell22 lrdn4837 isw130700 cell22 lrdn4838 isw130700 cell22 lrdn4839 isw130700 cell22 lrdn4840 isw130700 cell22 lrdn4841 isw130700 cell22 lrdn4842 isw130700 cell22 lrdn4843 isw130700 cell22 lrdn4844 isw130700 cell22 lrdn4845 isw130700 cell22 lrdn4846 isw130700 cell22 lrdn4847 isw130700 cell22 lrdn4848 isw130700 cell22 lrdn4849 isw130700 cell22 lrdn4850 isw130700 cell22 lrdn4851 isw130700 cell22 lrdn4852 isw130700 cell22 lrdn4853 isw130700 cell22 lrdn4854 isw130700 cell22 lrdn4855 isw130700 cell22 lrdn4856 isw130700 cell22 lrdn4857 isw130700 cell22 lrdn4858 isw130700 cell22 lrdn4859 isw130700 cell22 lrdn4860 isw130700 cell22 lrdn4861 isw130700 cell22 lrdn4862 isw130700 cell22 lrdn4863 isw130702 cell22 lrdn4864 isw130700 cell22 lrdn4865 isw130700 cell22 lrdn4866 isw130702 cell22 lrdn4867 isw130700 cell22 lrdn4868 isw130700 cell22 lrdn4869 isw130702 cell22 lrdn4870 isw130700 cell22 lrdn4871 isw130700 cell22 lrdn4872 isw130702 cell22 lrdn4873 isw130702 cell22 lrdn4874 isw130700 cell22 lrdn4875 isw130700 cell22 lrdn4876 isw130702 cell22 lrdn4877 isw130700 cell22 lrdn4878 isw130700 cell22 lrdn4879 isw130702 cell22 lrdn4880 isw130700 cell22 lrdn4881 isw130700 cell22 lrdn4882 isw130702 cell22 lrdn4883 isw130700 cell22 lrdn4884 isw130700 cell22 lrdn4885 isw130702 cell22 lrdn4886 isw130702 cell22 lrdn4887 isw130702 cell22 lrdn4888 isw130702 cell22 lrdn4889 isw130702 cell22 lrdn4890 isw130702 cell22 lrdn4891 isw130702 cell22 lrdn4892 isw130702 cell22 lrdn4893 isw130702 cell22 lrdn4894 isw130702 cell22 lrdn4895 isw130702 cell22 lrdn4896 isw130702 cell22 lrdn4897 isw130702 cell22 lrdn4898 isw130702 cell22 lrdn4899 isw130702 cell22 lrdn4900 isw130702 cell22 lrdn4901 isw130702 cell22 lrdn4902 isw130702 cell22 lrdn4903 isw130702 cell22 lrdn4904 isw130702 cell22 lrdn4905 isw130702 cell22 lrdn4906 isw130702 cell22 lrdn4907 isw130702 cell22 lrdn4908 isw130702 cell22 lrdn4909 isw130702 cell22 lrdn4910 isw130702 cell22 lrdn4911 isw130702 cell22 lrdn4912 isw130702 cell22 lrdn4913 isw130702 cell22 lrdn4914 isw130702 cell22 lrdn4915 isw130800 cell22 lrdn4916 isw130800 cell22 lrdn4917 isw130800 cell22 lrdn4918 isw130800 cell22 lrdn4919 isw130800 cell22 lrdn4920 isw130800 cell22 lrdn4921 isw130800 cell22 lrdn4922 isw130800 cell22 lrdn4923 isw130800 cell22 lrdn4924 isw130800 cell22 lrdn4925 isw130800 cell22 lrdn4926 isw130800 cell22 lrdn4927 isw130800 cell22 lrdn4928 isw130800 cell22 lrdn4929 isw130800 cell22 lrdn4930 isw130800 cell22 lrdn4931 isw130800 cell22 lrdn4932 isw130800 cell22 lrdn4933 isw130800 cell22 lrdn4934 isw130800 cell22 lrdn4935 isw130800 cell22 lrdn4936 isw130800 cell22 lrdn4937 isw130800 cell22 lrdn4938 isw130800 cell22 lrdn4939 isw130800 cell22 lrdn4940 isw130800 cell22 lrdn4941 isw130802 cell22 lrdn4942 isw130800 cell22 lrdn4943 isw130800 cell22 lrdn4944 isw130802 cell22 lrdn4945 isw130800 cell22 lrdn4946 isw130800 cell22 lrdn4947 isw130802 cell22 lrdn4948 isw130800 cell22 lrdn4949 isw130800 cell22 lrdn4950 isw130802 cell22 lrdn4951 isw130802 cell22 lrdn4952 isw130800 cell22 lrdn4953 isw130800 cell22 lrdn4954 isw130802 cell22 lrdn4955 isw130800 cell22 lrdn4956 isw130800 cell22 lrdn4957 isw130802 cell22 lrdn4958 isw130800 cell22 lrdn4959 isw130800 cell22 lrdn4960 isw130802 cell22 lrdn4961 isw130800 cell22 lrdn4962 isw130800 cell22 lrdn4963 isw130802 cell22 lrdn4964 isw130802 cell22 lrdn4965 isw130802 cell22 lrdn4966 isw130802 cell22 lrdn4967 isw130802 cell22 lrdn4968 isw130802 cell22 lrdn4969 isw130802 cell22 lrdn4970 isw130802 cell22 lrdn4971 isw130802 cell22 lrdn4972 isw130802 cell22 lrdn4973 isw130802 cell22 lrdn4974 isw130802 cell22 lrdn4975 isw130802 cell22 lrdn4976 isw130802 cell22 lrdn4977 isw130802 cell22 lrdn4978 isw130802 cell22 lrdn4979 isw130802 cell22 lrdn4980 isw130802 cell22 lrdn4981 isw130802 cell22 lrdn4982 isw130802 cell22 lrdn4983 isw130802 cell22 lrdn4984 isw130802 cell22 lrdn4985 isw130802 cell22 lrdn4986 isw130802 cell22 lrdn4987 isw130802 cell22 lrdn4988 isw130802 cell22 lrdn4989 isw130802 cell22 lrdn4990 isw130802 cell22 lrdn4991 isw130802 cell22 lrdn4992 isw130802 cell22 --- # Unknown .. \_hpc\_containers\_card: Singularity and Apptainer Containers ==================================== On CINECA's HPC clusters, the containerization \*platforms\* available can be eighter \*\*Singularity\*\* or \*\*Apptainer\*\*. Both containerizaion \*platforms\* are specifically designed to run scientific applications on HPC resources, enabling users to have full control over their environment. Singularity and Apptainer containers can be used to package entire scientific workflows, software, libraries and data. This means that you don’t have to ask your cluster admin to install anything for you - you can put it in a Singularity or Apptainer container and run. The official Singularity documentation for its last release is available \`here \`\_ while the official Apptainer documentation for its last release is available \`here \`\_. Differences between Singularity and Apptainer --------------------------------------------- In this section basic information about the history of the Singularity project are provided in order to help users which have no prior experience with both Singularity and Apptainer to better understand the differences between those platforms. \* The Singularity project first begun as an open-source project in 2015 form a team of researchers at Lawrence Berkeley National Laboratory lead by Gregory Kurtzer. \* In February 2018, the original leader of the Singularity project founded the Sylabs company to provide commercial support for Singularity. \* In May 2020, Gregory Kurtzer left Sylabs but retained leadership of the Singularity open source project: this event cause a major fork inside the Singularity project. \* In May 2021 Sylabs made a fork of the project and called SingularityCE while in November 30, 2021 when the move into the Linux Fundation of the Singularity open-source project has been \`announced \`\_ the Apptainer project born. Currently, there are three products derived from the original Singularity project from 2015: \* Singularity: the commercial software supported by Sylabs. \* SingularityCE: the open-source, community edition software also supported by Sylabs. \* Apptainer: the fully open-source Singularity port under the Linux Fundation. From a user perspective, quoting the announcement of the beginning of the Apptainer project: "\*\*Apptainer IS Singularity\*\*". The Apptainer project makes no changes at the image format level. This means that default metadata within Singularity Image Format (SIF image) and their filesystems will retain the Singularity name without change ensuring that containers built with Apptainer will continue to work with installations of Singularity. Moreover, \`\`singularity\`\` as a command line link, is provided, and maintains as much of the CLI and environment functionality as possible. .. important:: As a direct consequence of all the information previously reported, during the rest of the documentation all the command examples always use the \`\`singularity\`\` command. How to build a Singularity or Apptainer container image on your local machine ----------------------------------------------------------------------------- In this section the building procedure of container images with one between Singularity or Apptainer on your local machine is explained. A Singularity container image can be built in differnt ways. The simplest command used to build is: .. code-block:: bash $ sudo singularity build \[local options...\] the \`\`build\`\` command can produce containers in 2 different output formats. Format can be specified by passing the fllowing \`\`build option\`\`: 1. \*\*default\*\*: a compressed read-only \*\*Singularity Image Format (SIF)\*\*, suitable for production. This is an \*immutable object\*. 2. \*\*sandbox\*\*: a writable \*\*(ch)root directory\*\* called \*sandbox\*, used for interactive development. To create those kind of output format use the \`\`--sandbox\`\` build option. The build \`\`spec target\`\` defines the method that \`\`build\`\` uses to create the container. All the \*methods\* are listed in the table: .. list-table:: :widths: 30 70 :header-rows: 1 \* - \*\*Build method\*\* - \*\*Commands\*\* \* - Beginning with library to build from the \`Container Library \`\_ - \`\`sudo singularity build library://path/to/container\_img\[:tag\]\`\` \* - Beginning with docker to build from \`Docker Hub \`\_ - \`\`sudo singularity build docker://path/to/container\_img\[:tag\]\`\` \* - Path to an existing container on your local machine - \`\`sudo singularity build --sandbox \`\` \* - Path to a directory to build from a \*sandbox\* - \`\`sudo singularity build \`\` \* - Path to a \`SingularityCE definition file \`\_ - \`\`sudo singularity build \`\` Since build can accept an existing container as a target and create a container in any of these two formats, you can convert an existing .sif container image to a sandbox and viceversa. Some experienced Docker users may be in possession of Docker images not available on any container registry (\*e.g.\* custom container images): those users will take benefits from the possibility to convert Docker images into Singluarity image files. .. dropdown:: Convert Docker container images into Singularity image files .. important:: Before following this procedure, ensure that both Docker and one between Singularity or Apptiner are installed on your local system: the installation instructions can be found on the \`Docker official documentation \`\_ and on the \`Singularity official admin guide \`\_ or on the \`Apptaniner official admin guide \`\_ respectively. 1. Verify that the image exist by looking at the output of the docker command line utility. .. code-block:: bash sudo docker image ls Remember to annotate the ID of the desired image 2. Generate a \`\`.tar\`\` archive form the desired image using the following command: .. code-block:: bash sudo docker image save .tar 3. Build a Singularity image file starting from the freshly generated archive with: .. code-block:: bash sudo singularity build .sif .tar At the end of a successfull building process, if not needed for other purposes, remove the Docker image archive. 4. Change the ownership to the Singularity image file to be able to move it to a remote host without any permissions related issues. .. code-block:: bash sudo chown $(id -nu):$(id -ng) .sif For further informations on how to move files between local systems and one of the CINECA's clusters, visit the :ref:\`hpc/hpc\_data\_storage:Data Transfer\` section. | Understanding Singularity or Apptainer definition file for building container images ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The Definition File (or "def file" for short) is like a set of blueprints explaining how to build a custom container image including specifics about the base Operative System to build or the base container to start from, as well as software to install, environment variables to set at runtime, files to add from the host system, and container metadata. A definition file is divided into two parts: .. list-table:: :widths: 30 70 :header-rows: 1 \* - \*\*Parts\*\* - \*\*Purpose\*\* \* - Header - | Describes the core operating system to build within the container. | In the Header all the information to configure the base operating system features needed within the container are reported. | \*e.g. Linux distributions with its specific version or a base container image\* \* - Sections - | The rest of the definition is comprised of sections. | Each section is defined by a \`\`%\`\` character followed by the name of the particular section. | All sections are optional, and a def file may contain more than one instance of a given section. A definition file may look like this: .. code-block:: bash # This is a comment # -- HEADER begin -- Bootstrap: docker From: ubuntu:{{ VERSION }} Stage: build # -- HEADER end -- # -- SECTIONS begin -- %arguments VERSION=22.04 %setup touch /file1 touch ${APPTAINER\_ROOTFS}/file2 %files /file1 /file1 /opt %environment export LISTEN\_PORT=54321 export LC\_ALL=C %post apt-get update && apt-get install -y netcat NOW=\`date\` echo "export NOW=\\"${NOW}\\"" >> $APPTAINER\_ENVIRONMENT %runscript echo "Container was created $NOW" echo "Arguments received: $\*" exec echo "$@" %startscript nc -lp $LISTEN\_PORT # -- SECTIONS end -- For further informations on how to write a custom definition file, users are strongly encouraged to visit the dedicated page both on \`the official Singularity user guide \`\_ or \`the official Apptainer user guide \`\_. .. figure:: img/change\_formats.png :height: 300px :align: center | A quick outline over :ref:\`hpc/hpc\_enviroment:Spack\`, a package management tool compatible with Singularity, which can be used to deploy entire software stacks inside a container imageis provided. .. dropdown:: Advanced Singularity or Apptainer container build with Spack Spack (\`full documentation here \`\_) is a package manager for Linux and macOS, able to download and compile (almost!) automatically the software stack needed for a specific application. It is compatible with the principal container platforms (Docker, Singularity), meaning that it can be installed inside the container and in turn be used to deploy the necessary software stack inside the container image. This can be utterly useful in a HPC cluster environment, both to install applications as a root (inside the container), and to keep a pletora of ready-available software stacks (or even application built with different software stack versions) living in different containers (regardless of the outside environment). Getting Spack is an easy and fast three steps process: \* Install the necessary dependencies, eg. on Debian/Ubuntu: \`\`apt update; apt install build-essential ca-certificate coreutils curl enviroment-modules gfortran git gpg lsb-release python3 python3-distutils python3-venv unzip zip\`\`. \* Clone the repository: \`\`git clone -c feature.manyFiles=true https://github.com/spack/spack.git\`\`. \* Activate spack, eg: for \*bash/zsh/sh\*: \`\`source /spack/share/spack/setup-env.sh\`\`. The very same operations can be put in the \`\`%post\`\` section of a Singularity definition file to have an available installation of Spack at the completion of the built. Alternatively, one can bootstrap from an image containing spack only and start from there the built of the container. For example: .. code-block:: bash sudo singularity build --sandbox docker://spack/ubunty-jammy \*\*Spack Basic Usage\*\* .. code-block:: bash $ spack install openmpi@4.1.5+pmi fabrics=ucx,psm2,verbs schedulers=slurm %gcc@8.5.0 Generally speaking, the deployment of a software stack installed via spack is based on the following steps: 1. Build a container image. 2. Get spack in your container. 3. Install the software stack you need. In practice, and if foresight of building an \*immutable\* SIF container image for compiling and running an application, one can proceed as follow: 1. Get \`\`sandbox\`\` container image hodling an installation of spack and open a \*\*shell\*\* with \`\`sudo\`\` and writable privileges (\`\`sudo singularity shell --writable \`\`). 2. Write a \`\`spack.yaml\`\` file for a spack environment listing all the packages and compilers your application would need (more detaile \`here \`\_). 3. Execute \`\`spack concretize\`\` and \`\`spack install\`\`, if the installation goes through and you are application can compile and run you are set to go: a. either transform your sandobox \`\`.sif\`\` file fixing the changes to a conteiner image. b. or, for a clean build, copy the \`\`spacl.yaml\`\` file in the conteiner in the specific \`\`%\`\` in a \`\`\*.sif\`\` file fixing the changes to a conteiner imagefile section, activate spack and execute \`\`spack concretize\`\` and \`\`spack install\`\`. Following, a minimal example of a Singularity definition file: we bootstrap from a docker container holding a clean installation of ubuntu:22.04, we copy a ready made \`\`spack.yaml\`\` file in the container, get spack therein and use it to install the software stack as delineated in the \`\`spack.yaml\`\` file. .. code-block:: bash Bootstrap: docker From: ubuntu:22.04 %files /some/example/spack/file/spack.yaml /spacking/spack.yaml %post ### We install and activate Spack apt-get update apt install -y apt install build-essential ca-certificates coreutils curl environment-modules gfortran git gpg lsb-release python3 python3-distutils python3-venv unzip zip git clone -c feature.manyFiles=true https://github.com/spack/spack.git source /spack/share/spack/setup-env.sh ### We pretentiously deploy a software stack in a Spack environment spack env activate -d /spacking/ spack concretize spack install %environment ### Custom evironment variables should be set here export VARIABLE=MEATBALLVALUE Bindings -------- A Singularity container image provides a standalone environment for software handling. However, it might still need files from the host system, as well as write privileges at runtime. As pointed out above, this last operation is indeed available when working with a sandbox, but it is not for an (\*immutable\*) \*\*SIF\*\* object. To provide for these needs, Singularity grants the possibility to mount files and directories from the host to the container. \* In the default configuration, the directories \`\`$HOME\`\` , \`\`/tmp\`\` , \`\`/proc\`\` , \`\`/sys\`\` , \`\`/dev\`\`, and \`\`$PWD\`\` are among the system-defined bind paths \* The \`\`SINGULARITY\_BIND\`\` environment variable can be set (in the host) to specify the bindings. The argument for this option is a comma-delimited string of bind path specifications in the format \`\`src\[:dest\[:opts\]\]\`\` where src and dest are paths outside and inside of the container respectively; the dest argument is optional, and takes the same values as src if not specified. For example: \`\`$ export SINGULARITY\_BIND=/path/in/host:mount/point/in/container\`\`. \* Bindings can be specified on the command line when a container instance is started via the \`\`--bind\`\` option. The structure is the same as above, eg. singularity shell \`\`--bind /path/in/host:/mount/point/in/container \`\`. Enviroment variables -------------------- Environment variables inside the container can be set in a handful of ways, see also \`here \`\_. At build time they should be specified in the %environment section of a Singularity definition file. Most of the variables from the host are then passed to the container except for \`\`PS1\`\`, \`\`PATH\`\` and \`\`LD\_LIBRARY\_PATH\`\` which will ne modified to contain default values; to prevent this behavior, one can use the \`\`--cleanenv\`\` option, to start a container instance with a clean environment. Further environment variables can be set, and host variables can be overwritten at runtime in a handful of ways: .. list-table:: :header-rows: 1 \* - \*\*Scope\*\* - \*\*CLI Flag/Host-side variables\*\* \* - Directly pass an environment variables to the containerized application - \`\`--env MYVARIABLE="myvalue"\`\` \* - Directly pass a list of environment variables held in a file to the containerized application - \`\`--env-file
\`\` \* - Automatically pass host-side defined variable to the containerized application - | \`\`export SINGULARITYENV\_MYVARIABLE=myvalue\`\` \*on host machine\* | results in \`\`MYVARIABLE=myvalue\`\` \*inside the container\* With respect to special \`\`PATH\`\` variables: .. list-table:: :header-rows: 1 \* - \*\*Scope\*\* - \*\*Host-side variables\*\* \* - Append to the \`\`$PATH\`\` variable - \`\`export SINGULARITY\_APPEND\_PATH=
\`\` \* - Prepend to the \`\`$PATH\`\` variable - \`\`export SINGULARITY\_PREPEND\_PATH=\`\` \* - Override the \`\`$LD\_LIBRARY\_PATH\`\` variable - | \`\`export SINGULARITYENV\_LD\_LIBRARY\_PATH=\`\` | | \*\*NOTE\*\* | By default, inside the container the \`\`LD\_LIBRARY\_PATH\`\` is set to \`\`/.singularity/libs\`\`. | Users are strongly encouraged to inlude also this path when setting \`\`SINGULARITYENV\_LD\_LIBRARY\_PATH\`\` As a last disclaimer, we point out two additional variables which can be set in the host to manage the building process: .. list-table:: :header-rows: 1 \* - \*\*Scope\*\* - \*\*Host-side variables\*\* \* - Pointing to a directory used for caching data from the build process - \`\`export SINGULARITY\_CACHEDIR=\`\` \* - Pointing to a directory used for temporary build of the squashfs system - \`\`export SINGULARITY\_TMPDIR=\`\` .. note:: All the aforementioned variables containing the \`\`SINGULARITY\`\` word can be interpred and correctly applied by Apptainer. However, Apptainer may complain about using those variables instead of using the Apptainer's specific ones: to do so, users have to simply replace the occurance of \`\`SINGULARITY\`\` with \`\`APPTAINER\`\`. Containers in HPC environment ----------------------------- In this sections, all the information necessary for the execution of Singularity or Apptainer containers along with all the container \*platform\* flags are reported to perform their execution on CINECA's clusters. In order to move locally built SIF images on CINECA's clusters, consult the "Data Transfer" page under the :ref:\`hpc/hpc\_data\_storage:File Systems and Data Management\` section. However, Singularity allows pulling existing container images from container registries as the one seen in the third section. Pulling container images from registries can be done on CINECA's cluster via the following command synthax: .. code-block:: bash singularity pull registry://path/to/container\_img\[:tag\] This will create a SIF file in the directory where the command was run allowing the user to run the image just pulled. .. tab-set:: .. tab-item:: Parallel MPI Container The MPI implementation used in the CINECA clusters is OpenMPI (as opposed to MPICH). Singularity offers the possibility to run parallel applications compiled and installed in a container using the host MPI installation, as well as the bare metal capabilities of the host such as the Infiniband computer networking communication standard. This is the so called \*Singularity hybrid approach\* where the OpenMPI installed in the container and the one on the host work in tandem to instantiate and run the job, see also the \`documentation \`\_. .. note:: Keep in mind that when exploiting the \*Singularity hybrid approach\*, the necessary MPI libraries from the host are automatically bound above the ones present in the container. The only caveat is that the two installations (container and host) of OpenMPI have to be compatible to a certain degree. The (default) installation specifics for each cluster are here listed: .. list-table:: :widths: 30 30 30 30 30 :header-rows: 1 \* - \*\*Cluster\*\* - \*\*OpenMPI version\*\* - \*\*PMI implementation\*\* - \*\*Specifics\*\* - \*\*Tweaks\*\* \* - Galileo100 - 4.1.1 - pmi2 - | \`\`--with-pmi\`\` | \`\`--with ucx\`\` | \`\`--with-slurm\`\` - \* - Leonardo - 4.1.6 - pmix\_v3 - | \`\`--with ucx\`\` | \`\`--with-slurm\`\` | \`\`--with-cuda\`\` \* - \`\`export PMIX\_MCA\_gds=hash\`\` \*\* \* - Pitagora - 4.1.6 - pmix\_v3 - | \`\`--with ucx\`\` | \`\`--with-slurm\`\` | \`\`--with-cuda\`\` \* - | \`\`export PMIX\_MCA\_gds=hash\`\` \*\* \`\*\` only available in \`\`boost\_\*\`\` partitions. \*\* suppres PMIX WARNING when using srun. .. note:: Even if the host and container hold different versions of OpenMPI, the application might still run in parallel, but at a reduced speed, as it might not be able to exploit the full capabilities of the host bare metal installation. A suite of container images holding compatible OpenMPI versions for the CINECA clusters are available at the \`NVIDIA catalog \`\_, on which we dwell in the next section. .. tab-item:: GPU Aware Container To run GPU applications on accelerated clusters on first has to check his container image holds a compatible version of CUDA. The specifics are listed in the following table: .. list-table:: :widths: 30 30 30 30 :header-rows: 1 \* - - \*\*Driver Version\*\* - \*\*CUDA Version\*\* - \*\*GPU Model\*\* \* - Galileo100 - 470.42.01 - 11.4 - NVIDIA V100 PCIe3 32 GB \* - Leonardo - 535.54.03 - 12.2 - NVIDIA A100 SXM6 64 GB HBM2 \* - Pitagora - 565.57.01 - 12.7 - NVIDIA H100 SXM 80GB HBM2e while the \`CUDA compatibility \`\_ table is: .. list-table:: :widths: 50 50 :header-rows: 1 \* - \*\*CUDA Version\*\* - \*\*Required Drivers\*\* \* - CUDA 12.x - from 525.60.13 \* - CUDA 11.x - from 450.80.02 One can surely install a working version of CUDA on his own, for example via Spack. However, a simple and effective way to obtain a container image provided with a CUDA installation is to bootstrap from an NVIDIA HPC SDK docker container, which already comes equipped with CUDA, OpenMPI and the NVHPC compilers. Such containers are available at the \`NVIDIA catalog \`\_. Their tag follows a simple structure, \`\`$NVHPC\_VERSION-$BUILD\_TYPE-cuda$CUDA\_VERSION-$OS\`\`, where: 1. \`\`$BUILD\_TYPE\`\`: can either take the value devel or runtime. The first ones are usually heavier and employed to compile and install applications. The second ones are lightweight containers for deployment, stripped of all the compilers and applications not needed at runtime execution. 2. \`\`$CUDA\_VERSION\`\`: an either take a specific value (e.g. ) or be a \`\`multi\`\`. The multi flavors hold up to three different CUDA version, and as such are much heavier. However, they can be useful to deploy the same base container on HPC with different CUDA specifics or to try out the performance of the various versions. In the following we provide a minimal Singularity definition file following the above principles, namely: bootstrap from a develop NVIDIA HPC SDK container, install the needed applications, copy the necessary binaries and files for runtime, pass to a lightweight container. This technique is called multistage build, more information available \`here \`\_. .. code-block:: bash Bootstrap: docker From: nvcr.io/nvidia/nvhpc:23.1-devel-cuda\_multi-ubuntu22.04 Stage: build %files ### Notice the asterisk when copying directories /directory/with/needed/files/in/host/\* /destination/directory/in/container /our/application/CMakeLists.txt /opt/app/CMakeLists.txt /some/example/spack/file/spack.yaml /spacking/spack.yaml %post ### We install and activate Spack apt-get update apt install -y build-essential ca-certificates coreutils curl environment-modules gfortran git gpg lsb-release python3 python3-distutils python3-venv unzip zip git clone -c feature.manyFiles=true https://github.com/spack/spack.git . /spack/share/spack/setup-env.sh ### We pretentiously deploy a software stack in a Spack environment spack env activate -d /spacking/ spack concretize spack install ### Make and install our application cd /opt/app && mkdir build cd build cmake -DCMAKE\_INSTALL\_PREFIX=/opt/app\_binaries .. make -j make install ########################################################################################### ### We now only need to copy the necessary binaries and libraries for runtime execution ### ########################################################################################### Bootstrap: docker From: nvcr.io/nvidia/nvhpc:23.1-runtime-cuda11.8-ubuntu22.04 Stage: runtime %files from build /spacking/\* /spacking/ /opt/app\_binaries/\* /opt/app\_binaries/ Execute containerized application in an HPC environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ As explained in the previous section as well as in the \`documentation \`\_, if the MPI library installed in the container is compatible with that of the host system, Singularity will take care by itself of binding the necessary libraries to allow a parallel containerized application to run exploiting the cluster infrastructure. In practical terms, this means that one just need to launch it as: .. code-block:: bash mpirun -np $nnodes singularity exec In comparison, the following code snippet will launch the application using \*MPI inside the container\*, thus effectively running on a \*single node\*: .. code-block:: bash singularity exec mpirun -np $nnodes Regarding launching containerized applications needing GPU support, again Singularity is capable of binding the necessary libraries on its own, provided a compatible software version in the container and host has been deployed; full documentation is available \`here \`\_. To achieve this, one just need to add the \`\`--nv\`\` or the \`\`--nvccli\`\` flag on the command line, namely: .. code-block:: bash mpirun -np $nnodes singularity exec --nv .. important:: In most recent versions of both Singularty and Apptainer, the \`\`--nv\`\` flag used for NVIDIA GPUs, has been replaced by the \`\`--nvccli\`\` flag. .. note:: Similarly to what said about the \*Singularity hybrid approach\* in the"Parallel MPI Container" tab, for GPU parallel programs, the necessary CUDA drivers and libraries from the host are automatically bound and employed inside the container provided the \`\`--nv\`\` or \`\`--nvccli\`\` flag is used when starting a container instance. e.g. \`\`$ singularity exec --nv \`\`. Cluster specific tweaks ^^^^^^^^^^^^^^^^^^^^^^^ In this section the specific version of Singularity or Apptainer installed on each CINECA's cluster are reported along with some useful information to help users properly executing their containerized applications. .. tab-set:: .. tab-item:: Galileo100 On Galileo100, \`Singularity 3.8.0 \`\_ is available on the login nodes and on the partitions. Beware that, for the Galileo100 cluster, nodes with GPU are available under both the Interactive Computing service and by requesting the \`\`g100\_usr\_intercative\`\` Slurm partition with one main difference: .. list-table:: :widths: 50 50 :header-rows: 1 \* - \*\*Platform\*\* - \*\*Maximum number of GPUs per Job\*\* \* - Interactive Computing service - 2 \* - \`\`g100\_usr\_interactive\`\` Slurm partition - 1 The necessary MPI, Singularity and CUDA modules are the following: \* \`\`module load profile/advanced\`\` (profile with additional modules) \* \`\`module load autoload singularity/3.8.0--bind--openmpi--4.1.1\`\` \* \`\`module load cuda/11.5.0\`\` .. note:: The \`\`module load autoload singularity/3.8.0--bind--openmpi--4.1.1\`\` command automatically loads the following modules: \* \`\`singularity/3.8.0--bind–openmpi–4.1.1\`\` \* \`\`zlib/1.2.11--gcc–10.2.0\`\` \* \`\`openmpi/4.1.1--gcc--10.2.0-cuda–11.1.0\`\` The following code snippet is an example of a Slurm job script for running MPI parallel containerized applications on the Galileo100 cluster. Notice that the \`\`--cpus-per-task\`\` option has been set to \*\*48\*\* to fully exploit the CPUs in the \`\`g100\_usr\_prod\`\` partition. .. code-block:: bash #!/bin/bash #SBATCH --nodes=6 #SBATCH --ntasks-per-node=1 #SBATCH --cpu-per-task=48 #SBATCH --mem=30GB #SBATCH --time=00:10:00 #SBATCH --out=slurm.%j.out #SBATCH --err=slurm.%j.err #SBATCH --account= #SBATCH --partition=g100\_usr\_prod module purge module load profile/advanced module load autoload singularity/3.8.0--bind--openmpi--4.1.1 module load cuda/11.5.0 mpirun -np 6 singularity exec .. tab-item:: Leonardo \*\*Necessary modules and Slurm job script example\*\* On Leonardo, \`Singularity PRO 4.3.0 \`\_ is availabe on the login nodes and on the partitions. The necessary MPI, Singularity and CUDA modules are the following: \* \`\`module load hpcx-mpi/2.19\`\` \* \`\`module load cuda/12.2\`\` The following code snippet is an example of a Slurm job script for running MPI parallel containerized applications on the Leonardo cluster with GPU support. In order to equally and fully exploit the \*\*32 cores\*\* and \*\*4 GPUs\*\* of the \`\`boost\_usr\_prod\`\` partition, one needs to set \`\`--ntasks-per-node=4\`\`, \`\`--cpu-per-task=8\`\` and \`\`--gres=gpu:4\`\`. As a redundant but necessary measure, we also set the number of threads to eight manually via \`\`export OMP\_NUM\_THREADS=8\`\`. .. code-block:: bash #!/bin/bash #SBATCH --nodes=6 #SBATCH --ntasks-per-node=4 #SBATCH --cpu-per-task=8 #SBATCH --gres=gpu:4 #SBATCH --mem=30GB #SBATCH --time=00:10:00 #SBATCH --out=slurm.%j.out #SBATCH --err=slurm.%j.err #SBATCH --account= #SBATCH --partition=boost\_usr\_prod export OMP\_NUM\_THREADS=8 module purge module load hpcx-mpi/2.19 module load cuda/12.2 mpirun -np 6 singularity exec --nv As explained above, provided the container and host OpenMPI share a compatible pmi, the application can be launched via the srun command after having allocated the necessary resources. For example: .. code-block:: bash salloc -t 03:00:00 --nodes=6 --ntasks-per-node=4 --ntasks=24 --gres=gpu:4 -p boost\_usr\_prod -A export OMP\_NUM\_THREADS=8 srun --nodes=6 --ntasks-per-node=4 --ntasks=24 singularity exec --nv .. tab-item:: Pitagora \*tab under constuction\* On Pitagora, \`Apptainer 1.4.0 \`\_ is available on the login nodes and on the partitions. --- # Unknown .. \_dns\_guidelines\_card: DNS guidelines ============== DNS name -------- It is possible to ask CINECA for a DNS name association to the virtual machine by sending an email to superc@cineca.it. In CINECA DNS, it is necessary to comply with the following rules: - The \*\*reverse\*\* of the Floating IP (PTR record) must be set to the hostname of the VM, with the following naming convention: - for external users: .ext.cineca.it - for CINECA staff: .cineca.it - The \*\*record A\*\* in the DNS is set accordingly to the previous point. - If the service should be exposed with a different name, you can ask to set the \*\*CNAME\*\* with the chosen different name. If no other information is provided, only the record A will be set. Additionally, you might set up a CNAME with your DNS provider of choice. It is not possible to set the \*\*PTR record\*\* in CINECA DNS, if the record A has been set on an external DNS. --- # Unknown .. \_load\_balancers\_card: Load Balancer ============= \`OpenStack Octavia service \`\_ enables the deployment of load balancing solutions in OpenStack projects. A \*\*load balancer\*\* functions as a traffic intermediary, directing network or application traffic to multiple server endpoints. It helps manage capacity during high traffic periods and enhances the reliability of applications. The main components of a load balancer are the following: - \*\*Listener\*\*: The listener is a component that defines how incoming traffic is received. It listens for connection requests on a specific port and protocol (e.g., HTTP, HTTPS), and directs this traffic to the appropriate backend pool - \*\*Pool\*\*: The pool is a collection of backend servers (also known as members) that receive and process the incoming traffic distributed by the load balancer. The pool determines the load balancing algorithm and health check policies to manage traffic distribution effectively - \*\*Members\*\*: Members are the individual servers within a pool that handle the actual processing of the traffic. Each member represents a single endpoint (server) that performs the required tasks or services requested by the client. A load balancer determines which server to send a request to based on a desired algorithms (e.g., Round Robin, Least Connections, Random). The choice among the load balancing algorithms depends on the requirements of the specific use case. .. note:: The Octavia service is available but it is not enabled by default to all HPC Cloud projects. If you want to use it please ask access sending an email to superc@cineca.it. --- # Unknown .. \_lb\_troubleshooting\_card: LoadBalancer: troubleshooting ============================= Load Balancer in provisioning\_status = \`\`PENDING\_CREATE\`\` or \`\`PENDING\_UPDATE\`\` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When a load balancer is in provisioning\_status \`\`PENDING\_UPDATE\`\` or \`\`PENDING\_CREATE\`\`, any action on it is blocked from OpenStack. If the load balancer remains stuck in this state it can't be modified, recovered or deleted. In this case, the loadbalancer has to be deleted by our sys admins. To solve the issue please write to our support team (superc@cineca.it). Load Balancer in provisioning\_status = \`\`ERROR\`\` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When a load balancer is in provisioning\_status \`\`ERROR\`\` something has failed in one or more of its components (i.e. the amphora machines): if the error occurred on only one of the load balancer's amphora machines, the load balancer itself may be still operative but any modification operations (including deletion) performed by the user will be prevented. In order to solve this issue, the load balancer failover operation can be carried out by OpenStack admins. To solve the issue please write to our support team (superc@cineca.it). Load Balancer in operating\_status = \`\`DEGRADED\`\` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When a load balancer shows its operating\_status equal to \`\`DEGRADED\`\` this means that an error has occured to one or more of its pool member. This state does not necessarily compromise completely the loadbalacer, it just means that the loadbalancer is not operating at maximum capacity. The reason could be either: - at least one of the members is in \`\`ERROR\`\` state - all the members are in \`\`ACTIVE\`\` state but there is some configuration or network error inside the virtual machine Those problems should be solvable by the user since the problems depend on the inner workings of the member and users have access to the member via ssh. If the member is stuck in \`\`ERROR\`\` state, please write to our support team (superc@cineca.it). --- # Unknown .. \_security\_guidelines\_card: Security guidelines =================== This list of security guidelines is not meant to cover every possible case or scenario, but to serve as a starting point for keeping everyone secure: please read it carefully. A complete description of roles and responsibilities is provided in :ref:\`cloud/general/cineca\_cloud\_model:responsibilities\`. Additional information for the management of sensitive data is provided in :ref:\`cloud/tenant\_adm/store\_sens\_data:store sensitive data\`. .. important:: Concerning security in particular: - Users are responsible for the security of the virtualized resources under their control. This includes, but it is not limited to, virtual machines, network configuration, user accounts, disk volumes - If you discover a critical security flaw or believe that your machine has been compromised, please contact us immediately at superc@cineca.it - CINECA \*\*reserves the right to suspend connectivity for the IPs affected by security breaches\*\*, when necessary to protect the infrastructure or mitigate potential risks. Account and Credentials management ----------------------------------- Carefully manage the credentials that provide access to the service. General account guidelines ^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Set up additional user accounts to access the virtual machines using a SAML provider whenever possible. - Enable Multi-Factor Authentication (MFA) for enhanced security. - Always apply the principle of least privilege when assigning roles. - Utilize SAML accounts as much as possible and reserve the use of the Virtual Machine Admin for cases of actual necessity, - Promptly close any accounts to the service that are no longer necessary. - Periodically verify (at least once a year) that all active accounts are still necessary. Password management ^^^^^^^^^^^^^^^^^^^^ All users, especially system administrators, must comply with the following guidelines to ensure password security: - General Best Practices: - Choose strong, high-quality passwords as described below. - Keep passwords confidential: - Do not store them in plain text on local or remote systems. - Avoid writing them down unless stored in a locked, secure location. - Change passwords immediately if compromise is suspected. - After receiving a new account, perform the first login promptly and change the temporary password provided by the administrator. - Never share your password with anyone. - Rules for Strong Passwords - Minimum length: 8–12 characters. - Must include: Uppercase and lowercase letters, Numbers, Special characters. - Avoid: Common dictionary words, Personal information (name, birthday, phone number, tax code, car plate, relatives’ names, pet names, or work-related keywords), Simple sequences like 123456, abcdef, or repeated characters. - Secure Password Transmission - Passwords must never be sent on the same channel where they were requested (e.g., ticketing systems). - When sending initial credentials via email: - Use a generic subject line (e.g., “Requested Information”). - Include only the password in the email body. - Do not reference usernames, tickets, or account details. - Password Renewal - For accounts managing personal or sensitive data, users must change passwords before the expiration date imposed by the system to keep accounts active. User Account Management ^^^^^^^^^^^^^^^^^^^^^^^^ Regularly review the user accounts enabled in your system. Some applications create default accounts which are unnecessary or even directly insecure. Recommended setup: - \*\*root\*\* with ssh disabled and no password. This is the default in the images provided on CINECA HPC Cloud for the different OS (i.e. Ubuntu, Centos,..). - \*\*account for a sysadmin\*\* that can only be accessed via ssh keys and has sudo access. CINECA HPC Cloud VM images provide this user pre-configured as well, the name of the user depends on the distribution (cloud-user, centos or ubuntu), see the :ref:\`cloud/systems/index\_system\_specifics:cloud specifics\` of the system you are using for specific information. - \*\*user-level accounts\*\* that run a single service and have no login possible, neither remote nor local access. Do not enable password login, \*\*use SSH keys instead\*\* (see :ref:\`cloud/operative/compute\_ops/keypair\_create:key pair: create\`). Passwords can be, with enough time and compute power, guessed with brute force. The average SSH server deals with thousands of such attacks every week. When using SSH keys, challenge-response authentication is used instead. This means that for each login a different challenge is asked and a different response is the correct one. No secret (password or key) ever travels across the network. - Password protect your SSH keys and make sure your key never leaves the hardware where it was created. - Do not store public keys (much less private) on the image used to create the VM. Network ------- It is very important to keep your network configuration as secure as possible, as it is the gate any intruder will use to enter in your system. It is relatively simple to apply some good practices that will give a good extra security layer. Here below few strategies are advised. Restrictive firewall (white listing) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Your Virtual Machine instances should be configured so that they allow the minimum required access to run your application. By default, virtual machines have no external access (default security group rules in CINECA HPC Cloud), this means no single port is opened by default to the public Internet. In order to connect to them, or to provide any kind of service, access has to be explicitly granted. It is important to open only the ports that are needed and open them only for the least amount of IPs possible. Every virtual machine running in CINECA HPC Cloud comes with default Security groups. :ref:\`cloud/os\_overview/os\_components/network:security groups\` are the easiest way to apply a set of complex firewall rules to a set of virtual machines. This is an example of a security group that gives access to port 22/SSH to only 2 subnets (which could be the 2 public ranges that your organization uses in its office network): .. image:: /cloud/\_img/sec\_guidelines.png Security groups are easy to configure and easy to visualize in the Horizon Dashboard under the Network tab or in each virtual machine's instance page (see :ref:\`cloud/operative/network\_ops/secgroups\_create:security groups: create\`). Disable unneeded services ^^^^^^^^^^^^^^^^^^^^^^^^^ Do not run unnecessary services on your VM, even if they are not accessible from the outside. The more services you run, the more potential attack surfaces you have that top intruders might exploit. Use secure protocols ^^^^^^^^^^^^^^^^^^^^ Wherever possible, use encrypted and secure communication protocols to avoid man in the middle attacks; this is when someone get access to your communications and can read the data going through like in a public WIFI. For example: do not use HTTP, use instead HTTPS. Do not use FTP to transfer files, use instead FTPS, SFTP or S3. If you need a web certificate for your VM, we suggest to use the service provided by \`Let's Encrypt \`\_. Use intrusion detection software ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Tools such as \`denyhosts \`\_ or \`Fail2ban \`\_ can be used to analyse log files and ban IP addresses that are attempting to make brute-force attacks to your application. They are very powerful tools, but they have to be used with care as they can lead to false positives, i.e. Banning IPs that should not be banned. These tools are a best practice to provide 24/7 services, while may not be necessary for single user VMs. Software -------- Running secure software is also very important. It is not a trivial task to develop fully secure software, but there are some simple strategies that will help with the task. Automatic software updates ^^^^^^^^^^^^^^^^^^^^^^^^^^ All operating systems have the ability to apply updates automatically. If you run regular updates, you are less exposed to known security problems. It is common that the fix is available before the security problem is published. In Centos 8 and newer, you have \`\`dnf-automatic\`\`: .. code:: bash sudo yum install dnf-automatic -y systemctl enable --now dnf-automatic-install.timer For Centos 7, you have \`\`yum-cron\`\`: .. code:: bash sudo yum install yum-cron -y sudo systemctl enable yum-cron.service sudo systemctl start yum-cron.service For Ubuntu, you have \`\`unattended-upgrades\`\`: .. code:: bash sudo apt install unattended-upgrades Each OS version will have its own way to activate this. \*\*Kernel updates\*\*: Some updates, such as kernel upgrades, require rebooting the virtual machines. Please schedule this into your regular maintenance. If your use case does not support automatic updates, which is common for highly available setups, please make sure to schedule regular maintenance windows where the software upgrade is scheduled. \*\*Subscribe to security announcements for your OS\*\*, if there is a security problem in your operating system, you need to find it out as soon as possible. You can subscribe to an appropriate mailing list, RSS feed, ... to keep an eye out for anything that requires urgent action. Only install from reputable sources ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Be mindful of the sources for the software you install. Only install software from reputable sources. If possible, use the distribution's package manager (yum, dnf, apt, ...). Packages managers make it easy to install software, keep it updated, and uninstall it. If the desired software is not available in the distribution package manager repository, an official source must be used. Follow the instructions on the official website of the software you need. If more than one source is offered, think about using the one that provides an easier life-cycle (install/update/uninstall/...), like \`snap \`\_ or \`flatpak \`\_. Security for databases ^^^^^^^^^^^^^^^^^^^^^^^ If you have databases in your VM please make sure that these: - are not open to the whole internet (0.0.0.0/0) - are password protected - information is transferred via a secure connection. Keep logs of your applications ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Use the best practices for logging: - Make sure that the services are logging to a secure location, that is as tamper-proof as possible. - Keep the logs for a reasonably long amount of time. - Consider logging to a remote server as well. Images ------- The main reason behind the \*\*prohibition of uploading community images\*\*, to be used for instances creation, is the fact that they are visible and can be uploaded by any user on the cloud platform. CINECA HPC Cloud infrastructure administrators have thus no control on community image content. The use of a community image implies an acceptance of the risk that the image owner has uploaded, unconsciously or not, an image containing malicious software or vulnerability like \*\*backdoors\*\*, \*\*keyloggers\*\*, \*\*viruses\*\* or \*\*malware\*\*. Even if, the use of community images is not a guarantee of vulnerability, keeping in mind the aforementioned risks, the CINECA HPC Cloud infrastructure administrators have chosen to inhibit the possibility to upload images with such visibility. More information ---------------- If you are interested to learn more about security in cloud application, we advise to read the material provided by \`NeCTAR \`\_. \*\*Acknowledgements\*\*: CINECA Team would like to acknowledge the following source of information for this page: https://docs.csc.fi/cloud/pouta/security/ --- # Unknown .. \_store\_sensitive\_data\_card: Store sensitive data ==================== .. warning:: Following CINECA access policies, you must inform CINECA in case the activity requires the loading and processing of data that may fall under the GDPR (personal data), to identify the appropriate security level; in any case, \*\*sensitive or personal data shall not be loaded and processed with CINECA resources without CINECA written authorization\*\*. If your application or workflow is processing sensitive data, besides getting the required authorization and signing with CINECA the Data Processing Agreement (for the appointment of the Data Processor), you need to take the necessary technical precautions to safeguard the data from unauthorized access. On CINECA HPC Cloud infrastructure, sensitive data can be stored on special \*\*encrypted Cinder Volume\*\* of type LUKS. By using the OpenStack Horizon dashboard, every user can create such volumes and then attach them to a virtual machine. Due to a limitation of the crypto library, the \*\*maximum size of each volume is 15 TB\*\*. Since LUKS are encrypted volumes, the time needed to create one can vary greatly in association to the size of the volume (most of the time is needed to encrypt the data). Here are some indicative times for the creation of different sized LUKS volumes from the dashboard: - 1 TiB: 15 minutes - 7 TiB: 2 hours - 10 TiB: 3-4 hours The user can access the data stored in such LUKS volumes by login into the corresponding virtual machine. Only the users with authorization to login into the virtual machine will access the data "in clear", even if it is encrypted by key. The keys used by the OpenStack volume encryption feature are managed by Barbican, the official OpenStack Key Manager service. Barbican provides secure storage, provisioning and management of secret data. This includes keying material such as Symmetric Keys, Asymmetric Keys, Certificates and raw binary data. .. note:: CINECA HPC Cloud infrastructure is certified ISO 27001 since 2022 for \*\*"Servizi informatici HPC in cloud per la ricerca in ambito life science"\*\* and since 2025 for \*\*"Erogazione di servizi IaaS per ricerca e innovazione su HPC Cloud"\*\*. Details can be found \`here \`\_. --- # LoadBalancer: create — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [LoadBalancer operations](https://docs.hpc.cineca.it/cloud/operative/lb_ops/index_lb_ops.html) * LoadBalancer: create * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/lb_ops/lb_create.rst.txt) * * * LoadBalancer: create[](https://docs.hpc.cineca.it/cloud/operative/lb_ops/lb_create.html#loadbalancer-create "Link to this heading") ===================================================================================================================================== We exemplify here the procedures based on (1) [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) and (2) the [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) and the OpenStack Octavia plugin, for a simple use case: **the deployment of a basic HTTP load balancer with an associated Floating IP**. For further details and additional use cases, including step-by-step examples, refer to the [Octavia Basic Load Balancing Cookbook](https://docs.openstack.org/octavia/zed/user/guides/basic-cookbook.html) . This resource provides comprehensive instructions on applying the same procedure to various scenarios. Important * The Octavia service is available but it is not enabled by default to all HPC Cloud projects. * If you want to use it, please request access sending an email to [superc@cineca.it](mailto:superc%40cineca.it) . * Once the tenant is enabled to the service by the User Support Team, all users of the tenant will be able to use the service. In this section, we will walk you through the steps to create a setup where two instances running nginx servers are connected to an HTTP load balancer. The load balancer will use the round-robin algorithm to evenly distribute incoming HTTP traffic across the two servers. Important > 1. Before creating a load balancer, **ensure that the following resources are available** in your tenant: > > > * 1 network and subnet > > > > * 1 router > > > > * Desired security groups for the VMs: at the minimum, the Ingress rules for HTTP (port 80) and SSH (port 22) > > > > * 2 instances: in our example, these VMs host an nginx web server each > > > > * At least 2 floating IPs: one associated to one of the VMs and an additional one available to be associated to the load balancer. The internal IP of the second VM can be used to login from the first VM if needed for configuration. Note that an additional Floating IP could be directly associated to the second VM. However, this would entail using an additional (and not strictly necessary) resource, which we try to avoid. > > > > * 1 KeyPair: an SSH public key is needed to access the instances for their configuration > > > > 2. You can **setup a very simple nginx web server in each VM** by logging into each of the VM and running the following commands on the shell: > sudo apt-get update sudo apt-get install \-y nginx && \\ echo "Hello! This is $(hostname)" \> /var/www/html/index.html Copy to clipboard Horizon Dashboard > > * **Create the loadbalancer** by clicking on _“Network → Load Balancers → Create Load Balancer”_ and setting the following information. Once all the details are provided, click on _“Create Load Balancer”_. > > > > Load Balancer Details > > * Name. > > * Subnet. Select the desired subnet. > > > ![../../../_images/op_lb_create_img2.png](https://docs.hpc.cineca.it/_images/op_lb_create_img2.png) > > Listener Details > > * Name. > > * Protocol and Port. The protocol defines the type of network traffic the listener will handle, while the port specifies the network port on which the listener will accept incoming traffic. In our example we select protocol HTTP and port 80. > > > ![../../../_images/op_lb_create_img3.png](https://docs.hpc.cineca.it/_images/op_lb_create_img3.png) > > Pool Details > > * Name. > > * Algorithm. The algorithm determines how traffic is distributed across the members. We select ROUND\_ROBIN. > > > ![../../../_images/op_lb_create_img4.png](https://docs.hpc.cineca.it/_images/op_lb_create_img4.png) > > Pool Members > > * Add members. Choose the desired members among those available. We add VM-1 and VM-2, the names of the VMs in our example. > > * Port. For each VM, specify the port number on which the member will receive traffic. In our case, we expose the nginx server on port 80. > > * Weight. The weight of the member for load balancing purposes. The weight determines the relative portion of requests the member should handle compared to others. We use the default value. > > > ![../../../_images/op_lb_create_img5.png](https://docs.hpc.cineca.it/_images/op_lb_create_img5.png) > > Monitor Details > > * Decide whether you’d like to create a Health Monitor. In this example, we will not make use of a monitor. > > > ![../../../_images/op_lb_create_img6.png](https://docs.hpc.cineca.it/_images/op_lb_create_img6.png) * **Associate a floating IP** to the load balancer. > * Move to the _Network → Load Balancers”_ section > > * Display the options within the drop-down menu on the right side for the desired load balancer, and click on _“Associate Floating IP”_. > > > ![../../../_images/op_lb_create_img7.png](https://docs.hpc.cineca.it/_images/op_lb_create_img7.png) > > * Select the floating IP among those suggested in the drop-down menu _“Floating IP address or Pool”_ and click on _“Associate”_. > > > ![../../../_images/op_lb_create_img8.png](https://docs.hpc.cineca.it/_images/op_lb_create_img8.png) > > * The floating IP associated to the load balancer appears in the overview of its characteristics. In order to see the properties, click on the name of your load balancer in the _“Network → Load Balancers”_ section of the dashboard. > * **Test** your load balancer. Use the above floating IP to reach the nginx servers. The traffic will be managed by the load balancer following the algorithm ROUND\_ROBIN. ![../../../_images/op_lb_create_img1.png](https://docs.hpc.cineca.it/_images/op_lb_create_img1.png) Command Line Interface > Important > > * Follow the instructions on how to setup your cloud environment at [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) > > * Install the Octavia service plugin `pip install python-octaviaclient` > * **Create a Load Balancer** `openstack loadbalancer create --name --vip-subnet-id ` The __ can be found through the Horizon Dashboard. On the main menu, select the _Network → Networks_ tab. Then, click on your network and select the _Subnets_ tab. Finally, click on the desired subnet. This information can also be gathered using the CLI. Note: Wait until the creation is completed. It will take a while and the next steps will return an error if the load balancer is not available. * **Create a Listener** > `openstack loadbalancer listener create --name --protocol HTTP --protocol-port 80 ` * **Create a Pool** > `openstack loadbalancer pool create --name --lb-algorithm --listener --protocol HTTP` * **Add Members** to the pools > `openstack loadbalancer member create --subnet-id  --address --protocol-port 80 ` > > `openstack loadbalancer member create --subnet-id --address --protocol-port 80 ` You can find the _IPs_ of the VMs in the _Compute → Instances_ section of the lateral menu of Horizon Dashboard. The IPs can also be gathered using the CLI. * **Associate a Floating IP** to the load balancer > `openstack floating ip set --port ` You can find the __ if you navigate to the description of your load balancer on the Horizon Dashboard. Select the _Network → Load Balancers_ section on the left-hand menu of the dashboard. Then, click on your load balancer and go to the _Overview_ tab. You can also use the CLI to identify the value of the __. * **Test** your load balancer. Finally, use this floating IP to reach the nginx servers. The traffic will be managed by the load balancer following the algorithm . ![../../../_images/op_lb_create_img1.png](https://docs.hpc.cineca.it/_images/op_lb_create_img1.png) --- # Unknown .. \_shares\_ops\_card: Shares operations ================= For general information on the Shares component, visit the :ref:\`cloud/os\_overview/os\_components/shares:shares\` page. .. toctree:: :maxdepth: 2 :hidden: generic\_share\_create cephfs\_share\_create .. grid:: 2 .. grid-item-card:: |osmanila| \*\*Generic shares: create\*\* :link: shares\_generic\_create\_card :link-type: ref .. grid-item-card:: |osmanila| \*\*CEPHFS shares: create\*\* :link: shares\_cephfs\_create\_card :link-type: ref .. |osmanila| image:: /cloud/\_img/manila\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_database\_ops\_card: Database operations =================== For general information on the Database component, visit the :ref:\`cloud/os\_overview/os\_components/database:database\` page. .. toctree:: :maxdepth: 2 :hidden: db\_create db\_access .. grid:: 2 .. grid-item-card:: |ostrove| \*\*Database: create\*\* :link: db\_create\_card :link-type: ref .. grid-item-card:: |ostrove| \*\*Database: access\*\* :link: db\_access\_card :link-type: ref .. |ostrove| image:: /cloud/\_img/trove\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_network\_ops\_card: Network operations ================== For general information on the Network component, visit the :ref:\`cloud/os\_overview/os\_components/network:network\` page. .. toctree:: :maxdepth: 2 :hidden: network\_create secgroups\_create fip\_association .. grid:: 3 .. grid-item-card:: |osneutron| \*\*Network: create\*\* :link: network\_create\_card :link-type: ref .. grid-item-card:: |osneutron| \*\*floating IP: association\*\* :link: fip\_associate\_card :link-type: ref .. grid-item-card:: |osneutron| \*\*Security groups: create\*\* :link: secgroups\_create\_card :link-type: ref .. |osneutron| image:: /cloud/\_img/neutron\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_storage\_ops\_card: Storage operations ================== For general information on the storage component, visit the :ref:\`cloud/os\_overview/os\_components/storage:storage\` page. .. toctree:: :maxdepth: 2 :hidden: volume\_create volume\_mount volume\_resize .. grid:: 3 .. grid-item-card:: |oscinder| \*\*Volume: create and attach\*\* :link: volume\_create\_card :link-type: ref .. grid-item-card:: |oscinder| \*\*Volume: format and mount\*\* :link: volume\_mount\_card :link-type: ref .. grid-item-card:: |oscinder| \*\*Volume: resize\*\* :link: volume\_resize\_card :link-type: ref .. |oscinder| image:: /cloud/\_img/cinder\_logo.png :width: 35px :class: no-scaled-link --- # Unknown \-----BEGIN CERTIFICATE----- MIIFpDCCA4ygAwIBAgIQOcqTHO9D88aOk8f0ZIk4fjANBgkqhkiG9w0BAQsFADBs MQswCQYDVQQGEwJHUjE3MDUGA1UECgwuSGVsbGVuaWMgQWNhZGVtaWMgYW5kIFJl c2VhcmNoIEluc3RpdHV0aW9ucyBDQTEkMCIGA1UEAwwbSEFSSUNBIFRMUyBSU0Eg Um9vdCBDQSAyMDIxMB4XDTIxMDIxOTEwNTUzOFoXDTQ1MDIxMzEwNTUzN1owbDEL MAkGA1UEBhMCR1IxNzA1BgNVBAoMLkhlbGxlbmljIEFjYWRlbWljIGFuZCBSZXNl YXJjaCBJbnN0aXR1dGlvbnMgQ0ExJDAiBgNVBAMMG0hBUklDQSBUTFMgUlNBIFJv b3QgQ0EgMjAyMTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAIvC569l mwVnlskNJLnQDmT8zuIkGCyEf3dRywQRNrhe7Wlxp57kJQmXZ8FHws+RFjZiPTgE 4VGC/6zStGndLuwRo0Xua2s7TL+MjaQenRG56Tj5eg4MmOIjHdFOY9TnuEFE+2uv a9of08WRiFukiZLRgeaMOVig1mlDqa2YUlhu2wr7a89o+uOkXjpFc5gH6l8Cct4M pbOfrqkdtx2z/IpZ525yZa31MJQjB/OCFks1mJxTuy/K5FrZx40d/JiZ+yykgmvw Kh+OC19xXFyuQnspiYHLA6OZyoieC0AJQTPb5lh6/a6ZcMBaD9YThnEvdmn8kN3b LW7R8pv1GmuebxWMevBLKKAiOIAkbDakO/IwkfN4E8/BPzWr8R0RI7VDIp4BkrcY AuUR0YLbFQDMYTfBKnya4dC6s1BG7oKsnTH4+yPiAwBIcKMJJnkVU2DzOFytOOqB AGMUuTNe3QvboEUHGjMJ+E20pwKmafTCWQWIZYVWrkvL4N48fS0ayOn7H6NhStYq E613TBoYm5EPWNgGVMWX+Ko/IIqmhaZ39qb8HOLubpQzKoNQhArlT4b4UEV4AIHr W2jjJo3Me1xR9BQsQL4aYB16cmEdH2MtiKrOokWQCPxrvrNQKlr9qEgYRtaQQJKQ CoReaDH46+0N0x3GfZkYVVYnZS6NRcUk7M7jAgMBAAGjQjBAMA8GA1UdEwEB/wQF MAMBAf8wHQYDVR0OBBYEFApII6ZgpJIKM+qTW8VX6iVNvRLuMA4GA1UdDwEB/wQE AwIBhjANBgkqhkiG9w0BAQsFAAOCAgEAPpBIqm5iFSVmewzVjIuJndftTgfvnNAU X15QvWiWkKQUEapobQk1OUAJ2vQJLDSle1mESSmXdMgHHkdt8s4cUCbjnj1AUz/3 f5Z2EMVGpdAgS1D0NTsY9FVqQRtHBmg8uwkIYtlfVUKqrFOFrJVWNlar5AWMxaja H6NpvVMPxP/cyuN+8kyIhkdGGvMA9YCRotxDQpSbIPDRzbLrLFPCU3hKTwSUQZqP JzLB5UkZv/HywouoCjkxKLR9YjYsTewfM7Z+d21+UPCfDtcRj88YxeMn/ibvBZ3P zzfF0HvaO7AWhAw6k9a+F9sPPg4ZeAnHqQJyIkv3N3a6dcSFA1pj1bF1BcK5vZSt jBWZp5N99sXzqnTPBIWUmAD04vnKJGW/4GKvyMX6ssmeVkjaef2WdhW+o45WxLM0 /L5H9MG0qPzVMIho7suuyWPEdr6sOBjhXlzPrjoiUevRi7PzKzMHVIf6tLITe7pT BGIBnfHAT+7hOtSLIBD6Alfm78ELt5BGnBkpjNxvoEppaZS3JGWg/6w/zgH7IS79 aPib8qXPMThcFarmlwDB31qlpzmq6YR/PFGoOtmUW4y/Twhx5duoXNTSpv4Ao8YW xw/ogM4cKGR0GQjTQuPOAF1/sdwTsOEFy9EgqoZ0njnnkf3/W9b3raYvAwtt41dU 63ZTGI0RmLo= -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIGBTCCA+2gAwIBAgIQFNV782kiKCGaVWf6kWUbIjANBgkqhkiG9w0BAQsFADBsMQswCQYDVQQG EwJHUjE3MDUGA1UECgwuSGVsbGVuaWMgQWNhZGVtaWMgYW5kIFJlc2VhcmNoIEluc3RpdHV0aW9u cyBDQTEkMCIGA1UEAwwbSEFSSUNBIFRMUyBSU0EgUm9vdCBDQSAyMDIxMB4XDTI1MDEwMzExMTUw MFoXDTM5MTIzMTExMTQ1OVowYDELMAkGA1UEBhMCR1IxNzA1BgNVBAoMLkhlbGxlbmljIEFjYWRl bWljIGFuZCBSZXNlYXJjaCBJbnN0aXR1dGlvbnMgQ0ExGDAWBgNVBAMMD0dFQU5UIFRMUyBSU0Eg MTCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAKEEaZSzEzznAPk8IEa17GSGyJzPTj4c wRY7/vcq2BPT5+IRGxQtaCdgLXIEl2cdPdIkj2eyakFmgMjAtyeju8V8dRayQCD/bWjJ7thDlowg LljQaXirxnYbT8bzRHAhCZqBakYgi5KWw9dANLyDHGpXUdY259ab0lWEaFE5Uu6IzQSMJOAy4l/T wym8GUiy0qMDEBFSlm31C9BXpdHKKAlhvIjMiKoDeTWl5vZaLB2MMRGY1yW2ftPgIP0/MkX1uFIT lvHmmMTngxplH1nybEIJFiwHg1KiLk1TprcZgeO2gxE5Lz3wTFWrsUlAzrh5xWmscWkjNi/4Bpeu iT5+NExFczboLnXOfjuci/7bsnPi1/aZN/iKNbJRnngFoLaKVMmqCS7Xo34f+BITatryQZFEu2oD KExQGlxDBCfYMLgLucX/onpLzUSgeQITNLx6i5tGGbUYH+9Dy3GI66L/5tPjqzlOsydki8ZYGE5S BJeWCZ2IrhUe0WzZ2b6Zhk6JAQIDAQABo4IBLTCCASkwEgYDVR0TAQH/BAgwBgEB/wIBADAfBgNV HSMEGDAWgBQKSCOmYKSSCjPqk1vFV+olTb0S7jBNBggrBgEFBQcBAQRBMD8wPQYIKwYBBQUHMAKG MWh0dHA6Ly9jcnQuaGFyaWNhLmdyL0hBUklDQS1UTFMtUm9vdC0yMDIxLVJTQS5jZXIwEQYDVR0g BAowCDAGBgRVHSAAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATBCBgNVHR8EOzA5MDeg NaAzhjFodHRwOi8vY3JsLmhhcmljYS5nci9IQVJJQ0EtVExTLVJvb3QtMjAyMS1SU0EuY3JsMB0G A1UdDgQWBBSGAXI/jKlw4jEGUxbOAV9becg8OzAOBgNVHQ8BAf8EBAMCAYYwDQYJKoZIhvcNAQEL BQADggIBABkssjQzYrOo4GMsKegaChP16yNe6SckcWBymM455R2rMeuQ3zlxUNOEt+KUfgueOA2u rp4j6TlPbs/XxpwuN3I1f09Luk5b+ZgRXM7obE6ZLTerVQWKoTShyl34R2XlK8pEy7+67Ht4lcJz t+K6K5gEuoPSGQDPef+fUfmXrFcgBMcMbtfDb9dubFKNZZxo5nAXiqhFMOIyByag3H+tOTuH8zuI d9pHRDsUpAIHJ9/W2WBfLcKav7IKRlNBRD/sPBy903J9WHPKwl8kQSDA+aa7XCYk7bJtEyf+7GM9 F5cZ7+YyknXqnv/rtQEkTKZdQo5Us18VFe9qqj94tXbLdk7PejJYNB4OZlli44Ld7rtqfFlUych7 gIxFOmiyxMQQYrYmUi+74lEZvfoNhuref0CupuKpz6O3dLv6kO9T10uNdDBoBQTkge3UzHafTIe3 R2o3ujXKUGPwyc9m7/FETyKLUCwSU/5OAVOeBCU8QtkKKjM8AmbpKpe3pHWcyq3R7B3LmIALkMPT ydyDfxen65IDqREbVq8NxjhkJThUz40JqOlN6uqKqeDISj/IoucYwsqW24AlO7ZzNmohQmMi8ep2 3H4hBSh0GBTe2XvkuzaNf92syK8l2HzO+13GLCjzYLTPvXTO9UpK8DGyfGZOuamuwbAnbNpE3Rfj V9IaUQGJ -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIGwDCCBKigAwIBAgIQKmCG1NTeRcleS5j7vy+/JjANBgkqhkiG9w0BAQsFADCBpjELMAkGA1UE BhMCR1IxDzANBgNVBAcTBkF0aGVuczFEMEIGA1UEChM7SGVsbGVuaWMgQWNhZGVtaWMgYW5kIFJl c2VhcmNoIEluc3RpdHV0aW9ucyBDZXJ0LiBBdXRob3JpdHkxQDA+BgNVBAMTN0hlbGxlbmljIEFj YWRlbWljIGFuZCBSZXNlYXJjaCBJbnN0aXR1dGlvbnMgUm9vdENBIDIwMTUwHhcNMjEwOTAyMDc0 MTU1WhcNMjkwODMxMDc0MTU0WjBsMQswCQYDVQQGEwJHUjE3MDUGA1UECgwuSGVsbGVuaWMgQWNh ZGVtaWMgYW5kIFJlc2VhcmNoIEluc3RpdHV0aW9ucyBDQTEkMCIGA1UEAwwbSEFSSUNBIFRMUyBS U0EgUm9vdCBDQSAyMDIxMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAi8Lnr2WbBWeW yQ0kudAOZPzO4iQYLIR/d1HLBBE2uF7taXGnnuQlCZdnwUfCz5EWNmI9OAThUYL/rNK0ad0u7BGj Re5raztMv4yNpB6dEbnpOPl6DgyY4iMd0U5j1Oe4QUT7a69r2h/TxZGIW6SJktGB5ow5WKDWaUOp rZhSWG7bCvtrz2j646ReOkVzmAfqXwJy3gyls5+uqR23HbP8ilnnbnJlrfUwlCMH84IWSzWYnFO7 L8rkWtnHjR38mJn7LKSCa/AqH44LX3FcXK5CeymJgcsDo5nKiJ4LQAlBM9vmWHr9rplwwFoP1hOG cS92afyQ3dstbtHym/Uaa55vFYx68EsooCI4gCRsNqQ78jCR83gTz8E/NavxHREjtUMingGStxgC 5RHRgtsVAMxhN8EqfJrh0LqzUEbugqydMfj7I+IDAEhwowkmeRVTYPM4XK046oEAYxS5M17dC9ug RQcaMwn4TbSnAqZp9MJZBYhlhVauS8vg3jx9LRrI6fsfo2FK1ioTrXdMGhibkQ9Y2AZUxZf4qj8g iqaFpnf2pvwc4u5ulDMqg1CECuVPhvhQRXgAgetbaOMmjcx7XFH0FCxAvhpgHXpyYR0fYy2Iqs6i RZAI/Gu+s1AqWv2oSBhG1pBAkpAKhF5oMfjr7Q3THcZ9mRhVVidlLo1FxSTszuMCAwEAAaOCASEw ggEdMA8GA1UdEwEB/wQFMAMBAf8wHwYDVR0jBBgwFoAUcRVnyMjJvXVdctA4GGqd83EkVAswTAYI KwYBBQUHAQEEQDA+MDwGCCsGAQUFBzAChjBodHRwOi8vcmVwby5oYXJpY2EuZ3IvY2VydHMvSGFy aWNhUm9vdENBMjAxNS5jcnQwEQYDVR0gBAowCDAGBgRVHSAAMB0GA1UdJQQWMBQGCCsGAQUFBwMC BggrBgEFBQcDATA6BgNVHR8EMzAxMC+gLaArhilodHRwOi8vY3JsLmhhcmljYS5nci9IYXJpY2FS b290Q0EyMDE1LmNybDAdBgNVHQ4EFgQUCkgjpmCkkgoz6pNbxVfqJU29Eu4wDgYDVR0PAQH/BAQD AgGGMA0GCSqGSIb3DQEBCwUAA4ICAQDAze/rVni3cYGP1BWkxXnIVynqHcC6Cw7kbkXoXBZAqF9A NQAKoz2YQ+WyzrparKVilMG6p6mdWTKeLy6eQDRhFSu9DrMYr4TBYxIp2e5ZLQBk96xbG3wSepBC kTDA0MA0NybzM0gF5nTa82BMCED8EQXliWWsgqUorXuk7msadosX793M42llmAcj1zJkADzlyzi4 gcPvIc4skfmz1OW9jxxwyaR/41VYzPVyw7m0rsL9n3v5PCFJ1zrldYayKq9gOdwOZInNJQgPU6im XEFBZWdqdR+xdL7qHVMkdzBRoFqnYjr2bonw87Rxk7dAsdzmUUPJTIayzDTdtU78XkOl0FivJWBX DQPiVB2Z5NdOVPUANDnADA5octuC3YTG2ncpG9aNKM4ggAyr3imdnz5UZdioYkecv2TVJHT2lXE5 PH/ifJs8p9+WFgyzFsW97BUF1nKsZiUWvlyPY9C9PKuhI4OMeTEA99YIRXYjXDSZq300ko5sXUVK VG2osAR4Sr3wqS5hLGNBfym7FZwsQ+/ja+fenEALAbAiEMpI9h71lpMpIhdx7rdRznNeSOok/pVM aqvBZ0IvAq+yXb7d8C+tAND0IcUxYyyJqkE789RJdcGZO9JMXv4inBFIEuMAMlGPxzcroEx+ehDX yGR0asw7CooV2DCfiEJlIsKzLcPZew== -----END CERTIFICATE----- --- # Unknown .. \_compute\_ops\_card: Compute operations ================== For general information on the compute component, visit the :ref:\`cloud/os\_overview/os\_components/compute:compute\` page. .. toctree:: :maxdepth: 2 :hidden: instance\_create instance\_manage instance\_snap\_create instance\_download instance\_deletion instance\_resize instance\_rescue instance\_root\_storage\_increase image\_upload keypair\_create .. grid:: 3 .. grid-item-card:: |osnova| \*\*Instance: create\*\* :link: compute\_inst\_create\_card :link-type: ref .. grid-item-card:: |osnova| \*\*Instance: manage\*\* :link: compute\_inst\_manage\_card :link-type: ref .. grid-item-card:: |osnova| \*\*Instance: resize\*\* :link: compute\_inst\_resize\_card :link-type: ref .. grid:: 3 .. grid-item-card:: |osnova| \*\*Instance: snapshot create\*\* :link: compute\_inst\_snap\_create\_card :link-type: ref .. grid-item-card:: |osnova| \*\*Instance: download\*\* :link: compute\_inst\_download\_card :link-type: ref .. grid-item-card:: |osnova| \*\*Instance: delete\*\* :link: compute\_inst\_delete\_card :link-type: ref .. grid:: 3 .. grid-item-card:: |osnova| \*\*Instance: rescue\*\* :link: compute\_inst\_rescue\_card :link-type: ref .. grid-item-card:: |osnova| \*\*Instance: root storage increase\*\* :link: compute\_inst\_root\_storage\_increase\_card :link-type: ref .. grid-item-card:: |osnova| \*\*Image: upload\*\* :link: compute\_inst\_img\_upload\_card :link-type: ref .. grid:: 3 .. grid-item-card:: |osnova| \*\*KeyPair: create\*\* :link: compute\_keypair\_create\_card :link-type: ref .. |osnova| image:: /cloud/\_img/nova\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_dashboard\_card: Horizon Dashboard ================= The Horizon Dashboard is the main interface to interact with OpenStack. .. note:: Every cloud infrastructure at CINECA has a separate instance of the Horizon Dashboard, please refer to the :ref:\`cloud/systems/index\_system\_specifics:Cloud Specifics\` to know how to access a specific system. As a first step, you are asked to login using your HPC credentials and 2FA (see :ref:\`general/users\_account:hpc credentials\`). The dashboard gives users a graphical interface to monitor and manage the cloud resources associated to your projects. .. image:: ../../\_img/openstack\_dashboard\_overview.png You find the current active project on the top left of the screen, next to the CINECA logo. By clicking on the drop-down menu, you can choose the project you want to interact with, while the overview panel shows available quotas and the used resources for the selected project. On the left menu, you have various pages to interact with each OpenStack component. In each page, you have the list of all the resources of the corresponding type that have been created inside the project. For example, the \*"Instances"\* page shows all the virtual machine instances that are currently in the project, with some information on their status: .. image:: ../../\_img/openstack\_instances\_overview.png For each of these pages, you usually find on top of the page a search toolbar and a button to create new resources. The main tool to interact with OpenStack resources is the drop-down menu in the \*\*"Actions"\*\* column (on the right side). This gives you all the possible actions that can be performed on the specific resource. Clicking on the name of the resource opens a dedicated page that allows you to inspect all the resource's properties. --- # Unknown .. \_command\_line\_card: Command Line Interface ====================== The OpenStack Command Line Interface (CLI) serves as a powerful tool for users to interact with their OpenStack cloud environment directly from the terminal or command prompt. The CLI allows Users to perform the same operations that can be usually done via OpenStack dashboard, such as creating Instances, Volumes, Networks etc and many more, and it offers a convenient and efficient way to manage resources, automate tasks, and perform various operations within an OpenStack deployment. Access to the OpenStack CLI is granted thanks to a feature called Application Credentials (in the following AC) that is available on CINECA OpenStack infrastructure. .. important:: By default, the OpenStack CLI is not enabled to external users. In order to have access to the OpenStack CLI service, you need to provide to CINECA a static IP address of the machine from which you will launch the OpenStack commands. Please contact to CINECA support team requesting to add the static IP to the ones allowed to use this service. Installation ------------ To install the OpenStack CLI, you need to have a working installation of Python in your \*PATH\*. If you do not have it, you can download it from the official \`Python website \`\_ Using a python virtual environment will help you manage dependencies and avoid conflicts with other Python packages. The recommended method to install the CLI is using the python package in the PyPI repository. .. code:: bash pip install python-openstackclient==#.# .. important:: - Check that the \*\*version #.#\*\* is the one compatible with the specific cloud system your project is allocated on in :ref:\`cloud/systems/index\_system\_specifics:cloud specifics\`. With greater versions some commands might not work. - For some services, you might need to install additional packages, please see the \`OpenStack CLI Documentation \`\_ Alternatively, the CLI is installable via standard package manager on Linux machines. Check your own distribution's repositories to check if the package is available. Configuration ------------- To use the CLI, you need cloud-specific elements: 1. System \*\*SSL certificate chain\*\*: visit the :ref:\`cloud/systems/index\_system\_specifics:Cloud Specifics\` page of infrastructure where your project is allocated, to download the certificate 2. Valid \*\*OpenStack Application Credentials\*\*: see section :ref:\`cloud/os\_overview/management\_tools/command\_line:application credentials creation\` 3. \*\*Environmental variables\*\*: see section :ref:\`cloud/os\_overview/management\_tools/command\_line:setting the environment variables\` Application credentials creation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Application Credentials (ACs) are used to securely authenticate users with the OpenStack CLI without exposing user passwords. ACs use a unique \*\*"Application Credential ID"\*\* and corresponding \*\*"secret string"\*\* for authentication, ensuring privacy. Users can also delegate role assignments to ACs, controlling authorization levels. Each tenant has its own set of ACs, requiring separate generation ACs for each tenant. Each user has to generate their own application credentials. To generate an application credential, you need to login into the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`, select the project for which the Application Credentials are needed, then go to \*"Identity → Application credentials"\* page. You then need to select \*"Create Application Credential"\* and fill out the mandatory fields: - \*\*name\*\*: provide a name for your AC - \*\*expiration date\*\*: set an expiration for your AC -> you can create AC also without expiration date, but please be aware that these will be automatically cancelled at midnight of the creation day. - \*\*roles\*\*: If you want the AC to have all your available roles, please do not select anything. If by accident, you selected an item in the roles list, you have to re-create from scratch the AC. .. warning:: In CINECA HPC Cloud infrastructure, it is possible to create AC with duration up to \*\*7 days\*\*. Application Credentials with expiration time greater that 1 week will be \*\*automatically removed\*\*. After selecting the button \*"Create Application Credentials"\*, the interface shows you the ID and secret of the generated Application Credential. Download the Application Credential file by clicking on the button \*"Download openrc file"\* or, in alternative, \*"Download cloud.yaml file"\*. .. important:: Remember that the secret will not be available after closing the page, so you must capture it or download it. If you have not saved the AC secret, you have to re-create from scratch the AC. Setting the Environment Variables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The OpenStack CLI uses environment variables to store authentication details, such as the OpenStack Identity (Keystone) service endpoint, authentication method, and credentials. By setting these environment variables, you can configure the OpenStack CLI to authenticate with your cloud environment and perform various operations. In particular, you need to define an environment variable called \*"OS\_CACERT"\* with the the full path to your SSL certificate chain file (visit the :ref:\`cloud/systems/index\_system\_specifics:Cloud Specifics\` page of infrastructure where your project is allocated, to download it). You can set this variable in different ways depending if you have downloaded the AC in the previous step as \*openrc\* or \*yaml\* file. .. tab-set:: .. tab-item:: openrc file .. dropdown:: Linux Each time you would like to use the AC, you have to source the openrc file and export the \*"OS\_CACERT"\* variable: .. code:: bash source app-cred-...-openrc.sh export OS\_CACERT=/ .. dropdown:: Windows In order to use the app-cred-...-openrc.sh work in Windows Command Prompt, some changes are needed: - remove the first line with \*#!/usr/bin/env bash\* - replace all \*export\* commands in the file with \*set\* commands - remove all double quotes (\*"\*) - rename the file app-cred-...-openrc.bat Original app-cred-...-openrc.sh for Linux .. code:: bash #!/usr/bin/env bash export OS\_AUTH\_TYPE=v3applicationcredential export OS\_AUTH\_URL=https://clouddev.hpc.cineca.it:5000 export OS\_IDENTITY\_API\_VERSION=3 export OS\_REGION\_NAME="RegionOne" export OS\_INTERFACE=public export OS\_APPLICATION\_CREDENTIAL\_ID= export OS\_APPLICATION\_CREDENTIAL\_SECRET= New app-cred-...-openrc.bat for Windows .. code:: bash set OS\_AUTH\_TYPE=v3applicationcredential set OS\_AUTH\_URL=https://clouddev.hpc.cineca.it:5000 set OS\_IDENTITY\_API\_VERSION=3 set OS\_REGION\_NAME=RegionOne set OS\_INTERFACE=public set OS\_APPLICATION\_CREDENTIAL\_ID= set OS\_APPLICATION\_CREDENTIAL\_SECRET= Then, each time you would like to use the AC, you have to source the openrc file and export the \*"OS\_CACERT"\* variable: .. code:: bash call .\\path\\to\\app-cred-...-openrc.bat set OS\_CACERT=\\ If you prefer to use Windows PowerShell, you have to replace all \*export\* commands in the app-cred-...-openrc.sh file with \*$env:\* commands and remove the extension of the file. .. tab-item:: clouds.yaml You have to edit the \*yaml\* file and add the "cacert" line with the correct indentation as in the following: .. code:: bash clouds: : auth: auth\_url: application\_credential\_id: "" application\_credential\_secret: "" region\_name: "RegionOne" interface: "public" identity\_api\_version: 3 auth\_type: "v3applicationcredential" cacert: "/" \`\`\`\` is the name that will allow your system to refer to that specific authentication, while \`\`\`\` is the URL of the CINECA HPC Cloud infrastructure where the AC have been created. These two fields will be automatically compiled when you download the yaml file from OpenStack together with the AC secret and ID. Finally you must set the environment variable \`\`OS\_CLOUD=\`\` or the flag \`\`--os-cloud=\`\` in openstack command to use this AC. Using OpenStack CLI ------------------- Once you have set up the OpenStack CLI and configured the necessary environment variables, you can start using the CLI to interact with your cloud environment. The OpenStack CLI provides a wide range of commands for managing resources, performing operations, and automating tasks within an OpenStack deployment. Here are two basic OpenStack CLI commands that you can use to test if the CLI is working correctly: .. code:: bash openstack project list # List all the project associated with the user openstack server list # List all the servers in the project More OpenStack commands can be found at the following link: \`OpenStack CLI Commands \`\_ For CLI documentation refer to the following link: \`OpenStack CLI Documentation \`\_ --- # Unknown .. \_infrastructure\_as\_code\_card: Infrastructure as a Code ======================== The term Infrastructure as Code (IaC) refers to a methodology for the provisioning and management of cloud resources. In particular, it consists on treating infrastructure somewhat as software instead of relying on manual operations. The key advantages of IaC can be summarize as follows: - Automation. - Idempotency. - Version control. - CI/CD. - Documentation. Visit the section :ref:\`cloud/tutorials/index\_tutorials\_and\_repos:terraform/opentofu/ansible repositories\` to get further insight on the usage of IaC to manage resources on CINECA HPC cloud infrastructures. Declarative and procedural approaches to IaC -------------------------------------------- There are two primary approaches to Infrastructure as Code (IaC): declarative and procedural (or imperative). The \*\*declarative approach\*\* relies on the description of the desired final end state of the infrastructure, which is gathered in configuration files. Then, the tools used (e.g., Terraform, OpenTofu) are in charge of interpreting the configuration and apply all the actions needed to put it in place. This methodology is especially useful for tasks like provisioning cloud resources (e.g., servers, networks, load balancers). Terraform is an open-source infrastructure as code software tool created by HashiCorp. It enables users to define and provision datacenter infrastructure using a declarative configuration language known as HashiCorp Configuration Language (HCL), or optionally JSON. Terraform manages external resources such as public cloud infrastructure, private cloud infrastructure, network appliances, software as a service, and platform as a service with a code. For more information, visit the official \`Terraform \`\_ website. The \*\*procedural approach\*\* involves outlining the sequence of steps required to achieve the final end state of the infrastructure, rather of a description of the state itself. It is frequently used in configuration management tasks, such as installing software packages on newly provisioned servers. The most popular tools in this category include Ansible, Puppet, and Chef. In particular, Ansible is an open-source automation tool that simplifies IT tasks such as configuration management, application deployment, and cloud provisioning. Its agentless architecture makes it highly efficient for managing infrastructure. For more information about ansible, visit the official \`Ansible \`\_ website. Each method entails distinct advantages, making the choice among them dependent on project needs and team preferences. It is worth noting that, while presented as alternatives, these approaches are not mutually exclusive and can complement each other within different aspects of a single project. --- # Unknown Download RCM Software ^^^^^^^^^^^^^^^^^^^^^ Requirements: \* Microsoft Windows 7, 8, 10, 11. \* Linux: Ubuntu from 16.04, CentOS 7 (other distributions have been not tested) \* Apple MacOS Moajave or newer (both version, intel and apple silicon cpu are supported) .. note:: On Linux O.S. the user must modify the permission of the precompiled executable RCM with the command: \`\`chmod +x \`\` \*\*Download:\*\* \* \`GitHub repository \`\_ (github) \* \`Old version \`\_ Basic Usage of RCM ^^^^^^^^^^^^^^^^^^ \*\*Client/Server Interaction\*\* RCM operates as a client/server application. Every interaction with the application requires the server to execute specific tasks, which may take some time depending on your Internet connection's bandwidth and latency, as well as the current workload of the clusters. To start \*\*RCM\*\* application, follow the described steps: 1. \*\*Launch the RCM executable\*\* 2. Input the \*\*Connection Details\*\*: provide the required hostname (see the table below) and your username. In case you already have a previous saved session, select it from the drop-down menu. 3. \*\*Authenticate\*\*: If you have a valid autentication certificate (:ref:\`general/access:How to manage authentication \*certificates\*\`), leave the \[Password\] field empty. Then, press the :bdg-black-line:\`Login\` button. .. image:: img/rcm1.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link \*\*Hostname Table\*\* +----------------+---------------------------+ | \*\*Cluster\*\* | \*\*Hostname\*\* | +================+===========================+ | Leonardo | rcm.leonardo.cineca.it | +----------------+---------------------------+ | Galileo100 | rcm.g100.cineca.it | +----------------+---------------------------+ RCM can manage multiple sessions. To start a new one, i.e. one for each different cluster, just click on :bdg-black-line:\` + \` button in the tab bar (see the figure). Then, input the connection details for the new session. .. image:: img/rcm2.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link Once logged, a new tab will show the list of available remote display sessions. In case you did not create a display session yet, the list will be empty. .. image:: img/rcm3.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link How to create a new Display ^^^^^^^^^^^^^^^^^^^^^^^^^^^ To create a new display, inside a session, just click on :bdg-black-line:\` + \` button as shown in the figure below. .. image:: img/rcm4.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link Once the connection is estabilished, a new window for the setting of connection will appear (see the figure below). The configuration options are: .. list-table:: :widths: 35 65 :header-rows: 1 :class: tight-table \* - \*\*Option\*\* - \*\*Description\*\* \* - Scheduler - Choose between \*\*SSH\*\* (no scheduler) or \*\*Slurm\*\*. \* - Account - Select one of your project account \* - Queue - Specify the SLURM partition where the job will run. \* - QoS - Choose the proper quality of service (QoS) wich affects the job wall-time limits and permissions. \* - Memory - Define the RAM memory (GB) required for your job. \* - Time - Set the job walltime (up to 24 h). \* - GPU - Specify the number of GPUs \* - Service Type - The default serive is the latest version of \*\*VNC\*\*. \* - Window Manager - Choose a window manager for the VNC session: \* Openbox GPU only: Supported exclusively on GPU nodes. THis preloads the visualization stack (e.g. Paraview, VTK, and other pakages). \* Openbox/Fluxbox: Supported on all nodes. To use visualization applications, load the required module (e.g.: \`module load paraview\`). .. note:: Hover your cursor over any configuration element for additional information about its functionality. .. image:: img/rcm5.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link .. image:: img/rcm6.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link Click on :bdg-black-line:\` Ok \` button in the dialog window, a remote display session will be created and the user will automatically attached to it. Also, a Turbo VNC window will be open (see the figure below). .. image:: img/rcm7.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link .. note:: The creation of a display can take some time depending on the workload of the cluster. When a new display is created, a new item, showing the display details, will be added to the list of available remote display session on the RCM tab. Especially, the display details are: \* \*\*Name:\*\* name of the display session \* \*\*Status:\*\* the condition of the remote display session, pending (starting up the display session), valid (display session is running), killing (deleting the display session). \* \*\*Time:\*\* the remaining time before the session will ends (if the display session has no time limit, this value is replaced by the symbol "~" ). Note that each display session has a time limit: over that time limit, the display will be automatically killed and not saved data will be lost. \* \*\*Resources:\*\* the node of the cluster on which the remote display session has been created. .. image:: img/rcm8.png :align: center :class: no-scaled-link .. image:: ../general/img/spacer.png :align: center :height: 20 px :class: no-scaled-link How to share a Display ^^^^^^^^^^^^^^^^^^^^^^ Sharing a remote display means to give to another user the possibility to access to a specific remote display session you have created. The sharing of a remote display session is done by means of a \`\`.vnc\`\` file that as to be saved by the owner of the display session and opened by the user who has to access to the shared display session. To share a display session: \* click on :bdg-black-line:\`SHARE THE REMOTE DISPLAY SESSION VIA FILE\` button related to the remote display session you want to share. \* A dialog will prompt the user to select a location for saving a file. \* Send the saved file to the users who need to access to the shared display session. To connect to a shared display session click on :bdg-black-line:\`Open\` button from the :bdg-black-line:\`File\` menu and select the received \`\`.vnc\`\` file. How to kill a Display ^^^^^^^^^^^^^^^^^^^^^ Display sessions can be killed by pressing the :bdg-black-line:\`KILL THE REMOTE DISPLAY SESSION\` button. Just press the :bdg-black-line:\`x\` button in the row associated with the remote display session you don’t want to use anymore, and it will be removed from the list of the available displays. This operation can take some time, depending on the workload of the clusters. Note that by pressing it, the relative display will be not reachable anymore and you will lose not saved data. Running a GUI-based Software ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To execute a GUI-based application you have multiple options: .. tab-set:: .. tab-item:: On Standard Resources 1. Load the module for the required software (i.e. paraview) \* open the terminal within the graphical session of RCM .. code-block:: bash module load rcm module load paraview 2. Launch the software form the terminal, i.e.: .. code-block:: bash paraview .. tab-item:: On GPU Resources If you are using GPU resources (i.e. using the setting \*\*SSH\*\* or \*\*SLURM\*\*), additional steps are required: 1. Load all the required modules .. code-block:: bash module load rcm module load paraview module load virtualgl 2. Launch the software form the terminal by using \`\`vlgrun\`\`, i.e.: .. code-block:: bash vlgrun paraview .. tab-item:: For \*\*Openbox GPU Only\*\* Window Manager If you selected the Openbox GPU only window manager when creating the display, you only need to execute the visualization software using the vglrun command (i.e. for paraview): .. code-block:: bash vlgrun paraview These steps ensure proper execution of visualization tools, particularly when utilizing GPU resources for accelerated performance. --- # Unknown .. \_megaride\_card: MEGARIDE ========= \*\*PAGE UNDER CONSTRUCTION\*\* Now under configuration. The HPC cloud infrastructure, named MEGARIDE, is built using \`OpenStack Overview\` (version to be confirmed). .. note:: - It is possible to access MEGARIDE via Horizon Dashboard at \*\*TO BE ADDED\*\* - For CLI access, the certificate chain can be downloaded here \*\*TO BE ADDED\*\* .. grid:: 2 .. grid-item-card:: |oslogo| \*\*Openstack Overview\*\* :link: os\_overview\_card :link-type: ref .. grid-item-card:: |oslogo| \*\*Operative Manual\*\* :link: operative\_manual\_card :link-type: ref System architecture ^^^^^^^^^^^^^^^^^^^ .. image:: ../\_img/megaride.png \*\*CPU nodes\*\* - 150 Interactive OpenStack Nodes - 50 nodes, x2 Intel 8592V, 64c, 2 TB DDR5 - 100 nodes, x2 Intel 6766E, 144c, 512 GB DDR5 \*\*GPU nodes\*\* - 28 Interactive OpenStack Nodes (AMD 9534, 64c) - equipped with: - 96 GPUs Nvidia L40s (AI/Graph 3D) - 16 GPUs Nvidia H100 NVL2 (AI+HPC) \*\*Storage\*\* - 2 PB Capacity-optimized/HDD Ceph Storage System timeline ^^^^^^^^^^^^^^^ - to be announced: Early Availability - to be announced: Start of Pre-Production - to be announced: Start of Production Flavors/Images/Openstack services ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. dropdown:: Flavors TO BE ANNOUNCED .. dropdown:: Images TO BE ANNOUNCED .. dropdown:: OpenStack Services TO BE ANNOUNCED .. |oslogo| image:: /cloud/\_img/openstack\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_gaia\_card: GAIA ==== \*\*PAGE UNDER CONSTRUCTION\*\* Now under configuration, it is composed of 420 general purpose nodes and 80 GPU nodes, also built on OpenStack (version to be defined). It is not only the biggest of the two infrastructures, but also the only accelerated one, mounting Nvidia GPUs models: A30, L40s and H100NVL. The HPC cloud infrastructure, named GAIA, is built using \`OpenStack Overview\` (version to be confirmed). .. note:: - It is possible to access GAIA via Horizon Dashboard at \*\*TO BE ADDED\*\* - For CLI access, the certificate chain can be downloaded here \*\*TO BE ADDED\*\* .. grid:: 2 .. grid-item-card:: |oslogo| \*\*Openstack Overview\*\* :link: os\_overview\_card :link-type: ref .. grid-item-card:: |oslogo| \*\*Operative Manual\*\* :link: operative\_manual\_card :link-type: ref System architecture ^^^^^^^^^^^^^^^^^^^ .. image:: ../\_img/gaia\_architecture.png \*\*CPU nodes\*\* - 420 Interactive OpenStack Nodes - Each node consist of: - 2x CPU Intel Xeon Sierra Forrest 144 cores 2.2GHz - 1 TiB of DDR5 6400 MT/s \*\*GPU nodes\*\* - 80 Interactive OpenStack Nodes - Each node consist of: - 2x CPU Intel Xeon Emerald Rapids Platinum 8592+ 64 cores 1.9GHz - 1 TiB of DDR5 4800 MT/s - some equipped with 256 GPUs: - 80 GPUs Nvidia A30 (HPC) - 112 GPUs Nvidia L40s (AI/Graph 3D) - 64 GPUs Nvidia H100 NVL (AI+HPC) \*\*Storage\*\* - 2 PB IOPS-optimized/SSD Ceph Storage - 8 PB Capacity-optimized/HDD Ceph Storage System timeline ^^^^^^^^^^^^^^^ - to be announced: Early Availability - to be announced: Start of Pre-Production - to be announced: Start of Production Flavors/Images/Openstack services ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. dropdown:: Flavors TO BE ANNOUNCED .. dropdown:: Images TO BE ANNOUNCED .. dropdown:: OpenStack Services TO BE ANNOUNCED .. |oslogo| image:: /cloud/\_img/openstack\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_ada\_card: ADA === In production since 27 September 2021, it is the smaller of the two machines and is composed of 71 nodes. Built on OpenStack (version Zed) can host virtual machines up to 96 vCPUs. The HPC cloud infrastructure, named ADA, is built using \`OpenStack Overview\` (\`version Zed \`\_). .. note:: - It is possible to access ADA via Horizon Dashboard at https://adacloud.hpc.cineca.it - For CLI access: - the certificate chain can be downloaded here :download:\`ADA certificate \` - check that the \*\*version installed is 6.5 or 6.6\*\* with \`openstack --version\` command. With greater versions some commands might not work. .. grid:: 2 .. grid-item-card:: |oslogo| \*\*Openstack Overview\*\* :link: os\_overview\_card :link-type: ref .. grid-item-card:: |oslogo| \*\*Operative Manual\*\* :link: operative\_manual\_card :link-type: ref System architecture ^^^^^^^^^^^^^^^^^^^ .. image:: ../\_img/ada\_architecture.png Hardware Details - nodes ------------------------ .. list-table:: :widths: 30 50 :header-rows: 1 :class: tight-table \* - \*\*Type\*\* - \*\*Specific\*\* \* - Models - Dual-socket Dell PowerEdge \* - Nodes - 71 Interactive OpenStack Nodes \* - Processors - 2xCPU Intel CascadeLake 8260 24 cores 2.4GHz with hyperthreading \* - Cores - 48 cores/node \* - RAM - 768 GB \* - Internal Network - 100Gbs Ethernet interconnection \* - Storage - 2TB SSD Disks and Filesystems --------------------- - 1 PB NVMe/SSD Ceph Storage - This cloud infrastructure is tightly connected both to the LUSTRE storage of 20 PB raw capacity, and to the GSS storage of 6 PB seen by all other infrastructure. System timeline ^^^^^^^^^^^^^^^^ - \*\*05 Aug 2021\*\*: Early Availability - \*\*01 Sept 2021\*\*: Start of Pre-Production - \*\*27 Sept 2021\*\*: Start of Production Flavors/Images/Openstack services ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. dropdown:: Flavors .. list-table:: :widths: 50 50 50 50 50 :header-rows: 1 :class: tight-table \* - \*\*Flavor Name\*\* - \*\*vCPUs\*\* - \*\*RAM (GB)\*\* - \*\*Disk (GB)\*\* - \*\*Available\*\* \* - fl.ada.xxs - 1 - 7.5 - 10 - Yes \* - fl.ada.xs - 2 - 15 - 30 - Yes \* - fl.ada.s - 4 - 30 - 30 - Yes \* - fl.ada.m - 8 - 60 - 30 - Yes \* - fl.ada.l - 16 - 120 - 30 - Yes \* - fl.ada.xl - 24 - 180 - 30 - On-demand \* - fl.ada.xxl - 48 - 360 - 30 - On-demand \* - fl.ada.full - 96 - 7200 - 30 - On-demand .. dropdown:: Images .. list-table:: :widths: 50 50 50 50 :header-rows: 1 :class: tight-table \* - \*\*Image Name\*\* - \*\*Information\*\* - \*\*Default User\*\* - \*\*Default Access\*\* \* - CentOS-7-x86\_64-GenericCloud-2009 - CentOS-7-x86\_64-GenericCloud-2009.qcow2, last modified 2020-11-12 \`Source \`\_ - centos - SSH keypair \* - CentOS-8-GenericCloud-8.4.2105-20210603.0.x86\_64 - CentOS-8-GenericCloud-8.4.2105-20210603.0.x86\_64, last modified 2021-06-03 \`Source \`\_ - centos - SSH keypair \* - CentOS-Stream-GenericCloud-8-20220913 - CentOS-Stream-GenericCloud-8-20220913.x86\_64, last modified 2022-09-13 \`Source \`\_ - centos - SSH keypair \* - Ubuntu 18.04 LTS (Bionic Beaver) - Ubuntu server 18.04 (Bionic Beaver) LTS for cloud \`Source \`\_ - ubuntu - SSH keypair \* - Ubuntu Server 20.04 LTS (Focal Fossa) - focal-server-cloudimg-amd64.img, last modified 2021-07-20 \`Source \`\_ - ubuntu - SSH keypair \* - Ubuntu Server 21.04 (Hirsute Hippo) - hirsute-server-cloudimg-amd64.img, last modified 2021-07-20 \`Source \`\_ - ubuntu - SSH keypair \* - Ubuntu Server 22.04 LTS (Jammy Jellyfish) - jammy-server-cloudimg-amd64.img, last modified 2022-09-02 \`Source \`\_ - ubuntu - SSH keypair \* - Ubuntu Server 24.04 LTS (Noble Numbat) - noble-server-cloudimg-amd64.img, last modified 2025-03-13 \`Source \`\_ - ubuntu - SSH keypair \* - Rocky Linux 8.9 - File description: https://wiki.rockylinux.org/rocky/image/#about-cloud-images \`Source \`\_ - rocky - SSH keypair \* - Rocky Linux 9.3 - File description: https://wiki.rockylinux.org/rocky/image/#about-cloud-images \`Source \`\_ - rocky - SSH keypair \* - Rocky Linux 9.4 - File description: https://wiki.rockylinux.org/rocky/image/#about-cloud-images \`Source \`\_ - rocky - SSH keypair \* - Debian 12 generic 64-bit AMD/Intel - debian-12-generic-amd64.qcow2 \`Source \`\_ - debian - SSH keypair .. dropdown:: OpenStack Services In addition to the core OpenStack Components :ref:\`cloud/os\_overview/index\_openstack\_overview:openstack overview\`, on ADA we have - Barbican, for secure storage, provisioning and management - Manila, for shared filesystem management - Octavia, for load balancers deployment - Trove, for database management In ADA, the following :ref:\`cloud/os\_overview/os\_components/shares:shares\` types are available: - generic\_type (NFS protocol) - cephfs\_type (CEPHFS protocol) .. |oslogo| image:: /cloud/\_img/openstack\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_compute\_card: Compute ======= This component, based on the \`OpenStack Nova Service \`\_, allows the management of the computing resources. Instances --------- An instance (or virtual machine) is a software-based emulation of a physical computer. It runs an operating system and applications, just like a physical computer. With instances, users can run applications and workloads in the cloud. Instances can be created from an image, or from a snapshot. In the corresponding part of the OpenStack Horizon dashboard \*"Compute → Instances"\*, you can: - visualize the quota of the selected project and its usage in the \*"Overview"\* page - access the list of instances - create a new instance - perform operations (\*\*Actions\*\*) on the selected instance, like attaching/de-attaching volumes, suspend or shut off, creation of snapshots Virtual machines can be of two types: .. grid:: 2 .. grid-item-card:: \*\*Ephemeral\*\* Virtual Machines are booted from an OpenStack image. These images have fixed root volume dimension, dictated from the used flavor. Creating an ephemeral virtual machine is more lightweight for tenant resources, since the root volume won't be accounted on the tenant storage quota. .. grid-item-card:: \*\*Bootable\*\* Virtual Machines are created using a pre-existing volume as root volume. Users can choose freely the size of the root volume of the virtual machine. Creating a virtual machine from a bootable volume has two main advantages: - customizable root volume size - the data of the root volume won't be deleted when deleting the instance (if appropriately configured during creation) .. note:: \*\*Instance affinity and anti-affinity groups\*\* In some cases, users might want to be sure two or more instances run specifically on the same hypervisor (e.g. for ensuring better communication) or on different ones (e.g. to implement high availability). In OpenStack, it is possible to create affinity or anti-affinity groups for this purpose. When creating an instance, it is possible to define whether this will be part of one of the groups available in the tenant. Key Pairs --------- Key Pairs are a method for securing SSH access to instances by using public-key cryptography. When a Key Pair is associated with an instance, it allows users to SSH into the instance securely without using a password. This ensures a higher level of security compared to traditional password-based access. Public-Key Cryptography is a cryptographic system that uses pairs of keys - \*\*public keys\*\* (which can be shared) and \*\*private keys\*\* (which are kept secret). The public key is used to encrypt data, and the private key is used to decrypt it. The public key is injected into the instance at the time of creation. The private key is kept by the user and used to establish an SSH connection to the instance. Image Snapshots --------------- A snapshot is a point-in-time copy of data capturing the current state of a virtual machine. It preserves the state and the data of the VM including its power state (on, off, or suspended) and all its files (such as disks, memory, and network interfaces). Snapshots are generally used to restore a VM after a system failure, bad update, or error. You can find instances snapshots under the \*"Compute → Images"\* section in the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`. .. warning:: - Before taking a snapshot, log in to the virtual machine and shutdown the instance. - Snapshots are stored on the same cloud infrastructure of the instances, but can be downloaded to be stored at a different location. Images ------ An image is a template which contains a specific operating system and pre-installed software. Images hosted on the cloud platform have the \*\*“visibility”\*\* attribute which can assume 4 different values that define which users can use the image for the instance creation and provides information about who uploaded the image. .. list-table:: :widths: 50 50 :header-rows: 1 \* - \*\*Visibility\*\* - \*\*Characteristics\*\* \* - Public - - Uploaded only by CINECA HPC Cloud infrastructure administrators. - Usable by all the users on the cloud platform and appears in the default image list. \* - Private - - Uploaded by any user on the cloud platform (image owner). - Usable only by the image owner. \* - Shared - - Uploaded by any user on the cloud platform (image owner). - By default, usable only by all the users which work on the same project of the image owner. - The image owner can explicitly chose a user or a group of users to share the image to: the user or the group of user have to accept the sharing. \* - Community - - Uploaded by any user on the cloud platform (image owner). - Usable by all the users on the cloud platform and appears in the default image list. .. warning:: Since no control can be performed on \*Community\* images by the CINECA HPC Cloud infrastructure administrators, \*\*Community images are inhibited\*\*. Trying to \*\*upload a Community image\*\* will results in a \*\*blocking error\*\* both from Horizon dashboard and OpenStack CLI. Default images for common operating systems are provided by CINECA and available to be used. The updated list of all the images uploaded by CINECA HPC Cloud infrastructure administrators on each HPC cloud infrastructure can be found in the systems specifics pages at :ref:\`cloud/systems/index\_system\_specifics:Cloud Specifics\`. Alternatively, users can import their own images, as described in the :ref:\`cloud/operative/compute\_ops/image\_upload:image: upload\` .. warning:: On CINECA HPC Cloud machines, users are not allowed building windows virtual machines, even if they have their own windows license. --- # Unknown .. \_shares\_card: Shares ====== The \`OpenStack Manila service \`\_ allows the creation of a filesystem that can be shared among virtual machines in the same tenant (intra-tenant) or in different tenants (extra-tenant). This setup is particularly useful for applications that require consistent and simultaneous access to data across different instances. In CINECA HPC Cloud infrastructure, it is possible to create shares of the following types: - Generic type (NFS protocol) - Cephfs type (CEPHFS protocol) We suggest the users that need a shared filesystem to use the \*\*generic type\*\*. .. note:: \*\*Only intra-tenant\*\* shares are allowed on CINECA HPC Cloud infrastructures. --- # Unknown .. \_storage\_card: Storage ======= This component, based on the \`OpenStack Cinder Service \`\_, allows the management of the storage space in cloud projects. Storage in CINECA HPC Cloud machines is based on Ceph platform. Ceph provides reliable distributed and scalable storage without a single point of failure. Within OpenStack, it is possible to create different block storage devices, that can be used by instances (virtual machines) to store data persistently. Volumes ------- A volume is a block storage device that can be attached to instances. It provides persistent storage for data. Snapshots --------- A snapshot is a point-in-time copy of a volume. It can be used to create new volumes or restore existing volumes. Snapshots use the Copy On Write (COW) technique to create lightweight snapshots of volumes. Backups ------- A volume backup is a copy of a volume that can be used to restore data in case of data loss or corruption (\*\*NOT IN FULL PRODUCTION\*\*). --- # Unknown .. \_database\_card: Database ======== \`Trove \`\_ is a database as a service (DBaaS) component of OpenStack. It is a pluggable service that supports multiple database engines. Database as a service (DBaaS) is a cloud computing managed service offering that provides access to a database without requiring the setup of physical hardware, the installation of software or the need to configure the database. The Database service provides scalable and reliable cloud provisioning functionality for both relational and non-relational database engines. Users can quickly and easily use database features without the burden of handling complex administrative tasks. Cloud users and database administrators can provision and manage multiple database instances as needed. The Database service provides resource isolation at high-performance levels, and automates complex administrative tasks such as deployment, configuration, patching, backups, restorations, and monitoring. --- # Unknown .. \_network\_card: Network ======= \`OpenStack Neutron service \`\_ is a core component of the OpenStack cloud computing platform, which provides "networking as a service" between interface devices managed by other OpenStack components such as Nova (compute) and Cinder (block storage) services. Neutron allows users to create and manage various networking services like VLANs, SDNs (Software Defined Networks), and other complex network topologies. .. image:: ../../\_img/openstack\_network\_topology.png Project networks are isolated and private. These networks are not accessible from outside the OpenStack environment unless routed through an external network. The \*\*external network\*\* is the public-facing network that allows VMs within the tenant networks to access the internet. To operate, a network will need at least a \*\*subnet\*\* and a \*\*router\*\*. A subnet is a range of IP addresses in your project's network, while a router is the virtual device that forwards traffic between your project and the external network. Once the project has a network, it is possible to link to it virtual machines and other resources in order to connect them between each other and to the external network. Subnets ------- A subnet is a range of IP addresses in your project's network. You can create subnets to group instances according to security and operational needs. When you create a subnet, you specify the CIDR block for the subnet, which is a subset of the network CIDR block. .. note:: Typical CIDR reserved for private networks are: \`192.168.0.0/16\`, \`10.0.0.0/8\` or \`172.16.0.0/12\`. Routers ------- With OpenStack routers, you can create and manage multiple networks, define routing rules, and control the flow of traffic between subnets. This flexibility allows you to design and implement complex network topologies to meet the specific requirements of your cloud infrastructure. In addition to basic routing functionality, OpenStack routers support advanced features like \*\*floating IPs\*\*, which allow you to associate a public IP address with a private IP one to enable external access to instances on your private network. The \*\*gateway\*\* is the IP address of the router that connects the subnet to the external network. The gateway IP address is typically the first or last IP address in the subnet range. Floating IPs ------------ In OpenStack, instances are by default assigned private IP addresses from the internal network. These private IPs are not reachable from outside the OpenStack environment. However, by assigning a floating IP to an instance, the instance becomes reachable from the external network, enabling inbound and outbound communication via the internet. When a floating IP is allocated, it is reserved from a pool of available public IP addresses. This pool is configured by the cloud administrator and can be customized based on the organization's requirements. Each cloud project has a limited number of floating IPs controlled by the project quota and these need to be allocated to the project from the central pool prior to their use. Security Groups --------------- A security group acts as a virtual firewall for instances and other resources on a network controlling the inbound and outbound traffic to instances. One security group can be associated with one or more instances and a single instance can have multiple security groups associated to it. It is always possible to modify, add and remove security groups in a virtual machine after its creation. The security group contains one or more security rule which specify the network access. A security rule defines how traffic can enter/exit the VM instance via: - an IP address or CIDR block - an EtherType (IPv4 or IPv6) - a direction (INGRESS or EGRESS) - a port the traffic will pass through For example the rule: \`INGRESS IPv4 TCP 22(SSH) 131.175.44.1\` will imply that the virtual machine will be reachable via SSH on port 22 using TCP protocol from the IP \`131.175.44.1\` --- # Unknown .. \_load\_balancer\_ops\_card: LoadBalancer operations ======================= For general information on the compute component, visit the :ref:\`cloud/os\_overview/os\_components/load\_balancers:load balancer\` page. .. toctree:: :maxdepth: 1 :hidden: lb\_create lb\_troubleshooting .. grid:: 2 .. grid-item-card:: |osoctavia| \*\*Load balancer: create\*\* :link: lb\_create\_card :link-type: ref .. grid-item-card:: |osoctavia| \*\*Load balancer: troubleshooting\*\* :link: lb\_troubleshooting\_card :link-type: ref .. |osoctavia| image:: /cloud/\_img/octavia\_logo.png :width: 35px :class: no-scaled-link --- # Unknown .. \_interactive\_computing\_card: Interactive Computing --------------------- Basic concepts ^^^^^^^^^^^^^^ The interactive computing service provides an alternative approach to computational resources. The service is accessible via a web browser, with an extended JupyterLab interface based on the \`ICE4HPC suite \`\_ developed by E4 analytics. The resources requested through the browser interface are allocated on a dedicated set of nodes, all of which are equipped with GPUs. These GPUs are not shared among users, and their allocation is exclusively granted upon request. On the other hand, the allocated CPUs can be shared if the system is fully utilized. This allows for near-immediate access to the system without waiting time. Once the resources are allocated, the browser session can be closed and quickly restored by accessing the service URL. At the moment the service is available on Galileo100. .. preproduction phase warning .. warning:: The service is in pre-production phase, meaning all the resources are provided with no warranty. In such a pre-production phase the accounting is disabled. How to get access ^^^^^^^^^^^^^^^^^ Every user with computational resources on the cluster hosting the service can access to it. The service can be reached thought the following web address: https://jupyter.g100.cineca.it Requested resources and releases ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ At the opening page, the user will be asked to login with the CINECA cluster credentials. After the login, the system prompts the user to a form where he/she can request the resources needed during the interactive session. The form appears as follow: .. image:: img/iac-form.png :alt: Form to submit resources requests on the cluster :width: 600px :align: center In the form, in analogy to a sbatch SLURM script on cluster login nodes, the user has to select: .. list-table:: Form :widths: 20 80 :header-rows: 1 :class: tight-table \* - Field - How to fill \* - \*\*Slurm reservation\*\* - you can leave it to "None" unless you are assigned to some specific reservation; \* - \*\*Slurm account\*\* - the active account you want to be billed for the session; please note that during the pre-production phase, the accounting is inactive. \* - \*\*Number of cores\*\* - the number of cores requested for the interactive computing session; please note that cores are assigned in over-subscription, which means that in the unlucky scenario in which all the cores of the system are allocated, the user may share the same core with other users (currently maximum five users on the same CPU); \* - \*\*Memory\*\* - the amount of RAM memory requested for the session; \* - \*\*GPU configuration\*\* - the number of GPUs which the user requests; differently from CPUs, they are not shared among users and are assigned exclusively; the number of GPUs is limited, thus please be careful to release the resources you requested when you finish your work (see the session "Logout vs Session shutdown" here below) to let the other users to use them. You can check the availability of resources, in particular GPU ones, by looking at the table at the bottom of the page, wherein each row is displayed the number of nodes with no free GPUs, the ones with a single free GPU and the ones with both the GPUs available; \* - \*\*Time\*\* - the wall time of your interactive session; during this time, you can close and reopen your browser tab/windows with no issues: the session will stay active, and you can re-attach it simply by accessing to the Interactive Computing web url once again; \* - \*\*ICE4HPC Backend environment\*\* - the suite of tools you expect to find during the session execution; see "\`Tools and functionalities\`\_" section for details; \* - \*\*User interface\*\* - only the JupyterLab interface is available so far, so please ignore this menu for now; Once you have filled out the form with your preferred parameters, click the :bdg-black-line:\`Start\` button at the bottom. This action will redirect you to the JupyterLab interface, which runs on the cluster's compute nodes where the user can select the tool or functionality among the available. Tools and functionalities ^^^^^^^^^^^^^^^^^^^^^^^^^ The tools you will see in the JupyterLab interface are "packed" in releases: each tool in each release is pinned at the same version to guarantee compatibility. It is possible to choose the release in the initial form displayed after the login phase: in the drop-down menu, they are labelled with a release date, thus the more the date is recent, the more the tool versions are updated, so as a rule of thumb, you might want to test the most recent release with your code. Currently, the following services are up and running on the interface displayed after your session starts: .. image:: img/iac-launcher.png :alt: JupyterLab launcher :width: 600px :align: center depending on the release you chose at the login phase. Default kernels ~~~~~~~~~~~~~~~ Some default Python/Julia/R/C/C++ environments are provided by default in each release; the packages versions in each release are fixed, in order to guarantee retrocompatibility; updated versions of the environments will be added in new releases. Here below you can find the default kernels provided in the current releases: .. here below I open by default the latest release, and keep close by default the older (still availiable for retrocompatibility) ones .. dropdown:: Release 2024.04 :animate: fade-in-slide-down :color: primary :open: .. card:: Python Several Python environments are provided by default. You can click here below to their name if you need to check the versions of the main packages contained in each of them: .. tab-set:: .. tab-item:: Python 3.11 | python 3.11 .. tab-item:: Ray | ray 2.21.0 .. tab-item:: Dask | python 3.11 | dask 2024.04 .. tab-item:: Tensorflow | python 3.11 | tensorflow 2.12 .. tab-item:: Pytorch | python 3.11 | pytorch 2.3.0 .. tab-item:: Rapids | python 3.11 | rapids 24.04 .. tab-item:: Transformers | python 3.11 | huggingface\_hub 0.24.0 | transformers 4.40.2 .. tab-item:: MDAnalysis | python 3.11 | mdanalysis 2.7 You can obtain the complete list inside the environment by running \`\`!mamba list\`\` in a Jupyter notebook (after selecting the corresponding kernel in the top-right menu). You can also add your custom environments, as described in the section "\`User custom Python environments\`\_". .. card:: Julia | Julia kernel version 1.9.4 is automatically installed in the user's home directory (in the hidden path \`\`~/.julia\`\`) in background at the first login to the service, thus it won't be visible in the very first login. If you need Julia in the very first login of the service please wait some minutes and refresh the page to make it visible. | Being installed in your home directory, you can freely install your packages and in general manage your Julia environment as usual via Pkg package manager: check \`Pkg documentation \`\_ for the details. .. card:: C/C++ C/C++ kernel implementation is based on \`Xeus \`\_; you can use C/C++ instructions inline, like with Jupyter notebooks. You can check the Xeus documentation \`here \`\_. .. image:: img/iac-xeus.png :alt: Xeus C/C++ Jupyter kernel example :width: 300px :align: center .. card:: R R version 4.3.3 is currently provided via \`rlang \`\_ v1.1.4 package. Visual Studio Code ~~~~~~~~~~~~~~~~~~ Visual Studio Code (VSCode) is a very common code editor developed by Microsoft, which offers many advanced features for programming. You can find some tutorials for beginners in the \`official documentation \`\_. From the interactive Computing interface, you can see a VSCode entry in the launcher after you have started the session; clicking on :bdg-black-line:\`Visual Studio Code\`, a new tab/window (depending on your browser settings) will be opened with a web interface containing the VSCode. You can work with VSCode as long as your JupyterLab session is running and your resources are allocated; to stop your session in advance and release the resources, you need to stop the JupyterLab session in the original tab: see "Logout vs Session Shutdown" section for details. Monitoring tools ~~~~~~~~~~~~~~~~ On the (very) left side of your dashboard there is a vertical menu, which allows the user to access some additional functions; one of the buttons is called "GPU Dashboards". From here, you can monitor your resources usage in real-time; in particular: - in the "Machine Resources" section, you can monitor CPUs, memory utilization, and network and I/O bandwidth. - if you requested GPUs from the initial form, you would see several additional menu to monitor, for instance, GPUs utilization, memory bandwidth and occupation, PCI throughput. .. image:: img/iac-gpu\_monitoring.png :alt: Resources monitoring dashboard example :width: 700px :align: center The dashboard is developed by nVidia with the \`jupyterlab-nvdashboard \`\_ plugin. User custom Python environments ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can create your own customized Python environment and display them on the Jupyter launcher. You can proceed as follows: - open a terminal \*\*from your launcher\*\* .. warning:: Be careful to run the following commands from the terminal in the launcher, i.e. clicking on :bdg-black-line:\`Terminal\` in the Jupyter web interface) and not via ssh, in order not to have versions mismatches and/or errors in some cases. .. .. note:: Unless specifically reasons, it is suggested not to have others conda environments activated (e.g. the \*base\* environments from - in this terminal a conda instance is already active, specific for the release you chose at the login phase (see section "\`Requested resources and releases\`\_"), thus you can run the commands below: .. code-block:: bash source $CONDA\_PREFIX/etc/profile.d/conda.sh conda init bash conda create -n ipykernel -c conda-forge --override-channels conda activate conda install -c conda-forge --override-channels python -m ipykernel install --user --name --display-name After you will refresh the page, a new environment will appear in the launcher of the dashboard; the new kernel is also listed in the drop-down menu of every new Jupyter notebook so that you can use the packages you installed when you created the environment. .. note:: In case you need to delete the environment from your launcher, you can click on :bdg-black-line:\`Terminal\` in your Jupyter launcher and run the following commands (\*\*N.B. from the terminal in the Jupyter launcher\*\*): .. code:: bash # optional: delete the environment from your home directory source $CONDA\_PREFIX/etc/profile.d/conda.sh conda remove --name --all # remove kernel from Jupyter launcher jupyter kernelspec uninstall You can also further customize your kernels running specific bash script along with your custom Python environments; for instance, after the procedure described so far, a new json file is created in your home in the following path: .. code:: bash $HOME/.local/share/jupyter/kernels//kernel.json whose content is similar to the following: .. code:: json { "argv": \[\ "/.conda/envs//bin/python",\ "-m",\ "ipykernel\_launcher",\ "-f",\ "{connection\_file}"\ \], "display\_name": "", "language": "python", "metadata": { "debugger": true } } This JSON file describes what Jupyterlab runs when clicking on the related button in the launcher page; you can create a bash script to be added in this JSON file to be properly run before your environment execution; this allows for instance, to load any module from the cluster and make it visible during your Python environment execution. For instance, in the following example, we create a file called \`\`wrapper.sh\`\` in the same folder of the JSON file (you can choose the path and the name you prefer): - click on :bdg-black-line:\`Terminal\` on your launcher; - move inside the folder you want and open a new file with your favorite text editor (e.g. \`\`wrapper.sh\`\` using vim or nano or emacs); - write a bash script as the following: .. code:: bash #!/bin/bash ### You can add here whatever bash commands you like, like for example, "module load" commands module load module load ### Remember the next line! exec "$@" - Make this file executable: .. code:: bash chmod +x wrapper.sh - Edit your \`\`kernel.json\`\` file (e.g. \`\`nano kernel.json\`\`) by adding the following line: .. code:: bash "/.local/share/jupyter/kernels//wrapper.sh", as the first entry in your argv JSON field. Thus, in the end, your file should look like the following: .. code:: json { "argv": \[\ "/.local/share/jupyter/kernels//wrapper.sh",\ "/.conda/envs//bin/python",\ "-m",\ "ipykernel\_launcher",\ "-f",\ "{connection\_file}"\ \], "display\_name": "", "language": "python", "metadata": { "debugger": true } } .. warning:: You need to replace \`\`\`\`, \`\`\`\` and \`\`\`\` with your specific paths, without using environment variables (e.g. you cannot replace \`\`\`\` with \`\`$HOME\`\` since environment variables won't be expanded in the json file, thus you need to write the full path explicitely). - Now all the bash commands (and variables) you added inside \`\`wrapper.sh\`\` are visible by the Python kernel of your environment. You can check it by running bash commands directly from your Python environment in Jupyter notebooks using your kernel. .. note:: Bash commands can be run from Jupyter notebooks/consoles using "!" at the beginning of the line. Access to work and scratch areas ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In the left column of the principal page, you can see the content of your \*\*home\*\* area on the cluster. Your \*\*work\*\* and \*\*scratch\*\* areas are not visible from your default Interactive Computing interface. To make them visible and reachable, you need to create a symbolic link in your HOME directory pointing those areas. In the following example, we are creating a link to the \*\*scratch area\*\* and a link to the \*\*work\*\* area inside our home directory by launching the following commands in a terminal on the cluster: .. code:: bash ln -s $CINECA\_SCRATCH $HOME/scratch ln -s $WORK $HOME/work Thus now you can see a work and a scratch icons in the file manager on the left side of our interface, and you can access them. .. note:: It is strongly suggested to creating such links to make all the storage available to the Interactive computing sessions (and not just the home storage). Please remember that the $WORK variable refers to the work area of your current default account, so you should create different links for different accounts and keep them updated over time. Logout vs Session shutdown ^^^^^^^^^^^^^^^^^^^^^^^^^^ By default, your session is not shut down when you close your browser window (or tab); as long as the session is active (until the walltime is reached), the requested resources are not released to other users. In this way, by opening a new browser, the user can restore a still active session simply by re-accessing the \`Interactive Computing URL \`\_. .. commented so far since accounting is still disabled on IAC .. Until the session is finished by reaching the walltime or closed manually by the user, the requested resources will be billed on the budget account indicated by the user in the form. If you have finished and you want to close your session manually you need to click on :bdg-black-line:\`File\`, then select :bdg-black-line:\`Hub Control Panel\` and finally :bdg-black-line:\`Stop my server\`. .. image:: img/iac-shutdown\_01.png :alt: Hub control panel from "File" dropdown menu :width: 500px :align: center .. image:: img/iac-shutdown\_02.png :alt: "Stop my server" button :width: 500px :align: center Troubleshooting ^^^^^^^^^^^^^^^ .. dropdown:: "Kernel died unexpectedly" error message. :animate: fade-in-slide-down :color: warning Unfortunately, Jupyter kernels are not very verbose, but in many cases, this error can be related to a buffer overflow; please consider testing the code once again in a new session (see "Logout vs Session shutdown") requesting a larger amount of memory. .. dropdown:: Spawning job message hangs after requesting resources. :animate: fade-in-slide-down :color: warning The problem might occur when requesting an unavailable amount of resources, which might be the case of jobs requesting GPUs; you can take a look to the table in the bottom of the form for resources allocation, which lists all the nodes available with zero, one or two GPUs; if you requested, for instance, a session with 2 GPUs but there are no nodes with 2 GPUs currently available, then this issue might occur. Please consider to fit your requests to the available resources to have the session starts in short time. --- # Unknown .. \_lb\_create\_card: LoadBalancer: create ======================== We exemplify here the procedures based on (1) :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` and (2) the :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` and the OpenStack Octavia plugin, for a simple use case: \*\*the deployment of a basic HTTP load balancer with an associated Floating IP\*\*. For further details and additional use cases, including step-by-step examples, refer to the \`Octavia Basic Load Balancing Cookbook \`\_. This resource provides comprehensive instructions on applying the same procedure to various scenarios. .. important:: - The Octavia service is available but it is not enabled by default to all HPC Cloud projects. - If you want to use it, please request access sending an email to superc@cineca.it. - Once the tenant is enabled to the service by the User Support Team, all users of the tenant will be able to use the service. In this section, we will walk you through the steps to create a setup where two instances running nginx servers are connected to an HTTP load balancer. The load balancer will use the round-robin algorithm to evenly distribute incoming HTTP traffic across the two servers. .. important:: 1. Before creating a load balancer, \*\*ensure that the following resources are available\*\* in your tenant: - 1 network and subnet - 1 router - Desired security groups for the VMs: at the minimum, the Ingress rules for HTTP (port 80) and SSH (port 22) - 2 instances: in our example, these VMs host an nginx web server each - At least 2 floating IPs: one associated to one of the VMs and an additional one available to be associated to the load balancer. The internal IP of the second VM can be used to login from the first VM if needed for configuration. Note that an additional Floating IP could be directly associated to the second VM. However, this would entail using an additional (and not strictly necessary) resource, which we try to avoid. - 1 KeyPair: an SSH public key is needed to access the instances for their configuration 2. You can \*\*setup a very simple nginx web server in each VM\*\* by logging into each of the VM and running the following commands on the shell: .. code-block:: bash sudo apt-get update sudo apt-get install -y nginx && \\ echo "Hello! This is $(hostname)" > /var/www/html/index.html .. tab-set:: .. tab-item:: Horizon Dashboard - \*\*Create the loadbalancer\*\* by clicking on \*"Network → Load Balancers → Create Load Balancer"\* and setting the following information. Once all the details are provided, click on \*"Create Load Balancer"\*. .. dropdown:: Load Balancer Details - Name. - Subnet. Select the desired subnet. .. image:: /cloud/\_img/op\_lb\_create\_img2.png .. dropdown:: Listener Details - Name. - Protocol and Port. The protocol defines the type of network traffic the listener will handle, while the port specifies the network port on which the listener will accept incoming traffic. In our example we select protocol HTTP and port 80. .. image:: /cloud/\_img/op\_lb\_create\_img3.png .. dropdown:: Pool Details - Name. - Algorithm. The algorithm determines how traffic is distributed across the members. We select ROUND\_ROBIN. .. image:: /cloud/\_img/op\_lb\_create\_img4.png .. dropdown:: Pool Members - Add members. Choose the desired members among those available. We add VM-1 and VM-2, the names of the VMs in our example. - Port. For each VM, specify the port number on which the member will receive traffic. In our case, we expose the nginx server on port 80. - Weight. The weight of the member for load balancing purposes. The weight determines the relative portion of requests the member should handle compared to others. We use the default value. .. image:: /cloud/\_img/op\_lb\_create\_img5.png .. dropdown:: Monitor Details - Decide whether you'd like to create a Health Monitor. In this example, we will not make use of a monitor. .. image:: /cloud/\_img/op\_lb\_create\_img6.png - \*\*Associate a floating IP\*\* to the load balancer. - Move to the \*Network → Load Balancers"\* section - Display the options within the drop-down menu on the right side for the desired load balancer, and click on \*"Associate Floating IP"\*. .. image:: /cloud/\_img/op\_lb\_create\_img7.png - Select the floating IP among those suggested in the drop-down menu \*"Floating IP address or Pool"\* and click on \*"Associate"\*. .. image:: /cloud/\_img/op\_lb\_create\_img8.png - The floating IP associated to the load balancer appears in the overview of its characteristics. In order to see the properties, click on the name of your load balancer in the \*"Network → Load Balancers"\* section of the dashboard. - \*\*Test\*\* your load balancer. Use the above floating IP to reach the nginx servers. The traffic will be managed by the load balancer following the algorithm ROUND\_ROBIN. .. image:: /cloud/\_img/op\_lb\_create\_img1.png .. tab-item:: Command Line Interface .. important:: - Follow the instructions on how to setup your cloud environment at :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` - Install the Octavia service plugin \`\`pip install python-octaviaclient\`\` - \*\*Create a Load Balancer\*\* \`\`openstack loadbalancer create --name --vip-subnet-id \`\` The \*\* can be found through the Horizon Dashboard. On the main menu, select the \*Network → Networks\* tab. Then, click on your network and select the \*Subnets\* tab. Finally, click on the desired subnet. This information can also be gathered using the CLI. Note: Wait until the creation is completed. It will take a while and the next steps will return an error if the load balancer is not available. - \*\*Create a Listener\*\* \`\`openstack loadbalancer listener create --name --protocol HTTP --protocol-port 80 \`\` - \*\*Create a Pool\*\* \`\`openstack loadbalancer pool create --name --lb-algorithm --listener --protocol HTTP\`\` - \*\*Add Members\*\* to the pools \`\`openstack loadbalancer member create --subnet-id --address --protocol-port 80 \`\` \`\`openstack loadbalancer member create --subnet-id --address --protocol-port 80 \`\` You can find the \*IPs\* of the VMs in the \*Compute → Instances\* section of the lateral menu of Horizon Dashboard. The IPs can also be gathered using the CLI. - \*\*Associate a Floating IP\*\* to the load balancer \`\`openstack floating ip set --port \`\` You can find the \*\* if you navigate to the description of your load balancer on the Horizon Dashboard. Select the \*Network → Load Balancers\* section on the left-hand menu of the dashboard. Then, click on your load balancer and go to the \*Overview\* tab. You can also use the CLI to identify the value of the \*\*. - \*\*Test\*\* your load balancer. Finally, use this floating IP to reach the nginx servers. The traffic will be managed by the load balancer following the algorithm . .. image:: /cloud/\_img/op\_lb\_create\_img1.png --- # Instance: create — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: create * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_create.rst.txt) * * * Instance: create[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#instance-create "Link to this heading") ======================================================================================================================================== The creation of a virtual machine can be done using the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) , using the [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) or with customized Ansible and Terraform scripts to automate the process ([Declarative and procedural approaches to IaC](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/infrastructure_as_code.html#declarative-and-procedural-approaches-to-iac) ). You are free to choose any of these methods. However, there are a few mandatory steps that must be followed before creating a virtual machine. This page will guide you step by step in the creation of a virtual machine with the **Openstack Horizon Dashboard**. Important **Prerequisites**: In order to create and boot a virtual machine, you need have already created the following resources * A tenant network → [Network: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/network_create.html#network-create) * A security group → [Security groups: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/secgroups_create.html#security-groups-create) * A key-pair (it is possible to create a new key-pair during instance creation or upload a preexisting pair) → [Key Pair: create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/keypair_create.html#key-pair-create) Virtual machine creation[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#virtual-machine-creation "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------- Once you have completed the above steps, you can proceed to create a Virtual Instance. Go to _“Compute → Instances”_ and click on _“Launch Instance”_. A pop-up window will appear and you have to fill in the required information in the different tabs. ![../../../_images/op_instace_create_img1.png](https://docs.hpc.cineca.it/_images/op_instace_create_img1.png) Details Under the tab _“Details”_, you need to insert the instance name and how many copies of this virtual machine you want to create (_“count”_ field). Source Under the tab _“Source”_, you specify if you want to boot your virtual machine form an image/image snapshot or from a volume/volume snapshot. When booting from an image, you are able to decide if you want the virtual machine to be created as “ephemeral” or you want to create a root “bootable volume” contextually to the virtual machine creation by using the _Create New Volume_ checkbox (see [Compute](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#compute) for more information on bootable and ephemeral instances). If you select to create a _bootable_ instance you have also to specify if the volume will be deleted at deletion of the VM (_Delete Volume on Instance Delete_). If the volume is created with the option _Delete on termination_ active, this configuration can be changed later on only via [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) with this command openstack server volume set –preserve-on-termination In this section, you also need to choose from the available list which image you would like to use to build your instance. To select the one to be use simply click on the up arrow from the list of available resources. Flavor and network Under the respective tabs, you can choose the flavor of the virtual machine, and the network you want the virtual machine connected to. To select the one to be used, simply click on the up arrow from the list of available resources. Note The network should have been already created Security groups and key pair Under the respective tabs, you can choose the security groups you want to apply to the virtual machine, and the key-pairs you want to use to access. To select the one to be used, simply click on the up arrow from the list of available resources. Note The key pair should have been already created Server groups Under the _“Server Groups”_ tab you find the options to create the virtual machine under a previously created affinity or anti-affinity group (see [Compute](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#compute) for more information on affinity groups). Once you have setup the preferred configuration, click on _“Launch instance”_ to create your virtual machine. Warning Once the instance has been created, it is possible only to modify the security groups and the flavor. If you need to change the network, or the key pair, you will need to create a new instance. Associate a Floating IP[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#associate-a-floating-ip "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------- Once the virtual machine is created, you can associate a floating IP with the virtual machine. For more information on how to associate a floating IP with a virtual machine, refer to the [Floating IP: allocate and associate](https://docs.hpc.cineca.it/cloud/operative/network_ops/fip_association.html#floating-ip-allocate-and-associate) . Once the floating IP is associated with the virtual machine, you can access the virtual machine using it. Accessing the instance[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#accessing-the-instance "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------- Note If you created an instance using a default image available in the cloud computing, by default it is possible to login into the instance only by using the default user and key. Suppose you have used the default ubuntu cloud image, you can login as: $ ssh \-i MyKey.pem ubuntu@ Copy to clipboard --- # Network: create — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Network operations](https://docs.hpc.cineca.it/cloud/operative/network_ops/index_network_ops.html) * Network: create * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/network_ops/network_create.rst.txt) * * * Network: create[](https://docs.hpc.cineca.it/cloud/operative/network_ops/network_create.html#network-create "Link to this heading") ===================================================================================================================================== * In the Horizon Dashboard go to _“Network → Networks”_ tab and select _“Create network”_ button. ![../../../_images/op_network_create_img1.png](https://docs.hpc.cineca.it/_images/op_network_create_img1.png) * In the pop-up window add a name for your Network and select the _“Create Subnet”_ checkbox. ![../../../_images/op_network_create_img2.png](https://docs.hpc.cineca.it/_images/op_network_create_img2.png) * In the _Subnet tab_, assign a name to the Subnet and provide the Network address and Gateway IP. As an example you can set: > * Network Address: 192.168.0.0/24 > > * Gateway IP: 192.168.0.254 (the last address for subnet 192.168.0.0/24) > ![../../../_images/op_network_create_img3.png](https://docs.hpc.cineca.it/_images/op_network_create_img3.png) * In the last step, select _“Enable DHCP”_ checkbox. ![../../../_images/op_network_create_img4.png](https://docs.hpc.cineca.it/_images/op_network_create_img4.png) * Click the _“Create”_ button on the right bottom side of the window. --- # Volume: create and attach — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Storage operations](https://docs.hpc.cineca.it/cloud/operative/storage_ops/index_storage_ops.html) * Volume: create and attach * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/storage_ops/volume_create.rst.txt) * * * Volume: create and attach[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_create.html#volume-create-and-attach "Link to this heading") ======================================================================================================================================================== Create a volume[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_create.html#create-a-volume "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- * From the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) , go to the _“Volumes → Volumes”_ section and select _“Create Volume”_ ![../../../_images/op_volume_create_img1.png](https://docs.hpc.cineca.it/_images/op_volume_create_img1.png) * In the pop-up window, specify **Volume Name**: a name for the volume. **Description**: Optionally, provide a brief description for the volume. **Volume Source**: Select one of the following options: > * No source, empty volume: Creates an empty volume. An empty volume does not contain a file system or a partition table. > > * Image: If you choose this option, a new field for Use image as a source displays. You can select the image from the list. > **Type**: * \_\_DEFAULT \_\_ is a general cinder volume * LUKS is for encrypted volumes (see Storing sensitive data for more details). **Size (GiB)**: The size of the volume in gibibytes (GiB). ![../../../_images/op_volume_create_img2.png](https://docs.hpc.cineca.it/_images/op_volume_create_img2.png) * Finally, click on _“Create Volume”_ button. Attach/Detach a volume[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_create.html#attach-detach-a-volume "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------- * To attach a volume to an instance, go to the _“Volumes → Volumes”_ page and select the volume you would like to attach and then the action _“Manage Attachments”_ ![../../../_images/op_volume_create_img3.png](https://docs.hpc.cineca.it/_images/op_volume_create_img3.png) * Select the instance you would like the volume to be attached to: ![../../../_images/op_volume_create_img4.png](https://docs.hpc.cineca.it/_images/op_volume_create_img4.png) * Click _“Attach volume”_. At this point, you can view the status of a volume in the _“Volumes → Volumes”_ tab of the Horizon Dashboard. The volume can be is either in status _“Available”_ or _“In-Use”_. The same _“Manage Attachments”_ operation can be used to detach a volume from an instance. When the volume is attached, in order to use for storing data you need to log in to the instance to partition, format and mount it (see [Volume: format and mount](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_mount.html#volume-format-and-mount) ). --- # Instance: delete — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: delete * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_deletion.rst.txt) * * * Instance: delete[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_deletion.html#instance-delete "Link to this heading") ========================================================================================================================================== If you would like to delete one of the VMs, you have created, you can follow the steps below or follow the **Deletion tutorial** in [Tutorials for OpenStack dashboard](https://docs.hpc.cineca.it/cloud/tutorials/index_tutorials_and_repos.html#tutorials-for-openstack-dashboard) . Warning The order of the steps is important to avoid errors during the deletion. Horizon Dashboard * In the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) , select on _“Network → Floating IPs”_, then click on the button _“Disassociate”_ on the right side of the dashboard page for the IP associated to your VM. If desired, you can also release the floating IP. Note Once you release it, there is no guarantee the same IP can be allocated again. * Go to _“Compute → Instances”_, and display the drop-down menu of the VM you want to delete and then click the action _“Delete Instance”_ Command Line Interface * List all the VMs in the tenant openstack server list +----------------+------------------+--------+-------------------------------------------------------------+--------------+---------------+ | ID | Name | Status | Networks | Image | Flavor | +----------------+------------------+--------+-------------------------------------------------------------+--------------+---------------+ | | | ACTIVE | \=, ... | | | | | | ACTIVE | \=, ... | | | ... | | | ACTIVE | \=, ... | | | +--------------------------------------+------+--------+-----------------------------------------+------------------------+---------------+ Copy to clipboard * Shut down the VM using its ID from the previous step openstack server stop Copy to clipboard * Find the ID of the Floating IP associated to the VM based using the VM Floating IP Address openstack floating ip list \--floating-ip-address +------------------+-----------------------+--------------------+-----------+-----------------------+--------------+ | ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project | +------------------+-----------------------+--------------------+-----------+-----------------------+--------------+ | | | | | | | +------------------+-----------------------+--------------------+-----------+-----------------------+--------------+ Copy to clipboard * Disassociate the Floating IP openstack floating ip unset Copy to clipboard Note Once you release it, there is no guarantee the same IP can be allocated again. * Delete the VM openstack server delete Copy to clipboard If you are deleting the VM because of issues, and you would like to recreate it in a clean environment, it is recommended to remove also the network resources (router, interfaces, etc) that were associated to your instance. For this follow **Deletion tutorial** in [Tutorials for OpenStack dashboard](https://docs.hpc.cineca.it/cloud/tutorials/index_tutorials_and_repos.html#tutorials-for-openstack-dashboard) . --- # Instance: snapshot download — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: snapshot download * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_download.rst.txt) * * * Instance: snapshot download[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_download.html#instance-snapshot-download "Link to this heading") ================================================================================================================================================================ Prepare local system for download[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_download.html#prepare-local-system-for-download "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To avoid errors due to not enough disk space to accommodate the snapshot, check the available disk space on your system df \-h / Filesystem Size Used Avail Use% Mounted on /dev/ 466G 93G 349G 22% / Copy to clipboard In case an external drive is mounted on your system, or another partition is meant to be used, replace the _”/“_ character with the _/dev_ path of the drive. In case there is not enough disk space on your local system to store snapshots, it is possible to mount locally a remote host directory using the _sshfs_ utility. sshfs :/path/to/remote/directory /path/to/local/directory Copy to clipboard For CINECA users which have access to HPC Clusters, it is strongly suggested to mount the remote directory via the _datamover_ nodes (see [Data Transfer](https://docs.hpc.cineca.it/hpc/hpc_data_storage.html#data-transfer) ). sshfs @data..cineca.it:/path/to/remote/directory /path/to/local/directory Copy to clipboard Using _datamover_ nodes avoid process being killed by surpassing the CPU-time characteristic of long download processes. Also in this case, please check to have enough space to store the snapshot before starting the download. CINECA HPC Cluster users are strongly encouraged to use the _cinQuota_ command instead of _du_ to get information about the occupancy of a specific path to avoid stressing the Lustre filesystem: cinQuota ---------------------------------------------------------------------- Filesystem used quota grace files ---------------------------------------------------------------------- 50G \- 1T \- 1T \- Copy to clipboard HPC Cloud users without access to HPC Clusters can write an email to [superc@cineca.it](mailto:superc%40cineca.it) asking for information about how to obtain a budget and storage on HPC Clusters. Search and download snapshots[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_download.html#search-and-download-snapshots "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- To download an instance, it is necessary to create a snapshot of it (see [Instance: snapshot create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#instance-snapshot-create) ), and then save it locally. The download of a snapshot can be performed only via the [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) . Here are the steps to be followed. * The first step is obtaining the complete tabular list of all the images available on the tenant openstack image list +---------------+-----------------+--------+ | ID | Name | Status | +---------------+-----------------+--------+ | | | active | | | | active | | | | active | ... | | | active | +---------------+-----------------+--------+ Copy to clipboard * Use the ID corresponding to the image snapshot name to start the download procedure using the following command: openstack image save \--file /path/to/local/directory/ Copy to clipboard The OpenStack CLI does not show a progress bar for the download so the shell in which the last openstack command has been launched will hung until the download terminates. You can append to the previous command the ”&“ character to send the download in background: remember to not close the shell in which the command has been launched since the download process will be killed. It is possible to monitor the procedure by using the following watch command: watch \-n 1 ls \-lrth /path/to/local/directory/ Every 1,0s: ls \-lrth /path/to/local/directory/ ... total -rw-rw-r-- 1 Copy to clipboard The screen will refresh every second showing an increase in both and . * Check the downloaded image info to be sure the process has been executed correctly qemu-img info Copy to clipboard Limits of the procedure[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_download.html#limits-of-the-procedure "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------------------- The time needed to complete the download of the snapshot is strongly influenced by both the size of the snapshot itself and the bandwidth of internet connection. These are some examples of possible download times from OpenStack infrastructure to local server (ex. CINECA-PDL) for a 30 GigaBytes snapshot image: * ~ 60 minutes with nominal ~ 50 Mbps download speed connection: speed characteristics of a mobile hotspot 4G connection. * ~ 10 minutes with nominal ~ 300 Mbps download speed connection: speed characteristics of the wired optic fiber connection like the one available in CINECA (ex. Sede CINECA Casalecchio). --- # Key Pair: create — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Key Pair: create * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/keypair_create.rst.txt) * * * Key Pair: create[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/keypair_create.html#key-pair-create "Link to this heading") ======================================================================================================================================= * In the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) , go to the _“Compute → Key Pairs”_ tab. * Click on _“Create Key Pair”_. ![../../../_images/op_keypairs_create_img1.png](https://docs.hpc.cineca.it/_images/op_keypairs_create_img1.png) * Provide a name for the KeyPair and select the KeyPair type. ![../../../_images/op_keypairs_create_img2.png](https://docs.hpc.cineca.it/_images/op_keypairs_create_img2.png) * The private key will be automatically downloaded. Store this file securely as it will be needed for SSH access. Warning * The download of the private key will be done only when the keypair is created. It will not be possible to re-download it. If you lose the private key you will have to create a new keypair. * The public key can be seen in any moment by clicking on the generated Key Pair name. Best Practices[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/keypair_create.html#best-practices "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------ * **Secure Storage**: Store private keys securely. If lost, you won’t be able to access your instances using that KeyPair. * **Permissions**: Ensure private key files have restrictive permissions (chmod 600) to prevent unauthorized access. * **Rotation**: Periodically rotate your KeyPairs and update instances accordingly to maintain security. * **Backup**: Keep backups of your private keys in a secure location to prevent accidental loss. * **KeyPairs** in OpenStack provide a secure and efficient method for managing SSH access to instances. By leveraging public-key cryptography, KeyPairs ensure that only users with the appropriate private key can access the instances, enhancing overall security. --- # Instance: resize — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: resize * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_resize.rst.txt) * * * Instance: resize[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_resize.html#instance-resize "Link to this heading") ======================================================================================================================================== Note If you are trying to resize the root volume of your VM, please refer to this page: [Instance: root storage increase](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_root_storage_increase.html#compute-inst-root-storage-increase-card) Users are able to resize autonomously their VM resources, this operation can be done either via [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) or via [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) Important Before performing the resize operation: * The VM must be shut off. * If there are encrypted **LUKS VOLUMES** attached to the virtual machine, it is mandatory that the user: > * Unmount the volumes from the VM > > * Detach the volumes from the Horizon Dashboard (see [Attach/Detach a volume](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_create.html#attach-detach-a-volume) > ) > Note Remember to alert your users of the VM temporary shutdown during the operation, before starting the resize. Horizon Dashboard * Go to _“Compute → Instances”_, and find the VM you need to resize * From the drop-down menu on the right side, select _“resize instance”_ Note * If you have an **Ephemeral VM**, check the size of root disk of the original VM. Don’t resize the VM, if the new flavor has a disk smaller than the current one. * If you have a **VM with a Bootable Disk**, the resize will affect only vCPUs number and RAM. The bootable disk will not be changed by the operation. * A menu will popup where you can choose the new desired flavor and click _“resize”_ * OpenStack will prepare the operation and then wait for user input to confirm or revert the operation * From the drop-down menu on the right select either _“confirm resize/migration”_ if you want to continue, or _“revert resize/migration”_ if you want to keep the original flavor. * Confirm the success of the operation. To do that you will need to boot the VM, login, and verify the vCPUs number and Memory size are correct with the following commands: cat /proc/cpuinfo free \-g Copy to clipboard Command Line Interface To know how to configure and use the OpenStack CLI, please refer to the [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) page. * Identify the VM ID. openstack server list \--all | grep openstack server show < vm\_ID \> | grep flavor Copy to clipboard * Identify the ID of the new flavor the VM needs. openstack flavor list Copy to clipboard Note * If you have an **Ephemeral VM**, check the size of root disk of the original VM. Don’t resize the VM, if the new flavor has a disk smaller than the current one. * If you have a **VM with a Bootable Disk**, the resize will affect only vCPUs number and RAM. The bootable disk will not be changed by the operation. * Perform the resize. openstack server resize \--flavor \--wait Copy to clipboard * Wait for the operation to _“Complete”_. * Issue the resize confirmation in a separate command, since the option _–confirm_ on the command openstack server resize is deprecated. openstack server resize confirm Copy to clipboard * Verify the success of the operation. Since the Dashboard can have visualization bugs, it is best to check via CLI: openstack server show < vm\_ID \> | grep flavor Copy to clipboard * Confirm the success of the operation. To do that you will need to boot the VM, login, and verify the vCPUs number and Memory size are correct with the following commands: cat /proc/cpuinfo free \-g Copy to clipboard --- # Instance: rescue — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: rescue * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_rescue.rst.txt) * * * Instance: rescue[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_rescue.html#instance-rescue "Link to this heading") ======================================================================================================================================== Instance rescue provides a mechanism for accessing, even if an image renders the instance inaccessible. Two rescue modes are currently provided. Warning If the virtual machine has encrypted **LUKS VOLUMES** attached, it is mandatory to detach them before starting the rescue operation. Ephemeral Virtual Machine * Create a **rescuer** virtual machine with a **new key pair** ([Instance: create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#instance-create) ). Although this is not a fixed rule, it is suggested to create the rescuer machine using an image with **same OS** as the one on the inaccessible machine (same version or newer). * Login to the rescuer and update it. As an example, for Ubuntu virtual machines: sudo apt update sudo apt upgrade Copy to clipboard * Logout the rescuer and create a **snapshot image** of this virtual machine. * Select the instance you want to rescue, check that its openstack status is _“Active”_, and from the drop-down menu on the right select _“rescue instance”_: * In the menu that appears, select the image you just created from the rescuer machine. * Login via ssh to the broken machine using the rescuer username/key * Check that the boot of the machine has been correctly executed using the command `lsblk` * You should see the rescuer machine (/dev/vda1) mounted and the inaccessible machine on the device /dev/vdb1 * Mount such device /dev/vdb1 sudo mkdir /mnt/inaccessible\_vm sudo mount /dev/vdb1 /mnt/inaccessible\_vm Copy to clipboard * Now you can access the files in the inaccessible machine to fix the problems (lsblk, fsck, xfs\_repair, chroot, etc.) or backup important data * Once the operation is done, logout the virtual machine and from the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) select _“unrescue”_. Bootable Virtual Machine * Shutdown the instance. * In the tab _“Volumes”_, track which secondary volumes are attached to the VM to be rescued and detach them. * **IMPORTANT**: verify that the bootable volume won’t be erased when deleting the VM! > * using the [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) > , execute the command `openstack server volume list `, and have a look to the field _“delete\_on\_termination”_ that must be set to _‘False’_. (**Note** that this will work with openstack-cli >= 6.2.0) > > * If delete\_on\_termination option is set to _true_, it can be changed using the [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) > with the command `openstack server volume set --preserve-on-termination ` > * Keep track of the Flavor, Security Groups and FIP associated with the VM (FIP in particular if there is a DNS association). * Delete the instance. * Create a throwaway VM, attach the bootable volume to rescue as a secondary volume and associate a FIP to such VM. * Login via ssh to the throwaway VM and execute all the needed operations on the volume to rescue (lsblk, fsck, xfs\_repair, chroot, etc.). * Once the volume has been recovered, exit the throwaway VM and detach the secondary volume that has been rescued. * Restart the VM from the rescued bootable volume, reattaching the secondary volumes, FIP, and check the problem has been solved. --- # Image: upload — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Image: upload * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/image_upload.rst.txt) * * * Image: upload[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/image_upload.html#image-upload "Link to this heading") =============================================================================================================================== The upload of an image can be done using both the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) and the [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) : you are free to choose any of the available methods. This page will guide you, the image owner, step by step in the upload of an image with the OpenStack [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) . Once logged into the OpenStack Horizon Dashboard, go to _“Images”_ in the left panel and click on _“Create Image”_ button. ![../../../_images/image_upload_img1.jpg](https://docs.hpc.cineca.it/_images/image_upload_img1.jpg) You have to fill the information in the following pop-up form: ![../../../_images/image_upload_img2.jpg](https://docs.hpc.cineca.it/_images/image_upload_img2.jpg) Under the tab **“Image Details”**, a name for the image file have to be written in the text box under _“Image Name”_: it is also a good practice entering a brief description of the image in the text box under _“Image Description”_. Based on if the desired image is stored on your PC or is available online, indicate the appropriate _“Source Type”_ as _“File”_ or _“URL”_ Depending on the previous choice, browse your PC to the location in which is stored the image file or paste the URL to where the image file is hosted. From the _“Format”_ drop-down menu, select the file format of the desired image to upload. Subsequently, it is possible to specify optional advanced options as _“Kernel”_ image, _“Ramdisk”_ image, specify a string for the image _“Architecture”_, _“Minimum Disk”_ quota in GigaByte and a _“Minimum RAM”_ quota in MegaByte to boot the image. The next step is to select the _“Image Sharing”_ policies by choosing one of the three options under _“Visibility”_ keeping in mind: * **Private** images can be used in instance creation only by the image owner. * **Shared** images can be used, by default, by all the users collaborating to the project with the image owner. * **Community** images are **inhibited** even if the option is available to all users. By uploading an image with visibility set to “Community” will raise the following error message preventing the beginning of the upload procedure: ![../../../_images/Create_Community_Image_Error.png](https://docs.hpc.cineca.it/_images/Create_Community_Image_Error.png) Further information on image visibility can be found in [Images](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#images) , while a description of the risks associated in using community images can be found at [Images](https://docs.hpc.cineca.it/cloud/tenant_adm/security_guidelines.html#images) . Subsequently, it is possible to choose if the image is _protected_ or not from deleting operations. Finally, under the tab **“Metadata”**, it is possible to search the name of specific resources metadata in the _“Filter”_ research field at the top of the _“Available Metadata”_ list. Once found, clicking the blue “+” button will move the resource metadata under the _“Existing Metadata”_ list allowing the declaration of its value. ![../../../_images/image_upload_img3.jpg](https://docs.hpc.cineca.it/_images/image_upload_img3.jpg) To complete de creation of the image, simply create the _“Create Image”_ blue button at the bottom right corner of the pop-up window. --- # Instance: root storage increase — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: root storage increase * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_root_storage_increase.rst.txt) * * * Instance: root storage increase[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_root_storage_increase.html#instance-root-storage-increase "Link to this heading") ===================================================================================================================================================================================== Users are able to resize autonomously the root volume of their VM. The procedure differs wether the VM is ephemeral or created from a bootable volume (more info on the [Compute](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#compute) page). Important Before the procedure is advised to **shut down** the VM and detatch any secondary volumes. Ephemeral Ephemeral VMs have a fixed root volume size depending on the flavor available on the infrastructure. For this reason, the maximum size that can be reached by and ephemeral instance has an upper limit equal to the root volume size of the greates falvor avialable. All the flavor available for each CINECA’s HPC cloud infrastructures can be found in their dedicated pages grouped inside the [Cloud Specifics](https://docs.hpc.cineca.it/cloud/systems/index_system_specifics.html#cloud-specifics-card) page. To increase this storage space, the user will need to create a new VM with a larger root volume using a snapshot of the original VM. This can be achieved both via the Horizon Dashboard and the OpenStack command line inteface. Horizon Dashboard First of all, disassociate, if present, the floating IP from the VM (do not release it otherwise it will be lost) Warning If the floating IP is also released, it will be lost and a new allocated floating IP will have a different address ![../../../_images/op_fip_detach.jpg](https://docs.hpc.cineca.it/_images/op_fip_detach.jpg) Create a snapshot of the VM instance following the instruction reported in the [Instance: snapshot create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#compute-inst-snap-create-card) card. Command line interface First of all identify the ID of the bootable VM with: $ openstack server list +----------------+------------------+--------+-----+-----------------+------------------+ | ID | Name | Status | ... | Image | Falvor | +----------------+------------------+--------+-----+-----------------+------------------+ | | | ACTIVE | | | | | | | ACTIVE | | | | ... | | | ACTIVE | | | | +----------------+------------------+--------+-----+-----------------+------------------+ Copy to clipboard Then, disassociate, if present, the floating IP from the VM. $ openstack server remove floating ip Copy to clipboard Create a snapshot of the VM instance following the instruction reported in the [Instance: snapshot create](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#compute-inst-snap-create-card) card. Once the snapshotting procedure is completed, create a new VM from the snapshot using the option “create new volume” in the tab “source” and chosing the desired volume size (here the full guide on [Virtual machine creation](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_create.html#virtual-machine-creation) ) ![../../../_images/op_inst_from_snap.jpg](https://docs.hpc.cineca.it/_images/op_inst_from_snap.jpg) Finally, associate the old floating IP to the newly created VM ![../../../_images/op_fip_attach.jpg](https://docs.hpc.cineca.it/_images/op_fip_attach.jpg) Once the operation is completed, it is possible to remount any secondary volumes that were detatched at the start of this procedure. Note Remember: before deleting the old virtual machine, be sure the the new VM with increased volume size is up-and-running and is functioning correctly. Bootable For all VMs created from a bootable volume, it is necessary to perform the resize operation of VM’s Primary Volume to obtain a larger root volume size. The detailed procudure on how to perform such operation is illustrated in the [Resize a volume](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_resize.html#resize-a-volume) . --- # Volume: format and mount — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Storage operations](https://docs.hpc.cineca.it/cloud/operative/storage_ops/index_storage_ops.html) * Volume: format and mount * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/storage_ops/volume_mount.rst.txt) * * * Volume: format and mount[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_mount.html#volume-format-and-mount "Link to this heading") ===================================================================================================================================================== After a volume has been created and attached to a virtual machine (see [Volume: create and attach](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_create.html#volume-create-and-attach) ), in order to use for storing data you need to partition, format and mount it. To achieve this, these operations have to be performed inside the virtual machine: * Partition Table * Format the volume * Mount the volume Warning These operations is specific to the OS and software installed on the Virtual Machine. There are more than one software capable of partitioning your devices (e.g. fdisk, parted). Here you can find an example of how to perform these operations using fdisk. Partition Table[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_mount.html#partition-table "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------ Suppose that the volume is attached to the virtual machine as device _/dev/vdc_. Login in to the virtual machine and use fdisk to modify the partition table. * list the partition table sudo fdisk \-l Copy to clipboard * partition of device /dev/vdc > sudo fdisk /dev/vdc > > Copy to clipboard \# 1 new partition, primary, with default sector numbers and type “Linux” > \==> n; p; 1 ; …default ; \# check and write > \==> p; w Format the volume[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_mount.html#format-the-volume "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------- Format the device _/dev/vdc_ just partitioned as xfs: sudo mkfs \-t xfs /dev/vdc1 Copy to clipboard Mount the volume[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_mount.html#mount-the-volume "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------- sudo mkdir /mnt/stuff\_1 sudo mount /dev/vdc1 /mnt/stuff\_1 Copy to clipboard To mount the volume automatically at each boot of the virtual machine, please modify the _/etc/fstab_ file. Following the example, in the _/etc/fstab_ could be written: /dev/vdc1 /mnt/stuff\_1 xfs auto,nofail,defaults 0 0 Copy to clipboard --- # Instance: manage and monitor — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: manage and monitor * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_manage.rst.txt) * * * Instance: manage and monitor[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_manage.html#instance-manage-and-monitor "Link to this heading") ================================================================================================================================================================ To monitor your instance, login to the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) . Manage an instance[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_manage.html#manage-an-instance "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------- * Log in to the virtual machine and shutdown the instance * In the dashboard, go to _“Compute → Instances”_. * In the _“Actions”_ column drop down menu of the instance, you can find all possible operation to manage you instance. You can resize or rebuild an instance, edit instance or the security groups. Depending on the current state of the instance, you can pause, resume, suspend, soft or hard reboot, or terminate Track usage for instances[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_manage.html#track-usage-for-instances "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------- You can track usage for instances for each project. You can track usage per month by showing metrics like number of vCPUs, disks, RAM, and uptime for all your instances. * In the dashboard. choose the project, and go to _“Compute → Overview”_ * In the page you can see an overview of the used resources (for compute, storage and network) with respect to the quotas assigned to the project * To query the instances usage for a month, select a month and click “Submit”. * To download a summary, click _“Download CSV Summary”_. Monitor instances[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_manage.html#monitor-instances "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------- You can monitor the high-level actions (creation, start, stop) on the instances for each project via offered logs in the dashboard. * Go to _“Compute → Instances”_. * Select the instance name you would like to monitor > * go to the _“Action log”_ tab to see the high-level actions (creation, start, stop) > > * go to the _“Log”_ tab, to see the instance console log > More detailed monitoring logs can be set-up by you within the specific instance. --- # Security groups: create — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Network operations](https://docs.hpc.cineca.it/cloud/operative/network_ops/index_network_ops.html) * Security groups: create * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/network_ops/secgroups_create.rst.txt) * * * Security groups: create[](https://docs.hpc.cineca.it/cloud/operative/network_ops/secgroups_create.html#security-groups-create "Link to this heading") ======================================================================================================================================================= Security groups are a set of IP filter rules that are applied to an instance. These rules specify which type of traffic is allowed to reach the instance (see section [Security Groups](https://docs.hpc.cineca.it/cloud/os_overview/os_components/network.html#security-groups) for more information). * Go to the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) * Go to _“Network → Security Groups”_ and click on _“create security group”_ * Give a name to the security group and click on _“create security group”_ * The security group is created **by default with NO ingress rules** ![../../../_images/op_network_secgroups_img1.png](https://docs.hpc.cineca.it/_images/op_network_secgroups_img1.png) * Select _“Add rule”_ to specify the ingress rules. * A list of pre-defined rules is available for you to choose from (but you can create your own rule if you prefer). Common default rules are: * SSH (port 22) * ICMP (allow to “ping” a server) * HTTP (port 80) * HTTPS (port 443) ![../../../_images/op_network_secgroups_img2.png](https://docs.hpc.cineca.it/_images/op_network_secgroups_img2.png) * In the CIDR field, you can limit the range of IPs which are allowed to the rule. **NOTE**: 0.0.0.0/0 means all internet --- # Database: access — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Database operations](https://docs.hpc.cineca.it/cloud/operative/db_ops/index_db_ops.html) * Database: access * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/db_ops/db_access.rst.txt) * * * Database: access[](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_access.html#database-access "Link to this heading") ============================================================================================================================= Accessing the Database instance[](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_access.html#accessing-the-database-instance "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------ To access the database instance, the typical SQL command (depending on the datastore you chose for your database instance) can be used. In the below example of a MySQL database instance, it is shown how to access the database. After you successfully install the MySQL client, use the following commands to access the database: $ mysql \-h \-u \-p ------------------------------------------------------------------------------------ mysql: \[Warning\] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \\g. Your MySQL connection id is 16 Server version: 5.7.29 MySQL Community Server (GPL) Copyright (c) 2000, 2023, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement. mysql> Copy to clipboard Once you accessed the database, a mysql prompt will be available. You can check the list of databases using the following command: mysql> show databases; -------------- show databases -------------- +--------------------+ | Database | +--------------------+ | information\_schema | | test | +--------------------+ 2 rows in set (0,02 sec) Copy to clipboard How to list the available datastore and their version using OpenStack CLI[](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_access.html#how-to-list-the-available-datastore-and-their-version-using-openstack-cli "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ A datastore is a database engine that is supported by Trove and it is used to create the database instance. On CINECA HPC Cloud the following datastores namely MySQL, MariaDB, PostgreSQL are available. * You can check the available datastores with the command `openstack datastore list`. The ID and Name of the datastores supported will be shown. $ openstack datastore list \# shows all datastores available in Cloud infrastructure +--------------------------------------+------------+ | ID | Name | +--------------------------------------+------------+ | b2103dff-9331-4be2-8193-170f2a509e16 | mariadb | | ed541d5a-d260-4b6b-ac80-74ac38167d70 | mysql | | e8d12fef-3c54-4e83-818c-4a89a104780d | postgresql | +--------------------------------------+------------+ Copy to clipboard * With the command `openstack datastore version list `, you can check the list of datastores with their version. Below we show an example for mysql datastore. $ openstack datastore version list mysql +--------------------------------------+--------+---------+ | ID | Name | Version | +--------------------------------------+--------+---------+ | a6ee8255-2d71-4e22-b68d-1d0e5919e74a | 5.7.29 | 5.7.29 | | 434292f3-2074-4033-8f19-07874cbe5b7d | 8.0.29 | 8.0.29 | +--------------------------------------+--------+---------+ Copy to clipboard You will find Version, Name, and ID of the mysql datastores available in HPC Cloud. **Note**: The Version is important, as we have to specify the version of the datastore while creating the database instance. For more information about the version, use the following command: $ openstack datastore version show a6ee8255-2d71-4e22-b68d-1d0e5919e74a \# shows details of a version +-----------+--------------------------------------+ | Field | Value | +-----------+--------------------------------------+ | datastore | ed541d5a-d260-4b6b-ac80-74ac38167d70 | | id | a6ee8255-2d71-4e22-b68d-1d0e5919e74a | | name | 5.7.29 | | version | 5.7.29 | +-----------+--------------------------------------+ Copy to clipboard --- # Create and use a GENERIC_TYPE share — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Shares operations](https://docs.hpc.cineca.it/cloud/operative/shares_ops/index_shares_ops.html) * Create and use a GENERIC\_TYPE share * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/shares_ops/generic_share_create.rst.txt) * * * Create and use a GENERIC\_TYPE share[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/generic_share_create.html#create-and-use-a-generic-type-share "Link to this heading") ==================================================================================================================================================================================== The following sections describe the steps needed to create a share and mount it on two VMs attached to a local network. Note that the user needs to configure the VMs in a way that allows logging in via ssh. Request to be enabled to the service[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/generic_share_create.html#request-to-be-enabled-to-the-service "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The user willing to make use of the Manila service needs to send an email to [superc@cineca.it](mailto:superc%40cineca.it) , communicating * how many shares are needed. * their dimensions (GB). * the tenant’s name. Once the tenant is enabled to the service by the User Support Team, all users of the tenant will be able to use the service. Create share network[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/generic_share_create.html#create-share-network "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------- As a first step, in the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) you need to create the share network by clicking on _“Create Share Network”_ in _“Share → Share Networks”_ and set the value for the following attributes: * Share network name. * network: choose the desired network, in our example example\_share\_guide\_net. * subnet: choose the desired subnet, in our example example\_share\_guide\_subnet. * Click on the _“save”_ button. ![../../../_images/op_share_generic_img1.png](https://docs.hpc.cineca.it/_images/op_share_generic_img1.png) Create the share[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/generic_share_create.html#create-the-share "Link to this heading") --------------------------------------------------------------------------------------------------------------------------------------------- Create the share by clicking on _“Create Share”_ in _“Share → Shares”_ and setting the following information: * share name * share protocol == “NFS” * size (on the right side is visualized information about the actual available and used space within the tenant) * Type == “generic\_type” * Leave blank the option “Make visible for all projects” because it is not enabled * In the end, click on the _“create”_ button. ![../../../_images/op_share_generic_img2.png](https://docs.hpc.cineca.it/_images/op_share_generic_img2.png) Set the access rule(s) on the share just created. * On the OpenStack dashboard click on _“Share → Shares”_ * select the share just created * in the menu on the right select _“Manage Rules”._ ![../../../_images/op_share_generic_img3.png](https://docs.hpc.cineca.it/_images/op_share_generic_img3.png) Click on _“Add rule”_ and set: * access type: Choose “ip”, the rest of options displayed are not available for NFS share’s protocol. * access level: read-write or read-only (depending on your needs) * access to: write the IP with permission to access the share. Only one entry is allowed per rule, therefore, you will have to include a rule for the fixed-IP of each VM. * Finally, click on the “add” button. ![../../../_images/op_share_generic_img4.png](https://docs.hpc.cineca.it/_images/op_share_generic_img4.png) Mount the share on the VMs[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/generic_share_create.html#mount-the-share-on-the-vms "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- You are now ready to mount the share on VMs. In the following example, we will consider two VM with Ubuntu 22.04 OS. **Please refer to the network guide of the operating system of your VM to be sure about the actions to be performed.** * Login into the first VM. * Upgrade the packages installed in the VM sudo apt update sudo apt upgrade Copy to clipboard * Install the client. The package name is _“nfs-common”_. sudo apt install nfs-common Copy to clipboard * Identify or create the directory in which the share will be mounted (e.g., “/mnt/share\_manila”) sudo mkdir Copy to clipboard * To mount the share you will need the share displayed on the _“Share Overview”_ page on OpenStack dashboard under the keyword _“Export Location/Path”_. Gather this information and proceed. ![../../../_images/op_share_generic_img5.png](https://docs.hpc.cineca.it/_images/op_share_generic_img5.png) * Mount the share with the following command. Beware that different versions of nfs-common are available for different versions of Ubuntu and the syntax of the mount command could change. sudo mount \-t nfs \-v Copy to clipboard * Then, repeat the same steps for the second VM. --- # Floating IP: allocate and associate — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Network operations](https://docs.hpc.cineca.it/cloud/operative/network_ops/index_network_ops.html) * Floating IP: allocate and associate * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/network_ops/fip_association.rst.txt) * * * Floating IP: allocate and associate[](https://docs.hpc.cineca.it/cloud/operative/network_ops/fip_association.html#floating-ip-allocate-and-associate "Link to this heading") ============================================================================================================================================================================== Allocate a floating IP[](https://docs.hpc.cineca.it/cloud/operative/network_ops/fip_association.html#allocate-a-floating-ip "Link to this heading") ----------------------------------------------------------------------------------------------------------------------------------------------------- The allocation will reserve and return a floating IP from the available pool for the project. To allocate a floating IP to a project, in the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) * click on _“Network → Floating IPs”_ * then click on the button _“Allocate IP to Project”_ on the right side of the dashboard page. * Once allocated, a floating IP can be associated with running instances. ![../../../_images/op_fip_allocation_img1.png](https://docs.hpc.cineca.it/_images/op_fip_allocation_img1.png) ![../../../_images/op_fip_allocation_img2.png](https://docs.hpc.cineca.it/_images/op_fip_allocation_img2.png) At any moment, floating IPs can be de-allocated from a project using the available action for the floating IP. Important It should be noted that if you de-allocate a floating IP from a project and then allocate one again, the floating IP will not be same as before, as the floating IP is automatically select by the system from the available pool. Associate a floating IP to an instance[](https://docs.hpc.cineca.it/cloud/operative/network_ops/fip_association.html#associate-a-floating-ip-to-an-instance "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- With the association of the floating IP to an instance, the instance becomes reachable from the external network. * Click on _“Associate”_ action on the right of the page _“Network → Floating IPs”_. * In the popup, select your virtual machine by the menu in _“Port to be associated”_. ![../../../_images/op_fip_assocation_img1.png](https://docs.hpc.cineca.it/_images/op_fip_assocation_img1.png) ![../../../_images/op_fip_assocation_img2.png](https://docs.hpc.cineca.it/_images/op_fip_assocation_img2.png) The inverse action, _“Dissociate Floating IP”_, is available from the _“Compute → Instances”_ page. At any moment, floating IPs can be de-associated from an instance using the available action for the floating IP. --- # Create and use a CEPHFS_TYPE share — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Shares operations](https://docs.hpc.cineca.it/cloud/operative/shares_ops/index_shares_ops.html) * Create and use a CEPHFS\_TYPE share * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/shares_ops/cephfs_share_create.rst.txt) * * * Create and use a CEPHFS\_TYPE share[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/cephfs_share_create.html#create-and-use-a-cephfs-type-share "Link to this heading") ================================================================================================================================================================================= Request to be enabled to the service[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/cephfs_share_create.html#request-to-be-enabled-to-the-service "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ A user that would like to make use of the Manila service needs to send an email to [superc@cineca.it](mailto:superc%40cineca.it) , communicating how many shares are needed and for each share: * its dimensions (GB) * the instance name of the virtual machines (VMs) that will share that filesystem * the tenant’s name. Once enabled by the User Support Team, the user needs to create the share and mount it in a special network interface on the VMs attached to a dedicated storage network. As an example, in what follows, we will create a share in common between VM\_alice and VM\_bob belonging to the same tenant. Create the share[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/cephfs_share_create.html#create-the-share "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------------- * Create the share by clicking in the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) on _“Create Share”_ in _“Share → Shares_ * In the interface set: * share name * share protocol == “CephFS” (other cases not allowed) * size (on the right side is visualized information about the actual available and used space within the tenant) * Type == “cephfs\_type” * Leave blank the option “Make visible for all projects” because it is not enabled * In the end, click on the “save” button. > ![../../../_images/op_share_cephfs_img1.png](https://docs.hpc.cineca.it/_images/op_share_cephfs_img1.png) * Set the access rule on the share just created. Click on _“Share → Shares”_, then select the share just created and in the menu on the right select _“Manage Rules”_. > ![../../../_images/op_share_cephfs_img2.png](https://docs.hpc.cineca.it/_images/op_share_cephfs_img2.png) * Click on _“Add rule”_ and set: * access type: cephx * access level: read-write or read-only (depending on your needs) * access to: write the name of the client (in our example “charlie”) > ![../../../_images/op_share_cephfs_img3.png](https://docs.hpc.cineca.it/_images/op_share_cephfs_img3.png) * By clicking on the _“add”_ button the dashboard will show the _“access key”_ and _“access to”_ keys that must be used to mount the share on the virtual machines. Mount the share on the VMs[](https://docs.hpc.cineca.it/cloud/operative/shares_ops/cephfs_share_create.html#mount-the-share-on-the-vms "Link to this heading") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- You are now ready to mount the share on VMs. In the following example, we will consider two VM with Ubuntu 20.04 OS. **Please refer to the network guide of the operating system of your VM to be sure about the operations to be done.** * Login in the first VM, configure the network interface attached to the storage network * `ip a` command lists all the network interfaces. Find the new interface, attached to the storage network, and refer to the mac-address of the interface to be sure. > ![../../../_images/op_share_cephfs_img4.png](https://docs.hpc.cineca.it/_images/op_share_cephfs_img4.png) * Create a new file in the /etc/netplan directory to configure such new interface (in our example _“ens7”_), and enable it $ sudo su $ cd /etc/netplan $ cp \- example: cp 50\-cloud-init.yaml ens7.yaml \- $ vim \==\> Modify the value of the fields "ens", "mtu ", "macaddress " and "set-name < value>" with the values shown by "ip a" command. -- example of the ens7.yaml is \-- network: version: 2 ethernets: ens7: dhcp4: true match: macaddress: mtu: set-name: ens7 $ netplan apply \==\> to enable the new interface $ ip a \==\> check that the interface is enabled Copy to clipboard ![../../../_images/op_share_cephfs_img5.png](https://docs.hpc.cineca.it/_images/op_share_cephfs_img5.png) * Install the client, by installing the package named _“ceph-common”_ (**NOTE**: Beware that different versions of ceph-common are available for different versions of Ubuntu and the syntax of the mount command could change.) * Create the mount point in the virtual machines (in our example “/mnt/share\_manila”) and mount the share * To mount the share you will need some information contained in the _“Share Overview”_ page on OpenStack dashboard, in particular you will need the values of PATH, ACCESS\_TO and ACCESS KEY (here an example): ![../../../_images/op_share_cephfs_img6.png](https://docs.hpc.cineca.it/_images/op_share_cephfs_img6.png) Ubuntu 22.04 or higher The command is: `sudo mount.ceph @482d24d4-df47-11eb-8d80-0c42a1f53648.g100_fs=   -o  mon_addr=,secretfile=` Where `` and `` are the two parts of the “Path” string on OpenStack: > * `` is the first numeric part of the “Path” string, up to “:/volumes”, where each IP has to be separated using the character “/” instead of “,” > > * `` is everything else, from “/volumes/” to the end of the string > Finally, the `` is the path to a text file that contains the string ``. Following the same example that uses the picture from above: `sudo mount.ceph  charlie@482d24d4-df47-11eb-8d80-0c42a1f53648.g100_fs=/volumes/_nogroup/43aa4ecc-1db6-4952-b2dd-6336b45075d5 /mnt/share_manila/ -o mon_addr=10.35.1.9:6789/10.35.1.10:6789/10.35.1.11:6789/10.35.1.12:6789/10.35.1.13:6789,secretfile=/home/ubuntu/my_secret_file.txt` Ubuntu 20.04 The command is: `sudo mount -t ceph -v -o  name=,secret=` An example of the complete command is: `sudo mount -t ceph -v 10.35.1.9:6789,10.35.1.10:6789,10.35.1.11:6789,10.35.1.12:6789,10.35.1.13:6789:/volumes/_nogroup/43aa4ecc-1db6-4952-b2dd-6336b45075d5 /mnt/share_manila/ -o name=my-client-name,secret=AQBP07Nejv/RLhAABYqQ5tvgePh2EP7EL0UuhQ==` **NOTE**: If you are using a different Linux distribution, please refer to the ceph user manual to be sure that the syntax you are using is appropriate for the ceph version installed. > * Then repeat the same steps for the second VM as well. Now the two VMs share the same filesystem > --- # Database: create — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Database operations](https://docs.hpc.cineca.it/cloud/operative/db_ops/index_db_ops.html) * Database: create * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/db_ops/db_create.rst.txt) * * * Database: create[](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_create.html#database-create "Link to this heading") ============================================================================================================================= Horizon dashboard * Click on _“Database → Instances → Launch Instance”_ > ![../../../_images/op_db_create_img1.png](https://docs.hpc.cineca.it/_images/op_db_create_img1.png) * Fill in the fields described below for the different tabs Details tab * **Availability Zone**: nova * **Instance Name**: * **Volume Size**: . By default the maximum allowed is 10 GB. * **Volume Type**: choose between “\_\_DEFAULT\_\_” or “LUKS”. The second one is for encrypted volumes. * **Datastore**: choose among the available datastores. They are listed in the drop-down menu showing also the available versions. * **Flavor**: it is the dimension of the VM that will have the database volume attached. Insert fl.ada.xxs, since you will not be allowed by design to login into this VM. * **Locality**: None > ![../../../_images/op_db_create_img2.png](https://docs.hpc.cineca.it/_images/op_db_create_img2.png) Networking tab * **Selected networks**: , choose one among the available network in the project. * Make sure to create the network before creating the database instance (see [Network: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/network_create.html#network-create) ). > ![../../../_images/op_db_create_img3.png](https://docs.hpc.cineca.it/_images/op_db_create_img3.png) Database access tab * **Is public**: Check this box if you want to allow access to the database instance from the public network; otherwise leave blank. * **Allowed CIDRs**: . Specify the allowed IP or IP-ranges from which to access the database service. > ![../../../_images/op_db_create_img4.png](https://docs.hpc.cineca.it/_images/op_db_create_img4.png) Initialize Databases tab * **Initial Databases**: . Note that additional databases can be created later. * **Initial Admin Users**: * **Password**: * **Allowed Hosts**: optional value, to further restrict for this specific database the allowed Host or IP addresses able to connect to the database. > ![../../../_images/op_db_create_img5.png](https://docs.hpc.cineca.it/_images/op_db_create_img5.png) Advanced tab * **Configuration Group and Source from Initial State**: Fill these two fields only if you want to create the database using a previous backup, or as a replica of an other database instance. * **Replica Count**: fill in this field only if you want to have multiple replicas of this database instance. > ![../../../_images/op_db_create_img6.png](https://docs.hpc.cineca.it/_images/op_db_create_img6.png) * At the end, click on _“Launch”_ on the right bottom to launch the Database instance. * To access the created database you can refer to dedicated page [Accessing the Database instance](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_access.html#accessing-the-database-instance) Command Line Interface Important **Software required** To use [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) , the following additional packages are needed. It is recommended to install these packages in a virtual environment. pip install python-openstackclient\==5.8.0 pip install python-troveclient Copy to clipboard For more information, see python-troveclient, a command-line client for the Trove API. **Setting up the environment variables** Make sure to have a valid Application Credential for the project. Please refer to the [Application credentials creation](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#application-credentials-creation) page where it is described how to get OpenStack Application Credentials. **Create the database instance** To create a database instance, you need to execute a command as `openstack database instance create ` specifying at least the following parameters: > * **name**: The name of the database instance. > > * **flavor**: The flavor of the database instance. Insert fl.ada.xxs, since you will not be allowed by design to login into this VM. > > * **datastore**: The datastore of the database instance. > > * **datastore-version**: The version of the datastore to use. > > * **size**: The size of the instance disk volume in GB. By default the maximum allowed is 10 GB. > > * **nic**: The network interface card of the database instance. > > * **net-id**: The network id of the database instance. > > * **allowed-cidr**: The allowed cidr of the database instance. It is an IP or a range of IPs from which the database instance can be accessed. > > * **database**: The name of the initial database. > > * **users**: The username and password of the admin user in the database. > > * **is-public**: Add this flag to be able to access the database from internet. Otherwise, the database will be accessible only from an other VM present in the same network of the project. > For the full list of options please type the command: `openstack database instance create --help`. Make sure to create a network stack ([Network: create](https://docs.hpc.cineca.it/cloud/operative/network_ops/network_create.html#network-create) ) and copy the id of the network before creating a database instance. The example below shows how to create a database instance: openstack database instance create MyTroveDB \--flavor fl.ada.xxs \--datastore mysql \--datastore-version 5.7.29 \--size 10 \--nic net-id\= \\ --databases test \--is-public \--users : \--allowed-cidr xx.xx.xx.xx/y +--------------------------+--------------------------------------+ | Field | Value | +--------------------------+--------------------------------------+ | allowed\_cidrs | \[xx.xx.xx.xx/y\] | | created | 2023\-03-30T10:09:46 | | datastore | mysql | | datastore\_version | 5.7.29 | | datastore\_version\_number | 5.7.29 | | flavor | 3496e9e0-60c4-471a-99ce-51f3d0a8048b | | id | \--- the ID of the DB instance | | name | MyTroveDB | | operating\_status | | | public | True | | region | RegionOne | | service\_status\_updated | 2023\-03-30T10:09:46 | | status | BUILD | | updated | 2023\-03-30T10:09:46 | | volume | 10 | +--------------------------+--------------------------------------+ Copy to clipboard Once created, successfully, * if the flag _“–is-public”_ is specified, you will be provided with a public IP (also referred to as Floating IP) address attached to the database instance. You can use this IP address to reach the database instance from the internet. * Otherwise, only the IP of the internal network of the project will be presented. The allowed-cidr address will determine whether the database/s can be accessed from outside of the network. If the user wants to access the database from another VM in the same network then the user has to specify the CIDR of the network where the VM belongs to, whereas, for public internet access, the user has to specify the CIDR of the public network (for example 0.0.0.0/0 for all internet or a sub-range). **Check the status of the database instance** To check the status of a database instance, please use the following commands: $ openstack database instance list \# will list all the database instances present in the cluster +--------------------------------------+-----------------+-----------+-------------------+--------+------------------+--------+----------------------------------------------------------------------------------------------------+--------------------------------------+------+------+ | ID | Name | Datastore | Datastore Version | Status | Operating Status | Public | Addresses | Flavor ID | Size | Role | +--------------------------------------+-----------------+-----------+-------------------+--------+------------------+--------+----------------------------------------------------------------------------------------------------+--------------------------------------+------+------+ | | MyTroveDB | mysql | 5.7.29 | ACTIVE | HEALTHY | True | \[{'address': 'xx.xx.xx.xx', 'type': 'private'}, {'address': 'xx.xx.xx.xx', 'type': 'public'}\] | 7595d735-6de4-415f-a958-838089a09080 | 10 | | +--------------------------------------+-----------------+-----------+-------------------+--------+------------------+--------+----------------------------------------------------------------------------------------------------+--------------------------------------+------+------+ Copy to clipboard With the ID of the database instance just created, you can check its status with the following command: $ openstack database instance show +--------------------------+-----------------------------------------------------------------------------------------------------+ | Field | Value | +--------------------------+-----------------------------------------------------------------------------------------------------+ | addresses | \[{'address': 'xx.xx.xx.xx', 'type': 'private'}, {'address': 'xx.xx.xx.xx', 'type': 'public'}\] | | allowed\_cidrs | \['xx.xx.xx.xx/y', 'xx.xx.xx.xx/y'\] | | created | 2023\-06-09T09:19:11 | | datastore | mysql | | datastore\_version | 5.7.29 | | datastore\_version\_number | 5.7.29 | | flavor | 7595d735-6de4-415f-a958-838089a09080 | | id | \--- the ID of the DB instance | | ip | xx.xx.xx.xx, xx.xx.xx.xx | | name | MyTroveDB | | operating\_status | HEALTHY | | public | True | | region | RegionOne | | service\_status\_updated | 2023\-06-09T09:22:21 | | status | ACTIVE | | updated | 2023\-06-09T09:20:48 | | volume | 10 | | volume\_used | 0.22 | +--------------------------+-----------------------------------------------------------------------------------------------------+ Copy to clipboard Also, check the status and the operating\_status. If the status is _ACTIVE_, then the database instance is ready to use. It may takes some time to get the database instance in ACTIVE state after being created. To access the created database you can refer to dedicated page [Accessing the Database instance](https://docs.hpc.cineca.it/cloud/operative/db_ops/db_access.html#accessing-the-database-instance) --- # Instance: snapshot create — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Compute operations](https://docs.hpc.cineca.it/cloud/operative/compute_ops/index_compute_ops.html) * Instance: snapshot create * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/compute_ops/instance_snap_create.rst.txt) * * * Instance: snapshot create[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#instance-snapshot-create "Link to this heading") =============================================================================================================================================================== The creation of a snapshot image from an existing VM hosted on OpenStack will differ depending on whether the VM is ephemeral or instantiated from a bootable drive, namely bootable VM (see [Instances](https://docs.hpc.cineca.it/cloud/os_overview/os_components/compute.html#instances) ). Preliminary steps[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#preliminary-steps "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------------------ Independently from the type of VM, to avoid errors in the operation and inconsistent state of the VM, **it is mandatory to create the snapshot after the VM has been shut down**. In the [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) : * Shutdown the VM * Detach any secondary volume attached on the VM (remember the volume _/dev/vda_ is the bootable volume from which the VM is loaded). Ephemeral VM[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#ephemeral-vm "Link to this heading") -------------------------------------------------------------------------------------------------------------------------------------- If the VM is loaded from an image, there is no bootable volume _/dev/vda_. Horizon Dashboard * In the Horizon Dashboard, go to _Compute → Instances_ * Click on _“Create snapshot”_ action for the instance to snapshot ![../../../_images/op_snap_create_1.jpg](https://docs.hpc.cineca.it/_images/op_snap_create_1.jpg) * From the pop-up dialog, give a unique _“Snapshot Name”_ for the image snapshot file, then click on the _“Create Snapshot”_. ![../../../_images/op_snap_create_2.jpg](https://docs.hpc.cineca.it/_images/op_snap_create_2.jpg) * The procedure will bring you to the _“Compute → Images”_ section, where the snapshot image will appear after the generation process indicated by the transition of the image _“Status”_ from _“Queued”_ to _“Active”_, passing from _“Saving”_. * At the end the snapshot should appear with size different from zero. ![../../../_images/op_snap_create_3.jpg](https://docs.hpc.cineca.it/_images/op_snap_create_3.jpg) Command Line Interface * Configure your CLI following the steps in [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) * Obtain the complete tabular list of all the servers available on the tenant > openstack server list > +----------------+------------------+--------+-----+------------------+ > | ID | Name | Status | ... | Flavor | > +----------------+------------------+--------+-----+------------------+ > | | | ACTIVE | | | > | | | ACTIVE | | | > | | | ACTIVE | | | > ... > | | | ACTIVE | | | > +----------------+------------------+--------+-----+------------------+ > > Copy to clipboard * Use the ID corresponding to the server name from which is desired to generate a snapshot image to run the command > openstack server image create \--name \--wait > > Copy to clipboard * Even if the CLI does not show a progress bar for the snapshotting procedure, the shell will hang until the snapshot image _“Status”_ will become _“Active”_. * At the completion of the creation, you can check if the snapshot has been correctly created and in active state with the command > openstack image list \--name > +---------------+------------------+--------+ > | ID | Name | Status | > +---------------+------------------+--------+ > | | | active | > +---------------+------------------+--------+ > > Copy to clipboard Bootable VM[](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_snap_create.html#bootable-vm "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------ If the VM is loaded from a bootable volume, there is a volume attached as _/dev/vda_. Important For bootable VM, the snapshotting procedure can be done only if the **bootable volume is not set to be deleted** once the attached VM will be deleted. Before starting the creation of the VM snapshot, * save the ID of the bootable volume from which your VM has been created () * delete the VM instance to snapshot following the instructions in [Instance: delete](https://docs.hpc.cineca.it/cloud/operative/compute_ops/instance_deletion.html#instance-delete) . Horizon dashboard * Navigate to _“Volumes → Volumes”_ * On the row corresponding to the volume attached to the previously deleted instance, click on the action _“Upload to Image”_. > ![../../../_images/op_snap_create_5.jpg](https://docs.hpc.cineca.it/_images/op_snap_create_5.jpg) * From the pop-up dialog, give a unique _“Snapshot Name”_ for the image snapshot file, then click on _“Upload”_. > ![../../../_images/op_snap_create_6.jpg](https://docs.hpc.cineca.it/_images/op_snap_create_6.jpg) * The procedure will close the pop-up window and the status of the volume will change to _“uploading”_. > ![../../../_images/op_snap_create_7.jpg](https://docs.hpc.cineca.it/_images/op_snap_create_7.jpg) * Once the status of the volume will become once again _“Available”_, a snapshot image of the volume will appear in the _“Compute → Images”_ section with a size different from zero. Command Line Interface * Configure your CLI following the steps in [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) * Use the bootable , saved before, to upload a snapshot image openstack image create \--volume +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | container\_format | bare | | disk\_format | raw | | display\_description | | | id | | | image\_id | | | image\_name | | | protected | False | | size | | | status | uploading | | updated\_at | | | visibility | shared | | volume\_type | \_\_DEFAULT\_\_ | +---------------------+--------------------------------------+ Copy to clipboard * The creation will be completed, once the status of the image will become _“Available”_, you can check this with > openstack image list \--name > +---------------+------------------+--------+ > | ID | Name | Status | > +---------------+------------------+--------+ > | | | active | > +---------------+------------------+--------+ > > Copy to clipboard --- # Volume: resize — CINECA HPC Documentation 1.0 documentation * [](https://docs.hpc.cineca.it/index.html) * [Operative Manual](https://docs.hpc.cineca.it/cloud/operative/index_operative_manual.html) * [Storage operations](https://docs.hpc.cineca.it/cloud/operative/storage_ops/index_storage_ops.html) * Volume: resize * [View page source](https://docs.hpc.cineca.it/_sources/cloud/operative/storage_ops/volume_resize.rst.txt) * * * Volume: resize[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_resize.html#volume-resize "Link to this heading") ================================================================================================================================== Resize a volume[](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_resize.html#resize-a-volume "Link to this heading") ------------------------------------------------------------------------------------------------------------------------------------- Users are able to increase in size any of their volumes. This operation can be done either via [Horizon Dashboard](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/dashboard.html#horizon-dashboard) or via [Command Line Interface](https://docs.hpc.cineca.it/cloud/os_overview/management_tools/command_line.html#command-line-interface) . Important It is not possible to decrease the size of the volume (the volume can only be extended). The operations change depending on whether the volume to be resized is a primary volume (like the root volume of a bootable VM) or secondary volume. Primary Volume In order to perform the resize operation of a primary volume (_e.g._ the root disk of a bootable VM), it is strongly recommended shuting down the VM and it is mandatory to delete the instance attached to the volume. However, is it possible to ensure that the volume is not removed togheter with the instance to which is attached, by verifying that the volume’s `Delete On Termination` property is set to `False`. This flag is only visible via command line: First of all identify the ID of the bootable VM with: $ openstack server list +----------------+------------------+--------+-----+--------------------------+------------------+ | ID | Name | Status | ... | Image | Falvor | +----------------+------------------+--------+-----+--------------------------+------------------+ | | | ACTIVE | | N/A (booted from volume) | | | | | ACTIVE | | | | | | | ACTIVE | | | | ... | | | ACTIVE | | | | +----------------+------------------+--------+-----+--------------------------+------------------+ Copy to clipboard Then, visualize the details of the volume attached to the instance as follows: $ openstack server volume list +----------+---------------------------+------+------------------------+-----------------+-------------------------+ | Device | Server ID | Volume ID | Tag | Delete On Termination? | Attachment ID | BlockDeviceMapping UUID | +----------+-------------+-------------+------+------------------------+-----------------+-------------------------+ | /dev/vda | | | None | False | | | +----------+-------------+-------------+------+------------------------+-------------------------------------------+ Copy to clipboard In case, that the volume’s `Delete On Termination` property is set to `True`, it can be changed via the following command: $ openstack server volume set \--preserve-on-termination Copy to clipboard Horizon Dashboard Delete the instance ![cloud/_img/op_volume_resize_img8.png](https://docs.hpc.cineca.it/cloud/operative/storage_ops/cloud/_img/op_volume_resize_img8.png) Navigate to the _“Volumes”_ tab and proceed to resize the volume by selecting the action _“Extend Volume”_. ![../../../_images/op_volume_resize_img4.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img4.jpg) In the pop-up window that opens, you can select the New Size. ![../../../_images/op_volume_resize_img5.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img5.jpg) After entering the new size, click on _“Extend Volume”_ button. Command Line Interface First, list the servers and the volumes to identify the IDs of the bootable VM. $ openstack server list +----------------+------------------+--------+-----+--------------------------+------------------+ | ID | Name | Status | ... | Image | Falvor | +----------------+------------------+--------+-----+--------------------------+------------------+ | | | ACTIVE | | N/A (booted from volume) | | | | | ACTIVE | | | | | | | ACTIVE | | | | ... | | | ACTIVE | | | | +----------------+------------------+--------+-----+--------------------------+------------------+ Copy to clipboard Identify the volume ID used as boot source for the VM. $ openstack server volume list +----------+---------------------------+------+------------------------+-----------------+-------------------------+ | Device | Server ID | Volume ID | Tag | Delete On Termination? | Attachment ID | BlockDeviceMapping UUID | +----------+-------------+-------------+------+------------------------+-----------------+-------------------------+ | /dev/vda | | | None | False | | | +----------+-------------+-------------+------+------------------------+-------------------------------------------+ Copy to clipboard Remove the instance $ openstack server delete Copy to clipboard Resize the volume: $ openstack volume set \--size Copy to clipboard Once the volume has been resized, you can proceed in creating a new bootable VM using the newly sized volume as boot source. Important As opposed to the resize procedure of a secondary volume, the added disk space will be automaticaly used for the extension of the exsiting disk. Be aware that the `WARNING REMOTE HOST IDENTIFICATION HAS CHANGED` message will arise when trying to ssh into the new instance if the floating IP assigned to the deleted VM will be assigned to the new one. Secondary Volume > If the volume is attached to a virtual machine instance, we strongly recommend shutting down the instance and detatching the volume before performing the operation. > > The resize operation can be performed both via the Horizon Dashboard and the OpenStack command line interface. > > Horizon Dashboard > > Make sure that the volume you want to resize is not attached to any instance. If it is attached, detach it following the steps in [Attach/Detach a volume](https://docs.hpc.cineca.it/cloud/operative/storage_ops/volume_create.html#attach-detach-a-volume) > . > > ![../../../_images/op_volume_resize_img2.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img2.jpg) > > And select the volume you want to resize from the list. > > ![../../../_images/op_volume_resize_img3.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img3.jpg) > > Now you can proceed to resize the volume by selecting the action _“Extend Volume”_. > > ![../../../_images/op_volume_resize_img4.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img4.jpg) > > In the pop-up window that opens, you can select the New Size. > > ![../../../_images/op_volume_resize_img5.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img5.jpg) > > After entering the new size, click on _“Extend Volume”_ button. > > Once the volume has been resized, you can re-attach it to the instance using the Manage Attachments option. > > ![../../../_images/op_volume_resize_img6.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img6.jpg) ![../../../_images/op_volume_resize_img7.jpg](https://docs.hpc.cineca.it/_images/op_volume_resize_img7.jpg) > > Command Line Interface > > First, list the servers and the volumes to identify the IDs of the resources. > > $ openstack server list > +----------------+------------------+--------+-----+------------------+ > | ID | Name | Status | ... | Flavor | > +----------------+------------------+--------+-----+------------------+ > | | | ACTIVE | | | > | | | ACTIVE | | | > | | | ACTIVE | | | > ... > | | | ACTIVE | | | > +----------------+------------------+--------+-----+------------------+ > > Copy to clipboard > > $ openstack volume list > +----------------+------------------+----------+---------------------------------------+ > | ID | Name | Status | Attached to | > +----------------+------------------+----------+---------------------------------------+ > | | | | | > | | | | Attached to on /dev/vdX | > | | | | | > ... > | | | | Attached to on /dev/vdX | > +----------------+------------------+----------+---------------------------------------+ > > Copy to clipboard > > Detach the volume from the instance (if it is attached): > > $ openstack server remove volume > > Copy to clipboard > > Resize the volume: > > $ openstack volume set \--size > > Copy to clipboard > > Finally, you can re-attach the volume to the instance: > > $ openstack server add volume > +-----------------------+-------------+ > | Field | Value | > +-----------------------+-------------+ > | ID | | > | Server ID | | > | Volume ID | | > | Device | /dev/vdX | > | Tag | None | > | Delete On Termination | False | > +-----------------------+-------------+ > > Copy to clipboard After this, you can restart the instance. Once these operations are done, OpenStack will assume the volume has the new size and it will appear to the machine as a device of a different size. Important It is necessary to update the device partitions and then the filesystem must be extended to occupy all the free space that has been created. These operations depend on the operating system and the type of filesystem. For example, on a Rocky Linux system, you can resize an ext4 filesystem using the tools `fdisk`, `e2fsck`, and `resize2fs`. During these operations, all precautions must be taken to avoid data loss; therefore, we suggest you perform the necessary checks and ensure you use the specific tools depending on the volume’s filesystem. Refer to your operating system documentation for more details on resizing partitions and filesystems. Example: extending a partition formatted with XFS filesystem on Ubuntu As a reference, we provide here an example of extending a partition formatted with **XFS filesystem** on **Ubuntu 24.04**. We assume the volume has been already extended in OpenStack as shown above. We cannot cover all the combinations of OS and filesystems, so please refer to your OS documentation for the specific commands needed. Important These operations can lead to data loss if not done properly. Please ensure you have backups and understand the commands before executing them. The following example assumes the volume is attached as `/dev/sdb1` and is mounted in `/data`. The volume has been resized from 250GB to 400GB in OpenStack. * Check file system space usage ubuntu@vm:~$ df \-h Filesystem Size Used Avail Use% Mounted on tmpfs 34G 1.5M 34G 1% /run /dev/sda1 29G 25G 3.1G 90% / tmpfs 166G 4.0K 166G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda16 881M 117M 703M 15% /boot /dev/sda15 105M 6.2M 99M 6% /boot/efi tmpfs 34G 20K 34G 1% /run/user/1000 /dev/sdb1 250G 188G 63G 76% /data # <-- Filesystem to be extended Copy to clipboard * Show block devices and partitions ubuntu@vm:~$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 30G 0 disk ├─sda1 8:1 0 29G 0 part / ├─sda14 8:14 0 4M 0 part ├─sda15 8:15 0 106M 0 part /boot/efi └─sda16 259:0 0 913M 0 part /boot sdb 8:16 0 400G 0 disk # <-- Volume is 400GB └─sdb1 8:17 0 250G 0 part /data # <-- Partition is still 250GB Copy to clipboard * Grow the partition to occupy all the new space ubuntu@vm:~$ sudo growpart /dev/sdb 1 CHANGED: partition=1 start=2048 old: size=524285919 end=524287966 new: size=838858719 end=838860766 Copy to clipboard * Verify partition resized ubuntu@vm:~$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 30G 0 disk ├─sda1 8:1 0 29G 0 part / ├─sda14 8:14 0 4M 0 part ├─sda15 8:15 0 106M 0 part /boot/efi └─sda16 259:0 0 913M 0 part /boot sdb 8:16 0 400G 0 disk └─sdb1 8:17 0 400G 0 part /data # <-- Partition was extended Copy to clipboard * Verify filesystem size ubuntu@vm:~$ df \-h Filesystem Size Used Avail Use% Mounted on tmpfs 34G 1.5M 34G 1% /run /dev/sda1 29G 25G 3.1G 90% / tmpfs 166G 4.0K 166G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda16 881M 117M 703M 15% /boot /dev/sda15 105M 6.2M 99M 6% /boot/efi tmpfs 34G 20K 34G 1% /run/user/1000 /dev/sdb1 250G 188G 63G 76% /data # <-- Filesystem not yet updated Copy to clipboard * Grow the XFS filesystem ubuntu@vm:~$ sudo xfs\_growfs /data meta-data=/dev/sdb1 isize=512 agcount=7, agsize=9830336 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=1 = reflink=1 bigtime=1 inobtcount=1 nrext64=0 data = bsize=4096 blocks=65535739, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=19199, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 65535739 to 104857339 Copy to clipboard * Verify that the filesystem has been resized ubuntu@vm:~$ df \-h Filesystem Size Used Avail Use% Mounted on tmpfs 34G 1.5M 34G 1% /run /dev/sda1 29G 25G 3.1G 90% / tmpfs 166G 4.0K 166G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda16 881M 117M 703M 15% /boot /dev/sda15 105M 6.2M 99M 6% /boot/efi tmpfs 34G 20K 34G 1% /run/user/1000 /dev/sdb1 400G 191G 210G 48% /data # <-- Expected result Copy to clipboard As you can see, the partition and filesystem have been successfully resized to utilize the full 400GB of the volume. Also, in this specific case there’s no need to unmount the filesystem or stop the machine to perform the resize operation, but this may vary depending on the OS and filesystem type. Important There is a limit of 2TB for formatting volumes with the MBR partition table. If you need to create or resize a volume beyond this limit, you must use the GPT partition table. --- # Unknown .. \_compute\_inst\_create\_card: Instance: create ================ The creation of a virtual machine can be done using the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`, using the :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` or with customized Ansible and Terraform scripts to automate the process (:ref:\`cloud/os\_overview/management\_tools/infrastructure\_as\_code:declarative and procedural approaches to iac\`). You are free to choose any of these methods. However, there are a few mandatory steps that must be followed before creating a virtual machine. This page will guide you step by step in the creation of a virtual machine with the \*\*Openstack Horizon Dashboard\*\*. .. important:: \*\*Prerequisites\*\*: In order to create and boot a virtual machine, you need have already created the following resources - A tenant network → :ref:\`cloud/operative/network\_ops/network\_create:network: create\` - A security group → :ref:\`cloud/operative/network\_ops/secgroups\_create:security groups: create\` - A key-pair (it is possible to create a new key-pair during instance creation or upload a preexisting pair) → :ref:\`cloud/operative/compute\_ops/keypair\_create:key pair: create\` Virtual machine creation ------------------------ Once you have completed the above steps, you can proceed to create a Virtual Instance. Go to \*"Compute → Instances"\* and click on \*"Launch Instance"\*. A pop-up window will appear and you have to fill in the required information in the different tabs. .. image:: ../../\_img/op\_instace\_create\_img1.png .. dropdown:: Details :animate: fade-in-slide-down :color: light Under the tab \*"Details"\*, you need to insert the instance name and how many copies of this virtual machine you want to create (\*"count"\* field). .. dropdown:: Source :animate: fade-in-slide-down :color: light Under the tab \*"Source"\*, you specify if you want to boot your virtual machine form an image/image snapshot or from a volume/volume snapshot. When booting from an image, you are able to decide if you want the virtual machine to be created as "ephemeral" or you want to create a root "bootable volume" contextually to the virtual machine creation by using the \*Create New Volume\* checkbox (see :ref:\`cloud/os\_overview/os\_components/compute:compute\` for more information on bootable and ephemeral instances). If you select to create a \*bootable\* instance you have also to specify if the volume will be deleted at deletion of the VM (\*Delete Volume on Instance Delete\*). If the volume is created with the option \*Delete on termination\* active, this configuration can be changed later on only via :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` with this command \`openstack server volume set --preserve-on-termination \` In this section, you also need to choose from the available list which image you would like to use to build your instance. To select the one to be use simply click on the up arrow from the list of available resources. .. dropdown:: Flavor and network :animate: fade-in-slide-down :color: light Under the respective tabs, you can choose the flavor of the virtual machine, and the network you want the virtual machine connected to. To select the one to be used, simply click on the up arrow from the list of available resources. .. note:: The network should have been already created .. dropdown:: Security groups and key pair :animate: fade-in-slide-down :color: light Under the respective tabs, you can choose the security groups you want to apply to the virtual machine, and the key-pairs you want to use to access. To select the one to be used, simply click on the up arrow from the list of available resources. .. note:: The key pair should have been already created .. dropdown:: Server groups :animate: fade-in-slide-down :color: light Under the \*"Server Groups"\* tab you find the options to create the virtual machine under a previously created affinity or anti-affinity group (see :ref:\`cloud/os\_overview/os\_components/compute:compute\` for more information on affinity groups). Once you have setup the preferred configuration, click on \*"Launch instance"\* to create your virtual machine. .. warning:: Once the instance has been created, it is possible only to modify the security groups and the flavor. If you need to change the network, or the key pair, you will need to create a new instance. Associate a Floating IP ----------------------- Once the virtual machine is created, you can associate a floating IP with the virtual machine. For more information on how to associate a floating IP with a virtual machine, refer to the :ref:\`cloud/operative/network\_ops/fip\_association:floating ip: allocate and associate\`. Once the floating IP is associated with the virtual machine, you can access the virtual machine using it. Accessing the instance ---------------------- .. note:: If you created an instance using a default image available in the cloud computing, by default it is possible to login into the instance only by using the default user and key. Suppose you have used the default ubuntu cloud image, you can login as: .. code:: bash $ ssh -i MyKey.pem ubuntu@ --- # Unknown .. \_network\_create\_card: Network: create ================= - In the Horizon Dashboard go to \*"Network → Networks"\* tab and select \*"Create network"\* button. .. image:: /cloud/\_img/op\_network\_create\_img1.png - In the pop-up window add a name for your Network and select the \*"Create Subnet"\* checkbox. .. image:: /cloud/\_img/op\_network\_create\_img2.png - In the \*Subnet tab\*, assign a name to the Subnet and provide the Network address and Gateway IP. As an example you can set: - Network Address: 192.168.0.0/24 - Gateway IP: 192.168.0.254 (the last address for subnet 192.168.0.0/24) .. image:: /cloud/\_img/op\_network\_create\_img3.png - In the last step, select \*"Enable DHCP"\* checkbox. .. image:: /cloud/\_img/op\_network\_create\_img4.png - Click the \*"Create"\* button on the right bottom side of the window. --- # Unknown .. \_volume\_create\_card: Volume: create and attach ========================= Create a volume --------------- - From the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`, go to the \*"Volumes → Volumes"\* section and select \*"Create Volume"\* .. image:: /cloud/\_img/op\_volume\_create\_img1.png - In the pop-up window, specify \*\*Volume Name\*\*: a name for the volume. \*\*Description\*\*: Optionally, provide a brief description for the volume. \*\*Volume Source\*\*: Select one of the following options: - No source, empty volume: Creates an empty volume. An empty volume does not contain a file system or a partition table. - Image: If you choose this option, a new field for Use image as a source displays. You can select the image from the list. \*\*Type\*\*: - \_\_DEFAULT \_\_ is a general cinder volume - LUKS is for encrypted volumes (see Storing sensitive data for more details). \*\*Size (GiB)\*\*: The size of the volume in gibibytes (GiB). .. image:: /cloud/\_img/op\_volume\_create\_img2.png - Finally, click on \*"Create Volume"\* button. Attach/Detach a volume ---------------------- - To attach a volume to an instance, go to the \*"Volumes → Volumes"\* page and select the volume you would like to attach and then the action \*"Manage Attachments"\* .. image:: /cloud/\_img/op\_volume\_create\_img3.png - Select the instance you would like the volume to be attached to: .. image:: /cloud/\_img/op\_volume\_create\_img4.png - Click \*"Attach volume"\*. At this point, you can view the status of a volume in the \*"Volumes → Volumes"\* tab of the Horizon Dashboard. The volume can be is either in status \*"Available"\* or \*"In-Use"\*. The same \*"Manage Attachments"\* operation can be used to detach a volume from an instance. When the volume is attached, in order to use for storing data you need to log in to the instance to partition, format and mount it (see :ref:\`cloud/operative/storage\_ops/volume\_mount:volume: format and mount\`). .. COMMENTED TO REMOVE IF KNOWN ISSUES SECTION IS APPROVED .. Known issues .. ------------ .. There is a restriction imposed by libvirt which allows a maximum of 28 virtual PCI interfaces used for attaching block devices: 2 of these virtual PCIs are used for server needs (mainly boot device) which leaves 26 virtual PCI interfaces available for block device attaching. For this reason, it is possible to attach maximum 26 volumes to an instance created with the default Ubuntu images provided by CINECA. .. If you need to attach more than 26 volumes to an instance, you can: .. - download the Ubuntu image from the Ubuntu website (\`\`\_) .. - upload it to the HPC Cloud via the Horizon dashboard .. - edit its metadata from the Horizon interface setting the properties \*\*"hw\_scsi\_model = virtio-scsi"\*\* and the \*\*"hw\_disk\_bus = scsi"\*\* in the image metadata. .. - use the new image for creating your instance. .. It is possible to find a more detailed guide for the aforementioned operation inside the :ref:\`cloud/operative/compute\_ops/image\_upload:image: upload\` page. --- # Unknown .. \_compute\_inst\_delete\_card: Instance: delete ================== If you would like to delete one of the VMs, you have created, you can follow the steps below or follow the \*\*Deletion tutorial\*\* in :ref:\`cloud/tutorials/index\_tutorials\_and\_repos:tutorials for openstack dashboard\`. .. warning:: The order of the steps is important to avoid errors during the deletion. .. tab-set:: .. tab-item:: Horizon Dashboard - In the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`, select on \*"Network → Floating IPs"\*, then click on the button \*"Disassociate"\* on the right side of the dashboard page for the IP associated to your VM. If desired, you can also release the floating IP. .. note:: Once you release it, there is no guarantee the same IP can be allocated again. - Go to \*"Compute → Instances"\*, and display the drop-down menu of the VM you want to delete and then click the action \*"Delete Instance"\* .. tab-item:: Command Line Interface - List all the VMs in the tenant .. code-block:: bash openstack server list +----------------+------------------+--------+-------------------------------------------------------------+--------------+---------------+ | ID | Name | Status | Networks | Image | Flavor | +----------------+------------------+--------+-------------------------------------------------------------+--------------+---------------+ | | | ACTIVE | =, ... | | | | | | ACTIVE | =, ... | | | ... | | | ACTIVE | =, ... | | | +--------------------------------------+------+--------+-----------------------------------------+------------------------+---------------+ - Shut down the VM using its ID from the previous step .. code-block:: bash openstack server stop - Find the ID of the Floating IP associated to the VM based using the VM Floating IP Address .. code-block:: bash openstack floating ip list --floating-ip-address +------------------+-----------------------+--------------------+-----------+-----------------------+--------------+ | ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project | +------------------+-----------------------+--------------------+-----------+-----------------------+--------------+ | | | | | | | +------------------+-----------------------+--------------------+-----------+-----------------------+--------------+ - Disassociate the Floating IP .. code-block:: bash openstack floating ip unset .. note:: Once you release it, there is no guarantee the same IP can be allocated again. - Delete the VM .. code-block:: bash openstack server delete If you are deleting the VM because of issues, and you would like to recreate it in a clean environment, it is recommended to remove also the network resources (router, interfaces, etc) that were associated to your instance. For this follow \*\*Deletion tutorial\*\* in :ref:\`cloud/tutorials/index\_tutorials\_and\_repos:tutorials for openstack dashboard\`. --- # Unknown .. \_compute\_inst\_download\_card: Instance: snapshot download =========================== Prepare local system for download ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To avoid errors due to not enough disk space to accommodate the snapshot, check the available disk space on your system .. code-block:: bash df -h / Filesystem Size Used Avail Use% Mounted on /dev/ 466G 93G 349G 22% / In case an external drive is mounted on your system, or another partition is meant to be used, replace the \*”/“\* character with the \*/dev\* path of the drive. In case there is not enough disk space on your local system to store snapshots, it is possible to mount locally a remote host directory using the \*sshfs\* utility. .. code-block:: bash sshfs :/path/to/remote/directory /path/to/local/directory For CINECA users which have access to HPC Clusters, it is strongly suggested to mount the remote directory via the \*datamover\* nodes (see :ref:\`hpc/hpc\_data\_storage:data transfer\`). .. code-block:: bash sshfs @data..cineca.it:/path/to/remote/directory /path/to/local/directory Using \*datamover\* nodes avoid process being killed by surpassing the CPU-time characteristic of long download processes. Also in this case, please check to have enough space to store the snapshot before starting the download. CINECA HPC Cluster users are strongly encouraged to use the \*cinQuota\* command instead of \*du\* to get information about the occupancy of a specific path to avoid stressing the Lustre filesystem: .. code-block:: bash cinQuota ---------------------------------------------------------------------- Filesystem used quota grace files ---------------------------------------------------------------------- 50G - 1T - 1T - HPC Cloud users without access to HPC Clusters can write an email to superc@cineca.it asking for information about how to obtain a budget and storage on HPC Clusters. Search and download snapshots ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To download an instance, it is necessary to create a snapshot of it (see :ref:\`cloud/operative/compute\_ops/instance\_snap\_create:instance: snapshot create\`), and then save it locally. The download of a snapshot can be performed only via the :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\`. Here are the steps to be followed. - The first step is obtaining the complete tabular list of all the images available on the tenant .. code-block:: bash openstack image list +---------------+-----------------+--------+ | ID | Name | Status | +---------------+-----------------+--------+ | | | active | | | | active | | | | active | ... | | | active | +---------------+-----------------+--------+ - Use the ID corresponding to the image snapshot name to start the download procedure using the following command: .. code-block:: bash openstack image save --file /path/to/local/directory/ The OpenStack CLI does not show a progress bar for the download so the shell in which the last openstack command has been launched will hung until the download terminates. You can append to the previous command the ”&“ character to send the download in background: remember to not close the shell in which the command has been launched since the download process will be killed. It is possible to monitor the procedure by using the following watch command: .. code-block:: bash watch -n 1 ls -lrth /path/to/local/directory/ Every 1,0s: ls -lrth /path/to/local/directory/ ... total -rw-rw-r-- 1 The screen will refresh every second showing an increase in both and . - Check the downloaded image info to be sure the process has been executed correctly .. code-block:: bash qemu-img info Limits of the procedure ^^^^^^^^^^^^^^^^^^^^^^^ The time needed to complete the download of the snapshot is strongly influenced by both the size of the snapshot itself and the bandwidth of internet connection. These are some examples of possible download times from OpenStack infrastructure to local server (ex. CINECA-PDL) for a 30 GigaBytes snapshot image: - ~ 60 minutes with nominal ~ 50 Mbps download speed connection: speed characteristics of a mobile hotspot 4G connection. - ~ 10 minutes with nominal ~ 300 Mbps download speed connection: speed characteristics of the wired optic fiber connection like the one available in CINECA (ex. Sede CINECA Casalecchio). --- # Unknown .. \_compute\_keypair\_create\_card: Key Pair: create ================== - In the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`, go to the \*"Compute → Key Pairs"\* tab. - Click on \*"Create Key Pair"\*. .. image:: ../../\_img/op\_keypairs\_create\_img1.png - Provide a name for the KeyPair and select the KeyPair type. .. image:: ../../\_img/op\_keypairs\_create\_img2.png - The private key will be automatically downloaded. Store this file securely as it will be needed for SSH access. .. warning:: - The download of the private key will be done only when the keypair is created. It will not be possible to re-download it. If you lose the private key you will have to create a new keypair. - The public key can be seen in any moment by clicking on the generated Key Pair name. Best Practices -------------- - \*\*Secure Storage\*\*: Store private keys securely. If lost, you won't be able to access your instances using that KeyPair. - \*\*Permissions\*\*: Ensure private key files have restrictive permissions (chmod 600) to prevent unauthorized access. - \*\*Rotation\*\*: Periodically rotate your KeyPairs and update instances accordingly to maintain security. - \*\*Backup\*\*: Keep backups of your private keys in a secure location to prevent accidental loss. - \*\*KeyPairs\*\* in OpenStack provide a secure and efficient method for managing SSH access to instances. By leveraging public-key cryptography, KeyPairs ensure that only users with the appropriate private key can access the instances, enhancing overall security. --- # Unknown .. \_compute\_inst\_resize\_card: Instance: resize ================ .. note:: If you are trying to resize the root volume of your VM, please refer to this page: :ref:\`Instance: root storage increase \` Users are able to resize autonomously their VM resources, this operation can be done either via :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` or via :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` .. important:: Before performing the resize operation: - The VM must be shut off. - If there are encrypted \*\*LUKS VOLUMES\*\* attached to the virtual machine, it is mandatory that the user: - Unmount the volumes from the VM - Detach the volumes from the Horizon Dashboard (see :ref:\`cloud/operative/storage\_ops/volume\_create:attach/detach a volume\`) .. note:: Remember to alert your users of the VM temporary shutdown during the operation, before starting the resize. .. tab-set:: .. tab-item:: Horizon Dashboard - Go to \*"Compute → Instances"\*, and find the VM you need to resize - From the drop-down menu on the right side, select \*"resize instance"\* .. note:: - If you have an \*\*Ephemeral VM\*\*, check the size of root disk of the original VM. Don't resize the VM, if the new flavor has a disk smaller than the current one. - If you have a \*\*VM with a Bootable Disk\*\*, the resize will affect only vCPUs number and RAM. The bootable disk will not be changed by the operation. - A menu will popup where you can choose the new desired flavor and click \*"resize"\* - OpenStack will prepare the operation and then wait for user input to confirm or revert the operation - From the drop-down menu on the right select either \*"confirm resize/migration"\* if you want to continue, or \*"revert resize/migration"\* if you want to keep the original flavor. - Confirm the success of the operation. To do that you will need to boot the VM, login, and verify the vCPUs number and Memory size are correct with the following commands: .. code-block:: bash cat /proc/cpuinfo free -g .. tab-item:: Command Line Interface To know how to configure and use the OpenStack CLI, please refer to the :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` page. - Identify the VM ID. .. code-block:: bash openstack server list --all | grep openstack server show < vm\_ID > | grep flavor - Identify the ID of the new flavor the VM needs. .. code-block:: bash openstack flavor list .. note:: - If you have an \*\*Ephemeral VM\*\*, check the size of root disk of the original VM. Don't resize the VM, if the new flavor has a disk smaller than the current one. - If you have a \*\*VM with a Bootable Disk\*\*, the resize will affect only vCPUs number and RAM. The bootable disk will not be changed by the operation. - Perform the resize. .. code-block:: bash openstack server resize --flavor --wait - Wait for the operation to \*"Complete"\*. - Issue the resize confirmation in a separate command, since the option \*--confirm\* on the command openstack server resize is deprecated. .. code-block:: bash openstack server resize confirm - Verify the success of the operation. Since the Dashboard can have visualization bugs, it is best to check via CLI: .. code-block:: bash openstack server show < vm\_ID > | grep flavor - Confirm the success of the operation. To do that you will need to boot the VM, login, and verify the vCPUs number and Memory size are correct with the following commands: .. code-block:: bash cat /proc/cpuinfo free -g --- # Unknown .. \_compute\_inst\_rescue\_card: Instance: rescue ================ Instance rescue provides a mechanism for accessing, even if an image renders the instance inaccessible. Two rescue modes are currently provided. .. warning:: If the virtual machine has encrypted \*\*LUKS VOLUMES\*\* attached, it is mandatory to detach them before starting the rescue operation. .. tab-set:: .. tab-item:: Ephemeral Virtual Machine - Create a \*\*rescuer\*\* virtual machine with a \*\*new key pair\*\* (:ref:\`cloud/operative/compute\_ops/instance\_create:instance: create\`). Although this is not a fixed rule, it is suggested to create the rescuer machine using an image with \*\*same OS\*\* as the one on the inaccessible machine (same version or newer). - Login to the rescuer and update it. As an example, for Ubuntu virtual machines: .. code-block:: bash sudo apt update sudo apt upgrade - Logout the rescuer and create a \*\*snapshot image\*\* of this virtual machine. - Select the instance you want to rescue, check that its openstack status is \*"Active"\*, and from the drop-down menu on the right select \*"rescue instance"\*: - In the menu that appears, select the image you just created from the rescuer machine. - Login via ssh to the broken machine using the rescuer username/key - Check that the boot of the machine has been correctly executed using the command \`\`lsblk\`\` - You should see the rescuer machine (/dev/vda1) mounted and the inaccessible machine on the device /dev/vdb1 - Mount such device /dev/vdb1 .. code-block:: bash sudo mkdir /mnt/inaccessible\_vm sudo mount /dev/vdb1 /mnt/inaccessible\_vm - Now you can access the files in the inaccessible machine to fix the problems (lsblk, fsck, xfs\_repair, chroot, etc.) or backup important data - Once the operation is done, logout the virtual machine and from the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` select \*"unrescue"\*. .. tab-item:: Bootable Virtual Machine - Shutdown the instance. - In the tab \*"Volumes"\*, track which secondary volumes are attached to the VM to be rescued and detach them. - \*\*IMPORTANT\*\*: verify that the bootable volume won't be erased when deleting the VM! - using the :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\`, execute the command \`\`openstack server volume list \`\`, and have a look to the field \*"delete\_on\_termination"\* that must be set to \*'False'\*. (\*\*Note\*\* that this will work with openstack-cli >= 6.2.0) - If delete\_on\_termination option is set to \*true\*, it can be changed using the :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` with the command \`\`openstack server volume set --preserve-on-termination \`\` - Keep track of the Flavor, Security Groups and FIP associated with the VM (FIP in particular if there is a DNS association). - Delete the instance. - Create a throwaway VM, attach the bootable volume to rescue as a secondary volume and associate a FIP to such VM. - Login via ssh to the throwaway VM and execute all the needed operations on the volume to rescue (lsblk, fsck, xfs\_repair, chroot, etc.). - Once the volume has been recovered, exit the throwaway VM and detach the secondary volume that has been rescued. - Restart the VM from the rescued bootable volume, reattaching the secondary volumes, FIP, and check the problem has been solved. --- # Unknown .. \_compute\_inst\_img\_upload\_card: Image: upload ============== The upload of an image can be done using both the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` and the :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\`: you are free to choose any of the available methods. This page will guide you, the image owner, step by step in the upload of an image with the OpenStack :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`. Once logged into the OpenStack Horizon Dashboard, go to \*"Images"\* in the left panel and click on \*"Create Image"\* button. .. image:: /cloud/\_img/image\_upload\_img1.jpg You have to fill the information in the following pop-up form: .. image:: /cloud/\_img/image\_upload\_img2.jpg Under the tab \*\*“Image Details”\*\*, a name for the image file have to be written in the text box under \*“Image Name”\*: it is also a good practice entering a brief description of the image in the text box under \*“Image Description”\*. Based on if the desired image is stored on your PC or is available online, indicate the appropriate \*“Source Type”\* as \*“File”\* or \*“URL”\* Depending on the previous choice, browse your PC to the location in which is stored the image file or paste the URL to where the image file is hosted. From the \*“Format”\* drop-down menu, select the file format of the desired image to upload. Subsequently, it is possible to specify optional advanced options as \*“Kernel”\* image, \*“Ramdisk”\* image, specify a string for the image \*“Architecture”\*, \*“Minimum Disk”\* quota in GigaByte and a \*“Minimum RAM”\* quota in MegaByte to boot the image. The next step is to select the \*“Image Sharing”\* policies by choosing one of the three options under \*“Visibility”\* keeping in mind: - \*\*Private\*\* images can be used in instance creation only by the image owner. - \*\*Shared\*\* images can be used, by default, by all the users collaborating to the project with the image owner. - \*\*Community\*\* images are \*\*inhibited\*\* even if the option is available to all users. By uploading an image with visibility set to "Community" will raise the following error message preventing the beginning of the upload procedure: .. image:: /cloud/\_img/Create\_Community\_Image\_Error.png Further information on image visibility can be found in :ref:\`cloud/os\_overview/os\_components/compute:images\`, while a description of the risks associated in using community images can be found at :ref:\`cloud/tenant\_adm/security\_guidelines:images\`. Subsequently, it is possible to choose if the image is \*protected\* or not from deleting operations. Finally, under the tab \*\*“Metadata”\*\*, it is possible to search the name of specific resources metadata in the \*“Filter”\* research field at the top of the \*“Available Metadata”\* list. Once found, clicking the blue “+” button will move the resource metadata under the \*“Existing Metadata”\* list allowing the declaration of its value. .. image:: ../../\_img/image\_upload\_img3.jpg To complete de creation of the image, simply create the \*“Create Image”\* blue button at the bottom right corner of the pop-up window. --- # Unknown .. \_compute\_inst\_root\_storage\_increase\_card: Instance: root storage increase =============================== Users are able to resize autonomously the root volume of their VM. The procedure differs wether the VM is ephemeral or created from a bootable volume (more info on the :ref:\`cloud/os\_overview/os\_components/compute:compute\` page). .. important:: Before the procedure is advised to \*\*shut down\*\* the VM and detatch any secondary volumes. .. tab-set:: .. tab-item:: Ephemeral Ephemeral VMs have a fixed root volume size depending on the flavor available on the infrastructure. For this reason, the maximum size that can be reached by and ephemeral instance has an upper limit equal to the root volume size of the greates falvor avialable. All the flavor available for each CINECA's HPC cloud infrastructures can be found in their dedicated pages grouped inside the :ref:\`cloud\_specifics\_card\` page. To increase this storage space, the user will need to create a new VM with a larger root volume using a snapshot of the original VM. This can be achieved both via the Horizon Dashboard and the OpenStack command line inteface. .. tab-set:: .. tab-item:: Horizon Dashboard First of all, disassociate, if present, the floating IP from the VM (do not release it otherwise it will be lost) .. warning:: If the floating IP is also released, it will be lost and a new allocated floating IP will have a different address .. image:: /cloud/\_img/op\_fip\_detach.jpg Create a snapshot of the VM instance following the instruction reported in the :ref:\`compute\_inst\_snap\_create\_card\` card. .. tab-item:: Command line interface First of all identify the ID of the bootable VM with: .. code:: bash $ openstack server list +----------------+------------------+--------+-----+-----------------+------------------+ | ID | Name | Status | ... | Image | Falvor | +----------------+------------------+--------+-----+-----------------+------------------+ | | | ACTIVE | | | | | | | ACTIVE | | | | ... | | | ACTIVE | | | | +----------------+------------------+--------+-----+-----------------+------------------+ Then, disassociate, if present, the floating IP from the VM. .. code:: bash $ openstack server remove floating ip Create a snapshot of the VM instance following the instruction reported in the :ref:\`compute\_inst\_snap\_create\_card\` card. | Once the snapshotting procedure is completed, create a new VM from the snapshot using the option "create new volume" in the tab "source" and chosing the desired volume size (here the full guide on :ref:\`cloud/operative/compute\_ops/instance\_create:virtual machine creation\`) .. image:: /cloud/\_img/op\_inst\_from\_snap.jpg Finally, associate the old floating IP to the newly created VM .. image:: /cloud/\_img/op\_fip\_attach.jpg Once the operation is completed, it is possible to remount any secondary volumes that were detatched at the start of this procedure. .. note:: Remember: before deleting the old virtual machine, be sure the the new VM with increased volume size is up-and-running and is functioning correctly. .. tab-item:: Bootable For all VMs created from a bootable volume, it is necessary to perform the resize operation of VM's Primary Volume to obtain a larger root volume size. The detailed procudure on how to perform such operation is illustrated in the :ref:\`cloud/operative/storage\_ops/volume\_resize:resize a volume\`. --- # Unknown .. \_volume\_mount\_card: Volume: format and mount ========================= After a volume has been created and attached to a virtual machine (see :ref:\`cloud/operative/storage\_ops/volume\_create:volume: create and attach\`), in order to use for storing data you need to partition, format and mount it. To achieve this, these operations have to be performed inside the virtual machine: - Partition Table - Format the volume - Mount the volume .. warning:: These operations is specific to the OS and software installed on the Virtual Machine. There are more than one software capable of partitioning your devices (e.g. \`fdisk\`, \`parted\`). Here you can find an example of how to perform these operations using \`fdisk\`. Partition Table ---------------- Suppose that the volume is attached to the virtual machine as device \*/dev/vdc\*. Login in to the virtual machine and use \`fdisk\` to modify the partition table. - list the partition table .. code-block:: bash sudo fdisk -l - partition of device /dev/vdc .. code-block:: sudo fdisk /dev/vdc # 1 new partition, primary, with default sector numbers and type "Linux" ==> n; p; 1 ; ...default ; # check and write ==> p; w Format the volume ----------------- Format the device \*/dev/vdc\* just partitioned as xfs: .. code-block:: sudo mkfs -t xfs /dev/vdc1 Mount the volume ----------------- .. code-block:: sudo mkdir /mnt/stuff\_1 sudo mount /dev/vdc1 /mnt/stuff\_1 To mount the volume automatically at each boot of the virtual machine, please modify the \*/etc/fstab\* file. Following the example, in the \*/etc/fstab\* could be written: .. code-block:: /dev/vdc1 /mnt/stuff\_1 xfs auto,nofail,defaults 0 0 --- # Unknown .. \_compute\_inst\_manage\_card: Instance: manage and monitor ============================= To monitor your instance, login to the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`. Manage an instance ------------------ - Log in to the virtual machine and shutdown the instance - In the dashboard, go to \*"Compute → Instances"\*. - In the \*"Actions"\* column drop down menu of the instance, you can find all possible operation to manage you instance. You can resize or rebuild an instance, edit instance or the security groups. Depending on the current state of the instance, you can pause, resume, suspend, soft or hard reboot, or terminate Track usage for instances ------------------------- You can track usage for instances for each project. You can track usage per month by showing metrics like number of vCPUs, disks, RAM, and uptime for all your instances. - In the dashboard. choose the project, and go to \*"Compute → Overview"\* - In the page you can see an overview of the used resources (for compute, storage and network) with respect to the quotas assigned to the project - To query the instances usage for a month, select a month and click "Submit". - To download a summary, click \*"Download CSV Summary"\*. Monitor instances ----------------- You can monitor the high-level actions (creation, start, stop) on the instances for each project via offered logs in the dashboard. - Go to \*"Compute → Instances"\*. - Select the instance name you would like to monitor - go to the \*"Action log"\* tab to see the high-level actions (creation, start, stop) - go to the \*"Log"\* tab, to see the instance console log More detailed monitoring logs can be set-up by you within the specific instance. --- # Unknown .. \_secgroups\_create\_card: Security groups: create ========================= Security groups are a set of IP filter rules that are applied to an instance. These rules specify which type of traffic is allowed to reach the instance (see section :ref:\`cloud/os\_overview/os\_components/network:security groups\` for more information). - Go to the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` - Go to \*"Network → Security Groups"\* and click on \*"create security group"\* - Give a name to the security group and click on \*"create security group"\* - The security group is created \*\*by default with NO ingress rules\*\* .. image:: /cloud/\_img/op\_network\_secgroups\_img1.png - Select \*"Add rule"\* to specify the ingress rules. - A list of pre-defined rules is available for you to choose from (but you can create your own rule if you prefer). Common default rules are: - SSH (port 22) - ICMP (allow to "ping" a server) - HTTP (port 80) - HTTPS (port 443) .. image:: /cloud/\_img/op\_network\_secgroups\_img2.png - In the CIDR field, you can limit the range of IPs which are allowed to the rule. \*\*NOTE\*\*: 0.0.0.0/0 means all internet --- # Unknown .. \_db\_access\_card: Database: access ================ Accessing the Database instance ------------------------------- To access the database instance, the typical SQL command (depending on the datastore you chose for your database instance) can be used. In the below example of a MySQL database instance, it is shown how to access the database. After you successfully install the MySQL client, use the following commands to access the database: .. code-block:: bash $ mysql -h -u -p ------------------------------------------------------------------------------------ mysql: \[Warning\] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \\g. Your MySQL connection id is 16 Server version: 5.7.29 MySQL Community Server (GPL) Copyright (c) 2000, 2023, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement. mysql> Once you accessed the database, a mysql prompt will be available. You can check the list of databases using the following command: .. code-block:: bash mysql> show databases; -------------- show databases -------------- +--------------------+ | Database | +--------------------+ | information\_schema | | test | +--------------------+ 2 rows in set (0,02 sec) How to list the available datastore and their version using OpenStack CLI ------------------------------------------------------------------------- A datastore is a database engine that is supported by Trove and it is used to create the database instance. On CINECA HPC Cloud the following datastores namely MySQL, MariaDB, PostgreSQL are available. - You can check the available datastores with the command \`\`openstack datastore list\`\`. The ID and Name of the datastores supported will be shown. .. code-block:: bash $ openstack datastore list # shows all datastores available in Cloud infrastructure +--------------------------------------+------------+ | ID | Name | +--------------------------------------+------------+ | b2103dff-9331-4be2-8193-170f2a509e16 | mariadb | | ed541d5a-d260-4b6b-ac80-74ac38167d70 | mysql | | e8d12fef-3c54-4e83-818c-4a89a104780d | postgresql | +--------------------------------------+------------+ - With the command \`\`openstack datastore version list \`\`, you can check the list of datastores with their version. Below we show an example for mysql datastore. .. code-block:: bash $ openstack datastore version list mysql +--------------------------------------+--------+---------+ | ID | Name | Version | +--------------------------------------+--------+---------+ | a6ee8255-2d71-4e22-b68d-1d0e5919e74a | 5.7.29 | 5.7.29 | | 434292f3-2074-4033-8f19-07874cbe5b7d | 8.0.29 | 8.0.29 | +--------------------------------------+--------+---------+ You will find Version, Name, and ID of the mysql datastores available in HPC Cloud. \*\*Note\*\*: The Version is important, as we have to specify the version of the datastore while creating the database instance. For more information about the version, use the following command: .. code-block:: bash $ openstack datastore version show a6ee8255-2d71-4e22-b68d-1d0e5919e74a # shows details of a version +-----------+--------------------------------------+ | Field | Value | +-----------+--------------------------------------+ | datastore | ed541d5a-d260-4b6b-ac80-74ac38167d70 | | id | a6ee8255-2d71-4e22-b68d-1d0e5919e74a | | name | 5.7.29 | | version | 5.7.29 | +-----------+--------------------------------------+ --- # Unknown .. \_shares\_generic\_create\_card: Create and use a GENERIC\_TYPE share =================================== The following sections describe the steps needed to create a share and mount it on two VMs attached to a local network. Note that the user needs to configure the VMs in a way that allows logging in via ssh. Request to be enabled to the service ------------------------------------ The user willing to make use of the Manila service needs to send an email to superc@cineca.it, communicating - how many shares are needed. - their dimensions (GB). - the tenant's name. Once the tenant is enabled to the service by the User Support Team, all users of the tenant will be able to use the service. Create share network -------------------- As a first step, in the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` you need to create the share network by clicking on \*"Create Share Network"\* in \*"Share → Share Networks"\* and set the value for the following attributes: - Share network name. - network: choose the desired network, in our example example\_share\_guide\_net. - subnet: choose the desired subnet, in our example example\_share\_guide\_subnet. - Click on the \*"save"\* button. .. image:: /cloud/\_img/op\_share\_generic\_img1.png Create the share ---------------- Create the share by clicking on \*"Create Share"\* in \*"Share → Shares"\* and setting the following information: - share name - share protocol == "NFS" - size (on the right side is visualized information about the actual available and used space within the tenant) - Type == "generic\_type" - Leave blank the option "Make visible for all projects" because it is not enabled - In the end, click on the \*"create"\* button. .. image:: /cloud/\_img/op\_share\_generic\_img2.png Set the access rule(s) on the share just created. - On the OpenStack dashboard click on \*"Share → Shares"\* - select the share just created - in the menu on the right select \*"Manage Rules".\* .. image:: /cloud/\_img/op\_share\_generic\_img3.png Click on \*"Add rule"\* and set: - access type: Choose "ip", the rest of options displayed are not available for NFS share's protocol. - access level: read-write or read-only (depending on your needs) - access to: write the IP with permission to access the share. Only one entry is allowed per rule, therefore, you will have to include a rule for the fixed-IP of each VM. - Finally, click on the "add" button. .. image:: /cloud/\_img/op\_share\_generic\_img4.png Mount the share on the VMs -------------------------- You are now ready to mount the share on VMs. In the following example, we will consider two VM with Ubuntu 22.04 OS. \*\*Please refer to the network guide of the operating system of your VM to be sure about the actions to be performed.\*\* - Login into the first VM. - Upgrade the packages installed in the VM .. code-block:: bash sudo apt update sudo apt upgrade - Install the client. The package name is \*"nfs-common"\*. .. code-block:: bash sudo apt install nfs-common - Identify or create the directory in which the share will be mounted (e.g., "/mnt/share\_manila") .. code-block:: bash sudo mkdir - To mount the share you will need the share displayed on the \*"Share Overview"\* page on OpenStack dashboard under the keyword \*"Export Location/Path"\*. Gather this information and proceed. .. image:: /cloud/\_img/op\_share\_generic\_img5.png - Mount the share with the following command. Beware that different versions of nfs-common are available for different versions of Ubuntu and the syntax of the mount command could change. .. code-block:: bash sudo mount -t nfs -v - Then, repeat the same steps for the second VM. --- # Unknown .. \_fip\_associate\_card: Floating IP: allocate and associate ======================================= Allocate a floating IP ---------------------- The allocation will reserve and return a floating IP from the available pool for the project. To allocate a floating IP to a project, in the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` - click on \*"Network → Floating IPs"\* - then click on the button \*"Allocate IP to Project"\* on the right side of the dashboard page. - Once allocated, a floating IP can be associated with running instances. .. image:: /cloud/\_img/op\_fip\_allocation\_img1.png .. image:: /cloud/\_img/op\_fip\_allocation\_img2.png At any moment, floating IPs can be de-allocated from a project using the available action for the floating IP. .. important:: It should be noted that if you de-allocate a floating IP from a project and then allocate one again, the floating IP will not be same as before, as the floating IP is automatically select by the system from the available pool. Associate a floating IP to an instance -------------------------------------- With the association of the floating IP to an instance, the instance becomes reachable from the external network. - Click on \*"Associate"\* action on the right of the page \*"Network → Floating IPs"\*. - In the popup, select your virtual machine by the menu in \*"Port to be associated"\*. .. image:: /cloud/\_img/op\_fip\_assocation\_img1.png .. image:: /cloud/\_img/op\_fip\_assocation\_img2.png The inverse action, \*“Dissociate Floating IP”\*, is available from the \*"Compute → Instances"\* page. At any moment, floating IPs can be de-associated from an instance using the available action for the floating IP. --- # Unknown .. \_shares\_cephfs\_create\_card: Create and use a CEPHFS\_TYPE share ================================== Request to be enabled to the service ------------------------------------ A user that would like to make use of the Manila service needs to send an email to superc@cineca.it, communicating how many shares are needed and for each share: - its dimensions (GB) - the instance name of the virtual machines (VMs) that will share that filesystem - the tenant's name. Once enabled by the User Support Team, the user needs to create the share and mount it in a special network interface on the VMs attached to a dedicated storage network. As an example, in what follows, we will create a share in common between VM\_alice and VM\_bob belonging to the same tenant. Create the share ---------------- - Create the share by clicking in the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` on \*"Create Share"\* in \*"Share → Shares\* - In the interface set: - share name - share protocol == "CephFS" (other cases not allowed) - size (on the right side is visualized information about the actual available and used space within the tenant) - Type == "cephfs\_type" - Leave blank the option "Make visible for all projects" because it is not enabled - In the end, click on the "save" button. .. image:: /cloud/\_img/op\_share\_cephfs\_img1.png - Set the access rule on the share just created. Click on \*"Share → Shares"\*, then select the share just created and in the menu on the right select \*"Manage Rules"\*. .. image:: /cloud/\_img/op\_share\_cephfs\_img2.png - Click on \*"Add rule"\* and set: - access type: cephx - access level: read-write or read-only (depending on your needs) - access to: write the name of the client (in our example "charlie") .. image:: /cloud/\_img/op\_share\_cephfs\_img3.png - By clicking on the \*"add"\* button the dashboard will show the \*"access key"\* and \*"access to"\* keys that must be used to mount the share on the virtual machines. Mount the share on the VMs -------------------------- You are now ready to mount the share on VMs. In the following example, we will consider two VM with Ubuntu 20.04 OS. \*\*Please refer to the network guide of the operating system of your VM to be sure about the operations to be done.\*\* - Login in the first VM, configure the network interface attached to the storage network - \`\`ip a\`\` command lists all the network interfaces. Find the new interface, attached to the storage network, and refer to the mac-address of the interface to be sure. .. image:: /cloud/\_img/op\_share\_cephfs\_img4.png - Create a new file in the /etc/netplan directory to configure such new interface (in our example \*"ens7"\*), and enable it .. code-block:: bash $ sudo su $ cd /etc/netplan $ cp - example: cp 50-cloud-init.yaml ens7.yaml - $ vim ==> Modify the value of the fields "ens", "mtu ", "macaddress " and "set-name < value>" with the values shown by "ip a" command. -- example of the ens7.yaml is -- network: version: 2 ethernets: ens7: dhcp4: true match: macaddress: mtu: set-name: ens7 $ netplan apply ==> to enable the new interface $ ip a ==> check that the interface is enabled .. image:: /cloud/\_img/op\_share\_cephfs\_img5.png - Install the client, by installing the package named \*"ceph-common"\* (\*\*NOTE\*\*: Beware that different versions of ceph-common are available for different versions of Ubuntu and the syntax of the mount command could change.) - Create the mount point in the virtual machines (in our example "/mnt/share\_manila") and mount the share - To mount the share you will need some information contained in the \*"Share Overview"\* page on OpenStack dashboard, in particular you will need the values of PATH, ACCESS\_TO and ACCESS KEY (here an example): .. image:: /cloud/\_img/op\_share\_cephfs\_img6.png .. tab-set:: .. tab-item:: Ubuntu 22.04 or higher The command is: \`\`sudo mount.ceph @482d24d4-df47-11eb-8d80-0c42a1f53648.g100\_fs= -o mon\_addr=,secretfile=\`\` Where \`\`\`\` and \`\`\`\` are the two parts of the "Path" string on OpenStack: - \`\`\`\` is the first numeric part of the "Path" string, up to ":/volumes", where each IP has to be separated using the character "/" instead of "," - \`\`\`\` is everything else, from "/volumes/" to the end of the string Finally, the \`\`\`\` is the path to a text file that contains the string \`\`\`\`. Following the same example that uses the picture from above: \`\`sudo mount.ceph charlie@482d24d4-df47-11eb-8d80-0c42a1f53648.g100\_fs=/volumes/\_nogroup/43aa4ecc-1db6-4952-b2dd-6336b45075d5 /mnt/share\_manila/ -o mon\_addr=10.35.1.9:6789/10.35.1.10:6789/10.35.1.11:6789/10.35.1.12:6789/10.35.1.13:6789,secretfile=/home/ubuntu/my\_secret\_file.txt\`\` .. tab-item:: Ubuntu 20.04 The command is: \`\`sudo mount -t ceph -v -o name=,secret=\`\` An example of the complete command is: \`\`sudo mount -t ceph -v 10.35.1.9:6789,10.35.1.10:6789,10.35.1.11:6789,10.35.1.12:6789,10.35.1.13:6789:/volumes/\_nogroup/43aa4ecc-1db6-4952-b2dd-6336b45075d5 /mnt/share\_manila/ -o name=my-client-name,secret=AQBP07Nejv/RLhAABYqQ5tvgePh2EP7EL0UuhQ==\`\` \*\*NOTE\*\*: If you are using a different Linux distribution, please refer to the ceph user manual to be sure that the syntax you are using is appropriate for the ceph version installed. - Then repeat the same steps for the second VM as well. Now the two VMs share the same filesystem --- # Unknown .. \_db\_create\_card: Database: create ================ .. tab-set:: .. tab-item:: Horizon dashboard - Click on \*"Database → Instances → Launch Instance"\* .. image:: /cloud/\_img/op\_db\_create\_img1.png - Fill in the fields described below for the different tabs .. dropdown:: Details tab - \*\*Availability Zone\*\*: nova - \*\*Instance Name\*\*: - \*\*Volume Size\*\*: . By default the maximum allowed is 10 GB. - \*\*Volume Type\*\*: choose between "\_\_DEFAULT\_\_" or "LUKS". The second one is for encrypted volumes. - \*\*Datastore\*\*: choose among the available datastores. They are listed in the drop-down menu showing also the available versions. - \*\*Flavor\*\*: it is the dimension of the VM that will have the database volume attached. Insert fl.ada.xxs, since you will not be allowed by design to login into this VM. - \*\*Locality\*\*: None .. image:: /cloud/\_img/op\_db\_create\_img2.png .. dropdown:: Networking tab - \*\*Selected networks\*\*: , choose one among the available network in the project. - Make sure to create the network before creating the database instance (see :ref:\`cloud/operative/network\_ops/network\_create:network: create\`). .. image:: /cloud/\_img/op\_db\_create\_img3.png .. dropdown:: Database access tab - \*\*Is public\*\*: Check this box if you want to allow access to the database instance from the public network; otherwise leave blank. - \*\*Allowed CIDRs\*\*: . Specify the allowed IP or IP-ranges from which to access the database service. .. image:: /cloud/\_img/op\_db\_create\_img4.png .. dropdown:: Initialize Databases tab - \*\*Initial Databases\*\*: . Note that additional databases can be created later. - \*\*Initial Admin Users\*\*: - \*\*Password\*\*: - \*\*Allowed Hosts\*\*: optional value, to further restrict for this specific database the allowed Host or IP addresses able to connect to the database. .. image:: /cloud/\_img/op\_db\_create\_img5.png .. dropdown:: Advanced tab - \*\*Configuration Group and Source from Initial State\*\*: Fill these two fields only if you want to create the database using a previous backup, or as a replica of an other database instance. - \*\*Replica Count\*\*: fill in this field only if you want to have multiple replicas of this database instance. .. image:: /cloud/\_img/op\_db\_create\_img6.png - At the end, click on \*"Launch"\* on the right bottom to launch the Database instance. - To access the created database you can refer to dedicated page :ref:\`cloud/operative/db\_ops/db\_access:accessing the database instance\` .. tab-item:: Command Line Interface .. important:: \*\*Software required\*\* To use :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\`, the following additional packages are needed. It is recommended to install these packages in a virtual environment. .. code-block:: bash pip install python-openstackclient==5.8.0 pip install python-troveclient For more information, see python-troveclient, a command-line client for the Trove API. \*\*Setting up the environment variables\*\* Make sure to have a valid Application Credential for the project. Please refer to the :ref:\`cloud/os\_overview/management\_tools/command\_line:application credentials creation\` page where it is described how to get OpenStack Application Credentials. \*\*Create the database instance\*\* To create a database instance, you need to execute a command as \`\`openstack database instance create \`\` specifying at least the following parameters: - \*\*name\*\*: The name of the database instance. - \*\*flavor\*\*: The flavor of the database instance. Insert fl.ada.xxs, since you will not be allowed by design to login into this VM. - \*\*datastore\*\*: The datastore of the database instance. - \*\*datastore-version\*\*: The version of the datastore to use. - \*\*size\*\*: The size of the instance disk volume in GB. By default the maximum allowed is 10 GB. - \*\*nic\*\*: The network interface card of the database instance. - \*\*net-id\*\*: The network id of the database instance. - \*\*allowed-cidr\*\*: The allowed cidr of the database instance. It is an IP or a range of IPs from which the database instance can be accessed. - \*\*database\*\*: The name of the initial database. - \*\*users\*\*: The username and password of the admin user in the database. - \*\*is-public\*\*: Add this flag to be able to access the database from internet. Otherwise, the database will be accessible only from an other VM present in the same network of the project. For the full list of options please type the command: \`\`openstack database instance create --help\`\`. Make sure to create a network stack (:ref:\`cloud/operative/network\_ops/network\_create:network: create\`) and copy the id of the network before creating a database instance. The example below shows how to create a database instance: .. code-block:: bash openstack database instance create MyTroveDB --flavor fl.ada.xxs --datastore mysql --datastore-version 5.7.29 --size 10 --nic net-id= \\ --databases test --is-public --users : --allowed-cidr xx.xx.xx.xx/y +--------------------------+--------------------------------------+ | Field | Value | +--------------------------+--------------------------------------+ | allowed\_cidrs | \[xx.xx.xx.xx/y\] | | created | 2023-03-30T10:09:46 | | datastore | mysql | | datastore\_version | 5.7.29 | | datastore\_version\_number | 5.7.29 | | flavor | 3496e9e0-60c4-471a-99ce-51f3d0a8048b | | id | --- the ID of the DB instance | | name | MyTroveDB | | operating\_status | | | public | True | | region | RegionOne | | service\_status\_updated | 2023-03-30T10:09:46 | | status | BUILD | | updated | 2023-03-30T10:09:46 | | volume | 10 | +--------------------------+--------------------------------------+ Once created, successfully, - if the flag \*"--is-public"\* is specified, you will be provided with a public IP (also referred to as Floating IP) address attached to the database instance. You can use this IP address to reach the database instance from the internet. - Otherwise, only the IP of the internal network of the project will be presented. The allowed-cidr address will determine whether the database/s can be accessed from outside of the network. If the user wants to access the database from another VM in the same network then the user has to specify the CIDR of the network where the VM belongs to, whereas, for public internet access, the user has to specify the CIDR of the public network (for example 0.0.0.0/0 for all internet or a sub-range). \*\*Check the status of the database instance\*\* To check the status of a database instance, please use the following commands: .. code-block:: bash $ openstack database instance list # will list all the database instances present in the cluster +--------------------------------------+-----------------+-----------+-------------------+--------+------------------+--------+----------------------------------------------------------------------------------------------------+--------------------------------------+------+------+ | ID | Name | Datastore | Datastore Version | Status | Operating Status | Public | Addresses | Flavor ID | Size | Role | +--------------------------------------+-----------------+-----------+-------------------+--------+------------------+--------+----------------------------------------------------------------------------------------------------+--------------------------------------+------+------+ | | MyTroveDB | mysql | 5.7.29 | ACTIVE | HEALTHY | True | \[{'address': 'xx.xx.xx.xx', 'type': 'private'}, {'address': 'xx.xx.xx.xx', 'type': 'public'}\] | 7595d735-6de4-415f-a958-838089a09080 | 10 | | +--------------------------------------+-----------------+-----------+-------------------+--------+------------------+--------+----------------------------------------------------------------------------------------------------+--------------------------------------+------+------+ With the ID of the database instance just created, you can check its status with the following command: .. code-block:: bash $ openstack database instance show +--------------------------+-----------------------------------------------------------------------------------------------------+ | Field | Value | +--------------------------+-----------------------------------------------------------------------------------------------------+ | addresses | \[{'address': 'xx.xx.xx.xx', 'type': 'private'}, {'address': 'xx.xx.xx.xx', 'type': 'public'}\] | | allowed\_cidrs | \['xx.xx.xx.xx/y', 'xx.xx.xx.xx/y'\] | | created | 2023-06-09T09:19:11 | | datastore | mysql | | datastore\_version | 5.7.29 | | datastore\_version\_number | 5.7.29 | | flavor | 7595d735-6de4-415f-a958-838089a09080 | | id | --- the ID of the DB instance | | ip | xx.xx.xx.xx, xx.xx.xx.xx | | name | MyTroveDB | | operating\_status | HEALTHY | | public | True | | region | RegionOne | | service\_status\_updated | 2023-06-09T09:22:21 | | status | ACTIVE | | updated | 2023-06-09T09:20:48 | | volume | 10 | | volume\_used | 0.22 | +--------------------------+-----------------------------------------------------------------------------------------------------+ Also, check the status and the operating\_status. If the status is \*ACTIVE\*, then the database instance is ready to use. It may takes some time to get the database instance in ACTIVE state after being created. To access the created database you can refer to dedicated page :ref:\`cloud/operative/db\_ops/db\_access:accessing the database instance\` --- # Unknown .. \_compute\_inst\_snap\_create\_card: Instance: snapshot create ========================= The creation of a snapshot image from an existing VM hosted on OpenStack will differ depending on whether the VM is ephemeral or instantiated from a bootable drive, namely bootable VM (see :ref:\`cloud/os\_overview/os\_components/compute:instances\`). Preliminary steps ^^^^^^^^^^^^^^^^^ Independently from the type of VM, to avoid errors in the operation and inconsistent state of the VM, \*\*it is mandatory to create the snapshot after the VM has been shut down\*\*. In the :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\`: - Shutdown the VM - Detach any secondary volume attached on the VM (remember the volume \*/dev/vda\* is the bootable volume from which the VM is loaded). Ephemeral VM ^^^^^^^^^^^^^ If the VM is loaded from an image, there is no bootable volume \*/dev/vda\*. .. tab-set:: .. tab-item:: Horizon Dashboard - In the Horizon Dashboard, go to \*Compute → Instances\* - Click on \*"Create snapshot"\* action for the instance to snapshot .. image:: /cloud/\_img/op\_snap\_create\_1.jpg - From the pop-up dialog, give a unique \*“Snapshot Name”\* for the image snapshot file, then click on the \*“Create Snapshot”\*. .. image:: /cloud/\_img/op\_snap\_create\_2.jpg - The procedure will bring you to the \*"Compute → Images"\* section, where the snapshot image will appear after the generation process indicated by the transition of the image \*“Status”\* from \*“Queued”\* to \*“Active”\*, passing from \*“Saving”\*. - At the end the snapshot should appear with size different from zero. .. image:: /cloud/\_img/op\_snap\_create\_3.jpg .. tab-item:: Command Line Interface - Configure your CLI following the steps in :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` - Obtain the complete tabular list of all the servers available on the tenant .. code-block:: bash openstack server list +----------------+------------------+--------+-----+------------------+ | ID | Name | Status | ... | Flavor | +----------------+------------------+--------+-----+------------------+ | | | ACTIVE | | | | | | ACTIVE | | | | | | ACTIVE | | | ... | | | ACTIVE | | | +----------------+------------------+--------+-----+------------------+ - Use the ID corresponding to the server name from which is desired to generate a snapshot image to run the command .. code-block:: bash openstack server image create --name --wait - Even if the CLI does not show a progress bar for the snapshotting procedure, the shell will hang until the snapshot image \*“Status”\* will become \*“Active”\*. - At the completion of the creation, you can check if the snapshot has been correctly created and in active state with the command .. code-block:: bash openstack image list --name +---------------+------------------+--------+ | ID | Name | Status | +---------------+------------------+--------+ | | | active | +---------------+------------------+--------+ Bootable VM ^^^^^^^^^^^^^ If the VM is loaded from a bootable volume, there is a volume attached as \*/dev/vda\*. .. important:: For bootable VM, the snapshotting procedure can be done only if the \*\*bootable volume is not set to be deleted\*\* once the attached VM will be deleted. Before starting the creation of the VM snapshot, - save the ID of the bootable volume from which your VM has been created () - delete the VM instance to snapshot following the instructions in :ref:\`cloud/operative/compute\_ops/instance\_deletion:instance: delete\`. .. tab-set:: .. tab-item:: Horizon dashboard - Navigate to \*"Volumes → Volumes"\* - On the row corresponding to the volume attached to the previously deleted instance, click on the action \*“Upload to Image”\*. .. image:: /cloud/\_img/op\_snap\_create\_5.jpg - From the pop-up dialog, give a unique \*“Snapshot Name”\* for the image snapshot file, then click on \*“Upload”\*. .. image:: /cloud/\_img/op\_snap\_create\_6.jpg - The procedure will close the pop-up window and the status of the volume will change to \*“uploading”\*. .. image:: /cloud/\_img/op\_snap\_create\_7.jpg - Once the status of the volume will become once again \*“Available”\*, a snapshot image of the volume will appear in the \*“Compute → Images”\* section with a size different from zero. .. tab-item:: Command Line Interface - Configure your CLI following the steps in :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\` - Use the bootable , saved before, to upload a snapshot image .. code-block:: bash openstack image create --volume +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | container\_format | bare | | disk\_format | raw | | display\_description | | | id | | | image\_id | | | image\_name | | | protected | False | | size | | | status | uploading | | updated\_at | | | visibility | shared | | volume\_type | \_\_DEFAULT\_\_ | +---------------------+--------------------------------------+ - The creation will be completed, once the status of the image will become \*“Available”\*, you can check this with .. code-block:: bash openstack image list --name +---------------+------------------+--------+ | ID | Name | Status | +---------------+------------------+--------+ | | | active | +---------------+------------------+--------+ --- # Unknown .. \_volume\_resize\_card: Volume: resize ============== Resize a volume --------------- Users are able to increase in size any of their volumes. This operation can be done either via :ref:\`cloud/os\_overview/management\_tools/dashboard:horizon dashboard\` or via :ref:\`cloud/os\_overview/management\_tools/command\_line:command line interface\`. .. important:: It is not possible to decrease the size of the volume (the volume can only be extended). The operations change depending on whether the volume to be resized is a primary volume (like the root volume of a bootable VM) or secondary volume. .. tab-set:: .. tab-item:: Primary Volume In order to perform the resize operation of a primary volume (\*e.g.\* the root disk of a bootable VM), it is strongly recommended shuting down the VM and it is mandatory to delete the instance attached to the volume. However, is it possible to ensure that the volume is not removed togheter with the instance to which is attached, by verifying that the volume's \`\`Delete On Termination\`\` property is set to \`\`False\`\`. This flag is only visible via command line: First of all identify the ID of the bootable VM with: .. code:: bash $ openstack server list +----------------+------------------+--------+-----+--------------------------+------------------+ | ID | Name | Status | ... | Image | Falvor | +----------------+------------------+--------+-----+--------------------------+------------------+ | | | ACTIVE | | N/A (booted from volume) | | | | | ACTIVE | | | | | | | ACTIVE | | | | ... | | | ACTIVE | | | | +----------------+------------------+--------+-----+--------------------------+------------------+ Then, visualize the details of the volume attached to the instance as follows: .. code:: bash $ openstack server volume list +----------+---------------------------+------+------------------------+-----------------+-------------------------+ | Device | Server ID | Volume ID | Tag | Delete On Termination? | Attachment ID | BlockDeviceMapping UUID | +----------+-------------+-------------+------+------------------------+-----------------+-------------------------+ | /dev/vda | | | None | False | | | +----------+-------------+-------------+------+------------------------+-------------------------------------------+ In case, that the volume's \`\`Delete On Termination\`\` property is set to \`\`True\`\`, it can be changed via the following command: .. code:: bash $ openstack server volume set --preserve-on-termination .. tab-set:: .. tab-item:: Horizon Dashboard Delete the instance .. image:: /cloud/\_img/op\_volume\_resize\_img8.png Navigate to the \*"Volumes"\* tab and proceed to resize the volume by selecting the action \*"Extend Volume"\*. .. image:: /cloud/\_img/op\_volume\_resize\_img4.jpg In the pop-up window that opens, you can select the New Size. .. image:: /cloud/\_img/op\_volume\_resize\_img5.jpg After entering the new size, click on \*"Extend Volume"\* button. .. tab-item:: Command Line Interface First, list the servers and the volumes to identify the IDs of the bootable VM. .. code:: bash $ openstack server list +----------------+------------------+--------+-----+--------------------------+------------------+ | ID | Name | Status | ... | Image | Falvor | +----------------+------------------+--------+-----+--------------------------+------------------+ | | | ACTIVE | | N/A (booted from volume) | | | | | ACTIVE | | | | | | | ACTIVE | | | | ... | | | ACTIVE | | | | +----------------+------------------+--------+-----+--------------------------+------------------+ Identify the volume ID used as boot source for the VM. .. code:: bash $ openstack server volume list +----------+---------------------------+------+------------------------+-----------------+-------------------------+ | Device | Server ID | Volume ID | Tag | Delete On Termination? | Attachment ID | BlockDeviceMapping UUID | +----------+-------------+-------------+------+------------------------+-----------------+-------------------------+ | /dev/vda | | | None | False | | | +----------+-------------+-------------+------+------------------------+-------------------------------------------+ Remove the instance .. code:: bash $ openstack server delete Resize the volume: .. code:: bash $ openstack volume set --size | Once the volume has been resized, you can proceed in creating a new bootable VM using the newly sized volume as boot source. .. important:: As opposed to the resize procedure of a secondary volume, the added disk space will be automaticaly used for the extension of the exsiting disk. Be aware that the \`\`WARNING REMOTE HOST IDENTIFICATION HAS CHANGED\`\` message will arise when trying to ssh into the new instance if the floating IP assigned to the deleted VM will be assigned to the new one. .. tab-item:: Secondary Volume If the volume is attached to a virtual machine instance, we strongly recommend shutting down the instance and detatching the volume before performing the operation. The resize operation can be performed both via the Horizon Dashboard and the OpenStack command line interface. .. tab-set:: .. tab-item:: Horizon Dashboard Make sure that the volume you want to resize is not attached to any instance. If it is attached, detach it following the steps in :ref:\`cloud/operative/storage\_ops/volume\_create:Attach/Detach a volume\`. .. image:: /cloud/\_img/op\_volume\_resize\_img2.jpg And select the volume you want to resize from the list. .. image:: /cloud/\_img/op\_volume\_resize\_img3.jpg Now you can proceed to resize the volume by selecting the action \*"Extend Volume"\*. .. image:: /cloud/\_img/op\_volume\_resize\_img4.jpg In the pop-up window that opens, you can select the New Size. .. image:: /cloud/\_img/op\_volume\_resize\_img5.jpg After entering the new size, click on \*"Extend Volume"\* button. Once the volume has been resized, you can re-attach it to the instance using the Manage Attachments option. .. image:: /cloud/\_img/op\_volume\_resize\_img6.jpg .. image:: /cloud/\_img/op\_volume\_resize\_img7.jpg .. tab-item:: Command Line Interface First, list the servers and the volumes to identify the IDs of the resources. .. code:: bash $ openstack server list +----------------+------------------+--------+-----+------------------+ | ID | Name | Status | ... | Flavor | +----------------+------------------+--------+-----+------------------+ | | | ACTIVE | | | | | | ACTIVE | | | | | | ACTIVE | | | ... | | | ACTIVE | | | +----------------+------------------+--------+-----+------------------+ .. code:: bash $ openstack volume list +----------------+------------------+----------+---------------------------------------+ | ID | Name | Status | Attached to | +----------------+------------------+----------+---------------------------------------+ | | | | | | | | | Attached to on /dev/vdX | | | | | | ... | | | | Attached to on /dev/vdX | +----------------+------------------+----------+---------------------------------------+ Detach the volume from the instance (if it is attached): .. code:: bash $ openstack server remove volume Resize the volume: .. code:: bash $ openstack volume set --size Finally, you can re-attach the volume to the instance: .. code:: bash $ openstack server add volume +-----------------------+-------------+ | Field | Value | +-----------------------+-------------+ | ID | | | Server ID | | | Volume ID | | | Device | /dev/vdX | | Tag | None | | Delete On Termination | False | +-----------------------+-------------+ | After this, you can restart the instance. Once these operations are done, OpenStack will assume the volume has the new size and it will appear to the machine as a device of a different size. .. important:: It is necessary to update the device partitions and then the filesystem must be extended to occupy all the free space that has been created. These operations depend on the operating system and the type of filesystem. For example, on a Rocky Linux system, you can resize an ext4 filesystem using the tools \`\`fdisk\`\`, \`\`e2fsck\`\`, and \`\`resize2fs\`\`. During these operations, all precautions must be taken to avoid data loss; therefore, we suggest you perform the necessary checks and ensure you use the specific tools depending on the volume's filesystem. Refer to your operating system documentation for more details on resizing partitions and filesystems. .. dropdown:: Example: extending a partition formatted with XFS filesystem on Ubuntu As a reference, we provide here an example of extending a partition formatted with \*\*XFS filesystem\*\* on \*\*Ubuntu 24.04\*\*. We assume the volume has been already extended in OpenStack as shown above. We cannot cover all the combinations of OS and filesystems, so please refer to your OS documentation for the specific commands needed. .. important:: These operations can lead to data loss if not done properly. Please ensure you have backups and understand the commands before executing them. The following example assumes the volume is attached as \`\`/dev/sdb1\`\` and is mounted in \`\`/data\`\`. The volume has been resized from 250GB to 400GB in OpenStack. - Check file system space usage .. code:: console ubuntu@vm:~$ df -h Filesystem Size Used Avail Use% Mounted on tmpfs 34G 1.5M 34G 1% /run /dev/sda1 29G 25G 3.1G 90% / tmpfs 166G 4.0K 166G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda16 881M 117M 703M 15% /boot /dev/sda15 105M 6.2M 99M 6% /boot/efi tmpfs 34G 20K 34G 1% /run/user/1000 /dev/sdb1 250G 188G 63G 76% /data # <-- Filesystem to be extended - Show block devices and partitions .. code:: console ubuntu@vm:~$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 30G 0 disk ├─sda1 8:1 0 29G 0 part / ├─sda14 8:14 0 4M 0 part ├─sda15 8:15 0 106M 0 part /boot/efi └─sda16 259:0 0 913M 0 part /boot sdb 8:16 0 400G 0 disk # <-- Volume is 400GB └─sdb1 8:17 0 250G 0 part /data # <-- Partition is still 250GB - Grow the partition to occupy all the new space .. code:: console ubuntu@vm:~$ sudo growpart /dev/sdb 1 CHANGED: partition=1 start=2048 old: size=524285919 end=524287966 new: size=838858719 end=838860766 - Verify partition resized .. code:: console ubuntu@vm:~$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 30G 0 disk ├─sda1 8:1 0 29G 0 part / ├─sda14 8:14 0 4M 0 part ├─sda15 8:15 0 106M 0 part /boot/efi └─sda16 259:0 0 913M 0 part /boot sdb 8:16 0 400G 0 disk └─sdb1 8:17 0 400G 0 part /data # <-- Partition was extended - Verify filesystem size .. code:: console ubuntu@vm:~$ df -h Filesystem Size Used Avail Use% Mounted on tmpfs 34G 1.5M 34G 1% /run /dev/sda1 29G 25G 3.1G 90% / tmpfs 166G 4.0K 166G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda16 881M 117M 703M 15% /boot /dev/sda15 105M 6.2M 99M 6% /boot/efi tmpfs 34G 20K 34G 1% /run/user/1000 /dev/sdb1 250G 188G 63G 76% /data # <-- Filesystem not yet updated - Grow the XFS filesystem .. code:: console ubuntu@vm:~$ sudo xfs\_growfs /data meta-data=/dev/sdb1 isize=512 agcount=7, agsize=9830336 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=1 = reflink=1 bigtime=1 inobtcount=1 nrext64=0 data = bsize=4096 blocks=65535739, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=19199, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 65535739 to 104857339 - Verify that the filesystem has been resized .. code:: console ubuntu@vm:~$ df -h Filesystem Size Used Avail Use% Mounted on tmpfs 34G 1.5M 34G 1% /run /dev/sda1 29G 25G 3.1G 90% / tmpfs 166G 4.0K 166G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda16 881M 117M 703M 15% /boot /dev/sda15 105M 6.2M 99M 6% /boot/efi tmpfs 34G 20K 34G 1% /run/user/1000 /dev/sdb1 400G 191G 210G 48% /data # <-- Expected result As you can see, the partition and filesystem have been successfully resized to utilize the full 400GB of the volume. Also, in this specific case there's no need to unmount the filesystem or stop the machine to perform the resize operation, but this may vary depending on the OS and filesystem type. .. important:: There is a limit of 2TB for formatting volumes with the MBR partition table. If you need to create or resize a volume beyond this limit, you must use the GPT partition table. ---